Book excerpt: Robustness due to mismatched train/test conditions is the biggest challenge facing the speaker recognition community today, with transmission channel and environmental noise degradation being the prominent factors. Performance of state-of-the art speaker recognition methods aim at mitigating these factors by effectively modeling speech in multiple recording conditions, so that it can learn to distinguish between inter-speaker and intra-speaker variability. The increasing demand and availability of large development corpora introduces difficulties in effective data utilization and computationally efficient modeling. Traditional compensation strategies operate on higher dimensional utterance features, known as supervectors, which are obtained from the acoustic modeling of short-time features. Feature compensation is performed during front-end processing. Motivated by the covariance structure of conventional acoustic features, we envision that feature normalization and compensation can be integrated into the acoustic modeling. In this dissertation, we investigate the following fundamental research challenges: (i) analysis of data requirements for effective and efficient background model training, (ii) introducing latent factor analysis modeling of acoustic features, (iii) integration of channel compensation strategies in mixture-models, and (iv) development of noise robust background models using factor analysis. The effectiveness of the proposed solutions are demonstrated in various noisy and channel degraded conditions using the recent evaluation datasets released by the National Institute of Standards and Technology (NIST). These research accomplishments make an important step towards improving speaker recognition robustness in diverse acoustic conditions.