1. A computer-implemented method for speech recognition, the method comprising computer-implemented operations for: receiving a set of training speech utterances, wherein one or more of the training speech utterances comprises speech corrupted by noise or distortion; initializing parameters of a pseudo-clean speech model; initializing parameters of a distortion model by setting a channel mean to zero and setting a noise mean and a covariance according to one of a first or a last sample of an utterance where no speech is present; iteratively applying noise adaptive training using the set of training speech utterances to generate a speech recognition model based upon identifying a set of distortion model parameters from the distortion model and identifying a set of pseudo-clean model parameters from the pseudo-clean speech model; receiving a speech signal to be recognized; adapting the speech recognition model to a current environment associated with the received speech signal; and performing speech recognition on the received speech signal using the adapted speech recognition model.