1. A computer-implemented method comprising: selecting a candidate portion of a language model to evaluate for pruning; obtaining an entropy score representing information loss that would result from pruning the candidate portion of the language model; obtaining an acoustic score representing acoustic confusability of one or more words modeled by the candidate portion of the language model; evaluating whether to prune the candidate portion of the language model using the entropy score and the acoustic score; generating a pruned language model based on pruning the candidate portion from the language model; and generating, by a speech recognizer using the pruned language model, a transcription of an utterance.