5. A computer-implemented method comprising: as implemented by one or more computing devices configured with specific computer-executable instructions, obtaining a confusion probability that a first phoneme is recognized as a second phoneme; obtaining a first text comprising a first word; obtaining a second text comprising a second word; generating, using the confusion probability, an erroneous hypothesis for the second text, wherein generating the erroneous hypothesis comprises substituting the first word for the second word in the second text; obtaining a language model comprising a plurality of n-grams; for a first n-gram of the plurality of n-grams, wherein the n-gram comprises the first word in a context of one or more other words, determining a plurality of values using the language model, wherein the language model comprises the first n-gram, the plurality of values comprising: an n-gram probability for the first n-gram; a backoff probability for the first n-gram; a first probability that a true hypothesis comprising the first text is correct; and a second probability that the erroneous hypothesis is correct; computing an objective function value using the plurality of values, wherein the objective function value is based at least partly on a difference between (i) a first sum of values computed for individual true hypotheses including the true hypothesis and (ii) a second sum of values computed for individual erroneous hypotheses including the erroneous hypothesis, wherein the value computed for the true hypothesis is computed using the first probability, the n-gram probability and the backoff probability, and wherein the value computed for the erroneous hypothesis is computed using the second probability, the n-gram probability, and the backoff probability; based at least in part on the objective function value, pruning the first n-gram from the language model to generate a pruned language model; and recognizing user speech using the pruned language model and an audio signal corresponding to speech of a user.
6. The computer-implemented method of claim 5 , wherein: the first probability is computed using the n-gram probability; and the second probability is computed using the backoff probability.