15. The computer program product of claim 12 , wherein the operations further comprise calculating backoff scores for n-grams using the set of backoff factors and storing the backoff scores.
12. A computer program product, encoded on a computer-readable storage device, operable to cause data processing apparatus to perform operations comprising: generating a language model, including: identifying a collection of n-grams from a corpus of training data; determining, for each n-gram of the collection, a corresponding relative frequency of occurring in the corpus, each n-gram being a sequence of tokens of length k, wherein k is an integer between 1 and a maximum length N; and identifying a set of backoff factors α k , one backoff factor for each value of k from 2 to N, inclusive, where the backoff factors can be used to identify a backoff score S k for any n-gram of length k that is not present in the language model, wherein each n-gram of length k that is not present in the language model has a particular longest backoff n-gram that is present in the collection of n-grams and that has a particular relative frequency f and a particular length m, wherein the backoff score S k is determined as a function of the collection of backoff factors, the particular length m and the particular relative frequency f of the particular longest backoff n-gram.