15. A computer program product, encoded on a computer readable medium, operable to cause data processing apparatus to perform operations comprising: generating a compact language model, including: receiving a collection of n-grams from the corpus, each n-gram of the collection having a corresponding first probability of occurring in the corpus, and generating a trie representing the collection of n-grams including calculating a left word vector and a diversity count vector using the collection of n grams, the left word vector identifying each distinct left word for a given right context in the collection, the diversity count vector identifying a count of distinct left words for each right context in the collection, and using the language model to identify a second probability of a particular string occurring.
15. A computer program product, encoded on a computer readable medium, operable to cause data processing apparatus to perform operations comprising: generating a compact language model, including: receiving a collection of n-grams from the corpus, each n-gram of the collection having a corresponding first probability of occurring in the corpus, and generating a trie representing the collection of n-grams including calculating a left word vector and a diversity count vector using the collection of n grams, the left word vector identifying each distinct left word for a given right context in the collection, the diversity count vector identifying a count of distinct left words for each right context in the collection, and using the language model to identify a second probability of a particular string occurring. 16. The computer program product of claim 15 , further comprising: representing entries in the left word vector and diversity count vector as integers; and encoding the left word vector and the diversity count vector using block encoding, where the block encoding encodes blocks of integers in each respective vector.