1. A method in a computing system of updating population language models used by language recognition systems, the method comprising: receiving events associated with local language models that are used by language recognition systems in computing devices of a plurality of users; receiving information characterizing the computing devices or the plurality of users; identifying, by the computing system, from the received information, a cluster of users sharing similar characteristics; generating modifications to a population language model associated with the cluster of users by: identifying events associated with local language models that were received from users within the cluster; and aggregating the events to identify modifications to the population model; and providing the generated modifications to the population language model to a computing device of one or more users in the cluster.