21. A computer-implemented method comprising: transmitting, at a client device, audio data to a server system; identifying a first context of the client device; transmitting information indicating the first context to the server system; and receiving, at the client device, a transcription of at least a portion of the audio data at the client device, the server system having accessed at least one term, accessed information that indicates a second context, the second context being associated with the term, determined a similarity score that indicates a degree of similarity between the second context and the first context, adjusted a language model based on the accessed term and the determined similarity score to generate an adjusted language model, wherein the adjusted language model includes the accessed term and a weighting value assigned to the accessed term based on the similarity score, wherein the weighting value changes a probability for a transition between a first decision state in the language model and a second decision state in the language model, performed speech recognition on the audio data using the adjusted language model to select one or more candidate transcriptions for a portion of the audio data, and transmitted the transcription to the client device.
23. The computer-implemented method of claim 21 , wherein the server system has further determined that the accessed term was entered by a particular user, and wherein: the audio data encodes speech of the user; the first context includes the environment in which the speech occurred; and the second context includes the environment in which the accessed term was entered.