18. The system of claim 11 , wherein the one or more computers are configured to perform operations comprising: storing a plurality of scores for a plurality of contexts; and in response to adjusting a respective base weight for each of the one or more contexts, updating the adjusted scores for the one or more contexts.
11. A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving a voice input; determining a transcription for the voice input, wherein determining the transcription for the voice input includes, for a plurality of segments of the voice input: maintaining a plurality of contexts and respective base weights associated with the plurality of contexts; obtaining a first candidate transcription for a first segment of the voice input; determining one or more contexts, from the plurality of contexts, associated with the first candidate transcription; identifying one or more base weights respectively corresponding to the one or more contexts; adjusting a respective base weight of the one or more base weights for each of the one or more contexts based on the first candidate transcription; and determining a second candidate transcription for a second segment of the voice input based in part on the adjusted base weights; and providing the transcription of the plurality of segments of the voice input for output.