13. A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving, over a computer network, audio data for an utterance detected by a client device in communication with the one or more computers over the computer network; accessing association data that indicates a plurality of associations, each association indicating (i) uncorrupted audio data indicating characteristics of an uncorrupted audio segment, and (ii) a corresponding key based on a corrupted version of the same uncorrupted audio segment, the associations being determined before receiving the audio data for the utterance; selecting uncorrupted audio data based on a comparison of (i) one or more keys based on the audio data for the utterance, with (ii) the keys based on the corrupted audio data; constructing a representation of the utterance comprising the selected uncorrupted audio data; and performing speech recognition on the constructed representation of the utterance to determine a transcription of the utterance.
13. A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving, over a computer network, audio data for an utterance detected by a client device in communication with the one or more computers over the computer network; accessing association data that indicates a plurality of associations, each association indicating (i) uncorrupted audio data indicating characteristics of an uncorrupted audio segment, and (ii) a corresponding key based on a corrupted version of the same uncorrupted audio segment, the associations being determined before receiving the audio data for the utterance; selecting uncorrupted audio data based on a comparison of (i) one or more keys based on the audio data for the utterance, with (ii) the keys based on the corrupted audio data; constructing a representation of the utterance comprising the selected uncorrupted audio data; and performing speech recognition on the constructed representation of the utterance to determine a transcription of the utterance. 18. The system of claim 13 , wherein the operations comprise: before receiving the audio data for the utterance: accessing data that includes uncorrupted audio segments; adding noise to the uncorrupted audio segments to generate, for each uncorrupted audio segment, one or more corrupted versions of the uncorrupted audio segment, wherein each of the one or more corrupted versions of the uncorrupted audio segment has different noise added to the uncorrupted audio segment; generating, for each of the corrupted versions of the uncorrupted audio segments, association data that indicates an association of (i) uncorrupted audio data indicating characteristics of a particular uncorrupted audio segment, and (ii) a corresponding key based on a corrupted version of the particular uncorrupted audio segment; and storing the association data.