1. A method performed by data processing apparatus, the method comprising: receiving, by a server system that provides an automated speech recognition service over a computer network, audio data for an utterance detected by a client device in communication with the server system over the computer network; accessing, by the server system, association data that indicates a plurality of associations, each association indicating (i) uncorrupted audio data indicating characteristics of an uncorrupted audio segment, and (ii) a corresponding key based on a corrupted version of the same uncorrupted audio segment, the associations being determined before receiving the audio data for the utterance; selecting, by the server system, uncorrupted audio data based on a comparison of (i) one or more keys based on the audio data for the utterance, with (ii) the keys based on the corrupted audio data; constructing, by the server system, a representation of the utterance comprising the selected uncorrupted audio data; and performing, by the server system, speech recognition on the constructed representation of the utterance to determine a transcription of the utterance.
1. A method performed by data processing apparatus, the method comprising: receiving, by a server system that provides an automated speech recognition service over a computer network, audio data for an utterance detected by a client device in communication with the server system over the computer network; accessing, by the server system, association data that indicates a plurality of associations, each association indicating (i) uncorrupted audio data indicating characteristics of an uncorrupted audio segment, and (ii) a corresponding key based on a corrupted version of the same uncorrupted audio segment, the associations being determined before receiving the audio data for the utterance; selecting, by the server system, uncorrupted audio data based on a comparison of (i) one or more keys based on the audio data for the utterance, with (ii) the keys based on the corrupted audio data; constructing, by the server system, a representation of the utterance comprising the selected uncorrupted audio data; and performing, by the server system, speech recognition on the constructed representation of the utterance to determine a transcription of the utterance. 3. The method of claim 1 , wherein each key based on corrupted audio data is a corrupted feature vector indicating characteristics of a corrupted audio segment, and where selecting the uncorrupted audio data comprises: obtaining first feature vectors of the received audio data for the utterance; comparing the first feature vectors to the corrupted feature vectors; identifying, based on the comparison of the first feature vectors to the corrupted feature vectors, a particular corrupted feature vector for a particular first feature vector; and based on identifying the particular corrupted feature vector, selecting an uncorrupted feature vector that corresponds to the particular corrupted feature vector to include in the constructed representation of the utterance.