1. A computer-implemented method comprising: receiving a collection of speech data items, wherein each speech data item corresponds to an utterance that was previously submitted for transcription by a production speech recognizer, and wherein the production speech recognizer uses one or more initial production speech recognizer components in generating transcriptions of speech data items; generating a transcription for each speech data item of the collection using an offline speech recognizer, wherein the offline speech recognizer uses one or more offline speech recognizer components, and wherein the one or more offline speech recognizer components are configured to improve speech recognition accuracy in comparison with the one or more initial production speech recognizer components; selecting a subset of the transcriptions of the speech data items that are generated by the offline speech recognizer; training one or more updated production speech recognizer components for the production speech recognizer using the selected subset of the transcriptions of the speech data items that are generated by the offline speech recognizer; and providing the updated production speech recognizer component to the production speech recognizer for use in transcribing one or more subsequently received speech data items.