1. A method comprising: obtaining first data from a plurality of calls routed by a natural language call routing system that uses a first speech recognition model to recognize speech of the plurality of calls and a first action classification model to route calls based on the recognized speech, the first data comprising audio data from the plurality of calls, a first N-best list of word sequences generated by using the first speech recognition model to recognize at least a portion of the audio data and associated call classification data indicating how each of the plurality of calls were routed by the natural language call routing system; and modifying the first speech recognition model and the first action classification model based at least in part on the first data, including using the first N-best list of word sequences, to obtain a second speech recognition model and a second action classification model; and modifying the second speech recognition model and the second action classification model based at least in part on the first data to obtain a third speech recognition model and a third action classification model, wherein the modifying comprises modifying the second speech recognition model and the second action classification model by using the first N-best list or a second N-best list of word sequences generated by using the second speech recognition model to recognize at least the portion of the audio data.