1. A method, comprising: obtaining first data comprising audio data processed by a natural language understanding (NLU) system at least in part by using a first NLU model, a first N-best list of word sequences generated by using a first speech recognition model to recognize at least a portion of the audio data, and associated NLU data indicating how the audio data was understood by the NLU system; and modifying, using at least one hardware processor, the first speech recognition model and the first NLU model based at least in part on the first data, including using the first N-best list of word sequences, to obtain a second speech recognition model and a second NLU model.