7. A computer-implemented method comprising: under control of a server system comprising one or more computing devices configured with specific computer executable instructions, receiving audio data from a client device separate from the server system, wherein the audio data comprises data regarding an utterance of a user; producing first speech processing results using a base speech processing model and the audio data, wherein the base speech processing model is stored at the server system; obtaining a specialized speech processing model from a network-accessible data store separate from the server system and separate from the client device, wherein the obtaining is initiated based at least partly on an attribute of the specialized speech processing model and prior to completion of producing the first speech processing results; determining, based at least partly on a time at which the specialized speech processing model is obtained, that the server system is to produce second speech processing results using the specialized speech processing model subsequent to initiating production of the first speech processing results; and producing the second speech processing results using the specialized speech processing model and at least one of the audio data or the first speech processing results.
13. The computer-implemented method of claim 7 , wherein producing second speech processing results comprises re-scoring a subset of the first speech processing results, wherein the second speech processing results are produced using the subset.