1. A computer-implemented method comprising: receiving, at a computer system, an audio signal; initiating, by the computer system, a plurality of speech recognition tasks for the audio signal, wherein the speech recognition tasks each use a different one of a plurality of language models; detecting that a portion of the plurality of speech recognition tasks have completed, wherein a remaining portion of the plurality of speech recognition tasks have not completed; obtaining recognition results and confidence values for each of the plurality of speech recognition tasks included in the portion, wherein the recognition results identify one or more candidate transcriptions of the audio signal, and the confidence values identify one or more probabilities that the recognition results are correct; determining, by the computer system, whether at least one of the one or more confidence values is greater than or equal to a threshold confidence value; and in response to determining that the at least one of the one or more confidence values is greater than or equal to the threshold confidence value and before all of the remaining portion of the plurality of speech recognition tasks have completed, providing a final recognition result for the audio signal based on the recognition results and the one or more confidence values.