17. A system comprising: one or more processors and one or more computer-readable storage media storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising: performing a first training process comprising: obtaining, for a first sequence-training speech model, a first batch of training frames that represent speech features of first training utterances; obtaining, for the first sequence-training speech model, one or more first neural network parameters; and determining, for the first sequence-training speech model, one or more adjusted first neural network parameters based on (i) the first batch of training frames and (ii) the one or more first neural network parameters; performing a second training process comprising: obtaining, for a second sequence-training speech model, a second batch of training frames that represent speech features of second training utterances, wherein the obtaining of the second batch of training frames by the second sequence-training speech model is independent of the obtaining of the first batch of training frames by the first sequence-training speech model; obtaining, for the second sequence-training speech model, one or more second neural network parameters, wherein the obtaining of the second neural network parameters by the second sequence-training speech model is independent of (i) the obtaining of the first neural network parameters by the first sequence-training speech model and (ii) the determining of the one or more adjusted first neural network parameters by the first sequence-training speech model; and determining, for the second sequence-training speech model, one or more adjusted second neural network parameters based on (i) the second batch of training frames and (ii) the one or more second neural network parameters, wherein the determining of the one or more adjusted second neural network parameters for the second sequence-training speech model is independent of the determining of the one or more adjusted first neural network parameters for the first sequence-training speech model; providing the one or more adjusted first neural network parameters and the one or more adjusted second neural network parameters to a computing system comprising at least one of the one or more processors; and determining, by the computing system, neural network parameters for a third sequence-training speech model based on the one or more adjusted first neural network parameters and the one or more adjusted second neural network parameters; wherein the first training process is performed concurrently with the second training process and asynchronously with respect to the second training process.
17. A system comprising: one or more processors and one or more computer-readable storage media storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising: performing a first training process comprising: obtaining, for a first sequence-training speech model, a first batch of training frames that represent speech features of first training utterances; obtaining, for the first sequence-training speech model, one or more first neural network parameters; and determining, for the first sequence-training speech model, one or more adjusted first neural network parameters based on (i) the first batch of training frames and (ii) the one or more first neural network parameters; performing a second training process comprising: obtaining, for a second sequence-training speech model, a second batch of training frames that represent speech features of second training utterances, wherein the obtaining of the second batch of training frames by the second sequence-training speech model is independent of the obtaining of the first batch of training frames by the first sequence-training speech model; obtaining, for the second sequence-training speech model, one or more second neural network parameters, wherein the obtaining of the second neural network parameters by the second sequence-training speech model is independent of (i) the obtaining of the first neural network parameters by the first sequence-training speech model and (ii) the determining of the one or more adjusted first neural network parameters by the first sequence-training speech model; and determining, for the second sequence-training speech model, one or more adjusted second neural network parameters based on (i) the second batch of training frames and (ii) the one or more second neural network parameters, wherein the determining of the one or more adjusted second neural network parameters for the second sequence-training speech model is independent of the determining of the one or more adjusted first neural network parameters for the first sequence-training speech model; providing the one or more adjusted first neural network parameters and the one or more adjusted second neural network parameters to a computing system comprising at least one of the one or more processors; and determining, by the computing system, neural network parameters for a third sequence-training speech model based on the one or more adjusted first neural network parameters and the one or more adjusted second neural network parameters; wherein the first training process is performed concurrently with the second training process and asynchronously with respect to the second training process. 19. The system of claim 17 , wherein the operations comprise: before obtaining, by the first sequence-training speech model, the first batch of training frames: obtaining an initial batch of training frames, wherein each training frame represents a sequence of utterances spoken by a training speaker; pseudo-randomly selecting candidate training frames from the initial batch of training frames; and generating the first batch of training frames using the pseudo-randomly selected candidate training frames.