1. A method for asynchronously training a plurality of neural networks of sequence-training speech models, comprising: obtaining, by a first sequence-training speech model, a first batch of training frames that represent speech features of first training utterances; obtaining, by the first sequence-training speech model, one or more first neural network parameters; determining, by the first sequence-training speech model, one or more optimized first neural network parameters based on (i) the first batch of training frames and (ii) the one or more first neural network parameters; obtaining, by a second sequence-training speech model, a second batch of training frames that represent speech features of second training utterances, wherein the obtaining of the second batch of training frames by the second sequence-training speech model is independent of the obtaining of the first batch of training frames by the first sequence-training speech model; obtaining one or more second neural network parameters, wherein the obtaining of the second neural network parameters by the second sequence-training speech model is independent of (i) the obtaining of the first neural network parameters by the first sequence-training speech model and (ii) the determining of the one or more optimized first neural network parameters by the first sequence-training speech model; and determining, by the second sequence-training speech model, one or more optimized second neural network parameters based on (i) the second batch of training frames and (ii) the one or more second neural network parameters, wherein the determining of the one or more optimized second neural network parameters by the second sequence-training speech model is independent of the determining of the one or more optimized first neural network parameters by the first sequence-training speech model.