16. A system, comprising: a plurality of processors; a memory that includes a plurality of computer-executable components that are executable by the plurality of processors, comprising: a batch generation component that partitions training data into sample batches of a specific batch size; and an algorithm execution component that pipelines an execution of a pipelined algorithm through the plurality of processors to train deep neural networks (DNNs) using the sample batches, the execution including executing a model update prior to an input data forward propagation in a computation iteration of the pipelined algorithm, the DNNs being one of context-dependent DNNs or context-independent DNNs, wherein the algorithm execution component trains the DNNs based at least in part on performing gradient descent techniques, wherein the DNNs include multiple layers, and wherein the execution further includes streaming output data from a computation at a first processor of the plurality of processors that processes an upper layer to a second processor of the plurality of processors that processes a lower layer following a performance of an error back propagation of the computation iteration, the streaming of the output data occurring at least partially in parallel with one or more of the model update or the input data forward propagation.
16. A system, comprising: a plurality of processors; a memory that includes a plurality of computer-executable components that are executable by the plurality of processors, comprising: a batch generation component that partitions training data into sample batches of a specific batch size; and an algorithm execution component that pipelines an execution of a pipelined algorithm through the plurality of processors to train deep neural networks (DNNs) using the sample batches, the execution including executing a model update prior to an input data forward propagation in a computation iteration of the pipelined algorithm, the DNNs being one of context-dependent DNNs or context-independent DNNs, wherein the algorithm execution component trains the DNNs based at least in part on performing gradient descent techniques, wherein the DNNs include multiple layers, and wherein the execution further includes streaming output data from a computation at a first processor of the plurality of processors that processes an upper layer to a second processor of the plurality of processors that processes a lower layer following a performance of an error back propagation of the computation iteration, the streaming of the output data occurring at least partially in parallel with one or more of the model update or the input data forward propagation. 20. The system of claim 16 , further comprising a load balance component that groups a first layer and a second layer of the multiple layers of the DNNs based at least in part on a number of the plurality of processors.