1. A method for training a deep neural network, comprising: receiving and formatting speech data for the training; performing Hessian-free sequence training on a first subset of a plurality of subsets of the speech data; and iteratively performing the Hessian-free sequence training on successive subsets of the plurality of subsets of the speech data; wherein iteratively performing the Hessian-free sequence training comprises reusing information from at least one previous iteration; and wherein the receiving, formatting, performing and iteratively performing steps are performed by a computer system comprising a memory and at least one processor coupled to the memory.