1. A method for training a deep neural network, comprising: receiving and formatting speech data for the training; preconditioning a system of equations to be used for analyzing the speech data in connection with the training by using a non-fixed point quasi-Newton preconditioning scheme; and employing flexible Krylov subspace solvers in response to variations in the preconditioning scheme for different iterations of the training; wherein the receiving, formatting, preconditioning and employing steps are performed by a computer system comprising a memory and at least one processor coupled to the memory.