10. The speech recognition system of claim 7 the operations wherein providing the deep neural network with the adjusted one or more weights for use in processing audio comprises providing the deep neural network to a digital signal processor for processing audio signals.
7. A speech recognition system for training a deep neural network that comprises a low rank hidden input layer with m nodes and an adjoining hidden layer with o nodes, the low rank hidden input layer comprising a first matrix A and a second matrix B with dimensions i×m and m×o, respectively, to identify a key phrase, the speech recognition system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving, by the speech recognition system that includes the deep neural network trained to identify the key phrase, a feature vector comprising i values that represent features of an audio signal encoding an utterance; determining an output vector comprising o values using the m nodes in the low rank hidden input layer by combining the feature vector, the first matrix A in the low rank hidden input layer, and the second matrix B included in the low rank hidden input layer using a linear function, wherein m is a smaller number than o; determining, using a non-linear function for the adjoining hidden layer that has the o nodes, another vector using the output vector that comprises the o values; determining a confidence score that indicates whether the utterance includes the key phrase using the other vector; adjusting one or more weights for the low rank hidden input layer based on an accuracy of the confidence score; and providing, by the speech recognition system, the deep neural network with the adjusted one or more weights for use in processing audio.