1. A method for training a deep neural network that comprises a low rank hidden input layer and an adjoining hidden layer, the low rank hidden input layer comprising a first matrix A and a second matrix B with dimensions i×m and m×o, respectively, to identify a keyword, the method comprising: receiving a feature vector comprising i values that represent features of an audio signal encoding an utterance; determining, using the low rank hidden input layer, an output vector comprising o values using the feature vector; determining, using the adjoining hidden layer, another vector using the output vector; determining a confidence score that indicates whether the utterance includes the keyword using the other vector; and adjusting one or more weights for the low rank hidden input layer based on an accuracy of the confidence score.