1. A computer-implemented method comprising: receiving an audio signal encoding a portion of an utterance; providing, to a first neural network, data corresponding to the audio signal; and generating, by a processor, data representing a transcription for the utterance based on an output of the first neural network, wherein the first neural network is trained using features of multiple context-dependent states, the context-dependent states being derived from a plurality of context-independent states provided by a second neural network.