1. A method comprising: receiving an acoustic sequence, the acoustic sequence representing an utterance, and the acoustic sequence comprising a respective acoustic feature representation at each of a plurality of time steps; and generating, for use in a speech recognition system, a phoneme representation for the utterance, comprising: at a particular time step of the plurality of time steps: combining the acoustic feature representation at the particular time step and a preceding output generated by an acoustic modeling neural network for a preceding time step to generate a modified input; and processing the modified input using the acoustic modeling neural network to generate an output for the particular time step.