3. The method of claim 1 , wherein processing the representation for the input acoustic sequence using an attention-based Recurrent Neural Network (RNN) comprises, for the initial position in the output sequence order: processing a placeholder start of sequence token and a placeholder initial attention context vector using the attention-based RNN to update a hidden state of the attention-based RNN from an initial hidden state to a hidden state for the initial position in the output sequence order; generating an attention context vector for the initial position from the representation of the input acoustic sequence and the RNN hidden state for the initial position in the output sequence order; and generating the set of substring scores for the initial position using the attention context vector for the initial position and the RNN hidden state for the initial position.
1. A computer implemented method comprising: obtaining a representation of an input acoustic sequence, the input acoustic sequence representing an utterance; processing the representation of the input acoustic sequence using an attention-based Recurrent Neural Network (RNN) to generate, for each position in an output sequence order, a set of substring scores that includes a respective substring score for each substring in a set of substrings, comprising, for each position after an initial position in the output sequence order: processing a substring at the preceding position in the output sequence order and the attention context vector for the preceding position in the order using the attention-based RNN to update the hidden state of the attention-based RNN from the hidden state for the preceding position to a hidden state for the position; generating an attention context vector for the position from the representation and the RNN hidden state for the position in the output sequence order; and generating the set of substring scores for the position using the attention context vector for the position and the RNN hidden state for the position; and generating a sequence of substrings that represent a transcription of the utterance.