1. A method comprising: receiving a first sequence of words arranged according to a first order, wherein the first sequence of words comprises a plurality of sentences of words, the plurality of sentences being in an input order; for each word in the first sequence of words, beginning with a first word in the first order: determining a topic vector that is associated with the word, generating a combined input from the word and the topic vector, and processing the combined input through one or more sequence modeling layers to generate a respective sequence modeling output for the word; and processing one or more of the respective sequence modeling outputs through an output layer to generate a neural network output for the first sequence of words, wherein, for each word in each sentence after a first sentence in the input order, determining the topic vector that is associated with the word is based, at least in part, on a sequence modeling output for a last word in a sentence immediately before the sentence in the input order.
2. The method of claim 1 , wherein the one or more sequence modeling layers are recurrent neural network layers.