1. A computer-implemented method comprising: receiving, by a computer system, a request to predict a next word to occur in a phrase being uttered by a first user in a dialog between the first user and a second user; accessing, by the computer system, a neural network comprising i) an input layer that includes a first portion representing a local context for the phrase and a second portion representing a semantic context for the phrase, ii) one or more hidden layers that are at least partially interconnected with the input layer by first connections, and iii) an output layer that represents a vocabulary of candidate words and that is at least partially interconnected with at least one of the one or more hidden layers by second connections; identifying, by the computer system, the local context for the phrase being uttered by the first user; identifying, by the computer system, text of one or more previous messages communicated (i) between the first user and the second user, and (ii) before initiation of the dialog between the first user and the second user; determining, by the computer system and based at least on the identified text of the one or more previous messages, at least one vector that represent the semantic context for the phrase, the at least one vector including values for a plurality of dimensions; applying, by the computer system, input to the input layer of the neural network, the input comprising i) the local context of the phrase and ii) the values for the plurality of dimensions of the at least one vector that represents the semantic context of the phrase; generating, by the computer system, probability values for at least a portion of the candidate words in the vocabulary of the output layer based on propagation of the input through the neural network using, at least, the first connections and the second connections between layers of the neural network; using, by the computer system, the generated probability values to determine a transcription for the phrase uttered by the first user; and providing, by the computer system and as output of an automated speech recognizer, the transcription determined using the generated probability values.