1. A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: accessing, by the one or more computers, a neural network that has been trained, using speech in each of multiple languages, to be able to provide prosody information for each of the multiple languages; providing, by the one or more computers, input to the neural network that includes (i) a representation of a text in a first language and (ii) a language identifier for the first language; generating, by the one or more computers, audio data for a synthesized utterance of the text in the first language based on prosody information for the text that is output by the neural network in response to receiving the representation of the text and the language identifier for the first language; and providing, by the one or more computers, the audio data for the synthesized utterance of the text in the first language.
1. A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: accessing, by the one or more computers, a neural network that has been trained, using speech in each of multiple languages, to be able to provide prosody information for each of the multiple languages; providing, by the one or more computers, input to the neural network that includes (i) a representation of a text in a first language and (ii) a language identifier for the first language; generating, by the one or more computers, audio data for a synthesized utterance of the text in the first language based on prosody information for the text that is output by the neural network in response to receiving the representation of the text and the language identifier for the first language; and providing, by the one or more computers, the audio data for the synthesized utterance of the text in the first language. 10. The system of claim 1 , wherein the operations further comprise: providing, by the one or more computers, input to the neural network that includes (i) a representation of a second text in a second language that is different than the first language and (ii) a language identifier for the second language that is different from the language identifier for the first language; generating, by the one or more computers, audio data for a synthesized utterance of the second text in the second language based on prosody information for the second text that is output by the neural network in response to receiving the representation of the second text and the language identifier for the second language; and providing, by the one or more computers, the audio data for the synthesized utterance of the second text in the second language.