1. A method of modifying synthesized speech of a spoken dialogue system, the method comprising: receiving a user utterance; analyzing via a processor the user utterance using a natural language understanding model to determine an appropriate speech act for responding to the user utterance; selecting at least one phoneme from a catalogue of a plurality of phonemes to yield a selected at least one phoneme, wherein the catalogue organizes phonemes based on speech acts, wherein the speech acts used to organize the catalog of a plurality of phonemes are selected from the group of speech acts consisting of: detail information, general information, “wh” questions, yes/no questions, multiple choice questions, greetings, goodbyes, apologies, thanks, requests, directives, repeat, wait, confirmations, disconfirmations, positive exclamations, filled pause, and negative exclamations; and generating a response to the user utterance of a type associated with the appropriate speech act and using the selected at least one phoneme, wherein linguistic variables in the response are selected based on the appropriate speech act.