1. A mobile device for processing multi-modal natural language inputs, comprising: a conversational voice user interface that receives a multi-modal natural language input from a user, the multi-modal natural language input including a natural language utterance and a non-speech input, the conversational voice user interface coupled to a transcription module that transcribes the non-speech input to create a non-speech-based transcription; a conversational speech analysis engine that identifies the user that provided the multi-modal natural language input, the conversational speech analysis engine using a speech recognition engine and a semantic knowledge-based model to create a speech-based transcription of the natural language utterance, wherein the semantic knowledge-based model includes a personalized cognitive model derived from one or more prior interactions between the identified user and the mobile device, a general cognitive model derived from one or more prior interactions between a plurality of users and the mobile device, and an environmental model derived from an environment of the identified user and the mobile device; a merging module that merges the speech-based transcription and the non-speech-based transcription to create a merged transcription; a knowledge-enhanced speech recognition engine that identifies one or more entries in a context stack matching information contained in the merged transcription and determines a most likely context for the multi-modal natural language input based on the identified entries; and a response generating module that identifies a domain agent associated with the most likely context for the multi-modal input, communicates a request to the identified domain agent, and generates a response to the user from content provided by the identified domain agent as a result of processing the request.