1. A method for recognizing an utterance comprising: receiving an utterance comprising a first portion having a first mode and a second portion having a second mode; generating a first mode recognition lattice and a first finite-state transducer, based on the first mode; relating the first portion of the utterance to the second portion of the utterance based on the first finite-state transducer; generating a second finite-state transducer, comprising a gesture and speech recognition model finite-state transducer, based on the first mode recognition lattice and the first finite-state transducer; and outputting a recognition result based on the utterance and the second finite-state transducer.
1. A method for recognizing an utterance comprising: receiving an utterance comprising a first portion having a first mode and a second portion having a second mode; generating a first mode recognition lattice and a first finite-state transducer, based on the first mode; relating the first portion of the utterance to the second portion of the utterance based on the first finite-state transducer; generating a second finite-state transducer, comprising a gesture and speech recognition model finite-state transducer, based on the first mode recognition lattice and the first finite-state transducer; and outputting a recognition result based on the utterance and the second finite-state transducer. 5. The method of claim 1 wherein the generating the first mode recognition lattice is further based on a first mode feature lattice.