15. The method of claim 13 wherein the first mode is a gesture mode.
13. A method for extracting meaning from multimodal inputs comprising: receiving a multimodal input; relating a first portion of the multimodal input comprising a first mode to a second portion of the multimodal input comprising a second mode, based on a first finite-state transducer; generating a second finite-state transducer, comprising a gesture and speech recognition model finite-state transducer, based on a first mode recognition lattice and the first finite-state transducer; and outputting a recognition result based on the multimodal input and the second finite-state transducer.