12. The system of claim 10 , wherein extracting the visual point of interest from the gesture input is based on the speech input, the gesture input, and a location of the vehicle.
10. A system for multimodal human-vehicle interaction, comprising: a plurality of sensors for sensing interaction data from a vehicle occupant, wherein the interaction data includes a speech input and a gesture input; a multimodal recognition module for performing multimodal recognition of the interaction data; a point of interest identification module for determining a reference to a point of interest based on the speech input and to extract a visual point of interest based on the gesture input and the reference to the point of interest in the speech input, wherein the multimodal recognition module augments a recognition hypothesis based on the visual point of interest, a belief tracking module for determining a belief state of the occupant's intent, wherein the belief state is determined based on joint probability distribution tables of probabilistic ontology trees and the probabilistic ontology trees are based on the recognition hypothesis; and a dialog management and action module for selecting an action to take based on the determined belief state.