1. A computer-implemented method in a dialog system, comprising: receiving input from a plurality of input modalities; synchronizing the inputs based on the temporal order of events associated with the input; generating an initial hypothesis for an input for an event; receiving constraints for the input based on one or more other inputs of the plurality of inputs; combining the constraints with the initial hypothesis; and deriving a new hypothesis to interpret the input.