18. The method according to claim 17 , wherein the determining step includes using substantially a Viterbi algorithm to determine the path through the aggregate lattice that is mathematically optimal.
17. A method of providing a description of a video, the method comprising automatically performing the following steps using a processor: detecting one or more candidate object(s) in each of a plurality of frames of the video using image data of the plurality of frames; generating a candidate description including one or more participant(s) and one or more condition(s) applied to the participant(s), the conditions and participants selected from a linguistic model; constructing a plurality of respective component lattices corresponding to the participant(s) or condition(s), wherein at least one of the component lattices includes a node corresponding to one of the candidate objects detected in one of the frames of the video; producing an aggregate lattice having a plurality of nodes, each of which includes a respective factor computed from corresponding nodes in a respective plurality of corresponding ones of the component lattices; determining a score for the video with respect to the candidate description by determining an aggregate score for a path through the aggregate lattice; if the aggregate score does not satisfy a termination condition, altering the candidate description by adding to it one or more participant(s) or condition(s) selected from the linguistic model, and repeating the constructing, producing, and determining steps with respect to the altered candidate description.