1. A method of testing a video against an aggregate query, the video including image data of each of a plurality of frames, the method comprising automatically performing the following steps using a processor: receiving an aggregate query defining one or more participant(s) and one or more condition(s) with respect to the participant(s); detecting one or more candidate object(s) in each of the plurality of frames of the video; constructing a respective first lattice corresponding to each of the identified participant(s), each first lattice including a plurality of nodes, wherein each node in each first lattice includes a respective factor corresponding to one of the candidate objects detected in one of the frames of the video; constructing a respective second lattice corresponding to each of the identified condition(s), each second lattice including a plurality of nodes having respective factors; constructing an aggregate lattice using the respective first lattice(s) and the respective second lattice(s), the aggregate lattice including a plurality of nodes, wherein each of the nodes of the aggregate lattice includes a scoring factor computed using the factor in a corresponding one of the nodes in a corresponding one of the first lattice(s) and the factor in a corresponding one of the nodes in a corresponding one of the second lattice(s); determining whether the video corresponds to the aggregate query by determining respective aggregate score(s) of one or more path(s) through the aggregate lattice, each path including a respective plurality of the nodes in the aggregate lattice.