1. A computer-implemented method, comprising: receiving a request to view a scene depicted in a sequence of video frames; identifying and tracking at least one object between separate frames of the sequence of video frames; classifying each tracked object based on a known category of objects; generating a stream of context events associated with each tracked object; generating a sequence of primitive events based on the stream of context events; storing the stream of context events and the sequence of primitive events in a memory; storing detailed data in the memory related to an event based on the stream of context events and the sequence of primitive events; storing generalized data in the memory related to an event based on the stream of context events and the sequence of primitive events; and evaluating the stream of context events, the sequence of primitive events, the detailed data, and the generalized data to learn patterns of behavior that occur within the scene.