1. A computer-implemented method for processing data related to a scene depicted in a sequence of video frames, the method comprising: receiving input data describing one or more objects detected in the scene, wherein the input data includes at least a classification for each of the one or more objects; identifying one or more primitive events, related to a behavior engaged in by a corresponding one of the objects depicted in the sequence of video frames and wherein each primitive event has an assigned primitive event symbol; generating, for one or more objects, a primitive event symbol stream which includes the primitive event symbols corresponding to the primitive events identified for a respective object; forming a first vector representation of each object using the primitive event symbol stream for each respective object; and comparing the first vector representations to a stored vector representation to define a similarity measure.