16. A video surveillance system, comprising: a video input source configured to provide an input stream of video frames captured by a video camera, each depicting a scene; a processor; and a memory containing a program, which, when executed on the processor is configured to perform an operation for analyzing the scene depicted in the input stream of video frames, the operation comprising: receiving a first sequence storing an ordered string of labels, wherein each label in the string corresponds to a cluster in an adaptive resonance theory (ART) network, wherein the string of labels is generated by mapping kinematic data vectors generated for a foreground object detected in the input stream of video frames to nodes of a self-organizing map (SOM) and clustering the nodes of the SOM using the ART network, identifying one or more segments in the first sequence, wherein each segment includes a subsequence of the ordered string of labels in the first sequence, wherein the subsequence in the first one of the segments is determined to include a label not present in the ngram trie, determining a probability of observing each of the one or more segments relative to a probability distribution generated from an ngram trie, wherein the ngram trie is generated from a plurality of previously observed sequences, wherein each sequence of the plurality stores an ordered string of labels assigned to clusters in the ART network, upon determining a first one of the segments has a probability of being observed below a specified threshold, issuing an alert to a user of the video surveillance system.
16. A video surveillance system, comprising: a video input source configured to provide an input stream of video frames captured by a video camera, each depicting a scene; a processor; and a memory containing a program, which, when executed on the processor is configured to perform an operation for analyzing the scene depicted in the input stream of video frames, the operation comprising: receiving a first sequence storing an ordered string of labels, wherein each label in the string corresponds to a cluster in an adaptive resonance theory (ART) network, wherein the string of labels is generated by mapping kinematic data vectors generated for a foreground object detected in the input stream of video frames to nodes of a self-organizing map (SOM) and clustering the nodes of the SOM using the ART network, identifying one or more segments in the first sequence, wherein each segment includes a subsequence of the ordered string of labels in the first sequence, wherein the subsequence in the first one of the segments is determined to include a label not present in the ngram trie, determining a probability of observing each of the one or more segments relative to a probability distribution generated from an ngram trie, wherein the ngram trie is generated from a plurality of previously observed sequences, wherein each sequence of the plurality stores an ordered string of labels assigned to clusters in the ART network, upon determining a first one of the segments has a probability of being observed below a specified threshold, issuing an alert to a user of the video surveillance system. 22. The system of claim 16 , wherein the subsequence in the first one of the segments is determined to include a string of labels not present in the ngram trie.