9. A computer-readable storage medium containing a program which, when executed by a processor, performs an operation for identifying anomaly object types during classification of image data captured by a video camera, the operation comprising: receiving a micro-feature vector including multiple micro-feature values, each micro-feature value based on at least one pixel-level characteristic of a foreground patch that depicts a foreground object within the image data; classifying the foreground object as depicting a first object type corresponding to a first object type cluster of the object type clusters based on the micro-feature vector; computing a probability density function for the object type clusters; computing a probability density value for the micro-feature vector; evaluating a rareness measure of the micro-feature vector, wherein the rareness measure estimates a likelihood of observing the micro-feature vector, based on the probability density function and the probability density value; and identifying the foreground object as an anomaly object type when the rareness measure is below a specified threshold.
11. The computer-readable storage medium of claim 9 , wherein the foreground object is classified as depicting the first object type when a distance between the micro-feature vector and a center of the first object type cluster is less than a threshold value.