1. A method for using models learned from anomaly detection to rank detected anomalies, the method comprising: tracking movement of an object that is detected within a video data input from a camera through an image field of the camera, wherein the image field is partitioned into a matrix comprising a grid of a plurality of different local units, and wherein the tracking generates a trajectory of the object's motion that passes through a subset of the local units that is less than a totality of the plurality of the different local units; extracting image features from the video data from the camera with respect to each of the subset of the local units of the plurality of local units by using said trajectory; learning a plurality of local motion pattern models, one for each of the subset of the local units, wherein the plurality of the learned local motion pattern models comprise normal patterns that are defined by finding dominant distributions of the extracted image features within respective ones of the subset of the local units, and anomaly patterns that are defined by rare distributions of the extracted image features within the respective ones of the subset of the local units; generating anomaly confidence decision values for the tracked object for each of said subset of the local units as a function of fitting the image features extracted for each of the subset of the local units from the video data input of the tracked object to the plurality of the learned local motion pattern models of the respective subset of the local units by determining whether the features extracted relevant to the object's motion within the video data indicate that the object's motion within each particular local unit is one of said normal patterns or anomaly patterns in view of the plurality of the learned local motion pattern models for the local units; normalizing values of the image features that are extracted from the image field's subset of the local units that are associated with each of the plurality of anomaly confidence decision values; clustering the image feature values extracted from the image field's subset of the local units that are associated with each of the plurality of anomaly confidence decision values; learning weights for each of the anomaly confidence decision values as a function of a relation of their normalized values of the extracted image features to the clustered image feature values extracted from the each respective associated image field's subset of the local units by determining spatial locations of the clustered extracted image feature values of the subset local units of the anomaly confidence decision values within the field of view of the input video data as correlated to features of interest of a real-world scene represented within the field of view, and assigning a first weighting to a first anomaly of the anomaly confidence decision values that is higher than a second weighting assigned to a second anomaly of the anomaly confidence decision values in response to the determined spatial location of the clustered extracted image feature values of the subset local unit containing said first anomaly of the first anomaly confidence value being within a portion of the field of view of the input video that is correlated with a cordoned off area of the real-world scene and the determined spatial location of the clustered extracted image feature values of the subset local unit containing said second anomaly of the second anomaly confidence decision value being outside the portion, wherein the normalized extracted features of each of the subset local units of the first and the second anomaly confidence decision values are outliers from and have the same distance to a center of a cluster of extracted features of a one of the learned motion pattern local models; multiplying the normalized values of the extracted features of the anomaly confidence decision values of the subset of the local units by their respective learned weights to generate respective ranking values; and ranking the plurality of anomaly confidence decision values by their generated respective ranking values.