1. A method for extracting a low-rank descriptor of a video acquired of a scene, wherein the video includes a sequence of images, further comprising: extracting a set of descriptors for each image in the video; aggregating the sets of descriptors for the video to form a descriptor matrix; determining, iteratively until convergence, a low-rank descriptor matrix. from the descriptor matrix and a selection matrix that associates each column in the descriptor matrix to a corresponding column in the low-rank descriptor matrix; and outputting the low-rank descriptor matrix upon convergence, wherein the steps are performed in a processor.