1. A non-transitory computer readable medium comprising instructions that when executed by a processor execute a process comprising: receiving an image frame from a video sensing device; detecting motion in the image frame; tracking the motion in the image frame; generating an appearance model for an object detected in the image frame; storing the object appearance model in a database; building a tree-based structure including a fast search index, wherein the tree-based structure comprises one or more of a balanced tree and an unbalanced tree, wherein the balanced tree is based on a median distance between features of the object appearance model, and the unbalanced tree is based on clustering of features of the object appearance model; requesting the image frame and detecting a person in the image frame; generating an appearance model for a person detected in the image frame; and searching a queried person's appearance model in the image frame using the tree-based structure; wherein detecting the motion and tracking the motion operate on a number of image frames in a video sequence, and detecting a person operates on a subset of the number of image frames in the video sequence.