1. A method, comprising the steps of: receiving a video sequence comprising detection results from one or more detectors, the detection results identifying one or more objects; applying a clustering framework to the detection results to identify one or more clusters associated with the one or more objects, wherein the clustering framework is applied to the video sequence on a frame-by-frame basis; determining spatial and temporal information for each of the one or more clusters; associating the one or more clusters to the detection results based on the spatial and temporal information in consecutive frames of the video sequence to generate tracking information; generating one or more target tracks based on the tracking information for the one or more clusters; and consolidating the one or more target tracks to generate refined tracks for the one or more obj ects; wherein the steps are performed by at least one processor device coupled to a memory.