1. An image processing system configured to receive a plurality of input images and track objects within image frames, the system comprising: a hardware processing system configured to: perform a first level of association between detection responses in image frames, wherein the detection responses include a vector having position, size, and frame index components, including forming one or more tracklets between pairs of detection responses having suitable link probabilities; receive the one or more tracklets formed as a result of the first level of association, and perform a second level of association, including producing a tracklet association between tracklets; perform a third level of association, including estimating a scene structure model from the second level of association; and provide as an output a trajectory set of the tracklets.