1. A method for tracking pedestrians in a video sequence, wherein each image frame of the video sequence corresponds to a time step, comprising: using marginal space learning to sample a prior probability distribution of multi-person identity assignments for a current image frame in the video sequence given a set of feature measurements from all previous image frames in the video sequence; using marginal space learning to estimate an observation likelihood distribution of the set of features for the current image frame given a set of multi-person identity assignments for the current image frame sampled from the prior probability distribution of multi-person identity assignments for the current image frame; calculating a posterior probability distribution of multi-person identity assignments for the current image frame from the observation likelihood distribution for the current image frame and the prior probability distribution for the current image frame; and using marginal space learning to estimate the prior probability distribution of multi-person identity assignments for a next image frame in the video sequence given the posterior probability distribution of multi-person identity assignments for the current image frame and a probability p(x t+1 |x t ) of multi-person identity assignments for the next image frame given the multi-person identity assignments of the current image frame, wherein said posterior probability distribution of multi-person identity assignments corresponds to a set of pedestrian detection hypotheses for the video sequence.