1. A method for performing temporal motion vector filtering in a digital video sequence, comprising: receiving a plurality of vectors representing potential motion vectors for an image patch including one or more of an object and a background; partitioning the plurality of vectors into a plurality of vector clusters comprising a first vector cluster and a second vector cluster; determining a first representative vector for the first vector cluster and a second representative vector for the second vector cluster; determining which of the first representative vector and the second representative vector most accurately reflects a displacement of the image patch between a first frame and a second frame of the digital video sequence; and selecting as a motion vector the representative vector that most accurately reflects the displacement of the image patch.