1. A method for performing temporal motion vector filtering in a digital video sequence, comprising: receiving a plurality of vectors, the vectors representing potential motion vectors for an image patch including one or more of an object and a background; partitioning the plurality of vectors into two or more vector clusters, including: determining a first seed vector for a first cluster and a second seed vector for a second cluster by identifying two vectors among the plurality of vectors that are furthest apart from each other; and for each other vector in the plurality of vectors: placing the vector into the first cluster if the vector is closest to the first seed vector; and placing the vector into the second cluster if the vector is closest to the second seed vector; determining a representative vector for each vector cluster; testing each representative vector to determine which representative vector most accurately reflects a displacement of the image patch between a first frame and a second frame of the digital video; and selecting as a motion vector the representative vector that most accurately reflects the displacement of the image patch.