1. An apparatus for estimating motion vectors of a video, comprising: a computer readable medium storing the video comprising a plurality of video frames, each frame comprising a plurality of blocks; and a video signal processor comprising a multistage motion vector prediction engine configured to estimate one or more best block-matching motion vectors for each block in each video frame of the video in a number of stages, wherein the estimating comprises: for each stage of the number of stages of motion vector estimation for a block of a video frame of the video: selecting a test vector from a predictor set of test vectors, wherein the set of test vectors is selected from a plurality of predictor sets of vectors based on using at least one of a priori knowledge of the video and a priori knowledge of a plurality of video sequences stored in a database, and wherein each test vector is unique from other test vectors of the predictor set of the test vectors; computing a rate-distortion optimization (RDO) based metric for the selected test vector; selecting a subset of test vectors as individual best matched motion vectors based on the RDO based metric; selecting a total best matched motion vector from the individual best matched motion vectors based on comparisons of the individual best matched motion vectors; and iteratively applying one or more global matching criteria to the selected total best matched motion vector to select a best matched motion vector for the block of pixels.