1. A method for performing motion estimation on video data comprising a plurality of frames, the method comprising: reading, by a motion estimation unit, a current frame of the plurality of frames from a memory, the current frame comprising a plurality of rows; selecting a motion vector for each respective block of pixels in a current row of the current frame, wherein the step of selecting the motion vector comprises, for each respective block: selecting a candidate vector for at least one block directly surrounding the respective block based on a determination of whether the directly surrounding block has been processed for the current frame, calculating, for each candidate vector, a difference value indicating an accuracy of motion estimation when the candidate vector is used to represent motion of the respective block, and selecting, as the motion vector, the candidate vector with a lowest difference value; and processing at least one subsequent row by repeating the step of selecting a motion vector for each block of pixels in the at least one subsequent row, wherein processing the at least one subsequent row begins prior to completion of processing of at least one block of the current row.