1. A method for encoding a current block of a video stream having a plurality of blocks including the current block, comprising: identifying, from a plurality of blocks coded before the current block, a plurality of candidate motion vectors, each candidate motion vector associated with a respective reference frame; identifying, for the current block of a current frame, a set of reconstructed pixel values for a set of pixel positions within the current frame, but outside of the current block, coded before the current block; for each candidate motion vector of the plurality of candidate motion vectors: generating, by applying the candidate motion vector to the set of pixel positions, a set of predicted pixels for the reconstructed pixels within the respective reference frame associated with the candidate motion vector; and determining an error value between the set of reconstructed pixel values and values of the set of predicted pixels; and selecting, from the plurality of candidate motion vectors, a reference motion vector for the current block based on the error value for each candidate motion vector.