1. A method of decoding video data, the method comprising: determining, for each reference picture in one or more reference picture lists for a current picture, whether the reference picture is to be included in a plurality of reference pictures based at least in part on whether at least one of the reference pictures in the reference picture lists is a short-term reference picture or a long-term reference picture; comparing picture order count (POC) values of each of the plurality of reference pictures to a POC value of the current picture to determine at least one motion vector predictor for a current block of the current picture based on one or more motion vectors of at least one co-located block of video data in at least one of the plurality of reference pictures; determining whether a forward motion vector or a backward motion vector of the co-located block is to be initially used to derive the motion vector predictor; deriving the motion vector predictor based at least in part on the determination; and decoding a motion vector for the current block of video data relative to the motion vector predictor.