1. A method for coding video data, the method comprising: determining, based on a size characteristic of a prediction unit (PU) in a B slice of the video data being below a threshold, that the PU is restricted to uni-directional inter prediction; generating a merge candidate list for the PU; determining a selected merge candidate in the merge candidate list, wherein the selected merge candidate is a bi-directional merge candidate, wherein two different reference blocks are associated with motion information specified by the selected merge candidate; and based on the PU being restricted to uni-directional inter prediction, converting the selected merge candidate from being a bi-directional merge candidate to being a uni-directional merge candidate by keeping a list 0 motion vector of the selected merge candidate and ignoring a list 1 motion vector of the selected merge candidate such that a predictive video block for the PU is generated based on no more than one of the two different reference blocks associated with the motion information specified by the selected merge candidate.