1. A method of decoding video data, the method comprising: receiving in the video data a video parameter set (VPS) comprising one or more syntax elements that identify values for hypothetical reference decoder (HRD) parameters; receiving in the video data a first sequence parameter set (SPS) comprising a first syntax element identifying the VPS; receiving in the video data a second SPS comprising a second syntax element identifying the VPS; decoding, based on the one or more syntax elements, a first set of video blocks associated with the first SPS and a second set of video blocks associated with the second SPS; and outputting decoded video comprising the decoded first set of video blocks and the decoded second set of video blocks.