15. A video coding device comprising: a memory configured to store a plurality of context models for entropy coding transform coefficients of video data, wherein the plurality of context models includes one or more context models that are each used for a different transform unit size and at least one joint context model used for two or more transform unit sizes; and a processor configured to: select the joint context model shared by a first transform unit having a first size with transform coefficients that are zeroed out to generate a retained coefficient block of a second size and a second transform unit having the second size for entropy coding transform coefficients in one of the first transform unit with the retained coefficient block or the second transform unit, wherein the first size and the second size are different, and wherein each of the transform units comprises a residual video data unit including a plurality of blocks of transform coefficients, select contexts for the transform coefficients associated with the one of the first transform unit with the retained coefficient block or the second transform unit according to the joint context model, wherein the processor is configured to assign a context in the joint context model to a given one of the transform coefficients based on values of previously coded neighboring transform coefficients of the one of the transform units, and determine a probability estimate for a value of the given one of the transform coefficients associated with the assigned context in the joint context model, and entropy code the transform coefficients associated with the one of the transform units using context adaptive binary arithmetic coding (CABAC) based on the selected contexts.
28. The video coding device of claim 15 , wherein the processor is configured to arithmetically coding the given one of the transform coefficients based on the determined probability estimate for the value of the given one of the transform coefficients associated with the assigned context.