1. A method of coding video data, the method comprising: determining whether a coding unit (CU) depth of a current CU is smaller than a minimal CU depth of one or more neighboring CUs, or whether the CU depth of the current CU is greater than a maximum CU depth of the one or more neighboring CUs; selecting, as part of an entropy coding process, a context for a syntax element associated with the current CU based on information relating to CU depth of the one or more neighboring CUs relative to the current CU, wherein the information relating to the CU depth of the one or more neighboring CUs relative to the current CU includes whether the CU depth of the current CU is smaller than the minimal CU depth of the one or more neighboring CUs or whether the CU depth of the current CU is greater than the maximum CU depth of the one or more neighboring CUs, wherein the syntax element represents whether the current CU is split into sub-CUs; and entropy coding the syntax element based on the selected context.