1. A method of decoding video data, the method comprising: determining a prediction partitioning structure for predicting pixel values associated with a block of video data, wherein the prediction partitioning structure includes one or more non-square partitions; determining a transform partitioning structure for applying one or more transforms to the predicted pixel values, wherein determining the transform split structure includes: splitting a parent transform unit; upon determining the transform partitioning structure comprises splitting the parent transform unit into one or more square transforms, determining one or more square transforms such that each of the one or more square transforms correspond to exactly one prediction partition; and upon determining the transform partitioning structure comprises splitting the parent transform unit into one or more non-square transforms, determining whether to split the one or more non-square transforms based at least in part on the one or more non-square transforms being non-square; and decoding the predicted pixel values based on the determined transform partitioning structure.