1. A method of decoding a block of video data, the method comprising: deriving, from among a plurality of intra prediction modes, M most probable modes (MPMs) for intra prediction of the block of video data, wherein M is greater than 3; decoding a syntax element that indicates whether a MPM index or a non-MPM index is used to indicate a selected intra prediction mode of the plurality of intra prediction modes for intra prediction of the block of video data, wherein the MPM index indicates which of the M MPMs is the selected intra prediction mode, and wherein the non-MPM index indicates which of the plurality of intra prediction modes other than the MMPMs is the selected intra prediction mode; based on the syntax element indicating that the MPM index is used to indicate the selected intra prediction mode of the plurality of intra prediction modes for intra prediction of the block of video data, selecting, for each of one or more context-modeled bins of the MPM index and based on intra prediction modes used to decode one or more neighboring blocks, a context index for the context-modeled bin, wherein selecting the context index for the context-modeled bin comprises selecting the context index for the context-modeled bin from among a first context index, a second context index, a third context index, and a fourth context index such that: the first context index is selected when intra prediction modes used to decode a left neighboring block and an above neighboring block are the same and when neither of the intra prediction modes used to decode the left neighboring block nor the above neighboring block is DC or planar, the second context index is selected when intra prediction modes used to decode the left neighboring block and the above neighboring block are the same and when at least one of the intra prediction modes used to decode the left neighboring block and the above neighboring block is DC or planar, the third context index is selected when intra prediction modes used to decode the left neighboring block and the above neighboring block are different and when neither of the intra prediction modes used to decode the left neighboring block nor the above neighboring block is planar, and the fourth context index is selected when intra prediction modes used to decode the left neighboring block and the above neighboring block are different and when at least one of the intra prediction modes used to decode the left neighboring block and the above neighboring block is planar; and reconstructing the block of video data based on the selected intra prediction mode.