1. A method of video coding using intra prediction for a block of video data, the method comprising: determining one or more tables ranking first intra prediction modes according to a first priority order for a first neighboring block having a first block size and second intra prediction modes according to a second priority order for a second neighboring block having a second block size, wherein the first block size is different from the second block size, and wherein the first priority order is different from the second priority order; receiving a current intra prediction mode corresponding to a current block; receiving two or more neighboring intra prediction modes corresponding to two or more neighboring blocks, wherein one of said two or more neighboring block has a neighboring block size corresponding to the first block size and the other one of said two or more neighboring block has a neighboring block size corresponding to the second block size; determining a mode priority for each of said two or more neighboring intra prediction modes according to each neighboring intra prediction mode and each neighboring block size of said two or more neighboring blocks using said one or more tables; determining a highest-priority mode among said two or more neighboring intra prediction modes as a most probable mode by identifying a neighboring intra prediction mode with a lowest index relative to all of the two or more neighboring intra prediction modes ranked in their respective tables corresponding to different block sizes; and encoding the current mode using the most probable mode as a predictor.