Difference between HSI and YCbCr for pattern recognition by color
I'm not an expert, but judging from the Wikipedia articles, YCbCr is used for image compression (JPEG and television [YPbPr]), while HSI is used in computer vision. Since you're doing computer vision, if we believe Wikipedia then you should use HSI. Of course, you could always experiment with both representations and all six dimensions. See what works best. If you're fitting a linear model, then it doesn't really matter, and you can even use RGB.