Is there any color sensor that reads R,G,B and Clear channel values in parallel?
With S0 and S1 high the datasheet says the photodiode can run at 500-600kHz so I doubt your sensor is the problem. The Arduino is probably the limiting factor (I'm implying that's what you're using because of your tags). Have you changed the fuses to run the Arduino at 8Mhz instead of the default 1Mhz? If not that should give you the speed bump you need.