photodiode, ldr and light source
There are chips that already do what you want. There is really no need to build your own thing out of color filters and diodes.

Take a look at the TCS3103 or TCS3104 from AMS for example. They output three voltages which are proportional to the incoming red, green and blue light components.