How to compute average colours of the edges of a composite tv signal
I think an analog solution is the best. A hybrid digital-analog solution will be tricky, but not impossible.

Have a look at the chips used in a CRT TV. You may be able to hack one to get R, G, B outputs. To take the edge colours only might require storing the frame and grabbing pixels. It could be tricky, in other words.