How do our eyes detect light at different frequencies?
Short answer
Action potentials generated to different colors are indeed similar throughout the nervous system and do not encode color as such. Instead, the different color- sensitive cells in the retina are connected to different neurons and these color-specific signals are kept segregated up until the higher visual cortical areas.

Background
Action potentials are indeed pretty similar throughout the nervous system. However, the color-sensitive sensory cells in the retina, called the cones, come in three flavors: red, green and blue. These colors form the RGB system just like in your LED TV and can together make all the available millions of colors. These three cones synapse ultimately onto color-specific secondary sensory neurons (Fig. 1). 

Hence, R, G and B cones indeed generate identical action potentials in downstream neurons, the trick is they do so in different retinal ganglion cells, and at different firing rates depending on the intensity of light that particular cone is sensitive to. These different classes of retinal ganglion cells project onto different classes of neurons in the brainstem (lateral geniculate nucleus, or LGN) and ultimately onto different neurons in the higher cortical visual areas in the brain. 


Fig. 1. Different classes of cones synapse onto different classes of secondary sensory neurons in the retina. source: Discovery Eye Foundation

The reason why we can differentiate millions of colors can be explained by the Hering model of color vision (Fig. 2). Basically, the different cones converge pairwise onto opponent color-sensitive cells. The red-green opponent system, for example, operates by weighing the amount of red and green in the incoming signal. This weighting results in an analogue system that can code millions of colors along the red-green axis (Fig. 3).


Fig. 2. Hering model of color vision. source: Webvision   


Fig. 3. Red-green color axis. source: SO