Why is the threshold voltage kept large in an LED?
The forward voltage drop of an LED is indeed related to the color of light emitted.

As electrons traverse the N-P junction they lose energy. That energy is lost in the form of light. The more energy lost, the shorter the wavelength of light emitted. 

Historically, the development of LEDs has been the pursuit of higher band-gap voltages for shorter wavelengths of light. That is why early LEDs were all red or infrared, with yellow, green and finally blue coming later. 

BTW White LEDs are actually blue, but also contain a light emitting phosphor layer, similar in function to that in florescent light bulbs, to convert the blue light to white light.

This article shows the relationship between color and energy. Color and Energy Scroll down to the section called "Spectral Colors", there's an excellent table there.

Of course, the other parts of the LED still have resistance, so some more voltage will be needed to get a decent current to flow. That's why it is not uncommon to need more voltage than the minimum.