Temperature invariant color space I'd like to build a color difference metric that will be invariant to small changes in lighting conditions. In other words, I should be able to tell that a red shirt is not the same as a blue shirt, even if one image is lit differently (brighter / darker, outdoors / incandescent). 

Moving to YUV space and ignoring the luminance component I can deal with changing brightness conditions by considering the metric:
$$d = \sqrt{\Delta U^2  + \Delta V^2}$$
But this metric still gives large "distances" when different light sources are used, due to the change in chrominance along the Planckian locus.

My question is:

Is there a color difference metric that is invariant to temperature differences? in a sense, I'm looking for a color space in which one of the three axes is temperature. Does such a space exist?
Other than searching for a color space, why don't you just first apply a color correction and then get the hue space?

If illumination is an issue, you might consider an illumination invariant color correction:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.6859&amp;rep=rep1&amp;type=pdf

Then simple convert your image to HSV and get H channel.

Additionally, if you have some clue about what you are looking at (such as roads, football fields etc) then you might use that too. Such an algorithm is mentioned here:
https://stackoverflow.com/questions/10922467/illumination-invariant-image