Key out everything but one color?
This is quite simple.  Use any keyer of your choice that can give you a good mask for the layer, then place an "Alpha Adjust" layer below it in the Effects control and click the "Invert Alpha" checkbox.  This will invert the alpha channel produced by the keyer and reverse the effect the way you are looking for.