How do IR sensors detect color?
The kind of sensors used for line following robots (IR proximity sensors) don't sense color in the normal use of that word. They actually sense reflectivity, and in particular they sense the reflectivity of whatever is right in front of them at the IR wavelength of the sensor and emitter. (Strictly speaking, the reflectivity of an object at a particular wavelength is what gives it color, I know.)

So, your sensor would work best if you had a background that absorbed IR light and a line that reflected IR light. As it turns out, the colors that we call dark colors are pretty good at absorbing near IR as well as the visible spectrum, and the colors that we call bright colors are pretty good at reflecting near IR. If you really wanted a good contrast you could use a reflective material like aluminum foil for the line and have a deep dark void surrounding the line (maybe it's a line-following blimp). You don't normally think of metal foil or empty space as having a "color", but the IR sensor can easily tell the difference.