Comparator have hysteresis with pull-up resistor?
Edit: As it turns out from OP, the cause was unwanted feedback, likely causing high-frequency oscillation. Reducing the impedance at the non-inverting input by 10:1 by changing the resistor values cured the symptom.
In general it is a good idea to have a bypass capacitor across the supply rails near the chip and to bypass any slow-moving or DC input such as the reference voltage in the above circuit with a capacitor to ground. In this case, something like 100nF would be appropriate. OP could then most likely go back to the higher values and waste less power.
Solderless breadboards, as OP used, have a certain amount of parasitic capacitance between rows of connections, several pF or more, and although that is often not of much concern, it can cause issues in some cases, even where high frequencies are not intended to be present.
A coupling of only 5pF could cause this amount of oscillation with ~6.9K source impedance at only ~150kHz, which is well within the capabilities of that comparator.

Possibly the current drawn by the relay coil is dragging the 24V nominal supply down by 1.6V, which would cause the hysteresis you are seeing.
You can add a voltage regulator (series or shunt) or a zener diode regulator to stabilize your reference.
You must also have a flyback diode across the relay, which could conceivably be causing the problem. Try it without the relay connected at all and see what happens.