Art of electronics figure 1.18
If the red wire was not there, How would the comparator and LED get any voltage? That entire top line is the 'Power' rail. Usually, power is drawn at the top with GND at the bottom. 

The Op-Amp and the LED always have voltage, regardless of the temperature at any time. When the temperature gets high enough, the comparator pulls its output to GND, which allows the LED a path to GND, and allows it to light up. If the comparator output was high, there would be no voltage drop over the LED, so it would be unable to light up.

You are correct that the grounds would go to the negative terminal of the battery (if this were a battery powered circuit). 

Think about what would happen if that red wire were absent. Where would the comparator get its supply voltage? Without it, how could it provide a high output? Where would the LED get powered from? Assume it did get power from the comparator, how would it light up? It would be connected the wrong way. 

The comparator works so that when the inverting input (-) is below the non inverting (+) input, the comparator output is high. If the inverting input is higher than the non inverting input, the output goes to GND.

Here is how the 2 states would work:







simulate this circuit &ndash; Schematic created using CircuitLab