Why does my resistor consume so much power?
As indicated in comments I have prepared a low battery detector circuit that uses a comparator to sense the low battery state. This circuit will place less than 0.2mA of load on the battery until the battery voltage falls below about 3.75V. At that point the LED turns on and the load on the battery becomes primarily the LED current which can be adjusted by changing the value of R5.

The 10meg feedback resistor around the comparator adds hysteresis so that the detection point does not cause flickering of the LED.