How do I make a battery voltage indicator using the battery being tested?
I would consider using a MAX931 low-power comparator with an in-built voltage reference that is accurate to within 1% of 1.182 volts : -



V+ and Vin can connect to the battery and the two resistors (use high values to avoid excessive battery current) set the point at which the output goes low when the battery voltage drops below the required set-point.

The chip works from 2.5 volts to 11 volts so 5 volts is fine.

Choose the resistor divider so that the low threshold for the battery (say 4 volts) produces 1.18 volts at IN+.

What you drive with the output is up for grabs but, a simple scheme might be to illuminate a low power LED when the output switches low thus indicating that the battery has dropped below 4 volts (or whatever value you think is right).

The output stage can easily drive 10 mA into an LED and series resistor tied up to the battery supply. Or maybe you use the logic low output to trigger a low frequency oscillator that pulses the LED on for 0.5 seconds every (say) 20 seconds.

Or maybe you just drive a low power LED when the battery voltage is good. No LED illumination means battery bad.