What's a good estimate on how low an alkaline battery voltage can be to still make the battery useful?
A fully discharged alkaline cell (nominal voltage 1.5 Volts) still retains a voltage of 0.9 to 1.0 Volts. Therefore, voltage threshold for measurement can be taken as any value above 1.0 Volts.

Battery energy delivery capacity however would be limited by other factors: Internal resistance / electrode surface deterioration. A cell that has been left unused for a long period would show poor power delivery capability due to this. 

One way of evaluating these "depleted" batteries would be to measure current delivered due to a brief pulsed load, i.e. a low resistance across the terminals for a brief duration.

Such a measurement device can be built conveniently by connecting a resistor and a MOSFET in series across the terminals, feeding the MOSFET gate with a series of short pulses, and reading the voltage across the resistor each time the MOSFET is switched on.