MAX856 Low-Battery Indicator Question
Does this mean that I should take my battery + terminal, put it through a resistive divider and choose the resistors such that 3.4V at BAT+ gives 1.25V at the resisitve divider output ?


Yes, that's exactly right.

The MAX856 datasheet has this example of implementing this - see R3 and R4 in this schematic:









As an example, if you use E96 resistor values, then a quick calculation suggests 150 kΩ for R4 and 255 kΩ for R3 gets you close to your desired 3.4 V low-battery value and stays below the 300 kΩ suggested maximum resistor value - but do check my calculations.