Whether a battery is a constant voltage source or current source
A battery is considered to be a voltage source because the galvanic activity they use to store and deliver energy has a fixed voltage across it.

However, a battery is not an ideal voltage source.  All real sources have some built in resistance.  In the case of a battery, the effect is well modeled as an ideal voltage source in series with a small resistor (I don't know numbers, but I'd expect it to be single digit ohms).  Thus, when you draw current from the battery, the voltage across the resistor goes up which means the voltage across your circuit goes down.

Eventually you deplete the battery.  When this happens, we can no longer treat all of the parts of the battery in bulk.  Parts of the battery will remain charged, other parts will be fully discharged.  During this time, we have to use an even more complicated model to describe the battery.  Which model you use would depend on the particular battery failure characteristics.  However, the most visible effect is that, when there is no circuit connected, we can measure a voltage across the battery that is lower than the ideal expected voltage based on the galvanic properties of the battery.