Why do battery/power supply supply voltages drop under a load?
Batteries have an internal resistance, this is generally measured in milli-ohms.  As more current is drawn from the battery there will be a progressively larger voltage drop due to this internal resistance.  V = I*R.  This is obviously a high level answer, the details can vary based on battery chemistry.

A battery's voltage will also generally drop as the battery ages (runs out of juice).  The exact chemical process varies based on battery chemistry but simply put the battery is running out of charge.