Mysterious voltage drop then slows climbs back up when powered off
That's how batteries work. As you draw current from them, you deplete the chemical reactants near the plates, and it takes time for new reactants to take their place. This creates a gradient in the reactant concentration within the volume of the battery.

The terminal voltage is a function of both the reactant concentration at the plates and the voltage drop across the internal resistance of the battery (which rises a bit with reactant depletion). If you let it run long enough, the terminal voltage should stabilize at some lower value, although the overall depletion of the battery will eventually become a factor.

When you switch off, the voltage drop across the internal resistance disappears immediately, but it takes some time for the reactant concentration gradient to disappear, which is why you see the sudden jump up followed by the slow recovery to the original terminal voltage.