Bi-directional on/off toggle soft power latch
So, I needed to re-design the circuit to fix two issues: The step in current as described above, and a false default on for lower voltages of V1.

Edit: R10 should be 560kΩ to make sure the circuit works with button presses between 0.2...2 seconds.