Is it safe to leave a capacitor between the + and - wires of a circuit, while the rest of the circuit is turned off?
Caps are not short-circuits once charged to the DC level of the circuit. It is safe though but caps can have an operational  lifetime though it probably doesn't matter here.
Yes, due to leakage always decreasing the cap voltage requiring it to charge up again. It is a small amount though.
Yes because batteries. A factory would probably not go to this trouble though.

A or B technically don't matter in most cases but A is best since B is being used as the reference. It matters more for high voltage when everything is earthed to B so disconnecting B would make everything float and be unsafe.