How to flip polarity of battery with DIY device
You can do this with 4 diodes in a diode bridge configuration.

Parts with 4 diodes in this configuration are available in a single package rather than having to build it yourself.

You will however lose two voltage drops (about 1.2V) worth of voltage. Depending on the situation you could use schottky diodes to reduce this loss at the cost of higher leakage.