hot swapping battery
Your requirements are pretty simple. You'll need a pair of connectors for the batteries and a pair of diodes. Make a connection setup like this







simulate this circuit &ndash; Schematic created using CircuitLab

Now you can connect or disconnect either connector as you wish. As long as one battery is always connected, you should have no problems. 

Your diodes should be rated for at least 12 volts (20 or more is better), and at least the current required by the controller, probably at least 1 amp. Also, since the diodes will drop some voltage, Schottky diodes are preferred. Something like a 1N5820 should do you just fine. You can try sources like Jameco, Digikey or Mouser, or any number of other suppliers.