How to keep circuit powered when battery disconnected for short time
The battery and the capacitor should be connected in parallel. It is better to solder the capacitor to the main circuit board if possible. Considering the capacitor connection is always okay it will work just fine.

The beauty of parallel connection is that a damaged or disconnected or open branch does not affect other branches in parallel of the network.