Delay on DC-DC regulator
You show dropout voltage of the regulator as 0.1V and the battery voltage is 3.6V minimum, so you have only 0.2V drop allowed at the input. 

The minimum required capacitance is Cmin = \$ \frac {t_{HOLDUP}\cdot Iout}{\Delta V}\$ 

In your case that would be 0.05F (ignoring capacitor resistance). That's a problem. Inexpensive ultracaps tend to have far too high internal resistance to be usable (several ohms to tens of ohms). Useful in low current situations, but not here. AVX makes a 100mF cap with 80m\$\Omega\$ internal resistance, but it's very expensive (almost $7 in 1,000 quantity).  

Perhaps you can solve the problem using out-of-box thinking. I suggest either improving the battery connections or consider detecting an imminent drop-out and using an interrupt to put the circuit in a very low power survival state to ride out the power loss condition, then recover when power is restored. That way you may be able to use a moderately large electrolytic capacitor instead of a ultracap/supercap.