Cicuit to keep power supplied (12V DC) for a few seconds after power is cut?
The switch is getting stuck because of the surges of current that the capacitor draws. Each time the switch closes, the capacitor looks like a short circuit until it charges up. Without the capacitor, the current was limited by the coil in the relay; with it, it is limited only by the ESR of the capacitor.

One possible way to mitigate this in the short term would be to add a resistor in series with the switch. Select a value that doesn't cause more than, say, 0.5V drop with the coil alone. This will provide an upper limit on the surge that the capacitor can draw.

A better long-term solution would be to eliminate the capacitor altogether and instead use a different kind of float switch that has a hysteresis effect. With this kind of switch, the pump turns on at one (higher) water level, but turns off at a different (lower) level. This is how most sump pumps work.

This effect can be simulated by using two float switches of the type you have, plus an auxiliary relay.