Circuit to discharge capacitor after opening the circuit
The capacitor will eventually self discharge but this may be too long of a period. You can connect a resistor from the non battery side of the switch to the cathode of the LED to obtain a predictable discharge period.

The value of resistance determines the discharge rate of the capacitor. You can adjust this value in your simulation. Keep in mind that this resistor will draw current from your battery when the button is pushed so you should check this in your final design.