Only charge capacitor when button pushed then turn on LED momentarily with capacitor when button released
If you only have a push-to-make switch:-







simulate this circuit &ndash; Schematic created using CircuitLab