A capacitor experiment
Try wiring your circuit this way to get the effect you were thinking about.  Experiment with the component values a bit to see different timing changes and brightness levels, (just keep R1 over 100 ohm).   

Pressing SW1 slowly begins charging C1. When SW1 is released C1 discharges mainly through D1 &amp; R1 lighting the LED. (Some current also discharge through R2 but much less.)    







simulate this circuit &ndash; Schematic created using CircuitLab