Negative Voltage Output From Capacitor
The effect you are seeing is used for ‘flying capacitor’ voltage converters. Basically, the capacitor is storing charge in one polarity, then re-connnected in a different polarity to discharge.

This same principle is used in a now-obsolete LED flasher chip called the LM3909. At the high level, the LM3909 resembles your circuit, except it does something clever: connects the LED across the (-) flying cap to (+) battery, doubling the voltage across the LED. Its ability to run on very low battery for long stretches of time is the stuff of legend. 

BONUS: an analysis of the LM3909 here: https://cdn.hackaday.io/files/291791248394336/Discrete%20Version%20Of%20The%20LM3909%20Oscillator%20IC.pdf