How and why does this oscillator work?
Besides G36's suggestions for answers, here's an LTspice attempt, not using uic, or startup, or a pulsed supply, or ic:



It's a simulation ran with both a 100n and a 100\$\mu\$ cap in parallel with the 5k1 resistor, to see the effects of it. Also, I left the schematic as you showed it, but you need some sort of resistor in series with the LED, the spikes (even though short lived) from the supply are well over 0.5A.