Building NOT gate with transistor -- output remains always high
If SW is closed, then ... no current flows through R1 and LED1.

Not quite. Current always flows through R1, but when the transistor is switched on, all of the current flows through it, and since the collector voltage is now less than the LED's forward drop, no current flows through the LED.
When you inserted LED2 in the emitter lead of the transistor, you made it impossible for the transistor to pull its collector low enough to &quot;short out&quot; LED3. The emitter voltage is now equal to the forward drop of LED2, and the collector voltage is a few hundred mV above that.
If you take out LED2 and connect the emitter to ground again, the circuit will work as expected.