Why is my transistor ON when charge is only going to the base?
In such a circuit the emitter current equals the collector current PLUS the base current:

Ie = Ic + Ib

In normal situations the collector current is much larger than the base current, hence by good approximation

Ie = Ic

But in your case the collector current is zero, so the first equation degrades to

Ie = Ib

Which is exactly what you see: a very small current through the LED.

Another way to look at your circuit is to replace the b-e of the transistor with a diode. This gives the same results.