Why is my transistor switched on even though the collector is not connected
My question is: How can I fix this issue? Ideally I want to LED to not
light up at all unless there is current on the collector.

Move the LED to be in the collector circuit (with the appropriate series current limiting resistor) and connect the BC547 emitter to ground/0 volts. Feed the base via a 1 to 10 kohm series resistor.
As you currently have it connected, the base-emitter junction acts like a forward biased diode and will supply the LED with current via the base. With the above proposal this won't happen.