Light won't turn off, is there a short?
If the only paths in the circuit were those shown in your diagram, then your switch would turn off the light (but leave it in an unsafe condition where you could be shocked by touching the base of the bulb while changing it). Moving the switch to the hot side of the switch loop would make the wiring much safer. 

BUT. This is not the only problem you have. The symptom you describe proves that there is another path from the light's neutral to the breaker-box's neutral. The principal danger to you, and to any future maintainer, is that you don't know where this path is. This means that you're remodeling your wiring without fully understanding it first. 

I know it can be very difficult to analyze the wiring circuits in an old house.1 You have to open practically every j-box and even remove some wire nuts and separate the wires. You need to test voltage with the power on and check continuity with the power off, and get out of breath running back and forth to the breaker box. 

You can't just give up and wire it the best way you can. You must trace everything until you know where it all goes, or in the case of the stuff you already ripped out, where it all went. You can't "ignore the other 3 that come into the ceiling box", because you don't know which of them should be left connected to each other and which should be capped off. 

If you don't have the patience to trace all the old wiring, then you should hire a professional electrician to do it. Be aware, though, that this will be expensive -- unless you hire a guy who also doesn't have the patience to trace all the old wiring. 

1. Believe me, I know. My house was built in 1891 and still contains some inaccessible knob-and-tube wiring that was originally installed for Edison direct current service.