Can I power on my old MacBook with a no-lights battery?
Normally you should be able to, and even Apple says so, by providing a warning and a CPU clock speed reduction.

If it does not work on your computer.....

You might be having a problem with the MagSafe adapter or SMC. 

Your machine should turn on normally whether or not the battery is in it. 

The only problem you'll have with the battery removed is that machine will run slower - but that's by design. 

When you get the new battery, make sure you try another MagSafe adapter to be safe.

Try another SMC reset by Holding the power button for 5 or 10 seconds.