How should I correctly turn on my phone?
I'm assuming you have a power cable plugged in. 

In that case you'll have to unplug the cable, turn it on and plug it in as soon as something appears on the screen.

I've had this problem several times when trying to flash the phone while tethered to a computer and forgot to take out the USB cable when rebooting.