Pin change interrupt won't toggle OFF my LED
I can't see anything wrong with your code (although you do rely on registers being in their reset state which may not be true) but I do have a couple of points:

The pin change interrupt will occur if the pin either goes from low to high or high to low. Even if it worked as designed the led would change state both when you press the button and when you release it. You may not notice if you press and release the button quickly as the LED will end up in the same state.

in general it is a bad idea to use pushbuttons directly to create an interrupt. Whenever a normal switch connects (or in some cases disconnects) it does not do so cleanly, it is likely to bounce and cause many transitions over a period of e few hundred microseconds to a few milliseconds. This is called &quot;contact bounce&quot;.


A better approach that is normally used is to create a polling loop using a timer that runs at a few tens of Hertz to a few hundred times per second.
Within that timer ISR, you should create a state machine that polls the switch and performs the desired behaviour.
This is not suitable for waking up a processor from sleep but for that, it is acceptable to use the pin change interrupt bearing in mind that there may be multiple switch closures from a single button press.