How to read an input value from atmega32 pin?
As stated in the comment, the battery being reversed, causes the while statement to be true (because the button closure is "noisy"), but it fails to meet the next test (b00000010 mask), so the LED is not lit.
After the battery is corrected, you might still experience the same problem because the button is "noisy" (when you close the circuit, it will switch between 5v and ground several times, before "settling" on 5v, and when you break the circuit, the same thing will happen before settling on 0v.
If you want to prevent this, you need to "debounce" the switch.