Toggle a led with a switch
If you're connecting the button from pin 2 to ground and not using an external pull-up, it can lead to trouble. Either use an external pull-up or set the internal one:

pinMode(buttonPin, INPUT_PULLUP); // initialize the pushbutton pin as an input:


Also, you need to debounce the button.