Why does this button have a connection to ground?
That resistor is called a pull-down resistor. It's there to make sure pin 2 read low when the button isn't pressed. If you don't put it there pin 2 is left 'floating', and it could erroneously read a high value. Google it for more info. 

If you want to simplify you circuit you could instead use a pull-up resistor, and connect the button to ground instead of 5v. The ATMega chip on the arduino has these pull-up resistors build into the chip itself, so you don't need to add one to the breadboard. Just not that the input values will be reversed. So pin2 will read LOW when the button is pressed, and HIGH when not. To activate the internal pull-up use pinMode(2, INPUT_PULLUP)