Digital Read Serial tutorial - calculating the resistor value
You're very close. The resistor is there to act as a pull down resistor; If the input pin is floating it is just an antenna picking up random static. The key is that when positive is supplied (by pressing the button) the positive source needs to "overwhelm" the pull down resistor so that the input pin reads positive. 

If there were no resistance, than the input pin would be reading in the middle of a direct path that does not favor positive or negative. Asside from the direct short that would be occurring, Which state the arduino would read I suspect would either be random or some property of the power supply; It would probably still read ground. 

With the resistor, the signal is read directly connected to positive, before the load resistance. This way it will be sure to get a HIGH signal. 

The exact size of the resistor is of little consequence. Too small and you waste current, but too much resistance is effectively just disconnecting ground again. There is a very wide range of values that will work fine between those. 

I would like to note that when an arduino pin is set to input mode, it will effectively supply no current. It can be moddeled is being in series with a 100 megaOhm resistor. see http://arduino.cc/en/Tutorial/DigitalPins 
for details.