What's the difference between INPUT and INPUT_PULLUP?
The default is INPUT which sets up the pin as an INPUT.  If the pin is totally disconnected, it will randomly read HIGH and LOW.   If you tie it to +5V or 0V, it will obviously read HIGH or LOW.  

Internal to the Arduino, the Atmega chip has internal pullup resistors with a value around 20k ohm. (See the DigitalPins documentation for more details)  These resistors can be optionally connected internally using INPUT_PULLUP.   This is functionally (and electrically) equivalent to connecting a ~20k ohm resistor between the pin and +5V, the only difference is that it requires no external components and you can turn it on and off in software during the execution of your program.

So why pull-ups and not pull-downs?  There are likely several reasons for it, but when wiring buttons or switches or anything "normally open", you only have to tie them to ground, you don't need to run +5V out to them.   Since most boards are going to be designed with large ground pours for shielding reasons anyway, tying to ground is practically reasons.  

Some more featured ICs like ARM chips have both pull ups and pull downs, but the 8-bit AVR line only comes with pull-ups.  You just have to remember that HIGH is "open" and LOW is "closed".