Arduino receives HIGH when the button is LOW
You will need a pushbutton connected as follows:

The signal labeled Vout will connect to digital pin 4.



Explanation:

When the pushbutton is NOT pressed, no current flows through the resistor, and Vout = 0V. When the pushbutton IS pressed, current flows through the resistor, and Vout = 5V.

This resistor is called a pull-down resistor, because it pulls the signal down to ground. A pull-up resistor would reverse the operation of the pushbutton (i.e. Vout = 5V when not pressed, and 0V when pressed.)