Playing with a shift register and some LEDs
The switch is "bouncing":  Each time you open or close it, there is a period where it makes a bit of noise on the clock line.  The chip sees this noise as multiple transitions and clocks in more bits than you intended.  To fix this, you need a "debouncing" circuit.  Google should find you something.

edit:

You can't just switch voltage to an input like that.  Then the input is at +5V when the switch is closed, but floating when the switch is open.  You need at least a pulldown resistor after the switch.