What is the difference between a vertical stabilizer and a rudder?
The vertical stabilizer is immobile (if it starts to move, you're in deep shit), and provides stability in yaw (it keeps the aircraft pointed more or less in the direction that it's moving).

The rudder is mobile, and typically attached to the aft edge of the vertical stabilizer. It provides controllability in yaw (it allows the aircraft, if necessary, to point in a different direction than the one it's moving).

The rudder allows the aircraft to slip sideways when you want it to; the vertical stabilizer keeps it from slipping sideways when you don't want it to.


(Original via flickr.com)