Which directories should I back up to save all of my data?
A lot of applications store critical information in ~/Library so you might be better off with backing up your whole user folder (with the exception of Trash of course). If you use incremental backups the size will not be too big after the first run. And you can at least exclude ~/Library/Caches anyway. You also may want to back up any registration codes for applications (by keeping them in separate file for instance) you've bought so you can reinstall them easily.

OTOH you might want to consider the time required to rebuild all your configurations and installations from scratch and compare this to the price of an external USB drive.