In NXT, which files store the contact addressbook data? When upgrading NXT, which files do I need to copy over that contain the NXT address book contacts? What other vital files should i copy over from the old version?
The contacts are stored in local browser database. You do not need migrate them.

Regarding upgrade, in most cases when using the zip file installation the best strategy is:
1. Stop the NRS
2. Delete the /lib library
3. Unzip the new version on top of the old version
When you start the NRS again it will migrate the database to the new structure, this can take a while, sometimes up to an hour.

If you need to install from fresh, copy over from your old installation:
1. The blockchain in the nxt_db folder, also the nxt_test_db forlder if you are using the testnet
2. The relevant properties in your nxt.properties