How to graduate from Arduino to using a microcontroller directly?
Welcome to the wonderful world of Atmel. Let me offer you some answers to your questions based on my hobby and professional experience.

Do not bother with anything BUT in-circuit programming. Unless you are a perfect coder, removing a chip every time you want to program it is a nightmare. I recommend the AVRISPmkII as an entry level programming tool. The disadvantage is there is no hardware debug supported. An alternative is the dragon but I have no experience with that. I can say that the JTAGICEmk3 is a nice capable debug tool.

Atmel studio is good for programming. It is actually my favorite embedded development environment. You can use CLI tools such as AVRDUDE and AVR-GCC but the IDE takes care of that for you.

IN SUMMARY:

Buy an AVRISP for just programming (easier, plug&amp;play) or a DRAGON (I can not offer advice).

On your board, bring out the ICSP pins to a 6-pin dual row 0.100" header -- the connections are described in this datasheet.

Reset switches are not needed.