How do I debug on-chip with Arduino?
The Arduino Uno uses the Atmel Atmega328p microcontroller, which only has one debug option, DebugWire.  There are a few tools that can work with it, but the least expensive and most commonly available is the AVR Dragon.

You can use AVR Studio 6 for this.  You can use the .elf file file produced during the Arduino build process directly, but setting up the debugger is a little easier if you port the Arduino program to the AVR Studio.  It's not difficult, but there are a lot of little pieces to deal with.  There are guides online to help you through this.