In system programming for avr?
ISP works by using a serial connection (usually via the SPI and nRESET pins) to the MCU. The programmer talks to the device using the low-level protocol that the device expects, and connects to the host using a different, high-level protocol. The host then directs the programmer to perform the various programming actions on the device, and in turn the programmer reports to the host about its current status.

There are a number of programming protocols that already exist, such as the one described in AVR910. As long as both the connection type and the protocol itself are supported by AVRDUDE it will work transparently once the hardware is assembled.

Further details about ISP are available in both AVR910 as well as the "Serial Programming" section of the datasheet for the device in question.