Programming microcontrollers: JTAG, SPI, USB oh my!?
ATMEGA microcontrollers like the one in the arduino that are coming straight from the factory can only be programmed using the SPI or JTAG interface. 

SPI stands for Serial Peripheral Interface and is a way for microcontrollers to communicate with each other or with the outside world. It's also called 3-wire sometimes because it uses three wires to communicate. To program a chip, you will need a special programmer which reads commands from USB to drive the SPI lines to program the chip. A popular one seems to be the USBTinyISP from Adafruit. A very good introduction into SPI programming is at SparkFun. The most popular applications for programming Atmel AVR chips are avrdude (cmd line), ponyprog (doesn't work with newer programmers) and in some cases AVR Studio (if your programmer supports it). The advantage of SPI programming is that you can program any Atmel chip straight from the factory so you wouldn't always need an Arduino in your projects.

Where SPI is "just another" serial protocol, JTAG is a protocol which is especially designed to program and debug microcontrollers. Not all Atmel micro's support JTAG but the ones that are used in the Arduino do. The JTAG protocol can be used for cool things like "in circuit emulation" and debugging which means that it enables you to see the state of the program in your microcontroller while it's actually running. To do this you'll need a special programmer. Seen my answer for another question.

To program a chip using USB, you'll have to program it first with a "bootloader" using SPI or JTAG. Once loaded with the bootloader, the system will be programmable from any PC using a USB-Serial converter. The downside is that the bootloader takes up some memory space and this method does not allow you to see the state of the chip while it's running.