Tradeoffs when considering SPI or I2C?
Summary  


SPI is faster.  
I2C is more complex and not as easy to use if your microcontroller doesn't have an I2C controller. 
I2C only requires 2 lines.




I2C is a bus system with bidirectional data on the SDA line. SPI is a point-to-point connection with data in and data out on separate lines (MOSI and MISO).  

Essentially SPI consists of a pair of shift registers, where you clock data in to one shift register while you clock data out of the other. Usually data is written in bytes by having each time 8 clock pulses in succession, but that's not an SPI requirement. You can also have word lengths of 16 bit or even 13 bit, if you like. While in I2C synchronization is done by the start sequence in SPI it's done by SS going high (SS is active low). You decide yourself after how many clock pulses this is. If you use 13 bit words the SS will latch the last clocked in bits after 13 clock pulses.
Since the bidirectional data is on two separate lines it's easy to interface.  

SPI in standard mode needs at least four lines: SCLK (serial clock), MOSI (Master Out Slave In), MISO (Master In Slave Out) and SS (Slave Select).
In bideroctional mode needs at least three lines: SCLK (serial clock), MIMO (Master In Master Out) which is one of the MOSI or MISO lines and SS (Slave Select).
In systems with more than one slave you need a SS line for each slave, so that for \$N\$ slaves you have \$N+3\$ lines in standard mode and \$N+2\$ lines in bidirectional mode. If you don't want that, in standard mode you can daisy-chain the slaves by connecting the MOSI signal of one slave to the MISO of the next. This will slow down communication since you have to cycle through all slaves data.

Like tcrosley says SPI can operate at a much higher frequency than I2C.  

I2C is a bit more complex. Since it's a bus you need a way to address devices. Your communication starts with a unique start sequence: the data line (SDA) is pulled low while the clock (SCL) is high, for the rest of the communication data is only allowed to change when the clock is low. This start sequence synchronizes each communication.
Since the communication includes the addressing only two lines are required for any number of devices (up to 127).  


  edit
  It's obvious that the data line is bidirectional, but it's worth noting that this is also true for the clock line. Slaves may stretch the clock to control bus speed. This makes I2C less convenient for level-shifting or buffering. (SPI lines in standard mode are all unidirectional.) 


After each byte (address or data) is sent the receiver has to acknowledge the receipt by placing an acknowledge pulse on SDA. If your microcontroller has an I2C interface this will automatically be taken care of. You can still bit-bang it if your microcontroller doesn't support it, but you'll have to switch the I/O pin from output to input for each acknowledge or read data, unless you use an I/O pin for reading and one for writing.

At 400kHz standard I2C is much slower than SPI. There are high-speed I2C devices which operate at 1MHz, still much slower than 20MHz SPI.