Minimal external components for AVR CPU
For best results a 0.1uF capacitor on each Vcc line to the closest GND pin. This includes AVcc. A 10k ohm pullup on the reset line. The programming lines of course. 

About the clock, the datasheet says the following. 


  The device is shipped with Low Power Crystal Oscillator (8.0MHz-max) enabled and with the
  fuse CKDIV8 programmed, resulting in 1.0MHz system clock (with a 8MHz crystal). The default
  fuse configuration is CKSEL = "1110", SUT = "01", CKDIV8 = "0". This default setting ensures
  that all users can make their desired clock source setting using any available programming
  interface.


This implies that a crystal/caps do need to be connected to a virgin chip to talk to it.