Cheapest AVR With Built in I2C Support
A quick parametric search on Digikey shows that the ATTiny20 is the cheapest AVR with dedicated I2C capabilities in one-off quantities. Note that the ATTiny20 only provides hardware slave support, master TWI/I2C still needs a software implementation.

A slight step up is the ATTiny25 which has both master and slave TWI/I2C support, but operating in master mode the clock generation I believe has to be generated in software.

For complete TWI/I2C support, take a look at the ATTiny48.