Most power efficient way to constantly save 2 kilobytes of data per second
There are two constraints here, and I think it might back you into a corner. Size and then power.

2Kbyte/s is 2*60*60*24/1024 = 168.75Mbyte/day. This is a lot. The only readily available technology that can be used with a small microprocessor that can store this volume of data is an SD card. There are no serial EEPROM or flash chips this large, and the Arduino, as far as I know, can't deal with the NAND flash used in USB sticks and SD cards.

However, SD cards can be relatively power hungry. They draw between 5mA and 100mA when writing, typically around 30-50mA. The SD spec says the maximum is 200mA. Writing is also quite lengthy as they operate on blocks of data rather than individual cells. I've never done a full analysis of how much power they use, but it is not insignificant and much, much more than serial EEPROM or flash.

So your only choice, if you have to store that much, is to go with SD cards.

I would investigate compressing the data. Even very basic schemes can make a huge difference.