STM32 with USB accessible storage
Consider SPI flash memories. 2MB are definitely available. You could get an MCU with USB device capability and implement a mass storage device (there is a ready demo for that). Block read/write operations would map to the SPI flash memory. The external memory can be simply formatted with a FAT32 file system. That would be basically an overpriced USB thumb drive with mediocre performance ;)

Now, when the device is unplugged from the PC you would read out the files (for example using FatFs) from the external memory.

The only tricky bit: you can not (easily) open the same file system on the PC and your firmware at the same time (that is also why some digital cameras block all features when connected with a USB cable).