Which component to use for a few GBytes of memory available for several years?
In my projects I use Compact SD card directly soldered to pads on the board. This means there is no socket unreliability to contend with. The interface is standard, and looks like being so for many years, so even as individual cards will come and go, any commodity card of sufficient size can be substituted in (as long as you code to the common specification and don't try to be clever and employ some magic tweak from one particular vendor). 

The only downside with a permanent soldered connection to the card is that's you can't reckon on swapping it for software development, the board will need a connection for initialisation and debug, JTAG, Ethernet, wifi and USB are the obvious candidates here. Most designs will have have a need for at least one of those anyway for their normal operation. If not, then SPI or I2C are lightweight alternatives.