Removeable board sized ssd
There's something called NGFF as in next generation form factor, or M.2 (see link).

Passerby's suggestion is probably the best, especially if you opt for a usb stick since you can hotplug it into the host. With a PCI or SATA drive you need specific hardware, or you need to reboot your machine. If speed is not of great concern a usb2.0 stick will do, you can also opt for 3.0 if you need the additional MBps. I fear SD cards are a bit more unreliable than usb stick, and ofc the ssd option is the most reliable, but also the priciest.