Portable encrypted USB stick
Buy a USB flash drive that uses hardware encryption, where the unlock mechanism doesn't depend a software client, then format it however you prefer. An example of this would be a USB flash drive with a a biometric sensor which can scan a finger, or a built-in keypad where you can enter a PIN to unlock the drive. Several vendors sell hardware encrypted flash drives like these, including Apricorn and Imation.