Essential parts of a USB connector, stripping to bare bones
This has already been done. This is the basically the smallest you will get a USB Type A dongle,

You can get smaller Security dongles (Not storage ones) without the shielding like these,

We may start to see smaller Type C USB dongles in the next few years, but I struggle to see why anyone would want this?
EDIT - Okay so these actually exist for authentication,