Mount USB Drive under One User
If you format the drive as HFS+ (essentially Mac formatted) you can set permissions on it (which any admin user can change of course). If it is anything else (PC formats: FAT, exFAT, etc.) you can't set permissions

Probably the best thing would be to put an encrypted DMG on the thumb drive. Anyone could get to the DMG but only the person with the correct credentials could open it.