Mac Mini Insists on Shutting Down When I Unplug Mouse and Keyboard
In support of this, next time it happens, make a note of the time you pull the plugs out, and then when you are rebooted, run the Console application, and check your logs for the same timescale.  This should provide a handy clue.

Also, do you have any other devices on your USB hub that provide any more than just mouse/keyboard services?  Any card readers, storage devices etc?  These may need umounting properly before just pulling the plug and may be a part of it if you have them.