Why does my computer freeze when I unplug my headphones?
I faced a similar problem. I had a bluetooth headset and a wired one. Whenever I disconnected my wired one, sometimes the mac tried to connect to the bluetooth headset, and when it was switched off/out of range, the system would hang for sometime.

could be something similar. Can you give the list of speakers/headphones configured on your mac and their order (a screenshot would be convinient)