USB audio ducks when Notifications happen
The bluetooth receiver was also paired with my phone at the same time as the mac. 

I've unpaired the phone and it stops the ducking. 



I figured this out when a call came in and the ring tone played full blast and started trying to figure out how to prevent that from ever happening again.