Buzzing power supply
When load is low or very light, most SMPS will either reduce the frequency, or work in skip-cycle mode. 

Skip-cycle mode is a clever way to save power: the very light load slowly discharges the output capacitor. When voltage reaches a lower threshold, the supply pumps a few switching cycles to recharge it, then goes back to sleep.

When the frequency of this event is in the audible range, then what you hear will be:


A ceramic capacitor acting piezoelectric
Magnetostriction or other "voice coil" effects in the transformer


You can add a load at the output (a resistor) to shift the frequency to an inaudible one.

An alternate solution can be to put some insulating viscoelastic caulk on the noisy components (ie, polyurethane, but not water-based acrylic). But if it's the transformer, there's not much you can do...