24VAC relay coil buzzing when switched on
AC relays will hum or buzz. Their coil windings are not so very different from the voice coil of a loudspeaker, but without a large diaphragm to drive. They are often dipped in varnish or wax to fix the winding in place and minimise the noise, but that's an imperfect process. If it's not drawing excessive current or getting excessively hot, I would consider it normal. 

Poor quality AC waveform will have higher harmonic content, at frequencies where this loudspeaker is more efficient, so you will hear it get louder or change tone at certain times of day as usage of the supply changes.

If the noise was intolerable I'd consider running it off 24VDC instead (if the relay datasheet permits that)