Is "humming" normal in an AC relay?
Figure 1. Source: Machine Design.

The relay coil, when energised, pulls in the armature to actuate the contacts.
Since the coil is powered by alternating current the magnetic field collapses to zero at each mains crossing and the armature tends to start to release. Its inertia is high enough that the contacts remain actuated long enough to maintain contact through each zero-cross of the mains.

The buzz is normal. It is caused by the vibration of the armature on the yoke on each half cycle.

Just a note on relay terminology: "I have a 240V relay that uses a 120VAC coil" is a little confusing. "I have a relay with 240 V contacts and a 120 V AC coil" would be clearer.



Update:

Spehro and Tony's answers both address the use of 'shading' poles on the armature to help maintain force during zero-cross. This in turn will reduce the vibration.



Figure 2. The yoke of an AC 'contactor' (high-powered relay generally used for AC motor circuits, etc.) showing two shaded poles. Image source.