What causes a loud tone/noise in Blackmagic ATEM talkback?
tl;dr
Make sure none of the SDI outputs of the ATEM are routed back to its inputs.
Talkback feedback loop
The ATEM uses SDI channel 15 and 16 for talkback. If you route one of your SDI outputs to the SDI inputs (either directly via a simple SDI cable or trough a system of devices like record/playback devices) you will get a feedback loop that causes this high tone.
Possible solutions
The simplest solution is to break the loop by removing the cable/devices causing it. Obviously this way you will loose the functionality you tried to add trough it, like having M/E 1 PGM as one of your inputs so you can use it as an input on M/E 2, 3 or 4, so if you would like to keep the loop without the noise you have to deembed or redirect the audio from channel 15 and 16. You can use something like the Decimator MD-HX Cross Converter that can reroute SDI audio channels.