iTunes AirPlay to AppleTV works perfectly; System Preferences audio output to AppleTV jitters horribly?
The problem stopped occurring since updating OSX last night to 10.9.2.

In the fix lists is one ominous line:
"Fixes an issue that may cause audio distortion on certain Macs"

Here is the support link for reference:
http://support.apple.com/kb/HT6114