How can I prevent my Mac from selecting my Apple TV as an audio source by default
If you use airplay then this might work for you.

Change your Apple TV Name (Apple TV software version 6.1 and later)

Go to Settings > AirPlay > Apple TV Name > Custom. 

Use this setting to change the name of the Apple TV to distinguish it from other Apple TVs on the same network.

Or in iTunes select the output