Does the new AppleTV need to be connected to the internet to access content on my Mac?
Turns out it needs to be on the internet initially, in order to set up Home Sharing.

But once Home Sharing is set up, it can access content on your Mac as long as your Mac is on the same local network, and has iTunes running.