Does the DNS service need to be on in order for the caching service to work on OS X Server?
It depends:

If you plan to provide the Caching Service to clients with public IP addresses (in other words, not behind a NAT) that are not on the server’s same subnet, you need to create a TXT DNS record in your zone file for the server with the IP address ranges the server is responsible for. Source

If you plan to provide Caching Service in all or any combinations or portions of subnets of the local network that share a common public IP address you don't need a DNS-Server.

Please check the limitations of the Caching Service here.