not using Apple Mail, but using Calendar
In short, you don't. Calendar needs mail to send and receive appointments.

Have you tried simply telling Calendar to not use Mail for automatically retrieving CalDAV appointment requests via Mail?

Wanting to use Calendar for OS X implies you must use Apple's Mail for CalDAV appointment invitations as things stand today on Mountain Lion. You could either forego the Mail integration or select another Calendar like Fantastical or BusyCal.