AVR Timer without affecting Pins
Unless someone more knowledgeable about your device enters here (and I can hope they do), I'll point you to a few places where I think the answer becomes "yes, you can."

Take a look at the XMEGA C MANUAL on page 137. In particular, "Table 12-10. Clock Output Port Configurations" on that page. There, you can see how to disable the clock output, itself. I don't believe this addresses timer events, though. But on the same page, at the top where it says "Table 12-8. Event Output Pin Selection," you will also find out that you can similarly disable event outputs. Those two bits are part of the CLKEVOUT register.

What adds to my own questions is the fact that they also appear (prior page or two) to discuss the ability to map ports to virtual ports. And this brings in a whole other set of routing questions for me. This device family appears to have full crossbars on it and that makes telling you exactly what to do and how to do it very difficult for someone who has zero experience with this device. I'd have to do a lot of reading to make sure I knew what I was doing.

In the meantime, I think you can do what you want. But hold this as a tentative answer. Perhaps someone well informed with this device (or better able to interpret the datasheet) can provide a more comprehensive answer for you.