xmega external crystal won't stabilize
Note that 18pF load capacitance from the data sheet does not mean to put two 18pF capacitors on the board.

Its the total capacitance the crystal should "see". Assuming a stray capacitance of 5pF, you would have

Cl = (18pF*18pF)/(36pF) + 5pF = 14pF

You may want to try something like 24pF for the load capacitors.

Furthermore, the ESR of the crystal you choose it pretty high. This may become an issue if the oscillator circuit cannot adequately drive the crystal.

Higher frequency crystal tend to have lower ESR, so try a crystal with 8 or 16MHz from that series.