ARM Processor board and Programming
I'm not familiar with the ATSAM's, but I know the STM32F4's (as a comparative ARM Cortex-M) wouldn't work with an external 96MHz source - their working range is 4 to 26MHz.

I'd try a slower crystal or oscillator and use the PLL (assuming the ATSAM has one) to increase the system clock to 96MHz.

Re your problems with odd behaviour following an erase/program cycle, maybe try a different toolset.  The Coocox IDE (at v.1.7.7) appears to support your MCU (AT91SAM3U4C?), although I'm less sure that it supports the SAM-ICE.  If it doesn't then you can buy a CoLinkEx debugger that is definitely supported and should work with almost any Cortex-M for around US$20.

Disclaimer: I have no relationship with Coocox, I just think they have an awesome free product.

I suggest this because I've encountered a situation in the past where if I used vendor A's tools then I had to power cycle between erase/program cycles; but if I used vendor B's tools then I didn't, despite flashing exactly the same binary.