What is the difference between the spectra of white Gaussian noise and of a Dirac impulse?
They have very different phase spectra. For a dirac the phase of the spectrum is zero for white noise, it's uniformly distributed over  $[0, 2 \pi]$

They are both constant over the whole frequency range

That's only true for the spectrum of the Dirac. For white noise, it's more complicated. The FFT of a chunk of white noise does NOT have constant spectrum, it's actually white noise as well (the real part and the imaginary part). It only becomes constant if you average every over a large enough number of chunks (or a long enough period of time) or if you apply some sort of spectral smoothing.