White noise vs. delta pulse and Ultraviolet catastrofe
As Jason was saying in the comments, the power spectral density of white noise is flat.  This is equivalent to saying that the autocorrelation of white noise is a delta dirac function (i.e. that there is no correlation, positive or negative, between one noise sample and another), not that the noise itself is a delta dirac function.