Is excess noise in a resistor only present with DC?
There is excess noise in a carbon resistor when current flows through it. It doesn't matter whether that current changed direction in the past and will do again in the future, or is steady, the noise at any given current is the same.
The question is perhaps, why do people emphasise that noise is generated even when the current is DC?
One possibility is that it's unexpected to most people that a passive device like a resistor should generate noise in excess of its thermal noise when something as uninteresting as a DC current is flowing.
Another possibility is that when trying to measure or demonstrate the noise, it's trivial to get a very low noise source of DC. You only have to use larger and larger capacitors to ground, and you can suppress the supply noise to any given degree. It's rather more difficult to generate low noise AC.
The main reason is that the noise is 1/f noise, that it gets weaker as the frequency increases. So although the noise may be generated by an AC current, the noise will be modulated by that current, and will appear as sidebands around the drive frequency. This confounds the problem of measurement, as although it's relatively straightforward to measure noise at 1 Hz and 0.1 Hz from DC, getting that close to an AC signal requires exquisitely low phase noise sources, which themselves are corrupted by 1/f noise.