In what way VBW and trace averaging are different in spectrum analyzer?
Without getting into DSP and filter theory:

Video bandwidth works by 'smoothing' based on frequency points at higher and lower frequencies within the same sweep - It limits how 'sharp' peaks can be.

Trace averaging works by averaging a frequency point with previous versions of the same frequency point from different sweeps. This means it doesn't care about the samples at other frequencies, but you need more than one sweep to be able to do it.