Quantifying white noise with a spectrum analyzer
When measuring noise you must always specify the bandwidth in which you measured it !

As you lower the Resolution Bandwith (RBW), the noise level you see on the Spectrum Analyzer (SA) will go down as well. This makes sense as there is less power present in a smaller frequency band (and more frequencies outside the RBW are suppressed).

For flat noise, -100 dBm in 1 MHz is the same as -80 dBm in 10 MHz or -120 dBm in 0.1 MHz.

The Video Bandwidth (VBW) provides a sort of averaging on the image, leave it on "auto" for the moment.

What model SA are you using ? Maybe it has a noise measuring option ?