Why does increasing FFT length (narrowing bandwidth) not decrease Noise Power per bin?
A spectrum analyzer typically has more than one mode. One mode normalizes the frequency output such that a pure tone of amplitude “one” with frequency centered on a bin center has an amplitude of “one” in the spectrum display.  

For a pure tone, nonrandom, centered on a bin frequency of amplitude “one” , the corresponding uniformly weighted  N sample DFT bin magnitude will be “N”, so all the bins are reduced by dividing by N.  (before magnitude squaring) 

An actual Spectrum Analyzer will include a fudge factor correction for nonuniform windows, and average multiple DFTs . 

I haven’t seen too many spectrum analyzers that don’t have a DFT under the hood.  The last analog swept frequency analyzer I used was a very long time ago. I have seen a few modern versions at frequencies where converters aren’t good enough but DFT versions dominate most of the market.