Curved noise floor and ugly unexplained peaks after FFT It has been so long since I studied this at University, so I'll better ask here if I'm doing something really wrong here.

I'm using a USRP frontend for L1 band surveillance. I want a 20MHz bandwidth, but I found through testing that it's better to use a sampling frequency slightly higher than twice that bandwidth (I'm using 41MHz), so I can use a local oscillator to move the DC out of the spectrum. Probably there is some additional error of concept here, but this isn't the problem I'm facing now.

First thing to note: I'm using an FFT over a big chunk of data. Depending on the configuration this size may vary, but for example I'm currently using 8192 samples per FFT. Maybe this is too much and I should compose the whole FFT using many smaller ones, somehow?

In order to reduce noise, I'm performing 100 of these FFTs, summing their outputs and dividing by 100. Probably there's a better method, but so far ok.

After I apply the FFT (with FFTW library), I convert the output to dBW/Hz with the following formula (I'm assuming I have Vpeak):

10 * log10(psd_output / (2 * 50 Ohm)) - 60


Now, if I plot this vector, I get something like this:


The noise floor is getting some curvation (up to 5 dBs). I tried applying a hamming or a blackman-harris window to the FFT output with the hope or removing this effect, but it didn't work.

If I test with a jammer, I can see the peak, but I also see some other minor peaks which are unwanted (and that are not seen in an spectrum analyzer with the same configuration):


What am I missing? These should be simple operations, but I'm out of ideas at this point.

EDIT: Adding other plot
Does the 'same configuration of your analyzer' mean the 'same sampling rate'? Is that equal to 41 MHz? It looks like you faced a well-known problem with a Nyquist criterion practical implementation. I mean that some time ago in digital oscilloscope ads there was an exact Nyquist ratio between a sampling rate and channel bandwidth (especially for cheap ones). Now it is about 10. (You may want to look at any ads of digital oscilloscopes) I remember there were several articles with figures similar to those posted here. To check this hypothesis just check a sampling rate of your spectrum analyzer (you wrote that artificial pikes are not seen in a spectrum analyzer with the same configuration.) or try to use a higher sampling rate in your data processing procedure if possible. In order to get at least an intuitive idea of the reason lies in the foundation of the problem perhaps you want to check this: Why is it a bad idea to filter by zeroing out FFT bins? 
I suppose the following paragraph from the cited answer works for you: 'So if your original FFT input data is a window on any data that is somewhat non-periodic in that window (e.g. most non-synchronously sampled "real world" signals), then those particular artefacts will be produced' This phenomenon is called resolution bias error, or more commonly, the picket fence effect http://www.azimadli.com/vibman/thepicketfenceeffect.htm. Here you can find an illustration http://www.mechanicalvibration.com/More_on_spectral_leakge.html