Idea for Noise Level Estimation / Automatic Thresholding in the Presence of Peaks I have the fft of some signal, and want a rough estimate of the noise level in order to choose an appropriate threshold for our peak detection algorithm. In general, the fft contains mostly noise with a handful of peaks (which are usually pretty high compared to said noise). For reference, I've attached a screenshot of a pretty typical fft:



Now to my proposed algorithm for noise level estimation, which is based on the following assumptions:


What i want is the mean and standard deviation of the fft
If the fft contained only (gaussian) noise, the median would be very close to the mean
Peaks shouldn't affect the median too much
The mean and standard deviation are independent of the order of the samples
Therefore i can sort the fft and still get the same result


So the resulting algorithm looks like this:


Sort the fft (ascending)
Calculate the mean and standard deviation of the sorted fft, but stop as soon as the mean exceeds the median
Use the calculated mean and standard deviation to choose the peak detection threshold
Profit?


I've tested this algorithm on some typical signals and the results seem to be pretty decent (keeping in mind that i don't need the exact noise level, but just something robust to choose an appropriate threshold). Sorting is a little expensive of course, but i don't expect the noise level to change too much, so i don't need to run it for every fft.

That being said, the algorithm itself feels kind of "wrong", because i don't ask the actual question "What would the mean and standard deviation be without the peaks?" but rather "What would the standard deviation be if the mean was X?".

So what are your thoughts on this?
Are there better algorithms for this kind of problem? (i bet there are)
I'd do some small adjustments to your idea (You really nailed them).

Assumptions


The Signal Model - Signal + Additive White Gaussian Noise (AWGN)
Probably we could generalize it more but this is beyond the scope of this question.
The DFT of the signal contains Peaks with relatively small roll off
This is important as we're almost saying the Signal is a combination of sparse number of Harmonic signals. As the lobes width will hide the noise data. So the ratio between the observation window, signal lobes width and sampling rate means that the DFT of the data creates sharp peaks.


Estimation

Under the above assumptions we know that:


Noise is AWGN in the DFT (As Gaussian Vector stays Gaussian under linear transformations). It also suggests that the Mean Value and the Median are similar (Realization of Symmetric Distribution).
Without the Signal Data the Mean and the Median value of the beans should be similar.
Number of bins contaminated with signal (Funny to write this, ha?) is sparse.


So we can sort the bins, but sorting and evaluating the mean on sub set will create bias unless we make sure the sub set is symmetric around the Mean / Median.
So, I'd sort the data and keep bins in the 25-75 Percentile range only. I'd use Mean on those.