What is the best estimate of a signal given multiple noisy realisations I assume that the noise corrupting the sought-for signal is [Edit: Gaussian] white noise of zero mean and unknown variance. Is the optimal solution simply the mean over the realisations (the ensemble mean), or, we can have a better estimate, say, in terms of a smaller root-mean-square-error?
Edit: I found that my question have been asked already. Although the practical context is different, as in my case there is no concern that the different observations on different sensors could be correlated or that the corrupting process is slightly different. The answer to that question suggests that superposition, i.e., ensemble mean, is the best (“efficient&quot;?) estimate. However, it occurs to me that it might not be the case, as a moving-window temporal average of the ensemble mean would yield an even smaller RMSE if the signal is sufficiently smooth. Unfortunately, there was no published reference given there. It would be great to find the answer in a textbook or paper, as it would surely clarify the conditions under which the result/theorem applies.
In any case, even if the moving average of the ensemble mean yields a better RMSE than the ensemble mean alone, I wonder if there is an even better estimate. Intuitively -- and it might be a common failure of intuition -- there might be value (information) in having the relaisations available &quot;separately&quot; as opposed to only their superposition, the ensemble mean. Any further help would be much appreciated.
The term you're looking for is efficient estimator. That's the least-variance (== least RMS) estimator for a given amount of observation.
Whether or not that's the average depends on your noise distribution.
For example, consider the two noises:

continuous-valued uniformly distributed noise
Noise that only takes values $\in \{-\sigma, 0, \sigma\}$ with equal probability.

Both can have the same variance, both are zero mean. Both can be white. But while the average sounds like a good idea for the first one, for the second one you can get a 0 RMS estimator as soon as you've observed three different values by picking the middle one.