Condition for optimal value to be the average In many single parameter estimation process average is the optimal. Can anyone tell under what condition average is optimal? For example is average optimal if the data is iid, Gaussian or something else?
You don't need to assume anything about the data in order for the average to be optimal (in a least squares sense) for certain estimation problems. As an example, assume you want to estimate a constant value $x$ and you are given $N$ noisy measurements:

$$y_n=x+e_n,\quad n=0,1,\ldots,N-1\tag{1}$$

where $e_n$ is the error of the $n^{th}$ measurement. If our goal is to minimize the mean squared error

$$\frac{1}{N}\sum_{n=0}^{N-1}e_n^2=\frac{1}{N}\sum_{n=0}^{N-1}(y_n-x)^2\tag{2}$$

then the optimal estimate $\hat{x}$ is obtained by minimizing (2). Taking the derivative of the right-hand side of (2) with respect to $x$ and setting it to zero gives

$$\frac{2}{N}\sum_{n=0}^{N-1}(y_n-\hat{x})=0\tag{3}$$

From (3) the optimum estimate is obtained as

$$\hat{x}=\frac{1}{N}\sum_{n=0}^{N-1}y_n\tag{4}$$

which is simply the average of all observations.