Determining Loss of Information by Taking Average (Mean of Signal)
Consider observing $y(n) = w(n)$ where $w(n)$ are IID gausian distributed with mean 0 and variance $\sigma^2$. The entropy of each other observations $y(n)$ is thus: $$\log_22\pi e\sigma^2$$.

Now if you average $y(n)$ the resultant variable is also gaussian distributed with mean 0 but variance $\sigma^2/N$. Therefore the entropy is $$\log_22\pi e\sigma^2/N$$ so the entropy difference is $$\log_2N$$ and this would be the entropy reduction.

The amount of information lost will depend on the distribution of the random variable. However, there are some interesting problems in signal processing where there would essentially be no loss.

For ex: consider a set of observations taken as $$y(n) = x + w(n)$$ where $w(n)$ is normally distributed IID random varibles and suppose we take $N$ observations of this kind.

Then the mean of $y(n)$ essentially represents what we refer to as a sufficient statistic for $x$ and infact represents the minimum variance unbiased estimator, i.e. the most efficient estimator. So in this case the sample mean encompasses all the necessary information and no information is lost.

However, if the noise $w(n)$ were  not gaussian we couldn't claim this. So as you see the amount of information lost by sample mean depends on the distribution of the underlying variable.

Another example would be a trivial one of a deterministic random varaible. Here also no information would be lost by the sample mean.

Another way to look at it would be if the likelihood of posterior distribution (parameterized by the sample mean), after observing the sample mean renders the posterior useless, meaning encompasses no additional information about the random variable then there is no information loss, otherwise we would incur information loss.

Hope that helps