In signal averaging, will the noise be reduced if the noise is not Gaussian? My understanding is that signal averaging works this way. If we take the average of the waveform, we get the average of the signal component (which is the magnitude of the signal) and the average of the noise component (which becomes closer to zero as number of samples are increased).

However, this is based on the assumption that the noise has zero mean and constant variance (i.e. white or Gaussian noise). What if the noise is not white? For example, what if the noise decreases as frequency increases? Will the noise still be averaged out by signal averaging?
Consider two zero mean signals $x(n)$ and $y(n)$. Let $z(n)=x(n)+y(n)$. In the following $E()$ denotes the expectation.
$$E(z) = E(x+y)= E(x)+E(y)= 0$$
I've dropped the time index $n$ for convenience. So $z(n)$ has zero mean. Now let's look at the variance. Since $z(n)$ is zero mean - I'll just use the correlation.
$$E[z^2] =E[(x+y)(x+y)]=E[x^2+2xy+y^2]=E(x^2)+2E(xy)+E(y^2) $$

If the correlation between $x(n)$ and $y(n)$ is zero, then $E(xy)=0$, then the variances add. Usually this is done by stating the $x(n)$ and $(y)$ are independent which implies uncorrelated.

In the general case $E(xy)\neq 0$, so the resulting variance could increase or decrease depending on the sign of the correlation.

Note - stating that $x(n)$ and $y(n)$ are white can be a little unclear. Usually saying $x(n)$ is white implies it is uncorrelated with itself - it does not state how it is correlated to another noise process.

The above analysis is not dependent on the type of noise (Gaussian or not) and it does not make any assumptions on the autocorrelation functions of $x(n)$ and $y(n)$.