Understanding the definition of mean/autocorrelation I was studying about the definitions of mean, expected value and autocorrelation. I wanted to verify my understanding the evaluation of mean, expected value and autocorrelation. At the same time to verify the definition of autocorrelation as the convolution of the signal with this delayed copy mathematically which I was not able to.

Here is my analysis:

Basically, I see mean as an average of all values in a population. For example, consider an experiment where we measure the outcome of the random variable that can go anywhere between $[1,100]$. By performing the above experiment $50$ times, we get $50$ outcomes:$a_1, a_2, \ldots, a_{50}$ which has values ranging from $1$ to $100$. 

The mean of the above random variable is given by:
$$
\mu = \frac{a_1 + a_2 +\ldots + a_{50}}{50}
$$

The expected value is given by:
$$
E(x)=(1*\textrm{probability of getting 1})+(2*\textrm{probability of getting 2})+\ldots+(100*\textrm{probability of getting n})
$$

which can also be represented as:
$$
E(x)=\prod_{k=1}^{100} kP_k
$$

where $P_k$ is the probability of getting $k$. Now taking the actual definition of an expected value which is:
$$
E(X)=\int_{-\infty}^{\infty} xF_x(X)dx
$$

The above seems somewhat satisfying as we could relate the above with the example - taking $x$ as one of the outcome and $F_x(X)$ as the probabilities (actually it's a probability density function to be accurate).

For a random process, the above expands to (at time $t_1$):
$$
E(X_{t_1})=\int_{-\infty}^{\infty} x_{t_1}F_{X_{t_1}}(x_{t_1})dx
$$
Similarly, expected value can be computed at several time instances since the random variable changes every time instant.

Now, lets take a Strict-Sense Stationary Random Process where expected value (mean) is constant and autocorrelation depends up on the time difference. So, the definition of Autocorrelation is given as per wikipedia: 


  $$
R_x(\tau)=\frac{E\left[\left(X(t_1) - \mu\right)\left(X(t_1+\tau) - \mu\right)\right]}{\sigma_{t_1}\sigma_{t_2}}
$$


The above is with Normalization. Taking out the normalization, we have:


  $$
R_x(\tau)=E\left[X(t_1)X(t_1+\tau)\right]
$$


Till now it's pretty clear. Now applying the definition of an expected value in the above expression:
$$
R_x(\tau)=E\left[X(t_1)X(t_1+\tau)\right]=\int_{-\infty}^{\infty} x_{t_1}F_{x_{t_1}}(X_{t_1})x_{t_2}F_{x_{t_2}}(X_{t_2})dx\quad\text{where}\quad t_2 = t_1+\tau
$$

But in Wikipedia, it's given as:


  $$
R_x(\tau)=E\left[X(t_1)X(t_1+\tau)\right]=\int_{-\infty}^{\infty} x(t)x^*(t-\tau)dt
$$


This is where I am stuck at - I am not able to match the last two expressions. I am very well aware that autocorrelation of a random process is convolution with itself at a time lag of $\tau$ but I am not able to relate the expressions mathematically. I am stuck here for several days.
The definition of the autocorrelation function $R_x(\tau)$ depends on the nature of your $x$.


If $x$ is a deterministic signal with finite energy then: $$R_x(\tau)=\int_{-\infty}^{+\infty}x(t)x^*(t-\tau)dt$$
If $x$ is a deterministic  signal with finite average power$^{(1)}$ then: $$R_x(\tau)=\lim_{T\to+\infty}\frac{1}{T}\int_{-T/2}^{+T/2}x(t)x^*(t-\tau)dt$$
If $X$ is a (real-valued) random process then: $$R_X(t_1,t_2)=E[X(t_1)X(t_2)] = \int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}u_1u_2p_X(u_1,u_2)du_1du_2,$$


where $X(t_1)$ and $X(t_2)$ are two random variables (the realizations of the random process $X$ at times $t_1$ and $t_2$, respectively), $u_1$, $u_2$ are their respective values and $p_X(u_1,u_2)$ is the joint probability density function of these two random variables. This statistical autocorrelation becomes only dependent on $\tau = t_1-t_2$ if the random process is stationary and we can write:
$$R_X(\tau)=E[X(t)X(t-\tau)].$$

If, furthermore, the random process is ergodic then the statistical autocorrelation equals the temporal autocorrelation, i.e:

$$E[X(t)X(t-\tau)]= \lim_{T\to+\infty}\frac{1}{T}\int_{-T/2}^{+T/2}x(t)x(t-\tau)dt^{(2)}$$

$^{(1)}$: Signals with finite energy are also known as energy signals whereas signals with finite average power are also known as power signals. Moreover, the average power of an energy signal is zero and the energy of a power signal is infinite. 

$^{(2)}$A random signal is by convention considered as a power signal.