Gaussian White Noise - Relation Between Distribution and Correlation
When we say the stochastic process $X_{t_1}X_{t_2}\cdots X_{t_i}$ is a Gaussian Process with mean $\mu$ and variance $\sigma^2$ it only means
$$X_{t_i}\sim\mathcal{N}(\mu,\sigma^2)$$
So still we know nothing about the relation of the $X_{t_i}$'s with each other.
When we add that the process is also white, it means the $X_{t_i}$'s are uncorrelated. That is

$$\mathsf{E}[(X_{t_i}-\mathsf{E}X_{t_i})\mathsf{E}(X_{t_j}-\mathsf{E}X_{t_j})^*]=\begin{cases}\sigma^2,&amp;i=j\\0,&amp;i\neq j\end{cases}$$

Off topic, but notice that the $\tau$ notation is an implicit assumption of the process being stationary.

The process can be Gaussian but colored (rather than white). In such case, the random variables are Gaussian but correlated. For instance, the autocorrelation function in case of stationary Gaussian process you mentioned can be something like this:
$$R_{XX}(\tau)=\sigma^2e^{-|\tau|}$$