Covariance matrix associated with random DC level in Gaussian noise
It is important to add that $w[n]$ is uncorrelated with $A$. This means that

$$E\{Aw[n]\}=E\{A\}E\{w[n]\}=0\tag{1}$$

because we may assume that $w[n]$ is zero mean: $E\{w[n]\}=0$. In this case all cross-terms in the expectation are zero, and we're left with

$$C_{ij}=E\{A^2\}+E\{w[i-1]w[j-1]\}\tag{2}$$

Since $w[n]$ is white, you have $E\{w[i-1]w[j-1]\}=\sigma^2\delta_{ij}$. The expectation of $A^2$ is given by

$$E\{A^2\}=\sigma_A^2+\mu_A^2\tag{3}$$

where $\mu_A=E\{A\}$ is the mean of $A$.

So we finally have

$$C_{ij}=\sigma_A^2+\mu_A^2+\sigma^2\delta_{ij}\tag{4}$$

The expression in your question is only valid if $\mu_A=E\{A\}=0$, i.e., if $A$ is a zero mean random variable.