Can the law of iterated expectation be used on the inner expectation of the DQN cost function described in the DQN paper
Using the related question and simplifying notation
$$\begin{align} E_{\mu,\pi}[E_{\mathcal{E}}[s'|s,a]^2] &amp;= \sum_\limits{s,a}\left(\sum_\limits{s'}s'p(s'|s,a)\right)^2p(s,a) \\
&amp;=\sum_\limits{s,a}\left(\sum_\limits{s_1'}s_1'p(s'_1|s,a)\cdot\sum_\limits{s_2'}s_2'p(s'_2|s,a)\right)p(s,a) \\
&amp;= \sum_\limits{s,a,s_1',s_2'}s_1's_2'p(s_1'|s,a)p(s_2'|s,a)p(s,a) \\
&amp;= E_{\mathcal{E}\sim (s_1',s_2'),\mu,\pi}[s_1'\cdot s_2']
\end{align}$$
This is the closest I could come up with, basically instead of the same sample squared, we pick two samples independently given the pair $(s,a)$.