Why a single trajectory can be used to update the policy network $\theta$ in A3C?
I guess the gradient of the expectation of the Utility function, $\nabla_{\theta}J(\theta)$ in policy gradient methods where $\nabla_{\theta}J(\theta) = E_{\tau \sim p(\tau ; \theta)}[r(\tau)\nabla_{\theta}log p(\tau;\theta)]$ can be approximated using a single sample trajectory as shown in a deep reinforcement learning lecture by stanford where $J(\theta) \approx \sum_{t &gt; 0}r(\tau)\nabla_{\theta}log\pi_{\theta}(a_t |s_t)$ and an average over sampled trajectories is not needed to compute the gradient for $\theta$ update in the direction of gradient.