Title: Stochastic Variance Reduction for Policy Gradient Estimation,
Abstract: Recent advances in policy gradient methods and deep learning have
demonstrated their applicability for complex reinforcement learning problems.
However, the variance of the performance gradient estimates obtained from the
simulation is often excessive, leading to poor sample efficiency. In this
paper, we apply the stochastic variance reduced gradient descent (SVRG) to
model-free policy gradient to significantly improve the sample-efficiency. The
SVRG estimation is incorporated into a trust-region Newton conjugate gradient
framework for the policy optimization. On several Mujoco tasks, our method
achieves significantly better performance compared to the state-of-the-art
model-free policy gradient methods in robotic continuous control such as trust
region policy optimization (TRPO)