Title: Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning,
Abstract: We present an algorithm for rapidly learning controllers for robotics
systems. The algorithm follows the model-based reinforcement learning paradigm,
and improves upon existing algorithms; namely Probabilistic learning in Control
(PILCO) and a sample-based version of PILCO with neural network dynamics
(Deep-PILCO). We propose training a neural network dynamics model using
variational dropout with truncated Log-Normal noise. This allows us to obtain a
dynamics model with calibrated uncertainty, which can be used to simulate
controller executions via rollouts. We also describe set of techniques,
inspired by viewing PILCO as a recurrent neural network model, that are crucial
to improve the convergence of the method. We test our method on a variety of
benchmark tasks, demonstrating data-efficiency that is competitive with PILCO,
while being able to optimize complex neural network controllers. Finally, we
assess the performance of the algorithm for learning motor controllers for a
six legged autonomous underwater vehicle. This demonstrates the potential of
the algorithm for scaling up the dimensionality and dataset sizes, in more
complex control tasks.