This paper studies the problem of Reinforcement Learning (RL) using as few real-world samples as possible. A naive application of RL algorithms can be inefficient in large and continuous state spaces. We present two versions of Multi-Fidelity Reinforcement Learning (MFRL) algorithm that leverage Gaussian Processes (GPs) to learn the optimal policy in a real-world environment. In MFRL framework, an agent uses multiple simulators of the real environment to perform actions. With increasing fidelity in a simulator chain, the number of samples used in successively higher simulators can be reduced. By incorporating GPs in MFRL framework, further reduction in the number of learning samples can be achieved as we move up the simulator chain. We examine the performance of our algorithms with the help of real-world experiments for navigation with a ground robot.
@cite_30: Target tracking is a fundamental problem in robotics research and has been the subject of detailed studies over the years. In this paper, we introduce a new formulation, based on the mathematical concept of random finite sets, that allows for tracking an unknown and dynamic number of mobile targets with a team of robots. We show how to employ the Probability Hypothesis Density filter to simultaneously estimate the number of targets and their positions. Next, we present a greedy algorithm for assigning trajectories to the robots to actively track the targets. We prove that the greedy algorithm is a 2-approximation for maximizing submodular tracking objective functions. We examine two such functions: the mutual information between the estimated target positions and future measurements from the robots, and the expected number of targets detected by the robot team. We provide extensive simulation evaluations using a real-world dataset.
@cite_28: We present a novel Bayesian approach to the problem of value function estimation in continuous state spaces. We define a probabilistic generative model for the value function by imposing a Gaussian prior over value functions and assuming a Gaussian noise model. Due to the Gaussian nature of the random processes involved, the posterior distribution of the value function is also Gaussian and is therefore described entirely by its mean and covariance. We derive exact expressions for the posterior process moments, and utilizing an efficient sequential sparsification method, we describe an on-line algorithm for learning them. We demonstrate the operation of the algorithm on a 2-dimensional continuous spatial navigation domain.
@cite_3: 
@cite_2: We exploit some useful properties of Gaussian process (GP) regression models for reinforcement learning in continuous state spaces and discrete time. We demonstrate how the GP model allows evaluation of the value function in closed form. The resulting policy iteration algorithm is demonstrated on a simple problem with a two dimensional state space. Further, we speculate that the intrinsic ability of GP models to characterise distributions of functions would allow the method to capture entire distributions over future values instead of merely their expectation, which has traditionally been the focus of much of reinforcement learning.
@cite_15: This book examines Gaussian processes in both model-based reinforcement learning (RL) and inference in nonlinear dynamic systems. First, we introduce PILCO, a fully Bayesian approach for efficient RL in continuous-valued state and action spaces when no expert knowledge is available. PILCO takes model uncertainties consistently into account during long-term planning to reduce model bias. Second, we propose principled algorithms for robust filtering and smoothing in GP dynamic systems.
In model-based approaches, GPs are commonly used to learn transition models for agents moving in the real world @cite_30 and have been used in RL to learn the transition function @cite_2 and the reward function @cite_15 . GPs have also been used in model-free approaches for approximating the @math -values in continuous state-action spaces @cite_28 . This was extended to the GP-SARSA algorithm which includes an online action selection and policy improvement steps @cite_3 . The authors used the GP variance to compute confidence intervals around the value estimate. However, they also note that this measure could be used for various exploration strategies.