We propose a reinforcement learning solution to the soccer dribbling task, a scenario in which a soccer agent has to go from the beginning to the end of a region keeping possession of the ball, as an adversary attempts to gain possession. While the adversary uses a stationary policy, the dribbler learns the best action to take at each decision point. After defining meaningful variables to represent the state space, and high-level macro-actions to incorporate domain knowledge, we describe our application of the reinforcement learning algorithm Sarsa with CMAC for function approximation. Our experiments show that, after the training period, the dribbler is able to accomplish its task against a strong adversary around 58 of the time.
@cite_22: Learning directly on real world systems such as autonomous robots is a challenging task, especially if the training signal is given only in terms of success or failure (reinforcement learning). However, if successful, the controller has the advantage of being tailored exactly to the system it eventually has to control. Here we describe, how a neural network based RL controller learns the challenging task of ball dribbling directly on our middle-size robot. The learned behaviour was actively used throughout the RoboCup world championship tournament 2007 in Atlanta, where we won the first place. This constitutes another important step within our Brainstormers project. The goal of this project is to develop an intelligent control architecture for a soccer playing robot, that is able to learn more and more complex behaviours from scratch.
More closely related to our work are reinforcement learn-ing-based solutions to the task of conducting the ball (, @cite_22 ), which can be seen as a simplification of the dribbling task since it usually does not include adversaries.