Hierarchical Potential-based Reward Shaping from Task Specifications
The automatic synthesis of autonomous agents' policies through reinforcement learning relies on the definition of a reward signal that simultaneously captures many, possibly conflicting, requirements of a task specification. In this work, we introduce a novel, hierarchical, potential-based reward-shaping approach (HPRS) for defining effective rewards for a large family of problems. We formalize a task as a set of target, safety, and comfort requirements, and define an automated methodology to shape the reward signal by considering a natural priority among them. Building on top of potential-based shaping, we show that HPRS does not alter the optimal policy. Our experimental evaluation demonstrates the superiority of HPRS in capturing the intended behavior, resulting in task-satisfying policies with improved comfort, and converging to optimal policies faster than other state-of-the-art approaches.