Title: Multi-Advisor Reinforcement Learning,
Abstract: We consider tackling a single-agent RL problem by distributing it to $n$
learners. These learners, called advisors, endeavour to solve the problem from
a different focus. Their advice, taking the form of action values, is then
communicated to an aggregator, which is in control of the system. We show that
the local planning method for the advisors is critical and that none of the
ones found in the literature is flawless: the egocentric planning overestimates
values of states where the other advisors disagree, and the agnostic planning
is inefficient around danger zones. We introduce a novel approach called
empathic and discuss its theoretical aspects. We empirically examine and
validate our theoretical findings on a fruit collection task.