Interactive Policy Learning through Confidence-Based Autonomy
  We present Confidence-Based Autonomy (CBA), an interactive algorithm for
policy learning from demonstration. The CBA algorithm consists of two
components which take advantage of the complimentary abilities of humans and
computer agents. The first component, Confident Execution, enables the agent to
identify states in which demonstration is required, to request a demonstration
from the human teacher and to learn a policy based on the acquired data. The
algorithm selects demonstrations based on a measure of action selection
confidence, and our results show that using Confident Execution the agent
requires fewer demonstrations to learn the policy than when demonstrations are
selected by a human teacher. The second algorithmic component, Corrective
Demonstration, enables the teacher to correct any mistakes made by the agent
through additional demonstrations in order to improve the policy and future
task performance. CBA and its individual components are compared and evaluated
in a complex simulated driving domain. The complete CBA algorithm results in
the best overall learning performance, successfully reproducing the behavior of
the teacher while balancing the tradeoff between number of demonstrations and
number of incorrect actions during learning.
