KoGuN: Accelerating Deep Reinforcement Learning via Integrating Human
  Suboptimal Knowledge
  Reinforcement learning agents usually learn from scratch, which requires a
large number of interactions with the environment. This is quite different from
the learning process of human. When faced with a new task, human naturally have
the common sense and use the prior knowledge to derive an initial policy and
guide the learning process afterwards. Although the prior knowledge may be not
fully applicable to the new task, the learning process is significantly sped up
since the initial policy ensures a quick-start of learning and intermediate
guidance allows to avoid unnecessary exploration. Taking this inspiration, we
propose knowledge guided policy network (KoGuN), a novel framework that
combines human prior suboptimal knowledge with reinforcement learning. Our
framework consists of a fuzzy rule controller to represent human knowledge and
a refine module to fine-tune suboptimal prior knowledge. The proposed framework
is end-to-end and can be combined with existing policy-based reinforcement
learning algorithm. We conduct experiments on both discrete and continuous
control tasks. The empirical results show that our approach, which combines
human suboptimal knowledge and RL, achieves significant improvement on learning
efficiency of flat RL algorithms, even with very low-performance human prior
knowledge.
