The Research on deep reinforcement learning to estimate Q-value by deep learning has been active in recent years. In deep reinforcement learning, it is important to efficiently learn the experiences that a agent has collected by exploring the environment. In this research, we propose NEC2DQN that improves learning speed of a algorithm with poor sample efficiency by using a algorithm with good one at the beginning of learning, and we demonstrate it in experiments.
@cite_8: State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and its capacity for episodic memory. Here we investigate whether a simple model of hippocampal episodic control can learn to solve difficult sequential decision-making tasks. We demonstrate that it not only attains a highly rewarding strategy significantly faster than state-of-the-art deep reinforcement learning algorithms, but also achieves a higher overall reward on some of the more challenging domains.
Neural Episodic Control(NEC) is one of the method of efficient sampling from Replay Buffer. It is the algorithm based on episodic memory and improved Model-Free Episodic Control (MFEC) @cite_8 to learn end-to-end from state mappings to estimations of Q-values.