Skilled Experience Catalogue: A Skill-Balancing Mechanism for Non-Player
  Characters using Reinforcement Learning
  In this paper, we introduce a skill-balancing mechanism for adversarial
non-player characters (NPCs), called Skilled Experience Catalogue (SEC). The
objective of this mechanism is to approximately match the skill level of an NPC
to an opponent in real-time. We test the technique in the context of a
First-Person Shooter (FPS) game. Specifically, the technique adjusts a
reinforcement learning NPC's proficiency with a weapon based on its current
performance against an opponent. Firstly, a catalogue of experience, in the
form of stored learning policies, is built up by playing a series of training
games. Once the NPC has been sufficiently trained, the catalogue acts as a
timeline of experience with incremental knowledge milestones in the form of
stored learning policies. If the NPC is performing poorly, it can jump to a
later stage in the learning timeline to be equipped with more informed
decision-making. Likewise, if it is performing significantly better than the
opponent, it will jump to an earlier stage. The NPC continues to learn in
real-time using reinforcement learning but its policy is adjusted, as required,
by loading the most suitable milestones for the current circumstances.
