Learning to Shoot in First Person Shooter Games by Stabilizing Actions
  and Clustering Rewards for Reinforcement Learning
  While reinforcement learning (RL) has been applied to turn-based board games
for many years, more complex games involving decision-making in real-time are
beginning to receive more attention. A challenge in such environments is that
the time that elapses between deciding to take an action and receiving a reward
based on its outcome can be longer than the interval between successive
decisions. We explore this in the context of a non-player character (NPC) in a
modern first-person shooter game. Such games take place in 3D environments
where players, both human and computer-controlled, compete by engaging in
combat and completing task objectives. We investigate the use of RL to enable
NPCs to gather experience from game-play and improve their shooting skill over
time from a reward signal based on the damage caused to opponents. We propose a
new method for RL updates and reward calculations, in which the updates are
carried out periodically, after each shooting encounter has ended, and a new
weighted-reward mechanism is used which increases the reward applied to actions
that lead to damaging the opponent in successive hits in what we term "hit
clusters".
