ELF: An Extensive, Lightweight and Flexible Research Platform for
  Real-time Strategy Games
  In this paper, we propose ELF, an Extensive, Lightweight and Flexible
platform for fundamental reinforcement learning research. Using ELF, we
implement a highly customizable real-time strategy (RTS) engine with three game
environments (Mini-RTS, Capture the Flag and Tower Defense). Mini-RTS, as a
miniature version of StarCraft, captures key game dynamics and runs at 40K
frame-per-second (FPS) per core on a Macbook Pro notebook. When coupled with
modern reinforcement learning methods, the system can train a full-game bot
against built-in AIs end-to-end in one day with 6 CPUs and 1 GPU. In addition,
our platform is flexible in terms of environment-agent communication
topologies, choices of RL methods, changes in game parameters, and can host
existing C/C++-based game environments like Arcade Learning Environment. Using
ELF, we thoroughly explore training parameters and show that a network with
Leaky ReLU and Batch Normalization coupled with long-horizon training and
progressive curriculum beats the rule-based built-in AI more than $70\%$ of the
time in the full game of Mini-RTS. Strong performance is also achieved on the
other two games. In game replays, we show our agents learn interesting
strategies. ELF, along with its RL platform, is open-sourced at
https://github.com/facebookresearch/ELF.
