TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in
  the Full Game
  Starcraft II (SC2) is widely considered as the most challenging Real Time
Strategy (RTS) game. The underlying challenges include a large observation
space, a huge (continuous and infinite) action space, partial observations,
simultaneous move for all players, and long horizon delayed rewards for local
decisions. To push the frontier of AI research, Deepmind and Blizzard jointly
developed the StarCraft II Learning Environment (SC2LE) as a testbench of
complex decision making systems. SC2LE provides a few mini games such as
MoveToBeacon, CollectMineralShards, and DefeatRoaches, where some AI agents
have achieved the performance level of human professional players. However, for
full games, the current AI agents are still far from achieving human
professional level performance. To bridge this gap, we present two full game AI
agents in this paper - the AI agent TStarBot1 is based on deep reinforcement
learning over a flat action structure, and the AI agent TStarBot2 is based on
hard-coded rules over a hierarchical action structure. Both TStarBot1 and
TStarBot2 are able to defeat the built-in AI agents from level 1 to level 10 in
a full game (1v1 Zerg-vs-Zerg game on the AbyssalReef map), noting that level
8, level 9, and level 10 are cheating agents with unfair advantages such as
full vision on the whole map and resource harvest boosting. To the best of our
knowledge, this is the first public work to investigate AI agents that can
defeat the built-in AI in the StarCraft II full game.
