Combining Strategic Learning and Tactical Search in Real-Time Strategy
  Games
  A commonly used technique for managing AI complexity in real-time strategy
(RTS) games is to use action and/or state abstractions. High-level abstractions
can often lead to good strategic decision making, but tactical decision quality
may suffer due to lost details. A competing method is to sample the search
space which often leads to good tactical performance in simple scenarios, but
poor high-level planning.
  We propose to use a deep convolutional neural network (CNN) to select among a
limited set of abstract action choices, and to utilize the remaining
computation time for game tree search to improve low level tactics. The CNN is
trained by supervised learning on game states labelled by Puppet Search, a
strategic search algorithm that uses action abstractions. The network is then
used to select a script --- an abstract action --- to produce low level actions
for all units. Subsequently, the game tree search algorithm improves the
tactical actions of a subset of units using a limited view of the game state
only considering units close to opponent units.
  Experiments in the microRTS game show that the combined algorithm results in
higher win-rates than either of its two independent components and other
state-of-the-art microRTS agents.
  To the best of our knowledge, this is the first successful application of a
convolutional network to play a full RTS game on standard game maps, as
previous work has focused on sub-problems, such as combat, or on very small
maps.
