CFR-MIX: Solving Imperfect Information Extensive-Form Games with
  Combinatorial Action Space
  In many real-world scenarios, a team of agents coordinate with each other to
compete against an opponent. The challenge of solving this type of game is that
the team's joint action space grows exponentially with the number of agents,
which results in the inefficiency of the existing algorithms, e.g.,
Counterfactual Regret Minimization (CFR). To address this problem, we propose a
new framework of CFR: CFR-MIX. Firstly, we propose a new strategy
representation that represents a joint action strategy using individual
strategies of all agents and a consistency relationship to maintain the
cooperation between agents. To compute the equilibrium with individual
strategies under the CFR framework, we transform the consistency relationship
between strategies to the consistency relationship between the cumulative
regret values. Furthermore, we propose a novel decomposition method over
cumulative regret values to guarantee the consistency relationship between the
cumulative regret values. Finally, we introduce our new algorithm CFR-MIX which
employs a mixing layer to estimate cumulative regret values of joint actions as
a non-linear combination of cumulative regret values of individual actions.
Experimental results show that CFR-MIX outperforms existing algorithms on
various games significantly.
