Solving infinite-horizon Dec-POMDPs using Finite State Controllers
  within JESP
  This paper looks at solving collaborative planning problems formalized as
Decentralized POMDPs (Dec-POMDPs) by searching for Nash equilibria, i.e.,
situations where each agent's policy is a best response to the other agents'
(fixed) policies. While the Joint Equilibrium-based Search for Policies (JESP)
algorithm does this in the finite-horizon setting relying on policy trees, we
propose here to adapt it to infinite-horizon Dec-POMDPs by using finite state
controller (FSC) policy representations. In this article, we (1) explain how to
turn a Dec-POMDP with $N-1$ fixed FSCs into an infinite-horizon POMDP whose
solution is an $N^\text{th}$ agent best response; (2) propose a JESP variant,
called \infJESP, using this to solve infinite-horizon Dec-POMDPs; (3) introduce
heuristic initializations for JESP aiming at leading to good solutions; and (4)
conduct experiments on state-of-the-art benchmark problems to evaluate our
approach.
