On Blame Attribution for Accountable Multi-Agent Sequential Decision
  Making
  Blame attribution is one of the key aspects of accountable decision making,
as it provides means to quantify the responsibility of an agent for a decision
making outcome. In this paper, we study blame attribution in the context of
cooperative multi-agent sequential decision making. As a particular setting of
interest, we focus on cooperative decision making formalized by Multi-Agent
Markov Decision Processes (MMDP), and we analyze different blame attribution
methods derived from or inspired by existing concepts in cooperative game
theory. We formalize desirable properties of blame attribution in the setting
of interest, and we analyze the relationship between these properties and the
studied blame attribution methods. Interestingly, we show that some of the well
known blame attribution methods, such as Shapley value, are not
performance-incentivizing, while others, such as Banzhaf index, may over-blame
agents. To mitigate these value misalignment and fairness issues, we introduce
a novel blame attribution method, unique in the set of properties it satisfies,
which trade-offs explanatory power (by under-blaming agents) for the
aforementioned properties. We further show how to account for uncertainty about
agents' decision making policies, and we experimentally: a) validate the
qualitative properties of the studied blame attribution methods, and b) analyze
their robustness to uncertainty.
