Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to
  Stop Worrying and Love the Social and Behavioural Sciences
  In his seminal book `The Inmates are Running the Asylum: Why High-Tech
Products Drive Us Crazy And How To Restore The Sanity' [2004, Sams
Indianapolis, IN, USA], Alan Cooper argues that a major reason why software is
often poorly designed (from a user perspective) is that programmers are in
charge of design decisions, rather than interaction designers. As a result,
programmers design software for themselves, rather than for their target
audience, a phenomenon he refers to as the `inmates running the asylum'. This
paper argues that explainable AI risks a similar fate. While the re-emergence
of explainable AI is positive, this paper argues most of us as AI researchers
are building explanatory agents for ourselves, rather than for the intended
users. But explainable AI is more likely to succeed if researchers and
practitioners understand, adopt, implement, and improve models from the vast
and valuable bodies of research in philosophy, psychology, and cognitive
science, and if evaluation of these models is focused more on people than on
technology. From a light scan of literature, we demonstrate that there is
considerable scope to infuse more results from the social and behavioural
sciences into explainable AI, and present some key results from these fields
that are relevant to explainable AI.
