How is the "right to explanation" reasonable? There has been recent uptick in interest in eXplainable Artificial Intelligence (XAI). Here is XAI's mission as stated on its DARPA page:


  The Explainable AI (XAI) program aims to create a suite of machine learning techniques that:
  
  
  Produce more explainable models, while maintaining a high level of learning performance (prediction accuracy); and
  Enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners.
  


The New York Times article Can A.I. Be Taught to Explain Itself? does a good job at explaining the need for XAI from a human interest perspective as well as providing a glancing outlook on the techniques being developed for the same. The force behind XAI movement seems to center around the (up and coming?) concept of right to explanation, that is, the requirement that AI applications that significantly impact human lives via their decisions be able to explain to stakeholders the factors/reasons leading up to said decision. 

How is the right to explanation reasonable, given the current standards at which we hold each other accountable?
How is the right to explanation reasonable, given the current standards at which we hold each other accountable?


In short, it is quite reasonable.

More specifically, making AI accountable and responsible for explaining the decision seems reasonable because


Humans (DARPA in this case) has chosen to create, raise and evolve AI with the tax-payers money. In our society as well, whenever humans have come together for some purpose (some formal arrangement like government or otherwise), accountability and responsibility is assumed and assigned. Thus, expecting AI (who will take decisions on our behalf) to explain their decisions seems only natural extension of how humans currently operate.
Humans (generally at least) don't have super-power or resources to control and manipulate the rest of the population. But when they do (like our political leaders), we want them to be accountable. 
In rare cases, powerful humans become powerful due to things which are accessible to everyone, while AI's super-power won't be. So, our civilization would feel safer and thus less insecure with an AI which doesn't shut the door on face when questions are asked.


Other benefits of AI that offers an explanation on how a decision is reached


Far easier to debug and improve in its early stage.
Can be customized further (and along the way) to amplify certain aspects like social-welfare over profitability, etc.