Title: Quantifying Interpretability and Trust in Machine Learning Systems,
Abstract: Decisions by Machine Learning (ML) models have become ubiquitous. Trusting
these decisions requires understanding how algorithms take them. Hence
interpretability methods for ML are an active focus of research. A central
problem in this context is that both the quality of interpretability methods as
well as trust in ML predictions are difficult to measure. Yet evaluations,
comparisons and improvements of trust and interpretability require quantifiable
measures. Here we propose a quantitative measure for the quality of
interpretability methods. Based on that we derive a quantitative measure of
trust in ML decisions. Building on previous work we propose to measure
intuitive understanding of algorithmic decisions using the information transfer
rate at which humans replicate ML model predictions. We provide empirical
evidence from crowdsourcing experiments that the proposed metric robustly
differentiates interpretability methods. The proposed metric also demonstrates
the value of interpretability for ML assisted human decision making: in our
experiments providing explanations more than doubled productivity in annotation
tasks. However unbiased human judgement is critical for doctors, judges, policy
makers and others. Here we derive a trust metric that identifies when human
decisions are overly biased towards ML predictions. Our results complement
existing qualitative work on trust and interpretability by quantifiable
measures that can serve as objectives for further improving methods in this
field of research.