reliability of human-level evaluation of the interpretability quality of a model
This is specifically for interpretability of outcomes, i.e. a task where non-expert humans outperform machines.
There is a problem in collecting labels in machine learning, whereby labelling datasets is very expensive and time consuming (due to size of datasets &amp; cost of experts' time).
So it's less about trust, its more about practicality. Consider hiring a data scientist to develop an algorithm to automatically label a dataset based of expert heuristics (e.g. &quot;label the data as cancerous if it looks red&quot;), it might take 6 months to collect data, plan, develop &amp; test - therefore for certain use-cases hiring 10 non-experts and telling them the heuristic might be cheaper and faster.
The book uses an example &quot;show a user different explanations and the human would choose the best.&quot; in the context of radiology, it could be something like:
&quot;Look at the images of the patient, and compare it to this dictionary of images and diagnoses, combine multiple sources and then report what the diagnosis is&quot;
Of course if you have an algorithm which outperforms non-experts, you might just want some expert labels to validate your algorithm, and forget the non-experts.