Title: It Takes Two to Tango: Towards Theory of AI's Mind,
Abstract: Theory of Mind is the ability to attribute mental states (beliefs, intents,
knowledge, perspectives, etc.) to others and recognize that these mental states
may differ from one's own. Theory of Mind is critical to effective
communication and to teams demonstrating higher collective performance. To
effectively leverage the progress in Artificial Intelligence (AI) to make our
lives more productive, it is important for humans and AI to work well together
in a team. Traditionally, there has been much emphasis on research to make AI
more accurate, and (to a lesser extent) on having it better understand human
intentions, tendencies, beliefs, and contexts. The latter involves making AI
more human-like and having it develop a theory of our minds. In this work, we
argue that for human-AI teams to be effective, humans must also develop a
theory of AI's mind (ToAIM) - get to know its strengths, weaknesses, beliefs,
and quirks. We instantiate these ideas within the domain of Visual Question
Answering (VQA). We find that using just a few examples (50), lay people can be
trained to better predict responses and oncoming failures of a complex VQA
model. We further evaluate the role existing explanation (or interpretability)
modalities play in helping humans build ToAIM. Explainable AI has received
considerable scientific and popular attention in recent times. Surprisingly, we
find that having access to the model's internal states - its confidence in its
top-k predictions, explicit or implicit attention maps which highlight regions
in the image (and words in the question) the model is looking at (and listening
to) while answering a question about an image - do not help people better
predict its behavior.