Individual vs. Joint Perception: a Pragmatic Model of Pointing as
  Communicative Smithian Helping
  The simple gesture of pointing can greatly augment ones ability to comprehend
states of the world based on observations. It triggers additional inferences
relevant to ones task at hand. We model an agents update to its belief of the
world based on individual observations using a partially observable Markov
decision process (POMDP), a mainstream artificial intelligence (AI) model of
how to act rationally according to beliefs formed through observation. On top
of that, we model pointing as a communicative act between agents who have a
mutual understanding that the pointed observation must be relevant and
interpretable. Our model measures relevance by defining a Smithian Value of
Information (SVI) as the utility improvement of the POMDP agent before and
after receiving the pointing. We model that agents calculate SVI by using the
cognitive theory of Smithian helping as a principle of coordinating separate
beliefs for action prediction and action evaluation. We then import SVI into
rational speech act (RSA) as the utility function of an utterance. These lead
us to a pragmatic model of pointing allowing for contextually flexible
interpretations. We demonstrate the power of our Smithian pointing model by
extending the Wumpus world, a classic AI task where a hunter hunts a monster
with only partial observability of the world. We add another agent as a guide
who can only help by marking an observation already perceived by the hunter
with a pointing or not, without providing new observations or offering any
instrumental help. Our results show that this severely limited and overloaded
communication nevertheless significantly improves the hunters performance. The
advantage of pointing is indeed due to a computation of relevance based on
Smithian helping, as it disappears completely when the task is too difficult or
too easy for the guide to help.
