Cooperating with Machines
  Since Alan Turing envisioned Artificial Intelligence (AI) [1], a major
driving force behind technical progress has been competition with human
cognition. Historical milestones have been frequently associated with computers
matching or outperforming humans in difficult cognitive tasks (e.g. face
recognition [2], personality classification [3], driving cars [4], or playing
video games [5]), or defeating humans in strategic zero-sum encounters (e.g.
Chess [6], Checkers [7], Jeopardy! [8], Poker [9], or Go [10]). In contrast,
less attention has been given to developing autonomous machines that establish
mutually cooperative relationships with people who may not share the machine's
preferences. A main challenge has been that human cooperation does not require
sheer computational power, but rather relies on intuition [11], cultural norms
[12], emotions and signals [13, 14, 15, 16], and pre-evolved dispositions
toward cooperation [17], common-sense mechanisms that are difficult to encode
in machines for arbitrary contexts. Here, we combine a state-of-the-art
machine-learning algorithm with novel mechanisms for generating and acting on
signals to produce a new learning algorithm that cooperates with people and
other machines at levels that rival human cooperation in a variety of
two-player repeated stochastic games. This is the first general-purpose
algorithm that is capable, given a description of a previously unseen game
environment, of learning to cooperate with people within short timescales in
scenarios previously unanticipated by algorithm designers. This is achieved
without complex opponent modeling or higher-order theories of mind, thus
showing that flexible, fast, and general human-machine cooperation is
computationally achievable using a non-trivial, but ultimately simple, set of
algorithmic mechanisms.
