Artificially intelligent agents equipped with strategic skills that can negotiate during their interactions with other natural or artificial agents are still underdeveloped. This paper describes a successful application of Deep Reinforcement Learning (DRL) for training intelligent agents with strategic conversational skills, in a situated dialogue setting. Previous studies have modelled the behaviour of strategic agents using supervised learning and traditional reinforcement learning techniques, the latter using tabular representations or learning with linear function approximation. In this study, we apply DRL with a high-dimensional state space to the strategic board game of Settlers of Catan---where players can offer resources in exchange for others and they can also reply to offers made by other players. Our experimental results report that the DRL-based learnt policies significantly outperformed several baselines including random, rule-based, and supervised-based behaviours. The DRL-based policy has a 53 win rate versus 3 automated players ( bots'), whereas a supervised player trained on a dialogue corpus in this setting achieved only 27 , versus the same 3 bots. This result supports the claim that DRL is a promising framework for training dialogue systems, and strategic agents with negotiation abilities.
@cite_3: 
@cite_25: We build dialogue system policies for negotiation, and in particular for argumentation. These dialogue policies are designed for negotiation against users of different cultural norms (individualists, collectivists, and altruists). In order to learn these policies we build simulated users (SUs), i.e. models that simulate the behavior of real users, and use Reinforcement Learning (RL). The SUs are trained on a spoken dialogue corpus in a negotiation domain, and then tweaked towards a particular cultural norm using hand-crafted rules. We evaluate the learned policies in a simulation setting. Our results are consistent with our SUs, in other words, the policies learn what they are designed to learn, which shows that RL is a promising technique for learning policies in domains, such as argumentation, that are more complex than standard slot-filling applications. Index Terms: spoken dialogue systems, reinforcement learning, simulated users, argumentation, negotiation, culture.
@cite_12: Deception is a common and essential behavior in humans. Since human beings gain many advantages from deceptive capabilities, we can also assume that robotic deception can provide benefits in several ways. Particularly, the use of robotic deception in human-robot interaction contexts is becoming an important and interesting research question. Despite its importance, very little research on robot deception has been conducted. Furthermore, no basic metrics or definitions of robot deception have been proposed yet. In this paper, we review the previous work on deception in various fields including psychology, biology, and robotics and will propose a novel way to define a taxonomy of robot deception. In addition, we will introduce an interesting research question of robot deception in HRI contexts and discuss potential approaches.
@cite_11: We define grounding in terms of shared public commitments, and link public commitments to other, private, attitudes within a decidable dynamic logic for computing implicatures and predicting an agent’s next dialogue move.
Other related work has been carried out in the context of automated non-cooperative dialogue systems, where an agent may act to satisfy its own goals rather than those of other participants @cite_25 . The game-theoretic underpinnings of non-cooperative behaviour have also been investigated @cite_11 . Such automated agents are of interest when trying to persuade, argue, or debate, or in the area of believable characters in video games and educational simulations @cite_25 @cite_12 . Another arena in which strategic conversational behaviour has been investigated is negotiation @cite_3 , where hiding information (and even outright lying) can be advantageous.