We present the first complete attempt at concurrently training conversational agents that communicate only via self-generated language. Using DSTC2 as seed data, we trained natural language understanding (NLU) and generation (NLG) networks for each agent and let the agents interact online. We model the interaction as a stochastic collaborative game where each agent (player) has a role ("assistant", "tourist", "eater", etc.) and their own objectives, and can only interact via natural language they generate. Each agent, therefore, needs to learn to operate optimally in an environment with multiple sources of uncertainty (its own NLU and NLG, the other agent's NLU, Policy, and NLG). In our evaluation, we show that the stochastic-game agents outperform deep learning based supervised baselines.
@cite_10: This paper investigates the problem of policy learning in multiagent environments using the stochastic game framework, which we briefly overview. We introduce two properties as desirable for a learning agent when in the presence of other learning agents, namely rationality and convergence. We examine existing reinforcement learning algorithms according to these two properties and notice that they fail to simultaneously meet both criteria. We then contribute a new learning algorithm, WoLF policy hillclimbing, that is based on a simple principle: “learn quickly while losing, slowly while winning.” The algorithm is proven to be rational and we present empirical results for a number of stochastic games showing the algorithm converges.
concurrently learn two negotiator agents' dialogue policies in a setting where they negotiate allocation of resources. However, their agents do not interact via language, but rather via dialogue acts. They use PHC and WoLF-PHC @cite_10 to train their agents, who use two types of dialogue acts: and , each of which takes two numerical arguments. train agents on a similar task, but their agents are modelled as end-to-end networks that learn directly from text. However, the authors train their negotiator agent on supervised data and against a fixed supervised agent. Earlier works include , the first to train policies for two conversational agents, but with single-agent RL, and who applied co-adaptation on single-agent RL, using Inverse RL to infer reward functions from data.