An Application of Reinforcement Learning to Dialogue Strategy Selection
  in a Spoken Dialogue System for Email
  This paper describes a novel method by which a spoken dialogue system can
learn to choose an optimal dialogue strategy from its experience interacting
with human users. The method is based on a combination of reinforcement
learning and performance modeling of spoken dialogue systems. The reinforcement
learning component applies Q-learning (Watkins, 1989), while the performance
modeling component applies the PARADISE evaluation framework (Walker et al.,
1997) to learn the performance function (reward) used in reinforcement
learning. We illustrate the method with a spoken dialogue system named ELVIS
(EmaiL Voice Interactive System), that supports access to email over the phone.
We conduct a set of experiments for training an optimal dialogue strategy on a
corpus of 219 dialogues in which human users interact with ELVIS over the
phone. We then test that strategy on a corpus of 18 dialogues. We show that
ELVIS can learn to optimize its strategy selection for agent initiative, for
reading messages, and for summarizing email folders.
