Automatic optimization of spoken dialog management policies that are robust to environmental noise has long been the goal for both academia and industry. Approaches based on reinforcement learning have been proved to be effective. However, the numerical representation of dialog policy is human-incomprehensible and difficult for dialog system designers to verify or modify, which limits its practical application. In this paper we propose a novel framework for optimizing dialog policies specified in domain language using genetic algorithm. The human-interpretable representation of policy makes the method suitable for practical employment. We present learning algorithms using user simulation and real human-machine dialogs respectively. Empirical experimental results are given to show the effectiveness of the proposed approach.
@cite_4: A system for producing color water and colored ice cubes in various shapes, having a control unit with front panel buttons which allow a user to select a color. Dye reservoirs are provided which are selectively dispensed into a mixing chamber to provide the selected color, where they are combined with water and chilled. The colored water is provided to a dispensing unit which selectively dispenses the colored water, or creates colored ice cubes for subsequent dispensing. Ice trays are removably secured within the dispensing unit to allow various shaped ice cubes to be produced.
@cite_2: HighlightsWe present a new, hybrid modelling framework for dialogue management based on probabilistic rules.The probabilistic rules function as high-level templates for the generation of a directed graphical model.The rule parameters may be estimated from dialogue data via Bayesian inference.The OpenDial toolkit allows system designers to develop dialogue systems using probabilistic rules.User evaluation in a HRI domain shows that the approach outperforms traditional hand-crafted and statistical models. We present a new modelling framework for dialogue management based on the concept of probabilistic rules. Probabilistic rules are defined as structured mappings between logical conditions and probabilistic effects. They function as high-level templates for probabilistic graphical models and may include unknown parameters whose values are estimated from data using Bayesian inference. Thanks to their use of logical abstractions, probabilistic rules are able to encode the probability and utility models employed in dialogue management in a compact and human-readable form. As a consequence, they can reduce the amount of dialogue data required for parameter estimation and allow system designers to directly incorporate their expert domain knowledge into the dialogue models.Empirical results of a user evaluation in a human-robot interaction task with 37 participants show that a dialogue manager structured with probabilistic rules outperforms both purely hand-crafted and purely statistical methods on a range of subjective and objective quality metrics. The framework is implemented in a software toolkit called OpenDial, which can be used to develop various types of dialogue systems based on probabilistic rules.
The subject of automatically optimizing dialog policies is a hot topic, and many data-driven methods have been proposed among which RL-based ones are the most popular. There is some previous work on constraining the behavior of RL-based DM. In @cite_4 Williams proposed to construct a hand-crafted DM and it produces a set of candidate actions for given dialog state, from which the best one is chosen by a POMDP-DM. Lison @cite_2 proposed to use probabilistic rule' in specifying the transition and reward sub-models of the POMDP model. The probabilistic rules are human-readable and less parameterized than conventional probability distribution, thus reducing the free parameters of the POMDP model and allowing the system designers to make use of domain knowledge in designing DM. Our work bears some resemblance to @cite_2 . But we used the dialog policy template to specify a policy directly and utilized GA to train the free parameters.