abstract: Integration over non-negative integrands is a central problem in machine learning (e.g. for model averaging, (hyper-)parameter marginalisation, and computing posterior predictive distributions). Bayesian Quadrature is a probabilistic numerical integration technique that performs promisingly when compared to traditional Markov Chain Monte Carlo methods. However, in contrast to easily-parallelised MCMC methods, Bayesian Quadrature methods have, thus far, been essentially serial in nature, selecting a single point to sample at each step of the algorithm. We deliver methods to select batches of points at each step, based upon those recently presented in the Batch Bayesian Optimisation literature. Such parallelisation significantly reduces computation time, especially when the integrand is expensive to sample.
author: Ed Wagstaff
University of Oxford; Saad Hamid
University of Oxford; Michael Osborne
University of Oxford
bibliography: bibliography.bib
title: Batch Selection for Parallelisation of Bayesian Quadrature

# Introduction

Both within and without machine learning, we encounter integrals of the form $$\label{ZDef}
    Z = \int \ell(\boldsymbol{\theta})\pi(\boldsymbol{\theta})\text{d}\boldsymbol{\theta},$$ where both $\ell(\boldsymbol{\theta})$ (e.g. a likelihood), and $\pi(\boldsymbol{\theta})$ (e.g a prior) are non-negative. These integrals are usually intractable, and we must therefore turn to approximation to solve them.

One possibility is the Laplace approximation. For multimodal distributions it can be repeated on the residuals – the difference between the approximation and the true distribution – to obtain a better approximation . However, this requires optimisation and is not invariant to changes of representation. Another possibility is variational inference which provides a lower bound for the value of the integral, but there is limited underpinning theory and optimisation of the lower bound can be difficult . Markov Chain Monte Carlo (MCMC) methods have also seen broad use and success, but they are subject to a number of objections , the most pertinent of which is that they ignore the locations of samples from the likelihood when computing the integral. They therefore suffer from low sample efficiency and a poor $\frac{1}{\sqrt{N}}$ convergence rate. All of these approximation methods are unsuitable for applications where evaluating the desired likelihood is expensive (e.g. if we must make an estimate based on only a few dozen evaluations).

Bayesian Quadrature (BQ) offers an alternative framework for approaching the same integral problem. It belongs to a family of methods termed *probabilistic numerics* , which view the approximation of intractable quantities as a problem of inference from limited data, to which the tools of probability theory can be applied. In BQ, a probabilistic model of the integrand (typically a Gaussian Process) is maintained, from which the integral is inferred. Gunter et al. showed that BQ can achieve faster convergence – both in wall-clock time and sample complexity – than state of the art MCMC methods. Some deterministic quadrature techniques can be reinterpreted in terms of Bayesian Quadrature – for instance, the trapezoid rule (with $\frac{1}{N}$ convergence) arises from BQ with a linear spline kernel .

One drawback of BQ as developed thus far is that is not easily parallelised, in contrast to the comparatively straightforward situation with MCMC methods where, for instance, several independent Markov chains may be run in parallel. While the probabilistic model allows us to explore the space more efficiently, determining where next to evaluate the integrand depends on the current state of the model, i.e. on all evaluations made so far. Nonetheless, parallelisation is essential to realising competitive performance using modern computational hardware.

Using ideas applied in Batch Bayesian Optimisation, particularly those by Gonzalez et al. , we propose a novel technique for parallelising BQ by selecting new points in batches rather than one-by-one.

# Bayesian Quadrature

In Bayesian Quadrature, we work with a probabilistic model of an integrand which induces a probability distribution over the value of the integral. The early work on BQ identified that Gaussian Processes offer a well-studied method for placing a probability distribution over functions, so we work with these to provide our probabilistic model.

Recall from Equation that we are interested in the quantity $\int \ell(\boldsymbol{\theta})\pi(\boldsymbol{\theta})\text{d}\boldsymbol{\theta}$. We typically choose to place our Gaussian process prior on the likelihood $\ell$, rather than on the product $\ell\cdot\pi$ (though we could choose to do either).

Given a Gaussian prior $\pi$ and a GP model for our likelihood,

$$\begin{align}
\ell \mid \mathbf{y} &\sim \mathcal{GP}(m_\mathbf{y}, C_\mathbf{y})\\
m_\mathbf{y}(x) &= k(x, X)k(X, X)^{-1}\mathbf{y}\\
C_\mathbf{y}(x, x^\prime) &= k(x, x^\prime) - k(x, X)k(X, X)^{-1}k(X, x^\prime)
\end{align}$$

the value of the integral $Z = \int \ell(x)\pi(x)\text{d}x$ is Gaussian distributed – since Gaussian Processes are closed under linear operators (see e.g. ), the GP on the function induces a normal distribution on the value of the integral $Z$. By linearity of expectation and bilinearity of covariance, the mean and variance of $Z$ are simply :

$$\begin{align} \label{eq:11}
\mathbb{E}\left( Z \mid \mathbf{y} \right) &= \int m_\mathbf{y}\left(x\right)\pi(x)\text{d}x\\
 \label{eq:12}
\text{var}\left(Z \mid \mathbf{y} \right) &= \iint C_\mathbf{y}\left(x, x^\prime\right)\pi(x)\pi(x^\prime) \text{d}x\text{d}x^\prime
\end{align}$$

These integrals are, for certain choices of prior, $\pi$, and kernel, $K$, analytic. In particular, this is true when $\pi$ is a Gaussian and $K$ is the squared exponential covariance, and we use this combination throughout this paper.

At a high level, the advantages of Bayesian Quadrature are two-fold. First, with the stronger prior of BQ, evaluations provide stronger constraints on the integrand, and therefore more information about the integral. Secondly, we can use the uncertainty of the GP model to inform our choice of where next to evaluate the integrand, thus exploring the space more efficiently. We do this by defining and maximising an *acquisition function* which is high-valued at points which, according to our model, will reduce the uncertainty in the integral. In the standard BQ model, this second advantage does not apply – note that the posterior covariance only depends on the *locations* sampled, not on the function value at those points . However, as we shall see below, it is possible to develop more complex models where, among other advantages, the covariance of better-informed posteriors does depend on the sampled function values.

## Warped Bayesian Quadrature

The Gaussian process framework has some restrictions which interfere with our ability to model our true beliefs about the functions we wish to work with. Standard Bayesian Quadrature does not allow us to express our belief that the function to be modelled (in our case a likelihood) is non-negative, since the distribution at any point is a Gaussian. Recent work has investigated applying a warping to the output space of the GP to address this . Rather than modelling the likelihood $\ell$ itself as a GP, we model a transformed likelihood $g(\ell)$ such that the true likelihood $g^{-1}(g(\ell))$ is non-negative (i.e. such that the range of $g^{-1}$ is the positive reals). For example, $g=\text{log}(x)$ and $g=\sqrt{x}$ have been investigated. Though the final posterior on $\ell$ is not a GP, it is possible to derive a GP which closely approximates it.

For the purposes of this investigation we follow the method of Gunter et al. , dubbed WSABI, which makes use of the transformation $g=\sqrt{x}$. In particular we use the approximation dubbed WSABI-L, which has the virtue of providing a closed-form estimate for the final integral rather than requiring Monte Carlo estimation. In WSABI, the acquisition function is simply the variance of the posterior model – due to the form of the model, this automatically balances exploring areas of high uncertainty and exploiting in areas where the objective function is believed to be high-valued. WSABI has so far been considered only as a sequential process, selecting and evaluating one point at a time. We propose its parallelisation by turning to batch methods.

# Batch methods

Bayesian Quadrature as described above is an essentially serial process. That is, we must sample a single point at each step, before updating our model (and therefore our acquisition function) accordingly in order to recommend the next point. Since we expect each sample to be expensive, this is a serious bottleneck – we would rather be able to work in parallel, taking several samples at once in order to gain the maximum amount of information about our integrand. In order to enable this, we turn to ideas which have been successfully used in Bayesian Optimisation.

The field of Bayesian Optimisation has a clear relationship to Bayesian Quadrature. In particular, both fields use a probabilistic model to guide exploration of a search space by defining an acquisition function to be maximised. In Bayesian Optimisation, so-called batch methods have been proposed as an effective way of allowing parallelisation of this exploration (see e.g. Ginsbourger et al. , Gonzalez et al. , Nguyen et al. ).

In batch methods, rather than selecting one point to sample at each step, we select a batch of points, usually of a fixed size $n$. Naively one could try to select the $n$ highest points of the acquisition function, but this has the obvious issue that these will all be essentially equal to the maximum of the acquisition function, leading to an uninformative batch – the quality of the batch as a whole is more complex than simply the sum of the quality of each point individually. One way of viewing this issue is that we require an informative batch, and therefore we require a diverse batch.

To find the optimal batch we could, in principle, define an acquisition function over all sets of $n$ points (e.g. by considering the expected reduction in variance of our integral estimate) and maximise it, but in practice this is too expensive – it increases the dimensionality of the problem by a factor of $n$ and the acquisition function is also likely to be more costly to compute. It is therefore common to turn to heuristics where each point is selected sequentially, with the acquisition function modified after each point is selected.

A simple method for performing this sequential selection is the so-called Kriging Believer strategy . We "sample" our chosen point by setting its value to the posterior mean, update our Gaussian process model to take the new evidence into account, and select the most informative point from the posterior. We repeat this process until we have a sufficiently large batch, and then make the (real) evaluations at selected points and input them as observations into our model. This naturally forces further exploration of the space, as the region immediately around each "sampled" point will no longer be considered informative.

We may also consider a *local penalisation* strategy which directly penalises the acquisition function around the chosen point (in Bayesian Optimisation, this strategy was introduced by Gonzalez et al. ). With this strategy, we don't use our full model, as in Kriging Believer, but instead directly modify the acquisition function to lower its value around each selected point, encouraging diversity in the final batch.

In the next section, we will consider the application of batch methods to Bayesian Quadrature.

# Method

In this section we introduce, to our knowledge, the first Batch Bayesian Quadrature routine. We also introduce a novel version of local penalisation.

The core algorithm is simple – we repeatedly select a batch of points, evaluate the integrand at these points, and input the returned data into our model. At each iteration, we can calculate an estimate of the integral using BQ. There is a question here of the stopping criterion. Theoretically, Bayesian Quadrature offers a principled stopping criterion since it returns a variance for its estimate of the integral, and we can simply evaluate the variance at each iteration and stop once it has reached a certain threshold. For the specific case of WSABI-L, the model is overconfident away from the data, and we expect that the BQ variance will therefore be erroneously low and unsuitable as a stopping criterion. This is a question that clearly merits further investigation, but for the purposes of this paper we simply set a fixed computational budget.

The most important step of the algorithm is the batch selection process. Above we described the Kriging Believer approach, and in general terms described local penalisation. We now outline our approach to local penalisation.

## Local Penalisation

With local penalisation methods, there is a balance to be struck in how wide the penalisers are – if our penalisation affects a broad region, we may erroneously exclude genuinely informative points. If, however, our penalisation affects a narrow region, we may not sufficiently encourage diversity, and produce a batch of points which are clustered too closely together. Gonzalez et al. propose a multiplicative penaliser with a minimum at the sampled point. To determine how wide the penaliser should be, they make use of a Lipschitz constant, a measure of the maximum absolute slope of the acquisition function. In this method the Lipschitz constant is incorporated into a probablility density function which is used as part of the penaliser. The authors' proposition is less suited to quadrature than to optimisation, as it is not sufficiently exploratory – it heavily penalises regions surrounding a selection which has a high variance. Thorough exploration is more important in quadrature, since the result depends on the global behaviour of the function rather than a single optimum. We therefore consider an alternative approach, attempting to place an upper bound on the true acquisition function (i.e. the acquisition function had the integrand been evaluated at the previous selections).

Recall that a function $f$ is said to be Lipschitz continuous if there is a constant $L$ such that, for all $x$, $y$,

$$||f(x) - f(y)|| \leq L ||x - y||.$$

In our method, we apply the Lipschitz constant more directly. We expect our true acquisition function to be zero at the sampled point and increase away from it, and wish for our penalised function to reflect this. We find a local Lipschitz constant for a given point by locally optimising the norm of the gradient of the acquisition function. Viewing this as a bound on the slope of the acquisition function around that point, we then define a *Lipschitz Cone* which is zero at the point and increases linearly away from the point. i.e. for a Lipschitz constant $L$ and point $x_0$, we define the Lipschitz Cone,

$$C(x) = L||x-x_0||.$$

Our acquisition function is then taken to be the minimum of the original acquisition function and the Lipschitz Cones of all points so far added to the batch.

This minimum, however, is not differentiable everywhere. In particular, it is not differentiable where the cone intersects the original acquisition function (or where two cones intersect). These intersections may in fact be maxima of the penalised acquisition function, so ideally the penalised function would have stationary points on these intersection surfaces. We address this by applying a soft minimisation, rather than the true minimum, which leaves us with an acquisition function which is differentiable everywhere except at the centres of the Lipschitz cones (i.e. the already selected batch points).

Recall that the $\ell_p$ norm $||\cdot||_p$ is defined by

$$||x||_p = \left(\sum_i x_i^p\right)^\frac{1}{p}.$$

It is well known that the $\ell_p$-norm for large $p$ approaches the $\ell_\infty$ norm, $||x||_\infty = \text{max}_i(x_i)$. Large values of $p$ can therefore be viewed as applying a kind of soft maximisation. Large negative values of $p$, on the other hand, apply a soft minimisation to the input vector.[^1] We therefore take our final acquisition function to be an $\ell_p$-norm of the vector whose components are the function evaluation for each of the penalisers and the original acquisition function. There is a balance to be struck in the choice of $p$ here – large $p$ will more closely resemble the true minimum of all the functions, but will also be more prone to numerical errors (these errors will appear particularly around the important regions where the true minimum is not differentiable). We found that small values of $p$ already came reasonably close to the true minimum function, and selected $p=-6$ for our experiments.

Since we expect the true acquisition function to be smooth, rather than resembling a sharp cone near evaluated points, we set the gradient of our Lipschitz cone to be a fraction of the Lipschitz constant – this has the effect of encouraging (or rather not discouraging) exploration. We found $\frac{1}{2}$ to work well in practice. In principle, this parameter could be tuned to help determine the degree to which the algorithm explores diverse regions of the space, with smaller values encouraging exploration more strongly, but we did not investigate this possibility as part of this work.

## Optimising the acquisition function

To optimise the acquisition function, we run a local optimiser multiple times from randomly selected starting locations. We sampled these locations according to the prior $\pi$ against which we are integrating. We chose the number of starting locations to be 10 times the dimensionality of the integrand.

### Implementing Multi-start Optimisation

Our code for this method was written in python, using numpy and scipy. The most straightforward way to implement multi-start optimisation is to simply loop over the starting points and perform the optimisation many times. It is however possible to speed up this process when using numpy by taking advantage of fast vectorised computation in numpy rather than using slow python loops. The acquisition function can be evaluated at many points by passing these points as a numpy array, and this is many times faster than passing the points one at a time in a python loop. To take advantage of this in the optimisation process, we posed our multi-start optimisation problem as a single-start optimisation problem in higher dimensions. Each point in our high-dimensional space is the concatenation of multiple points in the original low-dimensional space. Denoting concatenation by $|$, we define a new function $g(x_1 | ... | x_n) = f(x_1) + ... + f(x_n)$. The Jacobian of this function is then obtained by concatenating the Jacobian of $f$ at each $x_i$, and we eliminate the python loop over starting points by simply starting this high-dimensional optimisation at $x_1 | ... | x_n$.

# Experiments

Prior work has demonstrated that in terms of wall-clock time, BQ methods are competitive with MCMC. Here we investigate the degree to which adding batch methods can improve the performance of BQ in terms of the number of batches required (which, with an objective function which can be effectively parallelised, is a better indicator of actual time taken than the number of samples taken). For a batch of size $n$, the best we can reasonably hope for is a factor-$n$ decrease in batch complexity against the sequential (i.e. batch size 1) method. We test both of our proposed batch selection methods on synthetic problems to assess the improvement achieved by batching, and compare both batch selection methods against MCMC methods in a test computing the model evidence of a Gaussian Process.

Our first experiment is on an in-model integral, i.e. a sample from a Gaussian Process prior with the WSABI warping applied to it. We integrated this function with WSABI for a number of different batch sizes. Each sample was taken in 2 dimensions, and integrated against an isotropic Gaussian prior. Because we are working in 2 dimensions, the ground truth was obtained by simply evaluating the function on a fine grid and evaluating the integral naively. We plot the average absolute error over 10 runs of this experiment in Figure .

The in-model test demonstrates effective parallelisation of the process – we can clearly see that larger batch sizes converge much more quickly to the true integral. We also see that both methods are similarly effective – Kriging Believer and Local Penalisation show comparable performance.

Our second experiment is on a synthetic integral. Here we construct our integrand as a mixture of isotropic Gaussians.

We do this in 4 dimensions, varying the number of components between 10 and 15, setting their variance uniformly at random between 1 and 4, and setting their means uniformly at random within the box bounded by $[-3, 3]$ in all dimensions. We set the hyperparameters of the underlying GP in our model (which uses an isotropic squared exponential kernel) by optimising the marginal likelihood.

Again we test both of our batch selection methods, with a varying number of batch sizes. We plot the average absolute error over 10 runs of this experiment in Figure . The results are similar to the in-model test, demonstrating an effective speedup with increasing batch size, and again showing comparable performance between the two batch selection methods.

Our final experiment is computing the model evidence for a Gaussian Process. The data for the GP was collected by running 20 iterations of Bayesian Optimisation on the Branin-Hoo function, a common benchmark function for Bayesian Optimisation . For the optimisation we used a GP with an isotropic squared exponential covariance, giving us two hyperparameters to marginalise over – we marginalise against a Gaussian prior on the hyperparameters. Since this integral is 2D, we once again obtained ground truth by evaluating on a fine grid.

In this experiment we also compare against two MCMC methods – Metropolis-Hastings (Metropolis et al., ; ), a popular standard method, and NUTS , a newer state-of-the-art method. Both of these methods were run with 500 samples of burn-in, and when comparing against our batched method with a batch size of $n$, we take the result obtained from running $n$ parallel chains. Figure shows the average absolute error in the log of the model evidence, over 50 runs of this experiment. Once again it is clear that larger batch sizes converge more quickly for our method, and our method clearly outperforms the MCMC benchmarks.

All three experiments show improved performance as batch size increases, demonstrating that the selected batches are genuinely informative sets of points.

# Conclusion

In this paper we have presented a routine for performing Bayesian Quadrature as a batched method, and introduced a novel form of local penalisation as part of the batch selection process. Our method allows parallelisation of methods which have so far only been available as serial processes. We have demonstrated that the performance of our method improves as the degree of parallelism increases, and shown that the method is competitive with state-of-the-art MCMC when the sampling budget is small.

[^1]: For negative $p$ this is no longer a norm, but we refer to it as an $\ell_p$-norm for brevity.


abstract: We present an empirical investigation of various ways to *automatically* identify phrases in a tagged corpus that are useful for dialogue act tagging. We found that a new method (which measures a phrase's deviation from an optimally-predictive phrase), enhanced with a lexical filtering mechanism, produces significantly better cues than manually-selected cue phrases, the exhaustive set of phrases in a training corpus, and phrases chosen by traditional metrics, like mutual information and information gain.
author: Ken Samuel, Sandra Carberry, and K. Vijay-Shanker
title: AUTOMATICALLY SELECTING USEFUL PHRASES FOR DIALOGUE ACT TAGGING

# INTRODUCTION

Although machine learning approaches have achieved success in many areas of Natural Language Processing, researchers have only recently begun to investigate applying machine learning methods to discourse-level problems (Litman 1994, Andernach 1996, Reithinger & Klesen 1997, Wiebe et al. 1997, DiEugenio, Moore, & Paolucci 1997). An important task in discourse understanding is to interpret an utterance's **dialogue act**, which is a concise abstraction of the speaker's intention; Figure  presents a hypothetical dialogue that has been labeled with dialogue acts. Recognizing dialogue acts is critical for discourse-level understanding and can also be useful for other applications, such as resolving ambiguity in speech recognition. However, computing dialogue acts is a challenging task, because often a dialogue act cannot be directly inferred from a literal interpretation of an utterance.

```latex
\begin{figure*}[ht]\centering
\begin{tabular}{|cclc|}
\multicolumn{1}{c}{\#} & Speaker & \multicolumn{1}{c}{Utterance} &
\multicolumn{1}{c}{Dialogue Act} \\
\hline
1 & John & Hello.                                          & Greet \\
2 & John & I'd like to meet with you on Tuesday at 2:00.   & Suggest \\
3 & Mary & That's no good for me,                          & Reject \\
4 & Mary & but I'm free at 3:00.                           & Suggest \\
5 & John & That sounds fine to me.                         & Accept \\
6 & John & I'll see you then.                              & Bye \\
\hline
\end{tabular}
\caption{A sample dialogue labeled with dialogue acts}
\label{ex-das}
\end{figure*}
```

We have investigated applying Transformation-Based Learning (Brill 1995) to the task of computing dialogue acts. Transformation-Based learning is a symbolic supervised machine learning method that generates a sequence of rules. This method, which has not been applied previously to discourse-level problems, has a number of attractive characteristics for our task, such as its intuitive learned model and its resistance to overfitting. (Brill 1995)

Our machine learning algorithm makes use of several abstract features extracted from utterances (Samuel et al. 1998a). In particular, one of the most effective features is the phrases[^1] in an utterance that provide useful information for dialogue act tagging, which we will call **dialogue act cues**. This paper presents our investigation of methods for identifying dialogue act cues.

We experimentally compared the effectiveness of various *automatic* methods for selecting phrases, by applying them to a VerbMobil tagged corpus (Reithinger & Klesen 1997). This corpus consists of appointment-scheduling dialogues in which each utterance has been manually labeled with one of eighteen dialogue acts, such as Greet, Suggest, and Accept. Although we understand that there may be problems[^2]$\mathrm{^{,}}$[^3] with the dialogue acts in this corpus, we will assume that they are correct, because these issues are beyond the scope of this project. In any event, if another tagged corpus were to become available, the methods presented here should be directly applicable.

Our results showed that a *new* metric (which measures how far a phrase deviates from an optimally-predictive phrase) enhanced with a simple lexical filtering mechanism can select phrases that are more effective for dialogue act tagging than phrases chosen by human intuitive approaches or traditional metrics (like mutual information and information gain).

# RELATED WORK

Several researchers (Cohen 1987, Fraser 1990, Grosz & Sidner 1986, Halliday & Hasan 1976, Heeman, Byron, & Allen 1998, Hirschberg & Litman 1993, Knott 1996, Marcu 1997, Reichman 1985, Schiffrin 1987, Warner 1985, Zukerman & Pearl 1986) identified **cue phrases** that are useful for discourse processing, such as "but", "so", and "by the way". In most cases, their research focused on selecting phrases that might be *generally* useful; however, we have found that many of the phrases that appear to be useful for our purposes were *not* included in the previous literature. By analyzing the phrases and tags in a corpus, automatic methods directly address three important factors:

1.  The domain of discourse affects which phrases are useful. In the appointment-scheduling dialogues of the VerbMobil corpus, phrases such as "what time" and "I'm busy" could be effective.

2.  The desired task of the system (dialogue act tagging, utterance segmentation, etc.) has a significant impact. For dialogue act tagging, "how about" and "sounds great" might serve as dialogue act cues.

3.  The specific dialogue acts that we want to identify can affect the usefulness of phrases. For example, one of the VerbMobil dialogue acts is Thank, so this motivates a need for phrases like "thank you" and "thanks".

Intuitively, all of the phrases in the above examples seem to be perfectly reasonable indicators of dialogue acts. However, to our knowledge, no one has previously identified these phrases as cue phrases. This leads us to suspect that the domain, task, and tags need to be considered when selecting phrases.

# PHRASE-SELECTION METHODS

The goal of our research is to devise a method that automatically identifies dialogue act cues. This section discusses two baseline approaches and several automatic methods, listed in Figure .

## Baseline Approaches

We used two sets of phrases as baselines for comparison. 1) The LIT set consists of the 687 different cue phrases proposed in twelve papers, dissertations, and books (Cohen 1987, Fraser 1990, Grosz & Sidner 1986, Halliday & Hasan 1976, Heeman, Byron, & Allen 1998, Hirschberg & Litman 1993, Knott 1996, Marcu 1997, Reichman 1985, Schiffrin 1987, Warner 1985, Zukerman & Pearl 1986). 2) The ALL set represents an extreme approach, selecting *all* sequences of up to three words found in a training corpus. Although this set is likely to include all of the useful phrases, it also includes many extraneous phrases, and we hypothesize that these irrelevant phrases can overwhelm a machine learning algorithm.

## Automatic Methods

Our general approach is to use *some metric* that estimates how useful a phrase is for dialogue act tagging by analyzing the dialogue acts of the utterances containing that phrase in a training corpus. In this section, we will discuss the motivations and limitations of several different metrics that we considered.

### Counting cooccurrences

It is reasonable to expect that a dialogue act cue would cooccur frequently with a specific dialogue act. For example, in the VerbMobil corpus, the phrase "see you" is found in 106 utterances that are labeled Bye, suggesting that "see you" is a dialogue act cue. A straightforward way to rank phrases is to count how often each phrase occurs in utterances labeled with each dialogue act. The **cooccurence** method sorts phrases in decreasing order by their COOC scores: $$\mathrm{COOC(p) = \max_{d} \#(p\&d)}$$ where p is a phrase, d is a dialogue act, and \#(x) is the frequency (in the training corpus) of an event x. This metric maximizes over dialogue acts in order to base the score on the best dialogue act for the phrase.

### Considering dialogue act distribution

The simple COOC metric does not take into account the *a priori* distribution of dialogue acts. Unless each dialogue act is equally likely, the most frequently-occurring dialogue acts will generate many high-scoring phrases, even though that may be inappropriate. It might be better to replace the joint frequency in COOC with the **conditional probability** of a phrase given a dialogue act. The conditional probability method sorts phrases in decreasing order by their CP scores:

$$\mathrm{CP(p) = \max_{d} P(p|d)}$$ where $\mathrm{P(x|y)}$ is the probability of x given y.

Since COOC and CP maximize over dialogue acts, these scores only account for one dialogue act for each phrase. But we might expect that a dialogue act cue should cooccur frequently with a few dialogue acts and infrequently with the others; a theoretically optimal dialogue act cue would correlate *perfectly* with a single dialogue act, as represented by the dashed line in Figure . So it might be worthwhile to consider how skewed the distribution of dialogue acts cooccurring with a phrase is. This criterion is captured by the **entropy** of the dialogue acts given a phrase. The entropy method[^4] sorts phrases in increasing order by their ENT scores:

$$\mathrm{ENT(p) = -\sum_{d} P(d|p)\log_{2} P(d|p)}$$

However, like COOC, ENT does not account for the *a priori* distribution of dialogue acts. Suppose the original dialogue act distribution has relatively low entropy, and a phrase p is completely independent[^5] of the dialogue acts. Then ENT(p) is also relatively low (since the dialogue act distribution is unaffected by the existence of p), incorrectly signifying that p conveys useful information. To account for the a priori dialogue act distribution, we examined four different metrics, which are based on the Kullback-Liebler distance, mutual information, the t test, and information gain.

The **selectional preference strength** method considers the difference between the distribution of dialogue acts given a particular phrase and the a priori distribution of dialogue acts to estimate the amount of information a phrase carries about the dialogue acts it cooccurs with.[^6] This is a special case of the **Kullback-Liebler distance** (also known as **relative entropy** or **divergence**),[^7] which measures how much information about a dialogue act would be lost by failing to recognize a specific phrase. The selectional preference strength method sorts phrases in decreasing order by their S scores:

$$\mathrm{S(p) = \sum_{d} P(d|p)[\log_{2} P(d|p) - \log_{2} P(d)]}$$ where $\mathrm{P(x)}$ is the probability of x.

**Mutual information** has been used to measure the reduction of uncertainty in one factor that results from the introduction of another factor. We consider the mutual information between the dialogue acts and a phrase, to compute the reduction of uncertainty in an utterance's dialogue act when the utterance contains the phrase. The mutual information method sorts phrases in decreasing order by their MI scores:

$$\mathrm{MI(p) = P(p) \sum_{d} P(d|p)[\log_{2} P(d|p) - \log_{2} P(d)]}$$

We note that, in this context, mutual information is closely related to selectional preference strength.

The **t test** is used to measure the statistical difference between two distributions. We ran a t test between the a priori distribution of dialogue acts and the distribution of dialogue acts given a phrase. The t test method sorts phrases in decreasing order by their TTEST scores:

$$\mathrm{TTEST(p) = \#(\overline{p})
\sqrt{\sum_{d} \frac{D^{2}-D}{[D\#(p\&d)-\#(p)]^{2} + [D\#(d)-U]^{2}}}}$$ where D is the number of different dialogue acts, U is the total number of utterances, and $\mathrm{\overline{p}}$ refers to the utterances where p does not appear.

**Information gain** is typically utilized to estimate the usefulness of a feature. For example, information gain has been used to determine how to split a node in a decision tree, by considering the distributions of data that fall along each branch. For our task, we are testing for the existence of a phrase, so we use information gain to measure the reduction in entropy of the dialogue acts resulting from partitioning utterances based on whether or not they contain the phrase. The information gain method sorts phrases in decreasing order by their IG scores:

$$\mathrm{IG(p) = \sum_{d}[P(p)P(d\&p)log_{2}P(d\&p) +
P(\overline{p})P(d\&\overline{p})log_{2}P(d\&\overline{p}) - P(d)\log_{2}P(d)]}$$

### Measuring deviation from optimal

In addition to adapting existing metrics, we also designed some new metrics, which evaluate phrases based on their estimated effectiveness in the hypothetical rule[^8] `p IFF d`. Recall that, if p is an optimal dialogue act cue, it correlates perfectly with a single dialogue act (like the dashed line in Figure ). And, if $\mathrm{d^{*}}$ is that dialogue act,[^9] then the rule `p IFF `$\mathtt{d^{*}}$ is *valid*. Therefore, this hypothetical dialogue act cue would be a *perfect* indicator for the dialogue act $\mathtt{d^{*}}$. Our new metrics measure how much each phrase deviates from this optimal design by assigning a penalty point for each utterance where the rule fails.

There are two ways that the rule may fail. First, the rule may be **unsound**, meaning that the left-to-right rule, `IF p THEN `$\mathtt{d^{*}}$, applies incorrectly. For each utterance that contains p but is not labeled $\mathrm{d^{*}}$, we assign a penalty point to the phrase, for a total of $\mathrm{\sum_{d \neq d^{*}} \#(p\&d)}$ points. However, unsoundness alone is not sufficient. A phrase may produce a perfect unsoundness score of 0, and yet still not be optimal. In the extreme case, any phrase that appears only once in the training corpus has an unsoundness score of 0.

One possible way to address this problem is to consider the thin line in Figure , which represents a phrase that only occurs in 25 utterances, where all 25 of those utterances are labeled Suggest. Although this is certainly a useful phrase, since it is as sound as the optimal phrase, we notice that there are 942 other Suggest utterances, which do not include the phrase. We expect that another equally-sound phrase that occurs more frequently should be ranked higher, and so we are considering the case where the rule `p IFF `$\mathtt{d^{*}}$ is **incomplete**, meaning that `IF `$\mathtt{d^{*}}$` THEN p` applies incorrectly. For each utterance that is labeled $\mathrm{d^{*}}$ but does not contain p, we assign one penalty point to the phrase, for a total of $\mathrm{\#(\overline{p}\&d^{*})}$ points.

It is unclear how to combine unsoundness and incompleteness in a general metric. Certainly, when choosing between two equally sound phrases, one would prefer the phrase that is more complete, and vice versa. (See the next section for a qualitative analysis of empirical results.) So, as an initial approach, we considered adding incompleteness and unsoundness together. The **deviation** method sorts phrases in decreasing order by their D scores:

$$\mathrm{D(p) = \min_{d^{*}} [\#(\overline{p}\&d^{*})+\sum_{d \neq d^{*}} \#(p\&d)]}$$

We minimize over dialogue acts, in order to base the score on the best $\mathrm{d^{*}}$ for p.

Like COOC and ENT, the D metric does not account for the a priori distribution of dialogue acts. So, we considered replacing the joint frequencies in D with conditional probabilities. The **deviation conditional probability** method sorts phrases in increasing order by their DCP scores:

$$\mathrm{DCP(p) = \min_{d^{*}} [P(\overline{p}|d^{*})+\sum_{d \neq d^{*}} P(p|d)]}$$

# QUALITATIVE ANALYSIS

We conducted some experiments to evaluate the merits of the various metrics discussed in the last section. First, we used each metric to order the phrases in the ALL set (all of the phrases in the VerbMobil corpus). Then, we manually examined the highest-ranking phrases to intuitively compare the methods. This qualitative analysis immediately revealed some problems.

Several methods suffer from an undesirable bias based on frequency. Many methods are susceptible to *infrequent* phrases; if a phrase appears only once or twice in the corpus, we cannot really draw any reliable conclusions about its usefulness. On the other hand, a number of methods are biased toward phrases that appear *very frequently* (such as "the"). These phrases may be cooccurring frequently with several (or all) of the dialogue acts, making them poor discriminators of dialogue acts. To address a frequency bias, we might want to remove any phrase with a frequency outside of some arbitrary range. However, we believe it may be difficult (or even impossible) to find an appropriate range, and so we would prefer to address this problem by developing some *automatic* mechanism.

In addition, we analyzed the tradeoffs between unsoundness and incompleteness. Figure  lists some phrases that we believe to be dialogue act cues, specifying how they would be ranked based on unsoundness or incompleteness alone.[^10] For example, the phrase "thanks" occurs in eleven utterances in the training corpus, and ten of these utterances are labeled with the Thank dialogue act. As a result, it is assigned a very good (though not perfect) unsoundness score. However, since every phrase that occurs only once in the training corpus gets a perfect unsoundness score, they all outrank "thanks" if only unsoundness is considered. Alternatively, using incompleteness, "thanks" is ranked $\mathrm{26^{th}}$, and the DCP method ranks it fifth.

The problem is that unsoundness is biased toward low-frequency phrases, while incompleteness is biased toward high-frequency phrases. It is not clear how to combine these two factors in order to balance their biases. The D and DCP methods simply sum them, although we have also considered weighting incompleteness and unsoundness in different ways.

Another potential problem is that, for several methods, many of the highest-ranking phrases appear to address the same goal. For example, all of the top eight ENT phrases ("how 'bout the", "'bout the", "okay how", etc.) signal the Suggest dialogue act in basically the same way. This is not surprising since, if one of these phrases receives a good score, then they all should. However, we hypothesize that the repetitions should be eliminated in order to produce a more concise set of phrases, since this may increase the effectiveness of the machine learning method in tagging dialogue acts. Furthermore, if we want to select a predetermined number of phrases, then a set with a wide variety of different phrases is probably more useful than a set with many redundant phrases.

As a starting point, we can easily eliminate some of the redundant phrases with a simple, lexical filtering mechanism, introduced in Samuel et al. (1998b). If one phrase contains another phrase as a subsequence, and the second phrase is ranked higher, then the first phrase is probably repetitious, and so it is unlikely to contribute anything useful. For example, suppose the phrase "see you" is ranked higher than "will see you", indicating that "see you" is more informative. Since "see you" appears in every utterance where "will see you" appears (and perhaps more), there is no good reason to keep the phrase "will see you". The phrase "see you" has better coverage and a better score, so it should always serve as a better feature for dialogue act tagging. The **lexical filter** removes a phrase if one of its subsequences is ranked higher.

# EXPERIMENTAL RESULTS

We ran several experiments to compare the methods in Figure  on the task of labeling utterances with dialogue acts. For all of these experiments, we applied Transformation-Based Learning (Brill 1995) using three classes of features that we have experimentally found to be particularly effective:

-   Applying one of the methods described above to rank the phrases, the system used the best-rated phrases as features of utterances.

-   An attractive characteristic of Transformation-Based Learning is that it generates preliminary tags during training. These tags can then be used as features to further refine the learned model. Ramshaw and Marcus (1994) referred to this as "leveraged learning". So, to help determine the dialogue act of a given utterance, our system used the preliminary dialogue act assigned to the *preceding* utterance as a feature.

-   Our system utilized a "change-of-speaker" feature that represented information about the speaker of a given utterance. This boolean feature is True for an utterance if the speaker of that utterance differs from the speaker of the preceding utterance, and False otherwise.

An effective heuristic is to cluster certain words into semantic classes, which can collapse several dialogue act cues into a single dialogue act cue. For example, in the appointment-scheduling corpora, there is a strong correlation between utterances that mention weekdays and the Suggest dialogue act, but to express this fact, it is necessary to consider five separate dialogue act cues, such as: "on Monday the", "on Tuesday the", "on Wednesday the", "on Thursday the", and "on Friday the". However, if the five weekdays are combined under one label, "\$weekday\$", then the same information can be captured by a single dialogue act cue that has five times as much data supporting it. The experiments presented in this paper use the following semantic clusters: "\$weekday\$", "\$month\$", "\$number\$", "\$ordinal-number\$", and "\$proper-name\$".

All of our experimental results were derived from a set of held-out data (328 utterances), which was completely disjoint from the training data (2701 utterances) that we used to select phrases.

## The Cutoff Points

Since the methods are supposed to rank dialogue act cues higher than other phrases, we should be able to separate the dialogue act cues from the other phrases. To test this, we applied various cutoff points to each method to determine how many lower-ranking phrases may be removed before accuracy begins to decrease. We wanted to investigate the cutoff points in isolation, so the lexical filter was not used in this set of experiments. Figure  presents the accuracy of each method as a function of the number of phrases used. The ALL and LIT sets are also included in the figure, for comparison. (For clarity, COOC, CP, and MI are not shown in the figure, because their curves are similar to IG's curve.)

Four methods, TTEST, D, S, and ENT, produced accuracies significantly[^11] below LIT when 25% (3558) of the 14,231 phrases were selected. This implies that many dialogue act cues were ranked in the bottom 75% by these methods, suggesting that there may be a problem with these phrase orderings. On the other hand, for four methods, IG, COOC, CP, and MI, we could remove more than 13,000 phrases without significantly affecting the accuracy. These methods also produced significantly higher accuracy scores than the LIT set. Therefore, automatic methods can select phrases that are better for dialogue act tagging than the cue phrases found in the literature.

However, DCP was the only method that produced a significant *rise* in accuracy over ALL. With cutoff points from 10% (1423) to 25% (3558), DCP's accuracy was significantly higher than ALL's accuracy. As we hypothesized above, it appears that the irrelevant phrases in ALL limit the accuracy of the machine learning method. And we expect that this effect would be more pronounced for a larger training corpus (with more phrases) or another machine learning method (that is more susceptible to irrelevant features).

However, for cutoff points of 5% (712) and lower, DCP is significantly worse than ALL. We believe that this is because DCP is susceptible to repetitive phrases. Since DCP assigns high scores to many redundant phrases, we require a relatively large set of phrases in order to capture the full variety of dialogue act cues. This is precisely the problem that the lexical filter was designed to address.

## The Lexical Filter

Our next set of experiments tested the lexical filter. If the phrases are ordered properly, then the filter should eliminate some redundant phrases without compromising accuracy in labeling dialogue acts. First, we ordered the phrases with each method and applied the filter. Then, we used various cutoff points to select the top-ranked phrases for training and testing our dialogue act tagger.

This produced some unexpected results, as shown in Figure .[^12] We found that, in some cases, the filter significantly *decreased* the accuracy on the dialogue act tagging task. Since we expected the filter to remove phrases *without* compromising accuracy, this result prompted us to analyze the filter's design more carefully.

We now believe that it is important for the filter to consider *why* a phrase is being selected. For example, while the phrase "hi" tends to signal the Greet dialogue act, an utterance with "hi I" is more likely to be an Init. Our filter would erroneously remove the phrase "hi I", losing some relevant information. So, we modified our filter to follow this new rule:

|                                                             |
|:------------------------------------------------------------|
| IF a phrase p has a subsequence p' that is ranked higher    |
| *AND both p and p' were selected for the same dialogue act* |
| THEN remove p                                               |

The second condition requires further explanation. For the COOC, CP, D, and DCP methods, the metrics maximize (or minimize) over dialogue acts. So, for a given phrase, we determine which dialogue act is producing the maximum (or minimum) value, and define that to be the dialogue act referred to in the second condition. For the other methods, the metrics sum over dialogue acts. In these cases, we follow Resnik (1996) by selecting the dialogue act that produces the greatest contribution to the sum.[^13]

The effect of this modified filter varies dramatically, removing 23% (3224) to 72% (10,237) of the 14,231 phrases, as shown in Figure . However, Figure  shows that, as expected, using the filter does not cause the accuracy to decrease. In addition, it allows the system to maintain a high accuracy with fewer phrases. In particular, DCP's accuracy is significantly higher than ALL's accuracy when using only 5% (712) of the phrases in ALL. This suggests that the filter is effectively removing redundant phrases, to produce a more parsimonious set of phrases.

# DISCUSSION

This paper presented an investigation of various methods for selecting useful phrases. We argued that the traditional method of selecting phrases, in which a human researcher analyzes discourse and chooses general cue phrases by intuition, could miss useful phrases. To address this problem, we introduced *automatic* methods that use a tagged training corpus to select phrases, and our experimental results demonstrated that these methods can outperform the manual approach. Another advantage of automatic methods is that they can be easily transferred to another tagged corpus.

Our experiments also showed that the effectiveness of different methods on the dialogue act tagging task varied significantly, when using relatively small sets of phrases. The method that used our new metric, DCP, produced significantly higher accuracy scores than any of the baselines or traditional metrics that we analyzed. In addition, we hypothesized that repetitive phrases should be eliminated in order to produce a more concise set of phrases. Our experimental results showed that our modified lexical filter can eliminate many redundant phrases without compromising accuracy, enabling the system to label dialogue acts effectively using only 5% of the phrases.

There are a number of research areas that we would like to investigate in the future, including the following: We intend to experiment with different weightings of unsoundness and incompleteness in the DCP metric; we believe that the simple lexical filter presented in this paper can be enhanced to improve it; we would like to study the merits of enforcing frequency thresholds for methods that have a frequency bias; for the semantic-clustering technique, we selected the clusters of words by hand, but it would be interesting to see how a taxonomy, such as WordNet, could be used to automate this process; since all of the experiments in this paper were run on a single corpus, in order to show that these results may generalize to other tasks and domains, it would be necessary to run the experiments on different corpora.

# ACKNOWLEDGMENTS

The members of the VerbMobil research group at DFKI in Germany, including Norbert Reithinger, Jan Alexandersson, and Elisabeth Maier, generously granted us access to the VerbMobil corpora. This work was partially supported by the NSF Grant \#GER-9354869.

[^1]: In this paper, the term **phrase** refers to any sequence of one of more words that may be found in a dialogue, such as "by the way" or "how about the".

[^2]: It may be relatively difficult for human coders to label utterances with dialogue acts in a consistent manner. Traditionally, intercoder reliability and intracoder reliability have been significant problems for dialogue act tagging.

[^3]: There is substantial disagreement about how to select an effective set of dialogue acts. Although several researchers are currently addressing this problem (DRI 1997, MATE 1998, JDTWG 1999), the research community still lacks a standardized set of dialogue acts.

[^4]: Our previous work (Samuel et al. 1998b) did not consider any metrics except for entropy.

[^5]: $\mathrm{P(p|d)}$ is constant for all d.

[^6]: Resnik (1996) introduced the selectional preference strength metric to measure how much information about an argument of a verb would be lost by not taking into account the verb itself.

[^7]: These metrics are related to the **L1 norm** and **information radius** (also known as **divergence from the average**).

[^8]: In other words, the rule states that phrase p appears in an utterance if and only if that utterance is assigned the dialogue act d.

[^9]: In Figure , $\mathrm{d^{*}}$ = Suggest.

[^10]: For this figure, we used the conditional probability scoring method discussed above.

[^11]: In all of the experiments in this paper, the differences were analyzed for statistical significance with the t test (Levine 1981) or the Tukey "honest significant differences" test, which is an extension of the t test that is appropriate for comparing more than two distributions. (Masterson 1997)

[^12]: In this figure, three methods, COOC, CP, and MI, are again omitted for clarity, because their results were similar to IG's results. Also, with the lexical filter, these four curves don't extend beyond 900 phrases, because the lexical filter removes 94% (13,342-13,347) of the phrases in each case.

[^13]: For TTEST, the sum is not located on the outside of the formula. However, since the square root function is monotonic and $\mathrm{\#(\overline{p})}$ is constant for a given phrase, we can use the same approach for selecting a dialogue act with the TTEST method.