{'fraction_non_alphanumeric': 0.028410719403840642, 'fraction_numerical': 0.01053310404127257, 'mean_word_length': 4.963042085024568, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Estimating User Communication Styles for Spoken Dialogue Systems

We present a neural network approach to estimate the communication style of spoken interaction, namely the stylistic variations elaborateness and directness, and investigate which type of input features to the estimator are necessary to achive good performance. First, we describe our annotated corpus of recordings in the health care domain and analyse the corpus statistics in terms of agreement, correlation and reliability of the ratings. We use this corpus to estimate the elaborateness and the directness of each utterance. We test different feature sets consisting of dialogue act features, grammatical features and linguistic features as input for our classifier and perform classification in two and three classes. Our classifiers use only features that can be automatically derived during an ongoing interaction in any spoken dialogue system without any prior annotation. Our results show that the elaborateness can be classified by only using the dialogue act and the amount of words contained in the corresponding utterance. The directness is a more difficult classification task and additional linguistic features in form of word embeddings improve the classification results. Afterwards, we run a comparison with a support vector machine and a recurrent neural network classifier.


Introduction
For humans, speech is the most natural form of interaction and it has been shown that people adapt their interaction styles to one another across many levels of utterance production when communicating, e.g. by matching each other's behaviour or synchronising the timing of behaviour (Burgoon et al., 2007;Niederhoffer and Pennebaker, 2002;Brennan, 1996;Pickering and Garrod, 2004;Nenkova et al., 2008). However, this adaptive behaviour has rarely been addressed and implemented in a live spoken dialogue system. With the aim of designing such a spoken dialogue system which adapts to the user's communication idiosyncrasies, we present a classification approach to automatically estimate the user's communication style during an ongoing dialogue. The estimated communication style can then be used in the dialogue management to adapt the system behaviour to the user, as depicted in Figure 1. To the best of our knowledge, this is the first work on automatic estimation of the user's communication style in a dialogue system. We present the task of estimating the user's communication style and investigate the influence of grammatical and linguistic features on its estimation. Our classifiers use only features that can be automatically generated during an interaction with a spoken dialogue system (i.e. without any manual annotation). Even though intelligent assistants like Amazon Alexa, Apple Siri, Google Assistant or Microsoft Cortana are becoming increasingly popular, they do not consider different communication styles to adapt their behaviour. Instead, current research in the field of spoken dialogue systems focuses on general user adaptivity like satisfaction or general user groups (Honold et al., 2014;Ultes et al., 2015;Casanueva et al., 2015;Pragst et al., 2015;Miehle et al., 2019). However, various studies suggest that adapting the communication styles of spoken dialogue systems to the individual users in a similar way to what humans do will lead to more natural interactions (Cassell and Bickmore, 2003;Forbes-Riley et al., 2008;Stenchikova and Stent, 2007;Reitter et al., 2006;Mairesse and Walker, 2010). To adapt the behaviour of the system to individual users, we consider the communication styles elaborateness and directness in this work as Pragst et al. (2019) have shown that they influence the user's perception of a dialogue and are therefore valuable candidates for adaptive dialogue management. The elaborateness thereby refers to the amount of additional information provided to the user and the directness describes how concretely the information that is to be conveyed is addressed by the speaker. This means that a direct and concise answer to the question "Can you tell me what the weather's gonna be like today?" is, for example: "It will rain." It answers the question concretely and gives only the requested information in the shortest possible way. The direct and elaborate version of the same answer provides some additional information: "Most of the time it is cloudy and in the afternoon it will rain." The indirect and concise version of this utterance also contains few information, yet addresses the fact that it is raining in a less concrete way: "Today is a good day for cosy activities at home." In this case, the interlocutor can infer that the weather won't be nice as it is better to stay at home. The elaborate and indirect version provides some more details: "Today is a good day for cosy activities at home. In the afternoon you could get wet outside." This example is taken from Miehle et al. (2018a)  spoken user interface are perceived by users and whether there exist global preferences in the communication styles elaborateness and directness. The authors could show that the system's communication style influences the user's satisfaction and the user's perception of the dialogue and that there is no general preference in the system's communication style. The authors conclude that spoken dialogue systems need to adapt their communication style to each user individually during every dialogue in order to achieve a high level of user satisfaction. A study presented by Miehle et al. (2016) investigated cultural differences between the Germans and the Japanese. The results revealed that communication idiosyncrasies in human-human interaction may also be observed during human-computer interaction in a spoken dialogue system context. Moreover, Miehle et al. (2018b) presented another study examining five European cultures whose communication styles are much more alike than the German and Japanese communication idiosyncrasies. The study explores not only the influence of the user's culture but also of the gender, the frequency of use of speech based assistants as well as the system's role. The results show that the system's role significantly influences the user's preference in the system's communication style whereas the frequency of use of speech based assistants has no influence. Moreover, the findings show differences among the cultures and, depending on the culture, there are gender differences with respect to the user's preference in the system's communication style. These studies show that adaptive behaviour regarding the user's communication style is an important aspect of spoken user interfaces. This is why we address the task of estimating user communication styles. The structure of the paper is as follows: In Section 2., related work in the field of the estimation of communication styles will be discussed. Afterwards, the corpus that has been used in this work will be described in Section 3. and our classification approach as well as the results will be presented in Section 4., before concluding in Section 5.

Related Work
Previous work has already explored approaches for the classification of elaborateness and directness in the context of related applications. Di Buccio et al. (2014) propose a methodology to automatically detect and process verbose queries submitted to search engines. It is shown that the information retrieval effectiveness can be significantly improved by considering the query verbosity. Moreover, Gharouit and Nfaoui (2017) suggest to use BabelNet as knowledge base in the detection of verbose queries and then present a comparative study between different algorithms to classify queries into two classes, verbose or succinct. However, both papers deal with the classification of queries submitted to search engines. To the best of our knowledge, there exists no previous work in the field of elaborateness classification for spoken language. Goel et al. (2018) explore different supervised machine learning approaches to automatically detect indirectness in tutoring conversations. The authors collected a corpus of tutoring dialogues from 12 American-English speaking pairs of teenagers whereby the conversations include social interaction as well as tutoring periods. They annotated four types of indirectness for the tutoring periods, namely apologising (e.g. "Sorry, its negative 2."), hedging language (e.g. "You just add 5 to both sides."), the use of vague category extenders (e.g. "You have to multiply and stuff.") and subjectivising (e.g. "I think you divide by 3 here."). Each utterance was then classified as direct or indirect based on its inclusion in any of these categories. Afterwards, they used different classification approaches to detect indirectness based on textual and visual features, reaching an F1 sore of 62 %. However, we claim that there are more aspects than the four types of indirectness annotated in this corpus and that indirectness cannot be broken down to rather simple key word spotting (e.g. "sorry", "just", "and stuff", "I think"). Neuliep (2011) describes the indirect style as "one where the speaker's intentions are hidden or only hinted at during interaction". For our corpus annotation, we used this definition and annotated the directness/indirectness in a global way and not based on fixed structures or key words. Other work in this field only focuses on a specific phenomena of indirect speech, like hedge detection (Prokofieva and Hirschberg, 2014;Ulinski et al., 2018)

Corpus Description
Our data set is based on recordings on health care topics containing spontaneous interactions in dialogue format between two participants: one is taking the role of the system while the other one is taking the role of the user. Each dialogue turn contains one or more dialogue acts. These dialogue acts are chosen out of a set of 43 distinct dialogue acts which have been predefined. A list of all dialogue acts can be found in Table 9 in the appendix. Along with the dialogue acts, the respective utterances are also added to the data set. Overall, the corpus covers 135 German dialogues containing 4,887 annotated dialogue acts. The average number of dialogue acts per dialogue is 36.20. We have annotated each dialogue act with the two communication styles directness and elaborateness. Both are assigned scores between 1 and 5 with 1 = extremely direct/concise and 5 = extremely indirect/elaborate. Each dialogue act has been annotated by three different raters. The class distribution of the annotated elaborateness and directness scores (median of the three ratings) is shown in Fig Table 1: Agreement (κ), correlation (ρ) and reliability (ICC) in elaborateness and directness of the three ratings (R1, R2, R3). All results are significant at the 0.001 level.
the classes 3, 4, and 5 contain utterances which are elaborate/indirect to a greater or lesser extent, we have combined them to one new class, thus reducing the corpus to three classes. For directness, the annotation has shown that it even makes sense to see it as a binary decision between direct/indirect utterances. As the classes 2-5 contain different degrees of indirectness (from slightly indirect to extremely indirect), we additionally combined these classes to one indirect class for binary classification. In order to analyse the quality of the annotated scores, we have used the following measures: Cohen's Kappa κ measures the relative agreement between two sets of ratings and is defined as where p 0 is the observed agreement, and p e is the chance agreement (Cohen, 1960). Hence, κ = 1 for perfect agreement and κ = −1 for perfect disagreement. Spearman's rank correlation coefficient Rho ρ is a nonparametric measure for the rank correlation between two variables and describes how well one variable can be ex-

Elaborateness Directness
Mean/Median Mean/Median κ 0.951 0.802 ρ 0.991 0.884 Table 2: Agreement (κ) and correlation (ρ) between the mean and the median of the three ratings for the elaborateness and the directness. All results are significant at the 0.001 level.
pressed by the other (Spearman, 1904). It is defined as where x i and y i are corresponding ranked ratings, whilex andȳ are the mean ranks. Thus, ρ = 1 if observations have identical ranks and ρ = −1 if observations have fully opposed ranks. The Intraclass Correlation Coefficient ICC measures the reliability between the ratings. We use the One-Way Random Average Measures ICC(1, k) as defined in Shrout and Fleiss (1979) with k = 3 raters who are randomly selected from a larger population of raters. ICC = 1 indicates maximum reliability, ICC = −1 maximum unreliability.
The results can be seen in Table 1. The original ratings (five classes) achieve an overall inter-rater agreement of κ = 0.47 for elaborateness and κ = 0.31 for directness, a correlation of ρ = 0.77 for elaborateness and ρ = 0.40 for directness and a inter-rater reliability of ICC = 0.91 for elaborateness and ICC = 0.64 for directness. If we reduce the classes to three or two (in case of directness), we obtain a higher agreement while the correlation and the inter-rater reliability do not change significantly. Overall, we have a good inter-rater reliability for both communication styles given the difficulty of the annotation task. In order to use the communication style annotations as target for our classification task, we need a final score to be calculated from the three ratings. Typical candidates are the mean and the median. To decide which one to use, we have calculated the mean and the median of the three ratings and analysed which of the two metrics better reflects the individual ratings. The comparison of the mean and the median in terms of Cohen's Kappa κ and Spearman's rank correlation coefficient Rho ρ is shown in Table 2. A strong correlation ρ can be observed. The values of κ indicate a high level of agreement, but still there seem to be some differences, especially for the directness. Therefore, we have compared the agreement and correlation between the mean and the median of the three ratings with the individual ratings (see Table 3). The values of κ indicate that the median better reflects the individual ratings, what might be explained by the fact that we have applied an ordinal scale. Therefore, the median of the ratings is used as final score for the classification in the following section.

Estimation of Communication Styles
The main contribution of this work is to address the problem of automatic estimation of the communication styles  Table 3: Agreement (κ) and correlation (ρ) between the mean (M) and the median (Md) of the three ratings for the elaborateness and the directness and the individual ratings (R1, R2, R3). All results are significant at the 0.001 level.
elaborateness and directness and to investigate how grammatical and linguistic features influence the performance. After defining and evaluating a baseline approach, different setups for adding linguistic information is described and their performance analysed. Afterwards, we run a comparison between different classifiers.

The Dialogue Act Baseline
As a baseline approach, an artificial neural network (ANN) classifier with one hidden layer has been trained using only dialogue act features (DA) that can directly be derived from the data 1 . These features contain the dialogue act and the amount of words in the utterance of the corresponding dialogue act. Note that the dialogue act is the output of the linguistic analysis while the text representation of the utterance is the output of the speech recogniser (see Figure 1). Hence, both features in this feature set can be automatically derived during an ongoing interaction in every spoken dialogue system and no annotation is necessary. The neural net is trained and evaluated with a 10-fold crossvalidation setting on the corpus described in Section 3. Grid search was used to find the best set of hyper parameters (i.e. the amount of nodes, the amount of epochs, the optimiser, the output function and the loss function). To take account for the imbalanced data during the grid search optimisation, the unweighted average recall (UAR) was used, which is the arithmetic average of all class-wise recalls. The results are shown in Table 4. Classification of the 3-class elaborateness reaches an UAR of 84 % only using dialogue act features, which is quite promising. Classification of the 3-class directness results in an UAR of 56 %, and the binary directness reaches an UAR of 75 %. These results clearly show the difficulty  of the task, which has already been shown by the corpus creation. There, it has been quite hard for the annotators to distinguish between different levels of directness so that the class distribution of the directness is sub-optimal for the classification task. However, comparing the results to a majority-class classifier clearly shows that there is still a lot of information encoded in the DA feature set achieving higher UAR. The majority-class classifier always predicts the most frequent class in the training set and achieves an UAR of 33 % for three classes and an UAR of 50 % for two classes. Furthermore, comparing the achieved κ and ρ with the results obtained for the human annotators (Table 1) shows that all, the results of the classifier for the elaborateness (three classes) and the results of the classifiers for the directness (three and two classes), outperform the agreement and correlation between the three human annotators. Hence, all trained classifiers for both the elaborateness and the directness constitute a reasonable baseline.

The Contribution of Grammatical and Linguistic Features
To address the question of whether grammatical features improve the estimation of the communication style, a second feature set is used containing the dialogue act features (as have been used for the baseline) as well as grammatical features. The grammatical features (G) are represented by Part-of-speech (POS) tags which have been assigned to the utterances by the RDRPOSTagger (Nguyen et al., 2014). As the utterance is the output of the speech recognition and this tagger can be used online during an ongoing interaction, there is also no annotation necessary for this feature set. The results are shown in Table 5. It can be seen that there is no improvement in comparison to the baseline. In addition to grammatical features, linguistic features may majorly contribute to the overall classification performance. In order to encode the linguistic features, a Bag-of-Words (BoW) approach has been used in combination with unigrams (U), unigrams and bigrams (UB) and word embeddings (WE). Using BoW, two distinct vocabularies have been created: • The BoW-U vocabulary contains every word occurring in the database of spoken dialogues.  • The BoW-UB vocabulary contains the BoW-U vocabulary (single words) as well as every two-wordsequence in the database.
These vocabularies and the combination with word embeddings led to three different linguistic feature sets: • U: This feature set contains a BoW-U vector for each utterance, thus encoding the number of times each word (of the overall vocabulary) appears in the corresponding utterance.
• UB: This feature set contains a BoW-UB vector for each utterance, thus encoding the number of times each word and each two-word-sequence (of the overall vocabulary) appear in the corresponding utterance.
• WE: For this feature set, the BoW-U vocabulary has been combined with the German pre-trained fastText word vectors by Grave et al. (2018) 2 . Matrix X of dimension u × w contains the BoW-U vectors (dimension 1 × w with w the amount of words in vocabulary BoW-U) for each utterance, where u is the total number of utterances. Matrix W of dimension w × p contains the fastText word vectors (dimension 1 × p with p the length of each word vector) for each word. By multiplying these matrices a new matrix Z = X ·W of dimension u×p is obtained, containing a vector representation for each utterance. These utterance vectors of dimension 1 × p can then be used as feature vectors for the classification task.
In addition to using these linguistic feature sets individually, we have used them in combination with the dialogue act features (DA) and the grammatical features (G). All results are shown in Table 6. For elaborateness, the best results are achieved with the baseline feature set. Grammatical and linguistic features do not seem to have any effect on the classification performance. This leads to the conclusion that for the elaborateness, analysing the utterance length dependent on the dialogue act seems to contain enough information to achieve good classification performance.  Table 6: The classification results using the ANN classifier and the linguistic features (separately and in combination with the dialogue act features and the grammatical features) in terms of the Unweighted Average Recall (UAR), the Accuracy (ACC), the F1-Score, Cohen's Kappa κ and Spearman's rank correlation coefficient Rho ρ. Linguistic features are beneficial for the estimation of the directness, but not for the estimation of the elaborateness.
For directness, the overall performance could be improved by using linguistic information encoded as word embeddings. This in combination with grammatical and dialogue act features (WE+DA+G) led to UARs of 59 % and 76 % for the estimation of directness using three classes and two classes, respectively. Using the BoW approach in combination with unigrams and bigrams could not improve the classification performance.
To sum up, linguistic features are beneficial for the estimation of the directness, but not for the estimation of the elaborateness. For the latter, the dialogue act features (i.e. the dialogue act and the amount of words in the utterance) seem to be sufficient.

Comparison with a SVM Classifier
In a further step, we have compared our ANN classifier with a support vector machine (SVM) classifier. For this evaluation, we have used the dialogue act features (DA), the grammatical features (G) as well as linguistic features encoded as word embeddings (WE) as these feature sets provided the best results with the ANN classifier. We have trained and evaluated our SVM with a 10-fold cross-validation setting (same as with the ANN classifier) and used grid search to find the best set of parameters (i.e. C and γ). The results are shown in Table 7. For the classification of the elaborateness, the SVM classifier performs comparable to the ANN classifier and reaches an UAR of 84 % when using the dialogue act features (DA) or the dialogue act features in combination with the grammatical features (DA+G). The directness classification using the SVM classifier yields worse results than the ANN classifier, for both classification in three and two classes.

Comparison with a RNN Classifier
Lastly, we have compared our ANN classifier with a recurrent neural network (RNN) classifier consisting of two long short-term memory (LSTM) layers followed by two hidden perceptron layers and one output layer. The LSTM layers extract and store temporal information that might be beneficial for the communication style estimation. As in Section 4.3., we have used the dialogue act features (DA), the grammatical features (G) as well as linguistic features encoded as word embeddings (WE). We have trained and evaluated our RNN classifier with a 10-fold crossvalidation setting (same as with the ANN and SVM classifier) and used grid search to find the best set of parameters (i.e. the amount of nodes of the LSTM layers, the dropout, the amount of nodes of the hidden perceptron layers, the amount of epochs, the optimiser, the output function and the loss function). The results are shown in Table 8. For the classification of the 3-class elaborateness and the 3-class directness, the RNN classifier yields worse results than the ANN classifier. However, for the estimation of the binary directness, the RNN classifier outperforms the ANN classifier, reaching an UAR of 78 % when using linguistic features encoded as word embeddings in combination with grammatical and dialogue act features (WE+DA+G). This shows that temporal information is beneficial for the estimation of the directness, but not for the estimation of the elaborateness.

Conclusion and Future Directions
In this work, we have presented a classification approach which, for the first time, addresses the estimation of the user's communication style in a spoken dialogue. We have considered the communication styles elaborateness and directness. First, we have described the annotated corpus based on recordings in the health-care domain which contain spontaneous interactions in dialogue format between two human participants. Each dialogue act has been annotated with the two communication styles elaborateness and directness. By analysing the corpus statistics in terms of the agreement, correlation and reliability of the ratings, we have achieved acceptable labels for the communication style annotations. Afterwards, we have used the corpus to estimate the elaborateness and the directness of each utterance. We have tested different feature sets as input for an ANN classifier and performed classification in two and three classes. The results show that the elaborateness can be classified quite well by only using the dialogue act and the amount of words contained in the corresponding utterance. The directness seems to be a more difficult classification task and additional linguistic features in form of word  Table 8: The classification results using the RNN classifier in terms of the Unweighted Average Recall (UAR), the Accuracy (ACC), the F1-Score, Cohen's Kappa κ and Spearman's rank correlation coefficient Rho ρ. Temporal information is beneficial for the estimation of the directness, but not for the estimation of the elaborateness.
embeddings give improvement in the classification results. Our classifiers for both the elaborateness and the directness use only features that can be automatically recognised during an ongoing interaction in any spoken dialogue system, without any prior annotation. Thereafter, we have run a comparison with a SVM and a RNN classifier. The results show that the SVM classifier performs comparable to the ANN classifier for the elaborateness estimation, but worse for the directness estimation. The RNN classifier outperforms the ANN classifier in the binary directness classification showing that temporal information is beneficial for the estimation of the directness, but not for the estimation of the elaborateness.
In future work, we will consider semantic features in form of sentence embeddings for our classification task. Moreover, we will evaluate our methods in a multilingual test set as we also have Polish, Spanish and Turkish data available.

Data
The feature sets used within this work will be made publicly available and can be downloaded from http://dx. doi.org/10.18725/OPARU-26061. Unfortunately, we are not able to publish the original corpus due to privacy reasons.

Acknowledgements
This work is part of a project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 645012. We thank our colleagues from the University of Tübingen, the German Red Cross in Tübingen and semFYC in Barcelona for organizing and carrying out the corpus recordings. Additionally, this work has received funding within the BMBF project "RobotKoop: Cooperative Interaction Strategies and Goal Negotiations with Learning Autonomous Robots" and the technology transfer project "Do it yourself, but not alone: Companion Technology for DIY support" of the Transregional Collaborative Research Centre SFF/TRR 62 "Companion Technology for Cognitive Technical Systems" funded by the German Research Foundation (DFG).