In this paper, we focus on online reviews and employ artificial intelligence tools, taken from the cognitive computing field, to help understand the relationships between the textual part of the review and the assigned numerical score. We move from the intuitions that (1) a set of textual reviews expressing different sentiments may feature the same score (and vice-versa), and (2) detecting and analyzing the mismatches between the review content and the actual score may benefit both service providers and consumers, by highlighting specific factors of satisfaction (and dissatisfaction) in texts. To prove the intuitions, we adopt sentiment analysis techniques and we concentrate on hotel reviews, to find polarity mismatches therein. In particular, we first train a text classifier with a set of annotated hotel reviews, taken from the Booking website. Then, we analyze a large dataset, with around 160k hotel reviews collected from TripAdvisor, with the aim of detecting a polarity mismatch, indicating if the textual content of the review is in line, or not, with the associated score. Using well-established artificial intelligence techniques and analyzing in depth the reviews featuring a mismatch between the text polarity and the score, we find that—on a scale of five stars—those reviews ranked with middle scores include a mixture of positive and negative aspects. The approach proposed here, beside acting as a polarity detector, provides an effective selection of reviews—on an initial very large dataset—that may allow both consumers and providers to focus directly on the review subset featuring a text score disagreement,which conveniently convey to the user a summary of positive and negative features of the review target.
@cite_23: The support-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data. High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
This brief overview of the literature shows heterogeneous techniques and applications for polarity detection, both supervised and unsupervised, in different contexts and for different goals. In this work, thanks to the availability of a labeled dataset, we exploit a supervised approach, which automatically learns a model from the annotated data. In order to choose the most effective algorithm, we test different supervised algorithms and we finally select a linear Support Vector Machine (SVM) @cite_23 , due to its efficiency in dealing with the task at hand.