Title: Really? Well. Apparently Bootstrapping Improves the Performance of Sarcasm and Nastiness Classifiers for Online Dialogue,
Abstract: More and more of the information on the web is dialogic, from Facebook
newsfeeds, to forum conversations, to comment threads on news articles. In
contrast to traditional, monologic Natural Language Processing resources such
as news, highly social dialogue is frequent in social media, making it a
challenging context for NLP. This paper tests a bootstrapping method,
originally proposed in a monologic domain, to train classifiers to identify two
different types of subjective language in dialogue: sarcasm and nastiness. We
explore two methods of developing linguistic indicators to be used in a first
level classifier aimed at maximizing precision at the expense of recall. The
best performing classifier for the first phase achieves 54% precision and 38%
recall for sarcastic utterances. We then use general syntactic patterns from
previous work to create more general sarcasm indicators, improving precision to
62% and recall to 52%. To further test the generality of the method, we then
apply it to bootstrapping a classifier for nastiness dialogic acts. Our first
phase, using crowdsourced nasty indicators, achieves 58% precision and 49%
recall, which increases to 75% precision and 62% recall when we bootstrap over
the first level with generalized syntactic patterns.