Sarcasm is often expressed through several verbal and non-verbal cues, e.g., a change of tone, overemphasis in a word, a drawn-out syllable, or a straight looking face. Most of the recent work in sarcasm detection has been carried out on textual data. In this paper, we argue that incorporating multimodal cues can improve the automatic classification of sarcasm. As a first step towards enabling the development of multimodal approaches for sarcasm detection, we propose a new sarcasm dataset, Multimodal Sarcasm Detection Dataset (MUStARD), compiled from popular TV shows. MUStARD consists of audiovisual utterances annotated with sarcasm labels. Each utterance is accompanied by its context of historical utterances in the dialogue, which provides additional information on the scenario where the utterance occurs. Our initial results show that the use of multimodal information can reduce the relative error rate of sarcasm detection by up to 12.9 in F-score when compared to the use of individual modalities. The full dataset is publicly available for use at this https URL
@cite_24: Prosodic features in spontaneous speech help disambiguate implied meaning not explicit in linguistic surface structure, but little research has examined how these signals manifest themselves in real conversations. Spontaneously produced verbal irony utterances generated between familiar speakers in conversational dyads were acoustically analyzed for prosodic contrasts. A prosodic contrast was defined as a statistically reliable shift between adjacent phrasal units in at least 1 of 5 acoustic dimensions (mean fundamental frequency, fundamental frequency variability, mean amplitude, amplitude variability, and mean syllable duration). Overall, speakers contrasted prosodic features in ironic utterances with utterances immediately preceding them at a higher rate than between adjacent nonironic utterance pairs from the same interactions. Across multiple speakers, ironic utterances were spoken significantly slower than preceding speech, but no other acoustic dimensions changed consistently. This is the first aco...
@cite_5: The present study was conducted to identify possible acoustic cues of sarcasm. Native English speakers produced a variety of simple utterances to convey four different attitudes: sarcasm, humour, sincerity, and neutrality. Following validation by a separate naive group of native English speakers, the recorded speech was subjected to acoustic analyses for the following features: mean fundamental frequency (F0), F0 standard deviation, F0 range, mean amplitude, amplitude range, speech rate, harmonics-to-noise ratio (HNR, to probe for voice quality changes), and one-third octave spectral values (to probe resonance changes). The results of analyses indicated that sarcasm was reliably characterized by a number of prosodic cues, although one acoustic feature appeared particularly robust in sarcastic utterances: overall reductions in mean F0 relative to all other target attitudes. Sarcasm was also reliably distinguished from sincerity by overall reductions in HNR and in F0 standard deviation. In certain linguistic contexts, sarcasm could be differentiated from sincerity and humour through changes in resonance and reductions in both speech rate and F0 range. Results also suggested a role of language used by speakers in conveying sarcasm and sincerity. It was concluded that sarcasm in speech can be characterized by a specific pattern of prosodic cues in addition to textual cues, and that these acoustic characteristics can be influenced by language used by the speaker.
@cite_0: This experiment investigated the relative contribution of context and tone of voice in the perception of sarcasm in short utterances. Eighty-two participants heard short stories read in a monotone voice reflecting either a positive or a negative context, followed by a relevant statement in a sarcastic or sincere tone of voice. Context and tone of voice were either congruent (e.g., positive context with sincere tone) or incongruent (e.g., positive context with sarcastic tone). Participants were asked to rate from 1 to 7 whether they perceived the statement as very sincere (1) or very sarcastic (7). Results indicated a significant context by tone interaction on both ratings and response time. Specifically, mid-range ratings and longer reaction times were obtained when context and tone were incongruent compared to when they were congruent. The discussion emphasizes the interplay of context and tone of voice in the perception of a sarcastic statement.
Sarcasm detection in speech has mainly focused on the identification of prosodic cues in the form of acoustic patterns that are related to sarcastic behavior. Studied features include mean amplitude, amplitude range, speech rate, harmonics-to-noise ratio, and others @cite_5 . presented one of the initial approaches to this problem that studied the vocal tonalities of sarcastic speech. They found slower speaking rates and greater intensity as probable markers for sarcasm. Later, studied prosodic and spectral features of sound --- both in and out of context --- to determine sarcasm. In general, prosodic features such as intonation and stress are considered important indicators of sarcasm @cite_24 @cite_0 . We take motivation from this previous research and include similar speech parameters as features in our dataset and baseline experiments.