love may make the world go round , but humor is the glue that keeps it together . our everyday experiences serve as evidence that humor plays a critical role in human interactions and composes a significant part of our linguistic , cognitive , and social lives . previous research has shown that humor is ubiquitous across cultures ( kruger , 1996 ; martin , 2010 ) , increases interpersonal attraction ( lundy , tan , & cunningham , 1998 ) , helps resolve intergroup conflicts ( smith , harrington , & neck , 2000 ) , and improves psychological wellbeing ( martin , kuiper , olinger , & dance , 1993 ) . however , little is known about the cognitive basis of such a pervasive and enjoyable experience . by providing a formal model of linguistic humor , we aim to solve part of the mystery of what makes us laugh . theories of humor have existed since the time of plato and aristotle ( see attardo , 1994 , for review ) . a leading theory in modern research posits that incongruity , loosely characterized as the presence of multiple incompatible meanings in the same input , may be critical for humor ( forabosco , 1992 ; hurley , dennett , & adams , 2011 ; koestler , 1964 ; martin , 2010 ; mcghee , 1979 ; vaid & ramachandran , 2001 ; veale , 2004 ) . however , despite relative consensus on the importance of incongruity , definitions of incongruity vary across informal analyses of jokes . as ritchie ( 2009 ) wrote ,  there is still not a rigorously precise definition that would allow an experimenter to objectively determine whether or not incongruity was present in a given situation or stimulus  ( p.  331 ) . this lack of precision makes it difficult to empirically test the role of incongruity in humor or extend these ideas to a concrete computational understanding . on the other hand , most work on computational humor focuses either on jokespecific templates and schemata ( binsted , 1996 ; taylor & mazlack , 2004 ) or surface features and properties of individual words ( kiddon & brun , 2011 ; mihalcea & strapparava , 2006 ; reyes , rosso & buscaldi , 2012 ) . one exception is mihalcea , strapparava , and pulman ( 2010 ) , which used features inspired by incongruity theory to detect humorous punch lines ; however , the incongruity features proposed did not significantly outperform a random baseline , leading the authors to conclude that jokespecific features may be preferable . while these dominant approaches in computational humor are able to identify humorous stimuli within certain constraints , they fall short of testing a more general cognitive theory of humor . in this work , we suggest that true measures of incongruity in linguistic humor may require a model that infers meaning from words in a principled manner . we build upon theories of humor and language processing to formally measure the multiplicity of meaning in puns  sentences  in which two different sets of ideas are expressed , and we are confronted with only one series of words ,  as described by philosopher henri bergson ( bergson , 1914 ) . puns provide an ideal test bed for our purposes because they are simple , humorous sentences with multiple meanings . here we focus on phonetic puns , defined as puns containing words that sound identical or similar to other words in english.1 the following is an example : ( 1 )  the magician got so mad he pulled his hare out . although the sentence 's written form unambiguously contains the word  hare ,  previous work has suggested that phonetic representations play a central role in language comprehension even during reading ( niznikiewicz & squires , 1996 ; pexman , lupker , & jared , 2001 ; pollatsek , lesch , morris , & rayner , 1992 ) . taking the lexical ambiguity of its phonetic form into account , this sentence thus implicitly expresses two  ideas ,  or meanings:2 
  ( 1a ) the magician got so mad he performed the trick of pulling a rabbit out of his hat . ( 1b ) the magician got so mad he pulled out the hair on his head . at the most basic level , however , the following sentence also contains a phonetically ambiguous word , but it is clearly not a pun : ( 2 )  the hare ran rapidly across the field . a critical difference between ( 1 ) and ( 2 ) is that hare and hair are both probable meanings in the context of sentence ( 1 ) , whereas hare is much more likely than hair in sentence ( 2 ) . from this informal analysis , it seems that what distinguishes a phonetic pun from a regular sentence is that both meanings are compatible with context in a phonetic pun , suggesting that a sentence must contain ambiguity to be funny . this sentence is also ambiguous between hare and hair , but it is unlikely to elicit chuckles . a critical difference between ( 1 ) and ( 3 ) is that while each meaning is strongly supported by distinct groups of words in ( 1 ) ( hare is supported by  magician  and  hare  ; hair is supported by mad  and  pulled  ) , both meanings are weakly supported by all words in ( 3 ) . this comparison suggests that in addition to ambiguity , distinctiveness of support may also be an important criterion for humor . observations on the putative roles of ambiguity of sentence meaning and distinctiveness of support will motivate our formal measures of humor.3 
  how should we represent the meaning of a sentence in order to measure its ambiguity and distinctiveness ? while formally representing sentence meanings is a complex and largely unsolved problem ( grefenstette , sadrzadeh , clark , coecke , & pulman , 2014 ; liang , jordan , & klein , 2013 ; socher , huval , manning , & ng , 2012 ) , we can utilize certain properties of phonetically ambiguous sentences to simplify the issue at hand . we notice that in sentence ( 1 ) , meaning ( 1a ) arises if the word  hare  is interpreted as hare , while meaning ( 1b ) arises if  hare  is interpreted as its homophone hair . each sentencelevel meaning directly corresponds to the meaning of a phonetically ambiguous word . as a result , we can represent sentence meaning ( 1a ) with the meaning of hare and ( 1b ) with the meaning of hair . this approximation is coarse and captures only the  gist  of a sentence rather than its full meaning . however , we will show that such a gist representation is sufficiently powerful for modeling the interpretation of sentences with only a phonetic ambiguity . given the space of candidate sentence meanings , a comprehender 's task is to infer a distribution over these meanings from the words she observes . formally , a phonetically ambiguous sentence such as ( 1 ) is composed of a vector of words w={w1,  ,wi , h , wi+1,  ,wn } , where h is phonetically confusable with its homophone h. the sentence meaning is a latent variable m , which we assume has two possible values m 
 a and m 
 b. these two sentence meanings can be identified with the homophones h and h , respectively . consistent with a noisy channel approach ( gibson , bergen , & piantadosi , 2013 ; levy , 2008 ; levy , bicknell , slattery , & rayner , 2009 ) , we construe the task of understanding a sentence as inferring m using probabilistic integration of noisy evidence given by w. we construct a simple probabilistic generative model that captures the relationship between the meaning of a sentence and the words that compose it ( fig .  1 ) . if a word is semantically relevant ( f i  =  1 ) , we assume that it is sampled based on semantic relatedness to the sentence meaning ; if the word is irrelevant , or  noise ,  it only reflects general language statistics and is sampled from an ngram model . because the comprehender maintains uncertainty about which words are relevant , it is possible for her to arrive at multiple interpretations of a sentence that are each coherent but incongruous with one another , a situation that we hypothesize gives rise to humor . to capture this intuition , we introduce two measures of humor derived from the distribution over sentence meanings ( details in methods section ) . if the indicator variable f 
 i has value 1 , w 
 t is generated based on semantic relatedness to the sentence meaning m ; otherwise , w 
 t is sampled from a trigram language model based on the immediately preceding two words . given words in a sentence , we infer the joint probability distribution over sentence meanings and semantically relevant words , which can be factorized into the following : ( 1)p(m , f|w)=p(m|w)p(f|m , w ) 
  we compute a measure of humor from each of the two terms on the righthand side . if entropy is high , then the sentence is ambiguous because both meanings are nearequally likely . distinctiveness captures the degree to which the semantically relevant words differ given different sentence meanings . given one meaning m 
 a , we can compute fa = p(f|ma , w ) . given another meaning m 
 b , we compute fb = p(f|mb , w ) . distinctiveness is quantified by the symmetrized kullbackleibler divergence between these two distributions , d 
 kl ( f 
 af 
 b )  +  d 
 kl ( f 
 bf 
 a ) . if the symmetrized kl distance is high , it suggests that the two sentence meanings are supported by distinct subsets of words in the sentence . derivation details of these two measures are in the methods section below . in what follows , we empirically evaluate ambiguity and distinctiveness as predictors of humor in a set of phonetically ambiguous sentences . we define the ambiguity of a sentence as the entropy of p(m|w ) , where w is a vector of observed content words in a sentence ( which contains a phonetically ambiguous word h ) and m is the latent sentence meaning . given the simplifying assumption that the distribution over sentence meanings is not affected by function words , each w 
 i in w is a content word . the distribution over sentence meanings given words can be derived using bayes rule : ( 2)p(m|w)=fp(m , f|w)fp(w|m , f)p(m)p(f)=fp(m)p(f)ip(wi|m , fi ) 
  each value of m is approximated by either the meaning of the observed phonetically ambiguous word h ( e.g. ,  hare  in sentence ( 1 ) ) or its unobserved homophone h ( e.g. ,  hair  ) . we can thus represent p(m ) as the unigram frequency of h or h. for example , p(m  =  hare ) is approximated as proportional to p(hare  ) . we assume equal prior probability that each subset of the words is semantically relevant , hence p(f ) is a constant . p(w 
 im , f 
 i ) depends on the value of the semantic relevance indicator variable f 
 i. if f 
 i  =  1 , w 
 i is semantically relevant and is sampled in proportion to its relatedness with the sentence meaning m. if f 
 i  =  0 , then w 
 i is generated from a noise process and sampled in proportion to its probability given the previous two words in the sentence . formally , ( 3)p(wi|m , fi)=p(wi|m)iffi=1p(wi|bigrami)iffi=0 
  we estimate p(w 
 im ) using empirical association measures described in the experiment section and compute p(w 
 ibigrami ) using the google ngrams corpus ( brants & franz , 2006 ) . once we derive m = p(m|w ) , we compute its informationtheoretic entropy as a measure of ambiguity : ( 4)amb(m)=k{a , b}p(mk|w)logp(mk|w ) 
  we next compute the distinctiveness of words supporting each sentence meaning . using bayes rule : ( 5)p(f|m , w)p(w|m , f)p(f|m ) 
  since f and m are independent , p(f|m)=p(f ) , which is a constant . we compute the symmetrized kullbackleibler divergence score d 
 kl ( f 
 af 
 b )  +  d 
 kl ( f 
 bf 
 a ) , which measures the difference between the distribution of supporting words given one sentence meaning and the distribution of supporting words given another sentence meaning . this results in the distinctiveness measure4 : ( 6)dist(fa , fb)=ilnfa(i)fb(i)fa(i)+lnfb(i)fa(i)fb(i ) 
  given these derivations , we conducted the following experiment to implement and test the ambiguity and distinctiveness measures . we collected 435 sentences consisting of phonetic puns and regular sentences that contain phonetically ambiguous words . ( http://www.punoftheday.com ) , which at the time of collection contained over a thousand puns submitted by users . we collected 40 puns where the phonetically ambiguous word has an identical homophone , for example  hare . since only a limited number of puns satisfied this criterion , a research assistant generated an additional 25 pun sentences based on a separate list of homophone words , resulting in a total of 65 identicalhomophone puns . we selected 130 corresponding nonpun sentences from an online version of heinle 's newbury house dictionary of american english ( http://nhd.heinle.com ) . of the 130 nonpun sentences , 65 sentences contain the ambiguous words observed in the pun sentences ( e.g. ,  hare  ) ; the other 65 contain the unobserved homophone words  ( e.g. , hair).5 to test whether our measures generalize to sentences containing phonetically ambiguous words that do not have identical homophones , we collected 80 puns where the phonetically ambiguous word sounds similar ( but not identical ) to other words in english ( e.g. ,  tooth the full set of sentences can be found here : http://web.stanford.edu/~justinek/pun-paper/results.html 
  example sentence from each category . identical homophone sentences contain phonetically ambiguous words that have identical homophones ; near homophone sentences contain phonetically ambiguous words that have near homophones . pun sentences were selected from a pun website ; nonpun sentences were selected from an online dictionary ( see main text for details ) we obtained funniness ratings for each of the 435 sentences . we asked 100 participants on amazon 's mechanical turk6 to rate the 195 sentences that contain identical homophones . each participant read roughly 60 sentences in random order , counterbalanced for the sentence types , and rated each sentence on funniness (  how funny is this sentence ?  ) on a scale from 1 ( not at all ) to 7 ( extremely ) . we removed seven participants who reported a native language other than english and zscored the ratings within each participant . a separate group of 160 participants on mechanical turk rated the 240 near homophone sentences . each participant read 40 sentences in random order , counterbalanced for the sentence types , and rated each sentence on funniness on a scale from 1 to 7 . we removed four participants who reported a native language other than english and zscored the ratings within each participant . we used the average zscored ratings across participants as human judgments of funniness for all 435 sentences . as described in the measure derivations , computing ambiguity and distinctiveness requires the conditional probabilities of each word given a sentence meaning , that is , p(w , this value is difficult to obtain reliably and accurately in an automated way , such as through wordnet distances or semantic vector space models ( gabrilovich & markovitch , 2007 ; mihalcea et  al . , 2010 ; zhang , gentile , & ciravegna , 2011).7 instead of tackling the challenging problem of automatically learning p(w i , m ) from large corpora , we observe that p(w 
 i , m ) is related to point wise mutual information ( pmi ) between w 
 i and m , an informationtheoretic measure defined mathematically as the following ( church & hanks , 1990 ) : ( 7)logp(wi , m)p(wi)p(m)=logp(wim)logp(wi ) 
  intuitively , pmi captures the relatedness between w 
 i and m , which can be measured empirically by asking people to judge the semantic relatedness between two words . this allows us to harness people 's rich knowledge of the relationships between word meanings without relying solely on cooccurrence statistics in corpora . we assume that the zscored human ratings of relatedness between two words , denoted r ( w 
 i , m ) , approximates true pmi . with the proper substitutions and transformations8 from eq . ( 7 ) , we derive the following : ( 8)p(wi|m)=er(wi , m)p(wi ) 
  to obtain r(w 
 i , m ) for each of the words in the stimuli sentences , function words were removed from each of the sentences in our data set , and the remaining words were paired with the phonetically ambiguous word h and its homophone h ( e.g. , for the pun in table  1 , [  magician ,   hare  ] is a legitimate word pair , as well as [  magician , this resulted in 1,460 distinct word pairs for identical homophone sentences and 2,056 word pairs for near homophone sentences . we asked 200 participants on amazon 's mechanical turk to rate the semantic relatedness of word pairs for identical homophone sentences . each participant saw 146 pairs of words in random order and were asked to rate how related each word pair is using a scale from 1 to 10 . since it is difficult to measure the relatedness of a word with itself , we assume that the value is constant for all words and treat it as a free parameter , r. after computing our measures , we fit this parameter to people 's funniness judgments ( resulting in r  =  13 ) . we used the average zscored relatedness measure for each word pair to obtain r(w 
 i , m ) and google web unigrams to obtain p(w 
 i ) . this allowed us to compute p(w 
 im ) for all word and meaning pairs . we define the ambiguity of a sentence as the entropy of p(m|w ) , where w is a vector of observed content words in a sentence ( which contains a phonetically ambiguous word h ) and m is the latent sentence meaning . given the simplifying assumption that the distribution over sentence meanings is not affected by function words , each w 
 i in w is a content word . the distribution over sentence meanings given words can be derived using bayes rule : ( 2)p(m|w)=fp(m , f|w)fp(w|m , f)p(m)p(f)=fp(m)p(f)ip(wi|m , fi ) 
  each value of m is approximated by either the meaning of the observed phonetically ambiguous word h ( e.g. ,  hare  in sentence ( 1 ) ) or its unobserved homophone h ( e.g. ,  hair  ) . we can thus represent p(m ) as the unigram frequency of h or h. for example , p(m  =  hare ) is approximated as proportional to p(hare  ) . we assume equal prior probability that each subset of the words is semantically relevant , hence p(f ) is a constant . p(w 
 im , f 
 i ) depends on the value of the semantic relevance indicator variable f 
 i. if f 
 i  =  1 , w 
 i is semantically relevant and is sampled in proportion to its relatedness with the sentence meaning m. if f 
 i  =  0 , then w 
 i is generated from a noise process and sampled in proportion to its probability given the previous two words in the sentence . formally , ( 3)p(wi|m , fi)=p(wi|m)iffi=1p(wi|bigrami)iffi=0 
  we estimate p(w 
 im ) using empirical association measures described in the experiment section and compute p(w 
 ibigrami ) using the google ngrams corpus ( brants & franz , 2006 ) . once we derive m = p(m|w ) , we compute its informationtheoretic entropy as a measure of ambiguity : ( 4)amb(m)=k{a , b}p(mk|w)logp(mk|w ) 
  we next compute the distinctiveness of words supporting each sentence meaning . using bayes rule : ( 5)p(f|m , w)p(w|m , f)p(f|m ) 
  since f and m are independent , p(f|m)=p(f ) , which is a constant . we compute the symmetrized kullbackleibler divergence score d 
 kl ( f 
 af 
 b )  +  d 
 kl ( f 
 bf 
 a ) , which measures the difference between the distribution of supporting words given one sentence meaning and the distribution of supporting words given another sentence meaning . this results in the distinctiveness measure4 : ( 6)dist(fa , fb)=ilnfa(i)fb(i)fa(i)+lnfb(i)fa(i)fb(i ) 
  given these derivations , we conducted the following experiment to implement and test the ambiguity and distinctiveness measures . we collected 435 sentences consisting of phonetic puns and regular sentences that contain phonetically ambiguous words . ( http://www.punoftheday.com ) , which at the time of collection contained over a thousand puns submitted by users . we collected 40 puns where the phonetically ambiguous word has an identical homophone , for example  hare . since only a limited number of puns satisfied this criterion , a research assistant generated an additional 25 pun sentences based on a separate list of homophone words , resulting in a total of 65 identicalhomophone puns . we selected 130 corresponding nonpun sentences from an online version of heinle 's newbury house dictionary of american english ( http://nhd.heinle.com ) . of the 130 nonpun sentences , 65 sentences contain the ambiguous words observed in the pun sentences ( e.g. ,  hare  ) ; the other 65 contain the unobserved homophone words  ( e.g. , hair).5 to test whether our measures generalize to sentences containing phonetically ambiguous words that do not have identical homophones , we collected 80 puns where the phonetically ambiguous word sounds similar ( but not identical ) to other words in english ( e.g. ,  tooth the full set of sentences can be found here : http://web.stanford.edu/~justinek/pun-paper/results.html 
  example sentence from each category . identical homophone sentences contain phonetically ambiguous words that have identical homophones ; near homophone sentences contain phonetically ambiguous words that have near homophones . pun sentences were selected from a pun website ; nonpun sentences were selected from an online dictionary ( see main text for details ) we obtained funniness ratings for each of the 435 sentences . we asked 100 participants on amazon 's mechanical turk6 to rate the 195 sentences that contain identical homophones . each participant read roughly 60 sentences in random order , counterbalanced for the sentence types , and rated each sentence on funniness (  how funny is this sentence ?  ) on a scale from 1 ( not at all ) to 7 ( extremely ) . we removed seven participants who reported a native language other than english and zscored the ratings within each participant . a separate group of 160 participants on mechanical turk rated the 240 near homophone sentences . each participant read 40 sentences in random order , counterbalanced for the sentence types , and rated each sentence on funniness on a scale from 1 to 7 . we removed four participants who reported a native language other than english and zscored the ratings within each participant . we used the average zscored ratings across participants as human judgments of funniness for all 435 sentences . as described in the measure derivations , computing ambiguity and distinctiveness requires the conditional probabilities of each word given a sentence meaning , that is , p(w im ) . in practice , this value is difficult to obtain reliably and accurately in an automated way , such as through wordnet distances or semantic vector space models ( gabrilovich & markovitch , 2007 ; mihalcea et  al . , 2010 ; zhang , gentile , & ciravegna , 2011).7 instead of tackling the challenging problem of automatically learning p(w i , m ) from large corpora , we observe that p(w 
 i , m ) is related to point wise mutual information ( pmi ) between w 
 i and m , an informationtheoretic measure defined mathematically as the following ( church & hanks , 1990 ) : ( 7)logp(wi , m)p(wi)p(m)=logp(wim)logp(wi ) 
  intuitively , pmi captures the relatedness between w 
 i and m , which can be measured empirically by asking people to judge the semantic relatedness between two words . this allows us to harness people 's rich knowledge of the relationships between word meanings without relying solely on cooccurrence statistics in corpora . we assume that the zscored human ratings of relatedness between two words , denoted r ( w 
 i , m ) , approximates true pmi . with the proper substitutions and transformations8 from eq . ( 7 ) , we derive the following : ( 8)p(wi|m)=er(wi , m)p(wi ) 
  to obtain r(w 
 i , m ) for each of the words in the stimuli sentences , function words were removed from each of the sentences in our data set , and the remaining words were paired with the phonetically ambiguous word h and its homophone h ( e.g. , for the pun in table  1 , [  magician ,   hare  ] is a legitimate word pair , as well as [  magician ,   hair  ] ) . this resulted in 1,460 distinct word pairs for identical homophone sentences and 2,056 word pairs for near homophone sentences . we asked 200 participants on amazon 's mechanical turk to rate the semantic relatedness of word pairs for identical homophone sentences . each participant saw 146 pairs of words in random order and were asked to rate how related each word pair is using a scale from 1 to 10 . since it is difficult to measure the relatedness of a word with itself , we assume that the value is constant for all words and treat it as a free parameter , r. after computing our measures , we fit this parameter to people 's funniness judgments ( resulting in r  =  13 ) . we used the average zscored relatedness measure for each word pair to obtain r(w 
 i , m ) and google web unigrams to obtain p(w 
 i ) . this allowed us to compute p(w 
 im ) for all word and meaning pairs . we computed an ambiguity and distinctiveness score for each of the 435 sentences ( see 2 ) . we found no significant differences between identical and near homophone puns in terms of funniness ratings ( t(130.91 )  =  0.13 , p  =  .896 ) , ambiguity scores ( t(137.80 )  =  1.13 , p  =  .261 ) , and distinctiveness scores ( t(134.91 )  =  0.61 , p  =  .543 ) , suggesting that ambiguity and distinctiveness are fairly robust to the differences between  puns that involve identical or near homophone words . as a result we found that  ambiguity was significantly higher for pun sentences than nonpun sentences ( t(159.48 )  =  7.89 , p  <  .0001 ) , which suggests that the ambiguity measure successfully captures characteristics distinguishing puns from other phonetically ambiguous sentences . distinctiveness was also significantly higher for pun sentences than nonpun sentences ( t(248.99 )  =  6.11 , p  <  .0001 ) . 2 shows the standard error ellipses for the two sentence types in a twodimensional space of ambiguity and distinctiveness . although there is a fair amount of noise in the predictors ( likely due to simplifying assumptions , the need to use empirical measures of relatedness , and the inherent complexity of humor ) , pun sentences ( both identical and near homophone ) tend to cluster at a space with higher ambiguity and distinctiveness , while nonpun sentences score lower on both measures . puns ( both identical and near homophone ) score higher on ambiguity and distinctiveness ; nonpun sentences score lower . we constructed a linear mixedeffects model of funniness judgments with ambiguity and distinctiveness as fixed effects , a byitem random intercept , and bysubject random slopes for entropy and distinctiveness . we found that ambiguity and distinctiveness were both highly significant predictors , with funniness increasing as each of ambiguity and distinctiveness increases ( table  2 ) . furthermore , the two measures capture a substantial amount of the reliable variance in funniness ratings averaged across subjects ( f(2 , 432 )  =  74.07 , r 
 =  0.25 , p  < .0001 ) . a linear mixedeffects model including a term for the interaction between ambiguity and distinctiveness ( both as fixed effect and bysubjects random slope ) showed no significant interaction between the two ( t  =  1.39 , p  >  .05 ) . regression coefficients using ambiguity and distinctiveness to predict funniness ratings for all 435 sentences ; pvalues are computed assuming that the t statistic is approximately normally distributed we then examined whether the measures are able to go beyond distinguishing puns from nonpuns to predicting finegrained levels of funniness within puns . we found that ambiguity does not correlate with human ratings of funniness within the 145 pun sentences ( r  =  .03 , p  =  .697 ) . however , distinctiveness ratings correlate significantly with human ratings of funniness within pun sentences ( r  =  .28 , p  <  .001 ) . by separating the puns into four equal bins based on their distinctiveness scores , we found that puns with distinctiveness measures in the topmost quartile were significantly funnier than puns with distinctiveness measures in the lower quartiles ( t(90.15 )  =  3.41 , p  <  .001 ) ( fig . this suggests that while ambiguity helps distinguish puns from nonpuns , high distinctiveness characterizes exceptionally humorous puns . to our knowledge , our model provides the first quantitative measure that predicts finegrained levels of funniness within humorous stimuli . , the model can also be used to reveal critical features of each pun that make it amusing . for each sentence , we identified the set of words that is most likely to be semantically relevant given w and each sentence meaning m. formally , we computed argmaxfp(f|ma , w ) and argmaxfp(f|mb , w ) . table  3 shows a group of identical homophone sentences and a group of near homophone sentences . sentences in each group contain the same pair of candidate meanings for the homophone ; however , they differ on ambiguity , distinctiveness , and funniness . words that are most likely to be relevant given sentence meaning m 
 a are in boldface ; words that are most likely to be relevant given m 
 b are in italics . qualitatively , we observe that the two pun sentences ( which are significantly funnier ) have more distinct and balanced sets of semantically relevant words for each sentence meaning than other sentences in their groups . nonpun sentences tend to have no words in support of the meaning that was not observed . furthermore , the boldfaced and italicized words in each pun sentence are what one might intuitively use to explain why the sentence is funny  for example , the fact that magicians tend to perform magic tricks with hares , and people tend to be described as pulling out their hair when angry . semantically relevant words , ambiguity / distinctiveness scores , and funniness ratings for sentences from each category . words in boldface are semantically relevant to m 
 a ; words in italics are semantically relevant to m 
 b in this paper , we presented a simple model of gistlevel sentence processing and used it to derive formal measures that predict human judgments of humor in puns . we showed that a noisy channel model of sentence processing facilitates flexible context selection , which enables a single series of words to express multiple meanings . our work is one of the first to integrate a computational model of sentence processing to analyze humor in a manner that is both intuitive and quantitative . in addition , it is the first computational work to our knowledge to go beyond classifying humorous versus regular sentences to predict finegrained funniness judgments within humorous stimuli . the idea of deriving measures of humor from a model of general language understanding is closely related to previous approaches , where humor is analyzed within a framework of semantic theory and language comprehension . raskin 's ( 1985 ) semantic script theory of humor ( ssth ) builds upon a theory of language comprehension in which language is understood in terms of scripts . under this analysis , a text is funny when it activates two scripts that are incompatible with each other . this theory explains a number of classic jokes where the punch line introduces a script that is incongruous with the script activated by the joke 's setup . attardo and raskin ( 1991 ) proposed a revision to ssth in the general theory of verbal humor ( gtvh ) , which details six hierarchically organized knowledge resources that inform the understanding of texts as well as the detection of humor . nirenburg and raskin ( 2004 ) further formalized the ideas proposed in ssth and gtvh by developing a system for computational semantics termed ontological semantics , which includes a large concept ontology , a repository of facts , and an analyzer that translates texts into an ontologybased knowledge representation . this system provides rich ontological knowledge to support indepth language comprehension and has been applied productively to a variety of domains ( beale , lavoie , mcshane , nirenburg , & korelsky , 2004 ; nirenburg & raskin , 2004 ; taylor , raskin , & hempelmann , 2011 ) . hempelmann , raskin , and triezenberg ( 2006 ) used a classic joke to show that an extension to the ontological semantics system can in principle detect as well as generate humorous texts . however , to our knowledge the system has not yet been tested on a larger body of texts to demonstrate its performance in a quantitative manner ( raskin , 2008 ; taylor , 2010 ) . while providing detailed analyses that reveal many important characteristics of humor , much of the work on formalizing humor theories falls short of predicting people 's finegrained judgments of funniness for a large number of texts ( attardo , hempelmann , & di maio , 2002 ; attardo , 2001 ; brne , feyaerts , & veale , 2006 ; hempelmann , 2004 ; raskin & attardo , 1994 ; ritchie , 2001 ; veale , 2006 ) . in this regard , we believe that our work advances the current state of formal approaches to humor theory . by implementing a simple but psychologically motivated computational model of sentence processing , we derived measures that distinguish puns from regular sentences and correlate significantly with finegrained humor ratings within puns . our approach also provides an intuitive but automatic way to identify features that make a pun funny . this suggests that a probabilistic model of general sentence processing ( even without the support of rich ontological semantics ) may enable powerful explanatory measures of humor . in addition to advancing computational approaches , our work contributes to cognitive theories of humor by providing evidence that different factors may account for separate aspects of humor appreciation . some humor theorists argue that while incongruity is necessary for humor , resolving incongruity  discovering a cognitive rule that explains the incongruity in a logical manner  is also key ( ritchie , 1999 , 2009 ; suls , 1972 , 1983 ) . we can construe our measures as corresponding roughly to incongruity and resolution in this sense , where ambiguity represents the presence of incongruous sentence meanings , and distinctiveness represents the degree to which each meaning is strongly supported by different parts of the stimulus . our results would then suggest that incongruity distinguishes humorous input from regular sentences , while the intensity of humor may depend on the degree to which incongruity is resolved by focusing on two different supporting sets of contexts . future work could more specifically examine the relationship between incongruity resolution and the measures presented in our framework . although our task in this paper was limited in scope , it is a step toward developing computational models that explain higher order linguistic phenomena such as humor . to address more complex jokes , future work may incorporate more sophisticated models of language understanding , for example to consider the time course of sentence processing ( kamide , altmann , & haywood , 2003 ; mcrae , spiveyknowlton , & tanenhaus , 1998 ) , effects of pragmatic reasoning and background knowledge ( kao , bergen , & goodman , 2014 ; kao , wu , bergen , & goodman , 2014 ) , and multisentence discourse ( chambers & jurafsky , 2008 ; polanyi , 1988 ) . our approach could also benefit greatly from the rich commonsense knowledge encoded in the ontological semantics system ( nirenburg & raskin , 2004 ) and may be combined with it to measure ambiguity and distinctiveness at the script level rather than only at the level of the sentence . previous research on creative language use such as metaphor , idioms , and irony has contributed a great deal to our understanding of the cognitive mechanisms that enable people to infer rich meanings from sparse and often ambiguous linguistic input ( gibbs & o'brien , 1991 ; lakoff & turner , 2009 ; nunberg , sag , & wasow , 1994 ; ricoeur , 2003 ) . we hope that our work on humor contributes to theories of language understanding to account for a wider range of linguistic behaviors and the social and affective functions they serve . by deriving the precise properties of sentences that make us laugh , our work brings us one step closer toward understanding that funny thing called humor ( pun intended ) .
abstracthumor plays an essential role in human interactions . 
 precisely what makes something funny , however , remains elusive . 
 while research on natural language understanding has made significant advancements in recent years , there has been little direct integration of humor research with computational models of language understanding . in this paper , we propose two informationtheoretic measures  ambiguity and distinctiveness  derived from a simple model of sentence processing . 
 we test these measures on a set of puns and regular sentences and show that they correlate significantly with human judgments of funniness . 
 moreover , within a set of puns , the distinctiveness measure distinguishes exceptionally funny puns from mediocre ones . 
 our work is the first , to our knowledge , to integrate a computational model of general language understanding and humor theory to quantitatively predict humor at a finegrained level . 
 we present it as an example of a framework for applying models of language processing to understand higher level linguistic and cognitive phenomena .