AI: In search of the sarcasm algorithm
Tue 25 Aug 2015
Sarcasm – the ‘lowest form of wit’ – promises one of the highest orders of challenge to researchers in artificial intelligence. The shortest route to the ‘sarcasm algorithm’ is empirical identification – the classification of ‘sarcasm properties’. But if that doesn’t work, computers may have to simply learn more about the world in order to develop a sense of humour…
Teaching computers how to recognise sarcasm is a more challenging task than one might think, but one that promises manifold applications. For instance, the New Yorker has been considering the possibility of getting a computer instead of interns to read cartoon contest entries.
Each issue of the New Yorker since 2005 has had a blank cartoon, for which readers submit captions, the funniest of which is then published in the next issue. They receive 5,000 entries every week, which cartoon editor Bob Mankoff’s assistant then has to read through and assess. Mankoff notes that such an overwhelming task usually burns his assistants out in two years, and then he has to get another one.
In order to address this problem, as well as to further AI research, Mankoff worked with Microsoft researchers to feed New Yorker archive cartoons and caption contest submissions into artificial intelligence software, in the hope of teaching a computer to have a sense of humour. They did this by breaking down each cartoon into two categories: context and anomalies.
Bloomberg reports, “The machine and the New Yorker editors don’t always align on shortlists. On average, though, all of the editors’ favourites appeared in the AI’s top 55.8 percent of choices, according to the study.”
Mankoff thinks that “a computer will probably never be able to beat his writers at being funny”. He’s impressed by the software’s sifting ability for humour, though he says it needs to become somewhat more accurate before he’d use it.
The etymology of sarcasm
Let’s look at sarcasm a little more abstractly, as a form of human expression.
Arguably, sarcasm is a subset of wit, which is a subset of writing, which is a way of engaging with life and its complexities. Sarcasm can be used for good or bad purposes, either aiming for something genuinely funny to amuse people, or from a place of bitterness and cynicism.
Some of the best writers make frequent use of sarcasm and irony more generally; not being sarcastic about literally everything in a nihilistic way like the Joker; but rather from a place of loving their characters, and exploring the nuances of life in a witty, empathetic way.
A single joke can work on four or five different levels beyond the obvious ‘laugh-out-loud’ category, incorporating such factors as character, character dynamics, and thematic resonance – as well as general principles of humour such as surprise, juxtaposition, and irony (of which sarcasm is a verbal form).
This is heightened by arc-based storytelling, which favours elements like character, character relationships, and themes, thus allowing any given carefully written scene or line of dialogue potentially to resonate with nuance. It’s all about context.
When sarcasm is no joke
Some potential applications of a successful sarcasm-recognition algorithm are more serious. For instance, there’s the matter of law enforcement determining whether something posted online is an actual threat intended seriously, or merely a joke. The issues at stake are public safety and civil liberties.
Being able to tell accurately whether or not a statement is sarcastic could potentially be very helpful, not only in preventing violence, but also in protecting people from misinterpretation and the consequences of that. Many of the widely reported cases where people have been prosecuted for posting certain things on social media give the impression, at least to the untrained eye, that principles such as the presumption of innocence, and the freedom of speech and freedom of expression, might not be given the weighting they deserve.
If an accurate method of detecting sarcasm were to be employed, this might help to clarify things (i.e. by making it very clear that a joke is, in fact, a joke), and thus redress the balance. But such methods are currently imperfect, and so such a system would need not merely a high degree of accuracy, but also the ability to defer to human experience and intuition, in those cases where the system falls short of accurately perceiving the nuances of sarcasm (or of any similar thing to which a similar system might be applied).
This still leaves us with the challenge of how to teach sarcasm to computers. Computer science and linguistics professor at Stanford, Christopher Manning, said, “In 2015, computers aren’t so bad at understanding language,” adding, “But they’re still pretty bad at understanding the world.”
Sarcasm as an object-oriented pursuit
In an episode of The Big Bang Theory, Leonard makes Sheldon a “sarcasm sign”, in order to denote when people are being sarcastic. Similarly, there’s an approach which essentially does the same thing for a computer. Huge amounts of data, from the likes of tweets featuring the hashtag “#sarcasm” or “#sarcastic”, are fed into self-learning pattern recognition programs, in order to find the topics, words, and phrases that tend to recur when people are being sarcastic. Mathieu Cliche’s online Sarcasm Detector is one example.
However, this approach has its shortcomings. As The Washington Post points out, typing in “I just love it when the office is quiet in the morning!” gives a high “sarcasm score” of 71 out of 100. “In other words, it’s pretty positive that I’m being sarcastic, simply because I used a statistically sarcastic word [‘just love’]; the algorithm has no way of even conceiving of a quiet office, let alone the fact that a quiet office is a good thing or that I might appreciate one.”
Understanding sarcasm, then, is not just about words per se; it’s about understanding the experience of life, in all its complexity.
Perhaps it’s a little like the difference between the French words ‘savoir’ and ‘connaitre’. ‘Savoir’ means ‘to know’, in the sense of intellectual knowledge, or knowing facts. ‘Connaitre’, on the other hand, means ‘to know’, in the sense of being really familiar with something or some situation or experience, or knowing someone.
The computational linguist David Bannam, who’s an assistant professor at the School of Information at UC Berkeley, said, “Sarcasm detection is a very difficult computation problem”.
Bannam’s latest attempt at solving this problem [PDF], along with Noah A. Smith and sponsored by the National Science Foundation, was around 85% accurate at detecting sarcasm on Twitter. The study took into account context, paying attention to things like the tweeter, the tweetee, and the relationship between them. It also drew on information from the tweeter’s profile and tweet history, in order to increase accuracy.
Interestingly, Bannam and Smith’s study noted the “important communicative function” of adding the hashtag “#sarcasm”, writing, “…in the absence of shared common ground required for their interpretation, explicit illocutionary markers are often necessary to communicate intent.” (Hence, sarcasm sign.)
However, there’s still quite a way to go. Future progress will rely on further contextualisation, to a much greater degree.
Christopher Manning said “A true sarcasm detector will need to understand people – what they like, what they think,” adding, “We’ve already made enormous advances in things like speech recognition, things we once thought of as artificial intelligence.”