Scientists devise algorithm that detects sarcasm better than humans

December 15, 2015 by Krissy Eliot

Think people know when you're being sarcastic? Yeah, right.

Studies show that most of us believe we are much better at communicating than we actually are, especially when interacting online. For instance, a 2005 study found that recipients correctly identified the behind email statements only 56 percent of the time. Furthermore, the participants remained confident they were being understood even when their actual ability to convey sarcasm varied significantly between email and .

For the past decade, data scientists have been trying to develop algorithms that can automatically detect sarcasm. Most of these programs focus solely on categorizing the text of the message to determine the emotion behind it. In at least one study, by UC Berkeley's David Bamman and the University of Washington's Noah A. Smith, computers showed an of 75 percent—notably better than the humans in the 2005 study.

But the researchers found they could make their algorithms even more accurate by including important contextual information about the topic being discussed, the targeted audience, and, most importantly, the author.

Bamman and Smith focused on items posted on the social networking platform Twitter, searching for tweets with the hashtags #sarcasm or #sarcastic, meaning the authors themselves displayed their intent. The scientists analyzed the tweets for sarcasm—first the text alone, and then while considering additional information, including details about the author, the audience to whom the tweet was directed, and if it was a response, the tweet to which it was responding. They found that by including such background information, their accuracy rate increased as high as 85 percent.

The element most responsible for the increase in accuracy was information about the author. Apparently, being "unverified, male, and from time zones in the United States" makes a tweeter more likely to be sarcastic. Topics most likely to be discussed with sarcasm included TV shows and art; and "users with historically negative sentiments" were more likely to be sarcastic.

Although such contextual information did not produce a big increase in accuracy, Bamman and Smith say their study points to the importance of considering that information. "This gets into what is, at heart, so difficult about recognizing sarcasm—not just for computers, but for humans as well," points out Bamman. "It just requires so much background knowledge between people to be understood."

They hope that in future studies they can refine their detector to be even more accurate. For instance, one factor they did not consider in their current experiment is that people are more likely to be sarcastic on some platforms than on others.

Bamman explains how Twitter invites a response, whereas some review sites may not. "That means there's a different kind of dynamic in place, which would make the models really hard to generalize from one domain to another." In other words, data scientists would have to factor in the type of platform being used and adjust data systems accordingly.

Bamman says sentiment analysis can be useful, for instance, when conducting an analysis of reviews on Amazon, to determine whether the reviewer actually liked a product. "One thing that can really interfere with that," he says, "is whether or not the person is being sarcastic."

Accurate sentiment analysis can also be valuable to national security. In 2014, the Secret Service posted a work order requesting analytics software that can detect sarcasm on social media—the idea being that the ability to identify sarcasm would help them discern jokes from actual emergencies.

On a lighter note, it might be nice if, when you receive an email that seems sarcastic, you could run a quick detector just to, you know, make sure.

Explore further: Do children understand irony?

Related Stories

Do children understand irony?

September 13, 2010

(PhysOrg.com) -- A new study reveals 4-year-old children understand and can even use certain types of irony. The study was published in the British Journal of Developmental Psychology by Stephanie Alexander, a PhD student ...

Empathy helps children to understand sarcasm

October 8, 2013

The greater the empathy skills of children, the easier it is for them to recognize sarcasm, according to a new study in the open-access journal Frontiers in Psychology.

How can I tell if she's lying?

November 27, 2015

Sarcasm, white lies and teasing can be difficult to identify for those with certain disorders – new video inventory developed at McGill may help

Recommended for you

Apple issues update after cyber weapon captured

August 26, 2016

Apple iPhone owners on Friday were urged to install a quickly released security update after a sophisticated attack on an Emirati dissident exposed vulnerabilities targeted by cyber arms dealers.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.