Scientists devise algorithm that detects sarcasm better than humans

Think people know when you're being sarcastic? Yeah, right.

Studies show that most of us believe we are much better at communicating than we actually are, especially when interacting online. For instance, a 2005 study found that recipients correctly identified the sarcasm behind email statements only 56 percent of the time. Furthermore, the participants remained confident they were being understood even when their actual ability to convey sarcasm varied significantly between email and verbal communication.

For the past decade, data scientists have been trying to develop algorithms that can automatically detect sarcasm. Most of these programs focus solely on categorizing the text of the message to determine the emotion behind it. In at least one study, by UC Berkeley's David Bamman and the University of Washington's Noah A. Smith, computers showed an accuracy rate of 75 percent—notably better than the humans in the 2005 study.

But the researchers found they could make their algorithms even more accurate by including important contextual information about the topic being discussed, the targeted audience, and, most importantly, the author.

Bamman and Smith focused on items posted on the social networking platform Twitter, searching for tweets with the hashtags #sarcasm or #sarcastic, meaning the authors themselves displayed their intent. The scientists analyzed the tweets for sarcasm—first the text alone, and then while considering additional information, including details about the author, the audience to whom the tweet was directed, and if it was a response, the tweet to which it was responding. They found that by including such background information, their accuracy rate increased as high as 85 percent.

The element most responsible for the increase in accuracy was information about the author. Apparently, being "unverified, male, and from time zones in the United States" makes a tweeter more likely to be sarcastic. Topics most likely to be discussed with sarcasm included TV shows and art; and "users with historically negative sentiments" were more likely to be sarcastic.

Although such contextual information did not produce a big increase in accuracy, Bamman and Smith say their study points to the importance of considering that information. "This gets into what is, at heart, so difficult about recognizing sarcasm—not just for computers, but for humans as well," points out Bamman. "It just requires so much background knowledge between people to be understood."

They hope that in future studies they can refine their detector to be even more accurate. For instance, one factor they did not consider in their current experiment is that people are more likely to be sarcastic on some platforms than on others.

Bamman explains how Twitter invites a response, whereas some review sites may not. "That means there's a different kind of dynamic in place, which would make the models really hard to generalize from one domain to another." In other words, data scientists would have to factor in the type of platform being used and adjust data systems accordingly.

Bamman says sentiment analysis can be useful, for instance, when conducting an analysis of reviews on Amazon, to determine whether the reviewer actually liked a product. "One thing that can really interfere with that," he says, "is whether or not the person is being sarcastic."

Accurate sentiment analysis can also be valuable to national security. In 2014, the Secret Service posted a work order requesting analytics software that can detect sarcasm on social media—the idea being that the ability to identify sarcasm would help them discern jokes from actual emergencies.

On a lighter note, it might be nice if, when you receive an email that seems sarcastic, you could run a quick detector just to, you know, make sure.