Banning words on Instagram doesn't help—it makes it worse

March 9, 2016
Stevie Chancellor is a doctoral student in Human Centered Computing.

A new Georgia Tech study finds that Instagram's decision to ban certain words commonly used by pro-eating disorder (pro-ED) communities has produced an unintended effect. The use of those terms decreased when they were censored in 2012. But users adapted by simply making up new, almost identical words, driving up participation and support within pro-ED groups by as much as 30 percent. The Georgia Tech researchers found that these communities are still very active and thriving despite Instagram's efforts to moderate discussion of the dangerous lifestyle.

People in pro-ED communities share content, and provide advice and support for those who choose eating disorders, such as anorexia or bulimia, as acceptable and reasonable ways of living. They use specific hashtags to form very connected groups, often using anonymous names to keep their lifestyle choice a secret from the families and friends.

Instagram banned some of the most common pro-ED tags four years ago. People can still post these censored , but the no longer show up in search results. Banned examples include "thighgap," "thinspiration" and "secretsociety." Other pro-ED words received advisories. They can be searched, but notifications about graphic content were added, along with public service links for people looking for help.

The Georgia Tech researchers looked at 2.5 million pro-ED posts from 2011 to 2014 to study how the community reacted to Instagram's content moderation.

"People pretty much stopped using the banned terms, but they gamed the system to stay in touch," said Stevie Chancellor, a doctoral student who led the study. "'Thinspiration' was replaced by 'thynspiration' and 'thynspo.' 'Thighgap' became 'thightgap' and 'thygap.'"

The 17 moderated terms morphed into hundreds of similar, new words. Each had an average of 40 variables. Some had more: the researchers found 107 variables of "thighgap."

Instagram's censorship polarized the pro-ED community.

"Likes and comments on these new tags were 15 to 30 percent higher compared to the originals," said Munmun De Choudury, assistant professor in Georgia Tech's School of Interactive Computing. "Before the ban, a person searching for hashtags would only find their intended word. Now a search produces dozens of similar, non-censored pro-ED terms. That means more content to view and engage with."

The team also found that the content on these so-called lexical variants discussed self-harm, isolation and thoughts of suicide more often than the larger community of sufferers of eating disorders.

Instagram has also blacklisted words related to sex, racism and self-harm. What is more effective than banning tags? The Georgia Tech team suggests a few alternatives.

"Allow them to be searchable. But once they're selected, the landing page could include links for help organizations," said Chancellor. "Maybe the search algorithms could be tweaked. Instead of similar terms being displayed, Instagram could introduce recovery-related terms in the search box."

The study, "#thyghgapp: Instagram Content Moderation and Lexical Variation in Pro-Eating Disorder Communities," was presented at the ACM Conference on Computer-Supported Cooperative Work and Social Computing on March 1 in San Francisco.

Explore further: User revolt causes Instagram to keep old rules

Related Stories

User revolt causes Instagram to keep old rules

December 21, 2012

Instagram on Thursday tried to calm a user rebellion by nixing a change that would have given the Facebook-owned mobile photo sharing service unfettered rights to people's pictures.

Instagram images no longer just squares

August 27, 2015

Instagram on Thursday broke its square mold with an update that adds portrait and landscape formats to the image-sharing smartphone application.

Excessive mortality observed in anorexia nervosa

January 23, 2016

(HealthDay)—Mortality is increased among patients with eating disorders, with higher mortality for those with anorexia nervosa (AN) compared with bulimia nervosa (BN), binge eating disorder (BED), and eating disorder not ...

Recommended for you

Where can I buy a chair like that? This app will tell you

August 23, 2016

If you think you have a knack for interior design, or just want to spruce up your own home, new technology developed by Cornell researchers may help you choose furnishings the way professionals do. And professionals may find ...

Sponge creates steam using ambient sunlight

August 22, 2016

How do you boil water? Eschewing the traditional kettle and flame, MIT engineers have invented a bubble-wrapped, sponge-like device that soaks up natural sunlight and heats water to boiling temperatures, generating steam ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.