Paying online community members to write product reviews backfires badly: study

May 1, 2017
Credit: Institute for Operations Research and the Management Sciences

Online user reviews have become an essential tool for consumers who increasingly rely on them to evaluate products and services before purchase. The business models of online review platforms such as Yelp and TripAdvisor, and e-commerce sites such as Amazon and Expedia critically depend on them. Should such sites pay users to encourage them to write reviews? According to a forthcoming study in the INFORMS journal Marketing Science, a leading academic marketing journal, that is a bad idea. Paying users suppresses the number of reviews on social platforms, especially among those users who are socially well-connected and likely to be more influential.

The study, "Motivating User Generated Content: Social Connectedness Moderates the Effects of Monetary Rewards," is authored by Yacheng Sun of University of Colorado and Tsinghua University, and Xiaojing Dong and Shelby McIntyre of Santa Clara University. The authors examine user response from a sample of customers following the introduction of a monetary payment program for user reviews by a social shopping platform in China. The payment was roughly the equivalent of 25 cents per in credit for purchases from sellers affiliated with the platform. To the company's surprise, the number of user reviews declined by over 30 percent in the month after the payment program was introduced, relative to the month before. "The familiar "Law of Supply," that implies supply increases in response to higher prices, does not seem to hold true when it comes to paying for reviews on a social platform," said Sun.

The paper explores why reviews drop in response to the monetary payments. The authors conjecture that the drop in reviews could be the result of community members' concerns that their honest reviews - motivated by an intrinsic motive to either help others with relevant information or to present themselves as knowledgeable about the product or service - may now be interpreted by the community as simply driven by the less honorable extrinsic motivation of making money. If this were true, the drop in user reviews would be greater among users who had more friends on the social network, who could potentially misinterpret the user's motivation for writing reviews.

The authors empirically test their conjecture by comparing the change in reviewing behavior among "socialites" (more than five friends on network) against "loners" (no friends on network) after the introduction of the payment scheme. Indeed, the reviews from socialites drop 85 percent, from just over 0.4 reviews a month to just under 0.06 reviews a month. In contrast, the loners who had little to lose in terms of social capital increase their reviews from close to zero to about 0.03 per month. The increase in reviews from the loners however does little to offset the massive drop among the socialites, who are the heavy contributors overall. Hence the aggregate drop of 30 percent.

"Nobody wants to be seen as a paid shill for brands, so the users with more friends and followers, who were likely more influential and wrote more originally, are the ones who stop writing. A real double-whammy," said Dong.

"Our results support the approach of industry leaders like Yelp or Amazon, who do not compensate for reviews. In fact, they tap into the intrinsic motive for social recognition through status badges for frequent contributors," said McIntyre. "There may be still 'under the radar' ways to pay only the less socially active users for their reviews, but such targeting can be risky as the heavy reviewers may perceive it to be unfair and therefore stop writing reviews, if and when they learn about it."

Explore further: Yelp to alert consumers on fake reviews

More information: Yacheng Sun et al. Motivation of User-Generated Content: Social Connectedness Moderates the Effects of Monetary Rewards, Marketing Science (2017). DOI: 10.1287/mksc.2016.1022

Related Stories

Recommended for you

Plague likely a Stone Age arrival to central Europe

November 22, 2017

A team of researchers led by scientists at the Max Planck Institute for the Science of Human History has sequenced the first six European genomes of the plague-causing bacterium Yersinia pestis dating from the Late Neolithic ...

How to cut your lawn for grasshoppers

November 22, 2017

Picture a grasshopper landing randomly on a lawn of fixed area. If it then jumps a certain distance in a random direction, what shape should the lawn be to maximise the chance that the grasshopper stays on the lawn after ...

Ancient barley took high road to China

November 21, 2017

First domesticated 10,000 years ago in the Fertile Crescent of the Middle East, wheat and barley took vastly different routes to China, with barley switching from a winter to both a winter and summer crop during a thousand-year ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.