Effects of recommender systems in e-commerce vary by product attributes and review ratings

Recommender systems are used in e-commerce to guide consumers with messages like "People who purchased this item also purchased ..." Past research has shown that these systems affect consumers' choices and generally boost sales, but few studies have examined how product-specific attributes or review ratings influence the effectiveness of such systems. A new study sought to determine how the impact of recommender systems (also called recommenders) is affected by factors such as product type, attributes, and other sources of information about products on retailers' websites. The study found that recommenders increased the number of consumer views of product pages as well as the number of products consumers consider, but that the increase was moderated by product attributes and review ratings.

The study, by researchers at Carnegie Mellon University and The Wharton School, appears in Management Science.

"Our findings can guide the effective use of recommender systems in e-commerce and provide insight into consumers' purchasing behavior," says Dokyun Lee, Assistant Professor of Business Analytics at Carnegie Mellon University's Tepper School of Business, who coauthored the study. "Understanding whether and how the effectiveness of recommender systems varies across product categories and by the number of reviews can help managers better understand how best to use these systems."

Researchers conducted an experiment on an e-commerce site of a top North American retailer with 184,375 users. In the experiment, about half the users were randomly selected to receive recommendations from a purchase-based filtering algorithm ("People who bought this also bought..."), while the other half, randomly selected to be in a control group, received no recommendations. The study used Amazon Mechanical Turk, a crowd-sourcing marketplace, to code the attributes of 37,125 unique products. The researchers then analyzed the resulting dataset to determine how factors that influence the costs, uncertainty, and risk related to searching for products interact with the positive impact of recommenders on customers' views of products and their purchasing decisions.

The study found that using recommenders increased both the volume of consumers' views of products and consumers' likelihood of buying a product. A recommender's positive impact on product views was greater for utilitarian products (e.g., a hammer) than it was for hedonic products (e.g., perfume), and greater for products with characteristics that can only be discerned by use (e.g., wine, movies) than for products for which consumers can easily judge the quality by reading descriptions (e.g., computers, phones).

In contrast, a recommender's positive impact on the likelihood of buying a product was greater for hedonic products than for utilitarian products. Contrary to past research, the attribute related to use or prior experience did not influence recommenders' effect on consumers' likelihood of purchasing a product.

The authors of the study note several limitations, including the use of just one type of recommender system. Also, they did not know which products were actually recommended by the recommender and therefore could not analyze whether a specific purchase resulted from a recommendation; instead, they compared the purchase behavior of consumers in the two groups and, because of randomization, attributed the difference to the recommender. Finally, they could not determine how long the impact of a recommender system would last.

"Our results suggest that the way recommenders help boost product sales differs by type of product," explains Kartik Hosanagar, Professor of Operations, Information and Decisions at The Wharton School, who coauthored the study. "We found that recommenders' positive impact on views is greater for products with high average review ratings, suggesting that a recommender complements review ratings, while the opposite is true for conversion rates conditional on views, that is, recommender and review ratings are substitutes."