How Amazon made star ratings more trustworthy
Removing the need for text-based reviews leads to more positive and useful product star ratings and decreases the impact of bought fake ratings.
This is according to new research by Professor Matthias D. Mahlendorf from Frankfurt School of Finance & Management, Oliver Hegers from Vrije Universiteit Amsterdam, and Fan Wu from The Chinese University of Hong Kong.
In 2019, Amazon removed written reviews as a requirement when leaving a product rating, introducing one-tap ratings of one to five stars.
Researchers investigated the impact of this policy change by following 500,000 products on Amazon week by week, 77 million observations in total, and used 50,000 books also rated on Goodreads as a comparison.
The researchers found that the previous written requirement had discouraged happier customers from leaving reviews. A year after the change, products tracked had accumulated over three times as many ratings as written reviews, and the average rating had risen from 4.17 to 4.32 stars.
This shows that customers which had previously been put off by the written requirement, on average, were more satisfied than those who had taken the trouble to write.
The research also shows that text reviews make fake ratings less effective. Using a detection method to identify products buying fake reviews, researchers found that these products had ratings one quarter of a star above comparable products before the change.
Afterwards, the gap narrowed significantly. As more authentic customers leave ratings, each purchased five-star rating has less impact on the overall average, so buying ratings becomes a worse investment.
Ratings also become more useful. After the change, Amazon ratings lined up more closely with independent product tests, and sales were more strongly influenced by a product’s rating. Reviews in which customers complained about having been misled by reviews declined from 36% to 21%.
“Our findings show that the way rating systems are designed can significantly influence which customers leave feedback and how accurately ratings reflect product quality.
“While Amazon’s decision to remove the written review requirement improved the informativeness of ratings in this case, design changes do not necessarily lead to better outcomes. Platform operators therefore play a crucial role in shaping the ratings and reviews consumers see and rely on when making purchasing decisions,” said Professor Matthias D. Mahlendorf from Frankfurt School.
