Should You Trust Online Reviews? Economists Weigh In.

The search for better economic policy.
Aug. 14 2012 7:00 AM

Should You Trust Online Reviews?

Yes and no.

(Continued from Page 1)

The researchers find that, even comparing hotels under the same brand, small owners are around 10 percent more likely to get five-star reviews on TripAdvisor than they are on Expedia (relative to hotels owned by large corporations). The study also examines whether these small owners might be targeting the competition with bad reviews. The authors look at negative reviews for hotels that have competitors within half a kilometer. Hotels where the nearby competition comes from small owners have 16 percent more one- and two-star ratings than those with neighboring hotels that are owned by big companies like Pillar.

This isn’t to say that consumers are making a mistake by using TripAdvisor to guide them in their hotel reservations. Despite the fraudulent posts, there is still a high degree of concordance between the ratings assigned by TripAdvisor and Expedia. And across the Web, there are scores of posters who seem passionate about their reviews.

Consumers, in turn, do seem to take online reviews seriously. By comparing restaurants that fall just above and just below the threshold for an extra half-star on Yelp, Harvard Business School’s Michael Luca estimates that an extra star is worth an extra 5 to 9 percent in revenue. Luca’s intent isn’t to examine whether restaurants are gaming Yelp’s system, but his findings certainly indicate that they’d profit from trying. (Ironically, Luca also finds that independent restaurants—the establishments that Mayzlin et al. would predict are most likely to put up fake postings—benefit the most from an extra star. You don’t need to check out Yelp to know what to expect when you walk into McDonald’s or Pizza Hut.)

The accumulating evidence on the gaming of review sites is a reminder of the limits of an open-source approach to rating quality. It works just fine in circumstances where there’s only a modest payoff to inflating a product’s rating, or where boosting the rating would require a massive effort by the cheaters. (Increasing a film’s rating on Netflix, for example, would require a coordinated effort by scores of Netflix subscribers to have much of an effect.) The quality of online reviews is also surely increased when they are tied into social-networking platforms, which construct recommendations based on social connections and which often lift the veil of anonymity that makes cheating easier (though the social-network approach also has the potential for abuse). It’s less likely to work in situations with little accountability and scores to be settled, or in circumstances where customers have a limited knowledge of the product they’re buying. You probably wouldn’t want to pick a brain surgeon by browsing online reviews.

Advertisement

So the techno-utopians were at least half right: The Internet has helped a lot of people make better decisions about which movies to watch, where to eat, and where to sleep. But on the other side of any sophisticated system designed to produce fair and honest appraisals will be equally resourceful schemers looking to game the system. In fact, given the subversive efforts of those working to undermine honest rating systems, it’s a testament to the fundamental good nature and public spiritedness of human beings that crowdsourced evaluation doesn’t unravel completely. The only way group feedback can survive is if there is a massive counterweight of reviewers taking the time to provide honest feedback despite the complete absence of economic incentives for doing so.

Ray Fisman is a professor of economics at the Columbia Business School and co-author of The Org: The Underlying Logic of the Office. Follow him on Twitter.

  Slate Plus
Slate Picks
Dec. 19 2014 4:15 PM What Happened at Slate This Week? Staff writer Lily Hay Newman shares what stories intrigued her at the magazine this week.