Should You Trust Online Reviews? Economists Weigh In.

The search for better economic policy.
Aug. 14 2012 7:00 AM

Should You Trust Online Reviews?

Yes and no.

(Continued from Page 1)

The researchers find that, even comparing hotels under the same brand, small owners are around 10 percent more likely to get five-star reviews on TripAdvisor than they are on Expedia (relative to hotels owned by large corporations). The study also examines whether these small owners might be targeting the competition with bad reviews. The authors look at negative reviews for hotels that have competitors within half a kilometer. Hotels where the nearby competition comes from small owners have 16 percent more one- and two-star ratings than those with neighboring hotels that are owned by big companies like Pillar.

This isn’t to say that consumers are making a mistake by using TripAdvisor to guide them in their hotel reservations. Despite the fraudulent posts, there is still a high degree of concordance between the ratings assigned by TripAdvisor and Expedia. And across the Web, there are scores of posters who seem passionate about their reviews.

Consumers, in turn, do seem to take online reviews seriously. By comparing restaurants that fall just above and just below the threshold for an extra half-star on Yelp, Harvard Business School’s Michael Luca estimates that an extra star is worth an extra 5 to 9 percent in revenue. Luca’s intent isn’t to examine whether restaurants are gaming Yelp’s system, but his findings certainly indicate that they’d profit from trying. (Ironically, Luca also finds that independent restaurants—the establishments that Mayzlin et al. would predict are most likely to put up fake postings—benefit the most from an extra star. You don’t need to check out Yelp to know what to expect when you walk into McDonald’s or Pizza Hut.)

The accumulating evidence on the gaming of review sites is a reminder of the limits of an open-source approach to rating quality. It works just fine in circumstances where there’s only a modest payoff to inflating a product’s rating, or where boosting the rating would require a massive effort by the cheaters. (Increasing a film’s rating on Netflix, for example, would require a coordinated effort by scores of Netflix subscribers to have much of an effect.) The quality of online reviews is also surely increased when they are tied into social-networking platforms, which construct recommendations based on social connections and which often lift the veil of anonymity that makes cheating easier (though the social-network approach also has the potential for abuse). It’s less likely to work in situations with little accountability and scores to be settled, or in circumstances where customers have a limited knowledge of the product they’re buying. You probably wouldn’t want to pick a brain surgeon by browsing online reviews.

Advertisement

So the techno-utopians were at least half right: The Internet has helped a lot of people make better decisions about which movies to watch, where to eat, and where to sleep. But on the other side of any sophisticated system designed to produce fair and honest appraisals will be equally resourceful schemers looking to game the system. In fact, given the subversive efforts of those working to undermine honest rating systems, it’s a testament to the fundamental good nature and public spiritedness of human beings that crowdsourced evaluation doesn’t unravel completely. The only way group feedback can survive is if there is a massive counterweight of reviewers taking the time to provide honest feedback despite the complete absence of economic incentives for doing so.

TODAY IN SLATE

Politics

Don’t Worry, Obama Isn’t Sending U.S. Troops to Fight ISIS

But the next president might. 

The Extraordinary Amicus Brief That Attempts to Explain the Wu-Tang Clan to the Supreme Court Justices

Amazon Is Officially a Gadget Company. Here Are Its Six New Devices.

The Human Need to Find Connections in Everything

It’s the source of creativity and delusions. It can harm us more than it helps us.

How Much Should You Loathe NFL Commissioner Roger Goodell?

Here are the facts.

Altered State

The Plight of the Pre-Legalization Marijuana Offender

What should happen to weed users and dealers busted before the stuff was legal?

Surprise! The Women Hired to Fix the NFL Think the NFL Is Just Great.

You Shouldn’t Spank Anyone but Your Consensual Sex Partner

Moneybox
Sept. 17 2014 5:10 PM The Most Awkward Scenario in Which a Man Can Hold a Door for a Woman
  News & Politics
Altered State
Sept. 17 2014 11:51 PM The Plight of the Pre-Legalization Marijuana Offender What should happen to weed users and dealers busted before the stuff was legal?
  Business
Business Insider
Sept. 17 2014 1:36 PM Nate Silver Versus Princeton Professor: Who Has the Right Models?
  Life
Outward
Sept. 17 2014 6:53 PM LGBTQ Luminaries Honored With MacArthur “Genius” Fellowships
  Double X
The XX Factor
Sept. 17 2014 6:14 PM Today in Gender Gaps: Biking
  Slate Plus
Slate Fare
Sept. 17 2014 9:37 AM Is Slate Too Liberal?  A members-only open thread.
  Arts
Brow Beat
Sept. 17 2014 8:25 PM A New Song and Music Video From Angel Olsen, Indie’s Next Big Thing
  Technology
Future Tense
Sept. 17 2014 9:00 PM Amazon Is Now a Gadget Company
  Health & Science
Medical Examiner
Sept. 17 2014 11:48 PM Spanking Is Great for Sex Which is why it’s grotesque for parenting.
  Sports
Sports Nut
Sept. 17 2014 3:51 PM NFL Jerk Watch: Roger Goodell How much should you loathe the pro football commissioner?