A year ago, Yelp released a list of the “100 best places to eat in America,” based on customer reviews from its millions of users. The result, as I wrote at the time, was a delightfully democratic and refreshingly random list that seated haute-cuisine stalwarts like New York City’s Le Bernardin (No. 45) next to greasy crowd-pleasers like Los Angeles’ El Chato Taco Truck (No. 46). No. 1, improbably, was Da Poke Shack, a cramped seafood counter in a condominium complex in Hawaii.
Last week, Yelp followed up with its second annual top-100 list. It calculated the rankings a little differently this time, giving extra weight to reviews from the past calendar year. That shook up the list more drastically than you might think: Not one of last year’s top 10 made this year’s top 10. Da Poke Shack fell all the way to 51, and a new “top-rated restaurant in America” was crowned: Copper Top BBQ.
Where will you find our great nation’s new top-rated restaurant? Next door to Ray’s Gun & Tackle shop on North Main Street in the global culinary mecca of Big Pine, California (pop. 1,756, elev. 3,989). The menu at Copper Top boasts such delicacies as Adam’s Favorite Pulled Pork sandwich ($8.25), the Saint Louis-Style Pork Ribs combo meal ($13.73), and Granny’s Potato Salad ($3.66), A 20-ounce soda is $1.83 and comes with free refills and “HIGH SIERRA CRUSHED ICE,” which must really be something to merit the all-caps treatment. According to its Yelp page, Copper Top has received an average of five stars out of five on 224 reviews.
Here’s the rest of the top 10:
2. Art of Flavors – Las Vegas
3. Soho Japanese Restaurant – Las Vegas
4. TKB Bakery and Deli – Indio, California
5. Ono Seafood – Honolulu, Hawaii
6. Shark Pit Maui – Lahaina, Hawaii
7. Gaucho Parilla Argentina – Pittsburgh
8. Bobboi Natural Gelato – La Jolla, California
9. Golden Bear Trading Company – San Francisco
10. Little Miss BBQ – Phoenix
It would be easy to dismiss a list like this as meaningless. Yelp reviews, in which customers rate businesses on a subjective five-star scale based on their experiences there, are far from scientific. Collectively they suffer from sampling bias, and individual reviews can exhibit just about any type of bias in the book. The restaurant ratings that really count, you might think, are the ones based on reviews by the experts at Michelin, Gayot, or the New York Times.
But to disregard online ratings would be folly. Professional food critics have their biases, too. What makes a list like Yelp’s worthwhile is that its biases are quite different from the ones we’re used to. And these days Yelp reviews carry more weight than any offline guidebook, except for the tiny fraction of world-class restaurants that receive Michelin stars. Fair or not, studies have found that a one-star difference in Yelp ratings can swing a restaurant’s fortunes.
More broadly, over the past few years customer reviews have begun to reshape industries ranging far beyond restaurants. They influence what we buy on Amazon, whose Uber car we hop into, and where we stay when we go on vacation. Like it or not, online reviews matter—so it’s increasingly important that we understand what they’re really telling us, and what they might obscure.
A lot of attention is paid to the fact that customer reviews can be gamed or skewed by fake reviews. But that’s unlikely to be much of a factor in Yelp’s top 100 lists, which cull only the very highest-rated and most-reviewed establishments from a massive pool of data. Instead, what you see when you dive into the rankings are subtler forms of bias at work.
What’s fascinating about Yelp’s 2015 list is that, despite all the new names, the types of restaurants that populate the top 100 feel eerily familiar to last year’s. (For reference, last year’s full list is here.) Da Poke Shack is no longer in the top 10, but two other Hawaiian seafood joints that serve their food on Styrofoam and paper plates are. (Congrats to fourth-ranked Ono Seafood in Honolulu and fifth-ranked Shark Pit Maui in Lahaina.) And No. 11 this year is a third Hawaiian restaurant called Dat Cajun Guy, which sounds so much like Da Poke Shack that I did a double-take to make sure it wasn’t a misprint.
As for this year’s No. 1? Well, it isn’t in Hawaii, and its name doesn’t start with the letters “Da.” But it’s a revealing choice nonetheless.
At first glance, Copper Top has several things in common with Da Poke Shack: low prices, a casual atmosphere, and unpretentious fare. Nor is the ascension of a barbeque restaurant to the top spot a surprise: Last year’s top 20 featured no fewer than four barbeque joints, including Oklahoma Joe’s Barbecue in Kansas City (No. 3) and Franklin Barbecue in Austin, Texas (No. 8). I have not eaten at Copper Top, but I have little doubt that these restaurants also share a consistently high quality of food for the money.
On Yelp, that’s usually a recipe for a four-star rating. Compared to professional critics, Yelp reviewers skew young and budget-conscious, which is part of the site’s appeal. By and large, they’re happier paying $8 for a very good burrito than $23 for a fancy one, and the ratings reflect that. In most cases, however, solid value alone is not enough for the near-perfect score required to make Yelp’s national top-100 list. That takes something extra—and it has little to do with culinary excellence. Here’s a sampler platter of individual reviews Copper Top has received in the past few months:
- “Lots of good food at a fair price, and Hank brought us samples of his goods for us to try.”
- “Copper Top is a solid place—nay, perhaps the only place in Big Pine?—for hearty nourishment and rehydration.”
- “We stopped off here on our way to Bishop and I have to say I am so glad we did!”
These are not the raves of diners bowled over by the most stupendously impressive meal of their lives. What Copper Top’s reviews share is a sense of pleasant surprise.
So why should that make for the best-rated restaurant in America?
Part of the explanation lies in the distribution of ratings on the site’s five-star scale. Only a handful or restaurants in the world rate three Michelin stars. But more than 40 percent of all Yelp reviews are perfect scores, suggesting that five stars on Yelp entails satisfaction rather than perfection. Average hundreds of reviews of the same establishment, and you’ll find that its overall rating is influenced far more by the number of dissatisfied customers than by how much the five-star reviewers loved it. The best-rated restaurants on Yelp, then, are not so much the most loved as the least hated.
No wonder Yelp’s top 100 restaurants tend to be down-home joints specializing in distinctive cuisines like poke, barbecue, tacos, and hot dogs. Customers know exactly what they’re looking for when they go there, reducing the chances that they’ll order something unfamiliar and end up disliking it. They also know not to expect the world when they pull over at outside a roadside stand on the highway home from Lake Tahoe, or a condominium complex in Hawaii.
More surprising is the importance of timing and location. In a 2014 study reported on by the Washington Post, researchers at Georgia Tech and Yahoo Labs found that online restaurant reviews are significantly influenced by at least three factors that have nothing to do with the operation of the business:
- Neighborhood demographics: Restaurants in neighborhoods with high education levels don’t get better reviews, but they do get more reviews. That matters, because Yelp’s top-100 rankings are based not only on average ratings, but on the number of ratings a restaurant has received. So a place with 100 five-star reviews will rank higher than one with 50.
- Time of year: Restaurants get more reviews in July and August than they do in the winter, but the average ratings in the summer months are lower.
- Weather: One of the strongest exogenous effects on restaurant ratings, according to the study, is the weather at the time of the review. As you might guess, warm temperatures and sunshine mean higher reviews. Cold temperatures or extreme heat mean lower reviews, as does precipitation of any kind. The researchers attribute this to weather’s well-documented effects on mood and memory.
Take all of this into account, and the odd similarities between Yelp’s 2014 and 2015 lists start to make a lot of sense. Both are dominated by restaurants from sunny locales like Hawaii and Southern California, while traditional culinary hotspots like New York and Chicago are under-represented. And if the researchers are right that restaurant reviews are heavily influenced by weather and mood, that could go a long way toward explaining how Da Poke Shack and Copper Top seized first place. Not only do they enjoy blessedly mild climates, they’re likely to be visited by a disproportionate share of happy vacationers. Da Poke Shack is right on the beach, while Copper Top caters to skiers on their way to Mammoth and Lake Tahoe.
None of this invalidates Yelp’s top 100 lists any more than we should discard the Michelin Guide because certain reviewers might have visited a top establishment on an off night or hold a grudge against the chef. I find them far more fun to peruse than a list of 100 restaurants I’ll never be able to afford unless I win the lottery. Just remember: Like poke, restaurant reviews are best served with a healthy dose of salt.
