Politics

How Nate Silver Missed Donald Trump

The election guru said Trump had no shot. Where did he go wrong?

silver trump.

Polls whiz kid Nate Silver and presidential candidate Donald Trump.

Photo illustration by Slate. Images by Slaven Vlasic/Getty Images and Ethan Miller/Getty Images.

For the past six months, one big question has loomed over the 2016 election: Is the candidacy of Donald J. Trump an amusing bit of reality TV or a terrifying and dangerous challenge to the country’s political system? At first, Trump’s popularity was easy to dismiss. It was nothing more than a phase, the result of Trump’s celebrity status and his talent for provocation. His antics made it hard to look away, but it was easy to convince yourself that Trump mania would never lead to anything serious, like the Republican nomination.

It was especially easy to come to that conclusion if you were reading FiveThirtyEight, the statistics-driven news website founded by Nate Silver. Since the beginning of Trump’s campaign last June, the election guru and his colleagues have been consistently bearish on Trump’s chances. Silver, who made his name by using cold hard math to call 49 out of 50 states in the 2008 general election and all 50 in 2012, has served as a reassuring voice in the midst of Trump’s shocking rise. For those of us who didn’t want to believe we lived in a country where Donald Trump could be president, Silver’s steady, level-headed certainty felt just as soothing as his unwavering confidence in Barack Obama’s triumph over Mitt Romney four years ago.

What exactly has Silver been saying? In September, he told CNN’s Anderson Cooper that Trump had a roughly 5-percent chance of beating his GOP rivals. In November, he explained that Trump’s national following was about as negligible as the share of Americans who believe the Apollo moon landing was faked. On Twitter, he compared Trump to the band Nickelback, which he described as being “[d]isliked by most, super popular with a few.” In a post titled “Why Donald Trump Isn’t A Real Candidate, In One Chart,” Silver’s colleague Harry Enten wrote that Trump had a better chance of “playing in the NBA Finals” than winning the Republican nomination.

Multiple times over the past six months, Silver has reminded his readers that four years ago, daffy fly-by-nighters like Herman Cain and Michele Bachmann led the GOP field at various points. Trump’s poll numbers, he wrote, would drop just like theirs had. In one August post, “Donald Trump’s Six Stages of Doom,” Silver actually laid out a schedule for the candidate’s inevitable collapse.

That collapse is running late. Here we are, a few days from the Iowa caucus, and Trump’s poll numbers haven’t gone down at all. The latest data suggest that he leads his closest rival, Ted Cruz, by about 5 points in Iowa and almost 20 points in New Hampshire. He has also recently become the top GOP contender according to the betting market Betfair. Meanwhile, members of the so-called GOP establishment, who previously expressed open contempt for Trump, now seem to be warming to him. On Jan. 16, the Washington Post quoted the former finance chairman for Mitt Romney’s 2012 campaign saying there was a “growing feeling” among many in the GOP that Trump “may be the guy.” Bob Dole praised Trump in the New York Times as a dealmaker who has the “right personality” to do business with Congress. Orrin Hatch, the most senior Republican in the Senate, told CNN he was “coming around” on Trump.

It’s clear, now, that Silver and his fellow analysts at FiveThirtyEight underestimated Trump. Silver himself recently admitted as much, writing in a blog post published last week that he’d been too skeptical about Trump’s chances. “Things are lining up better for Trump than I would have imagined,” he wrote, adding that “[i]f, like me, you expected” the show to have been over by now, “you have to revisit your assumptions.”

Everyone makes mistakes—even Nate Silver. It’s also entirely possible that the Trump collapse is still to come and that as soon as we see the actual voting process play out, the hollowness of his popularity will reveal itself. Still, Silver is right that his assumptions are worth revisiting. Maybe the Trump phenomenon is so unprecedented that no statistical model could have foreseen it. Or maybe it took a candidate as unique as Donald Trump to reveal the flaws and limitations of Silver’s prediction machine.

* * *

To understand how Silver got Trump wrong, it helps to understand what exactly he was skeptical about, and why. A look at his campaign coverage reveals that two basic beliefs guided Silver’s thinking.

The first centered on the polls showing Trump miles ahead of his rivals. These polls have been plentiful, and they have been consistent. To pick two more or less at random, CNN showed Trump’s support in Iowa grow from 22 percent in August to 37 percent this past week. According to national polls conducted by CBS and the New York Times, he has gone from polling at 24 percent nationally in August to 36 percent earlier this month.

None of this has impressed Silver. No matter what the polls said, as he wrote on FiveThirtyEight week after week, it was important to remember they were fundamentally unreliable and not at all indicative of how primary voters would ultimately cast their ballots. This has always been true of pre-primary polls, Silver argued, in part because primary voters have historically waited until the last minute to decide whom to support and in part because the people answering questions from pollsters are not necessarily the ones who will end up actually voting.

Anything Silver says about polling carries weight. Polls are his bread and butter—the raw materials he filters through his proprietary model to predict the outcomes of elections. His expertise on which polls to ignore, which ones to trust, and how much to trust them is central to his political wisdom. The early national polls showing Trump in the lead, Silver wrote, were basically worthless. As he put it in a post titled “Donald Trump Is Winning The Polls—and Losing the Nomination,” they not only “lack empirical power to predict the nomination” but “describe a fiction.”

Silver thought that it was foolish of reporters and columnists to act like Trump’s numbers were significant. The fact that pundits insisted on investing them with so much importance proved they were motivated more by the demands of the news cycle than by a commitment to truth—a tendency Silver has always taken pride in avoiding.   

The problem, Silver believed, wasn’t just that the media legitimized polls that didn’t deserve people’s attention. It was worse than that: By talking about Trump’s poll numbers like they mattered, the media risked distorting future polls, thereby reinforcing the false narrative of Trump’s dominance. “Some voters may be coughing up Trump’s name in polls because he’s the only candidate they’ve been hearing about,” Silver wrote in December, noting that the media has given Trump’s campaign “more coverage than literally all the other Republicans combined.”

Nate Silver.
Nate Silver at the Tribeca Film Festival, April 20, 2015, in New York City.

Photo by Dave Kotinsky/Getty Images

Silver’s error, in retrospect, was to conflate his doubts about the polls with his doubts about Trump’s viability as a candidate. In other words, it’s perfectly possible for Silver to have been correct in saying the early polls did not constitute proof of a massive Trump lead, while also being wrong about the likelihood that Trump would become the nominee. This mistake is illustrated most clearly in that post headlined “Donald Trump’s Six Stages of Doom,” in which Silver asserted that “Trump’s campaign will fail by one means or another” before ticking off a bunch of reasons to be suspicious of the early polls that showed him in the lead. While the post made brief mention of Trump’s “poor organization in caucus states, poor understanding of delegate rules,” and his lack of “support from superdelegates,” it didn’t offer much on why Silver found it so unlikely that lots and lots of people would vote for him.

This brings us to the second basic belief guiding Silver’s skepticism about Trump mania. Polls aside, the history of modern American politics made it clear to him that a “Trump-like candidate” could never win the nomination.

What is a “Trump-like candidate”? Under Silver’s definition, it’s someone who has low favorability ratings and, more important, is hated by party leaders. Citing a theory laid out in The Party Decides—an influential work of political science which says that primary candidates don’t win without the support of the party establishment—Silver has argued that Trump was an almost certain loser. Even if Trump managed to survive until the Republican National Convention, Silver wrote, “the Republican Party would go to extraordinary lengths to avoid nominating him.”

The race has not played out that way. Indications in December that GOP leaders were either powerless against Trump or unwilling to go after him struck Silver as “perplexing”—precisely the emotion you would expect from a quantitatively inclined thinker confronted with a reality that hasn’t conformed to his calculations. In a chat with colleagues published on FiveThirtyEight, Silver discussed possible reasons why GOP leaders had not been more aggressive in snuffing out Trump earlier, when he might have been an easier target. “From the get-go, they haven’t seemed to have any plan at all for how to deal with Trump,” he wrote.  

What’s crucial to note here is that Silver’s confidence about how the GOP would respond to Trump was never really based on any statistical calculations. Rather, in repeatedly citing The Party Decides, he was relying on a theory about how political parties work—one that’s been embraced by some of the very same pundits that Silver has defined himself against. And while it’s true that The Party Decides was an empirical work based on historical data, the notion that GOP leaders would find a way to kill Trump’s campaign is, on some level, premised on a belief that the individual actors who control the Republican Party would all act as rationally as Nate Silver would if he were in their shoes. When news reports came out this month that influential Republican donors were starting to think Trump wouldn’t be such a bad candidate, Silver wrote, with some exasperation, “the donor class is probably wrong.”

Maybe what happened here is that Silver was the one sober guy in a room full of drunks, powerless to stop irrational party leaders from taking unreliable polling data seriously. Even if you believe that establishment figures are only giving Trump a look because they despise his closest rival, Ted Cruz, it’s undeniable that their thinking is being informed by Trump’s numbers. The fact that Silver thinks those numbers are silly doesn’t matter. They were consequential, and now that the Iowa caucus is one week away, those consequences are becoming more and more serious. 

Why was Silver so confident that the “party decides” theory would hold? One reason, surely, is that it always had in the past—if you want a recent example, think back to the mavericks of the 2012 GOP contest, who were squashed like bugs until party favorite Romney was the last man standing. But it also seems possible that Silver believed the GOP would stop Trump for a simpler reason: It was what he wanted to happen.

Silver did not respond to multiple requests for comment for this story, so the best I can do is venture a guess. Maybe, like many people who have watched Trump’s rise with increasing horror, Silver latched onto a narrative that justified rejecting the Apprentice star’s achievements, identifying them as symptoms of a media bubble rather than a reflection of real popular sentiment. If that’s the case, Silver turns out to have a good bit in common with the pundits that he and his unemotional, numbers-driven worldview were supposed to render obsolete. Faced with uncertainty, Silver chose to go all in on an outcome that felt right, one that meshed with his preexisting beliefs about how the world is supposed to work.

* * *

There is another, more narrow explanation for why Trump eluded Silver. As effective as the FiveThirtyEight approach was when applied to Obama vs. McCain and Obama vs. Romney, perhaps it just doesn’t work nearly as well when applied to primaries. If Silver’s system depends largely on interpreting poll numbers, how reliable can that system be if the pre–Iowa and New Hampshire polls are basically worthless? Garbage in, garbage out.

“I think figuring out what’s going on in a primary is more of an art than a science,” says Steve Kornacki, a political analyst at MSNBC who has been covering presidential campaigns since 2002. “There’s just so much more volatility, coming from so many different levels in a primary. And there’s a lot more art involved in figuring out what’s going on than there is in a general election, especially in an era when 80 percent of the country knows whether they’re team blue or team red and which way they vote.”

Of course, Silver knows this, and he has taken certain steps to compensate for it. In a 2,800-word blog post laying out FiveThirtyEight’s methodology for forecasting primaries, he explained how he and his team use state and national polls alongside party leader endorsements. He also left room for the possibility that FiveThirtyEight’s prediction “might be totally wrong.” “Forecasting primaries and caucuses is challenging, much more so than general elections,” Silver wrote, adding that “an unusual candidate like Donald Trump tends to have especially uncertain forecasts.”

trump.
Republican presidential candidate Donald Trump at a campaign event, Jan. 23, 2016, in Pella, Iowa.

Photo by Joshua Lott/Getty Images

Where does all that uncertainty leave FiveThirtyEight? In the months leading up to Iowa and New Hampshire, frequent Silver critic Matt Bruenig told me the FiveThirtyEight founder is “in a situation where the only thing he’s really capable of doing—the thing that he’s exceptional at—is not really available to him, so he ends up doing what normal reporters do.” Bruenig added: “It just makes him like everyone else. … Anyone can read The Party Decides and be like, ‘Oh yeah, this is what social science says will happen.’ ”

(It may not be entirely true that this GOP primary was a hopeless exercise for data journalists. Though early polls may not be the most solid data points, it’s possible that FiveThirtyEight could have done a better job interpreting them: As Kornacki points out, Silver’s insistence on comparing Trump to Joe Lieberman and Rudy Giuliani—candidates with high name recognition who led in primary polls before imploding—failed to consider that both Lieberman and Giuliani led their respective races very early, while Trump has built his lead more gradually. RealClearPolitics, which averages multiple polls, shows Trump starting at just 6 percent last July.)   

In Silver’s defense, he has occasionally given voice to self-doubt. He concluded a Jan. 6 chat with FiveThirtyEight staff by saying, “Yeah, the pundits are probably full of shit, but there’s a chance we’re full of shit too, so let’s wait and see what happens.” For the most part, though, Silver his proclaimed his skepticism about Trump loudly, repeatedly, and unequivocally. His predictions of Trump’s collapse—at an event at the 92nd Street Y in September he literally told the audience to “calm down” about his supposed march to the nomination—have not betrayed much caution or uncertainty.

As irrational as it seems for a quantitative analyst to comment so confidently on something he knows he can’t reliably predict, it’s also not all that surprising. Silver has a website to run, after all, and that means covering Trump—and making predictions about him—whether the necessary data is available or not.

The theory that Donald Trump was a real threat to the status quo was a perfect target for Silver and his colleagues. Throughout 2015 and into 2016, they set out to prove that this media sensation was being amplified by a credulous, mathematically illiterate press corps. A Trump implosion would be a classic Silver victory, one that would demonstrate the superiority of rational, data-driven analysis over the chatter of insiders and vague notions of “momentum.”  

Instead, the rise of Trump might have demonstrated the limits of Silver’s powers. As Dave Weigel wrote in the Washington Post recently, Trump’s enormous popularity—a tidal wave of support that Silver has said will soon abate—has been the story of the campaign. In his piece, Weigel argued that it wasn’t the first time a primary bid turned out to signal a major shift in the political winds, from the campaign of George Wallace in 1964, which Weigel said represented “a historic moment in the politics of backlash,” to that of Pat Robertson in 1988, which “cemented the influence of the religious right in Republican electoral politics.” While none of those candidates won their party’s nomination, it would have been irresponsible for the media to ignore the significance of their campaigns, as Silver has encouraged his audience, and the press, to do with Trump.

While it’s true that “the rise of Trump” may not end with Trump becoming the nominee, it has revealed, or perhaps even caused, a profound shift in the nation’s political climate. As Kornacki put it to me, “It took Donald Trump saying all this stuff”—floating the idea of denying Muslims entry into the United States, for instance—“to reveal there was a massive constituency for it.”

Missing the significance of Trumpism is a different kind of failure than, say, calling the 2012 election for Mitt Romney. It also might be a more damning one. Botching your general election forecast by a couple of percentage points suggests a flawed mathematical formula. Actively denying the reality of Trump’s success suggests Silver may never have been capable of explaining the world in a way so many believed he could in 2008 and 2012, when he was telling them how likely it was that Obama would become, and remain, the president.

“This is an extraordinary, unusual, utterly bizarre election year, in which events that have never happened before are happening,” says Blake Zeff, the editor of the political news site Cafe and a former campaign aide to Obama and Hillary Clinton. “That’s a nightmare scenario for a projection model that is predicated on historical trends.” While Zeff cautioned it was premature to pillory Silver for missing out on Trumpism, the point stands: What was true yesterday is not necessarily true today, and that’s a problem for Silver and his team of prognosticators.

In 2008, Silver emerged as a new kind of journalist. His data-driven approach to political analysis was a necessary corrective to a media herd that too often relied on gut feelings and received wisdom. So long as punditry continues to exist, thinkers like Silver will remain essential. But the rise of FiveThirtyEight hasn’t changed the fundamental purpose of journalism: to pay attention as the world changes and to try to understand what’s driving that change.

You could argue Silver never promised he was capable of doing those things—that all he ever intended to do was predict the future, not explain it. But Trump’s campaign, which is forcing Americans to ask themselves how such a hateful, boorish candidate could capture the imagination of so many of their fellow citizens, makes it clear that truly revelatory analysis must tell us “why,” not just “what.” If only Nate Silver could give us both.