With Nate Silver’s move to ESPN in the news, I think it’s striking that most people still don’t really understand the source of his election forecasting success. But when you look at it, with all due respect to Silver, his ability to beat the armchair analysis of the TV pundits is much more a story about the TV pundits being morons than it is a story about Silver having an amazingly innovative analytic method.
Or alternatively, Silver’s amazing analytic innovation was to just do the obvious thing. To see what I mean, contrast Silver’s 2012 election forecasting track-record with that of Princeton’s Sam Wang. Silver got 50 states out of 50 correct in the presidential election. So did Wang. Silver got got every Senate race right except for North Dakota. Wang got every Senate race right. How did Wang do so well? It’s simple—he aggregated the available public polls and averaged them. How did Silver do so well? It’s also simple—he did the exact same thing. How did Silver end up doing slightly worse than Wang? Well, Silver added a few bells and whistles onto the simplistic poll averaging method and in the particular case of North Dakota that led to a wrong conclusion. Now of course if you understand why forecasting is hard you’d know it’d be dumb to conclude from this one example that Silver’s methods are worse than Wang’s. His more complicated model strikes me as fairly well-justified in theory, and ideally what we’d do is re-run the 2012 election a hundred times and see what shakes out.
But here’s the point. We had three approaches to the 2012 election:
— The polls are right.
— Nate Silver’s quantitative analysis.
— TV pundits pull guesses out of their asses.
The outcome conclusively demonstrates that option number three is inferior to options one and two, but it also seems clear that if Silver’s model represents an advance on simply looking at the polls it’s not a very large advance.
Now on my grumpy days I think to myself Nate Silver is overrated his election forecasting methods are superior to being a total moron but that’s an awfully low bar. On my non-grumpy days, I recall that the moron method of election forecasting had dominated the media landscape for years and that mainstream news media continues to insist on methodologically absurd conventions like exclusive reliance on in-house polls rather than aggregation and averaging. But most of all, from a non-grumpy viewpoint Silver’s success is a reminder that journalism is about more than getting the answers right. He’s a fantastic and engaging writer, who not only came up with an election forecasting method that far outpaces the TV pundits but more impressively he found a large audience for it. After all, even though the TV pundits’ methods are totally wrong and arbitrary they don’t do what they do for no reason. The idea is that it makes good television. And you don’t crowd out terrible analysis just by doing better analysis, you have to find the better analysis and find a way to make it compelling to people. That’s what Nate Silver accomplished.
