The official excuses for the bad exit poll numbers don't cut it.
Edison/Mitofsky, the co-venture that collected the exit-poll numbers and performed other heavy lifting of election data this year for big media, has taken its first stab at explaining why their figures consistently overstated Kerry support on Nov. 2.
Joe Lenski of Edison Media Research and Warren Mitofsky of Mitofsky International admit in a report they wrote that some of their exit pollsters failed to get close enough to voting places to gather adequate samples of voter opinion, and that the exit polls may have overstated Kerry support because his supporters were more willing to be surveyed than Bush supporters.
Their report, covered in today's New York Times (" Report Says Problems Led to Skewed Surveying Data," Nov. 5), "details systematic glitches that skewed the data in ways of which several news organizations, who paid tens of thousands of dollars for the service, were not aware," writes Jim Rutenberg, and "the surveys had the biggest partisan skew since at least 1988, the earliest election the report tracked."
Other exit-poll dirt was turned in yesterday's Washington Post, where reporter and poll maven Richard Morin writes that the "samples that at one point or another included too many women, too few Westerners, not enough Republicans. …"
What went wrong? Edison/Mitofsky can rightly say that nothing went wrong. The skewed exit poll data didn't cause any of the National Election Pool owners (CBS, NBC, ABC, CNN, Fox News, and the Associated Press) who commissioned the Edison/Mitofsky data to incorrectly project a winner in any contest. But Edison/Mitofsky was hired to do more than just not cause others to make mistakes. They were hired to restore confidence to the whole election projection game after Voter News Service botched the job in 2000 in Florida. VNS's goofs resulted in the networks calling the state for Gore, withdrawing the call, calling Bush the winner hours later, and then withdrawing that call.
To hear Lenski of Edison Media talk about it, the whole election brouhaha of 2004 can be blamed on the people who leaked the exit poll information and the outlets (Slate, drudgereport.com, wonkette.com, dailykos.com, mydd.com, et al.) that tossed the raw data out for consumption.
"I'm not designing polls for some blogger who doesn't even understand how to read the data," Lenski told the Los Angles Times yesterday. "It's like if you were graded by your readers on the first draft of your article."
Yet it is Lenski and the networks who are at fault for not telling viewers—and bloggers—the deeper meaning of exit poll data. The business of calling an election is as much an art as it is a science, and they've not been candid about that. In some elections, the exit-poll data is good enough that combining it with the secret sauce of computer models and calculations will produce a solid election projection before the polls close. In other cases, the exit-poll data can clash with what the wizards expected to see, and they want to wait for actual vote tabulations before predicting a winner. In other words, not all exit polls are created equal.
Lenski must know by now that the worst exploiters of exit polls are not bloggers but newscasters. The TV networks create exit-poll mania by cravenly hyping their importance in election after election while insisting that the figures stay under wraps. The election projection elite also lecture the public that the early exit-poll data is so flawed that viewers should be blocked from seeing it. But if that's the case, why is it collected and distributed to journalists in first place? Is it any wonder that the public hungers for a peek at the forbidden numbers after Dan, Peter, Tom, and Wolf hint-hint and wink-wink about them all day and evening long? For instance, in the early evening of Nov. 2, you could almost read the flattering-to-Kerry exit-poll numbers off the faces of the Crossfire gang and the Fox News Channel "All Star Panel."
I hope there is more to the Edison/Mitofsky report than what the New York Times reported, because the glitches they describe are well known within the industry. They complain that their exit pollsters couldn't get the sort of proximity they wanted to some voting places—but exit pollsters always encounter overzealous election officials enforcing electioneering laws. Can we really believe that a significant number of the 1,480 exit-poll precincts in 50 states and the District of Columbia that Edison/Mitofsky surveyed on Election Day were so affected? And in sufficient numbers to bend state-by-state exit polls in Kerry's favor?
As for the Edison/Mitofsky concession that Kerry voters might have distorted exit polls in his favor with their eagerness to complete a post-ballot questionnaire, this generic excuse was also used in the 1996 Arizona Republican presidential primary. The final exit poll gave candidate Patrick J. Buchanan 31 percent of the vote, while the actual count gave him only 27. This 4 percentage point differential exceeded the poll's margin of error by 1 percentage point. Voter News Service Editorial Director Murray Edelman attributed the error to enthusiastic Buchanan voters in a Feb. 29, 1996, Washington Post story. "Buchanan's voters are much more fired up. They're more eager to talk about it, more behind what they just did," he said. In 2000, VNS determined that in Kentucky, Bush voters were more likely to complete the exit pollster's questionnaire, according to this CBS News post-mortem from 2000. Blaming anomalous exit-poll data on enthusiastic voters seems to be a standard industry cop-out and ex-post facto reasoning at its worst.
The CBS News post-mortem should be read by everybody who cares about election coverage. It admits that broadcasters have been remiss in not educating viewers and recommends fuller disclosure about how the process works, even suggesting that the networks explain why some election calls are not made, "so that the audience knows we are not consulting a crystal ball." But why stop there? Why not release the exit poll numbers to the public at the same time they go to the news media? The whole rationale for keeping the numbers secret in the first place was that they might deter voters from casting their ballots. But after the experience of 2004, nobody is going to believe that raw exit polls are any better at predicting elections than conventional political pollsters, some of whom called the presidential election right, and some of whom called it wrong.
One conventional pollster, by the way, got the election wrong and right. John Zogby's final poll called the election for Bush. But then he switched to Kerry at 5 p.m. on Election Day. Is he blaming the exit polls, too?