In defense of exit polls.

In defense of exit polls.

In defense of exit polls.

Who's winning, who's losing, and why.
Nov. 4 2004 11:07 AM

In Defense of Exit Polls

You just don't know how to use them.

Four years ago Dan Rather famously declared that when CBS proclaims a winner on Election Night, "You can take it to the bank." Tuesday night Dan knew better than to offer up another such hostage to the gods of chance, but he might as well have. For neither he nor anyone at the other networks made a single call that was not good as gold. So, why are we hearing so much noise about "flawed" and "wildly inaccurate" exit polls in this year's election? Not because of anything heard from the people who commissioned them, who if anything were cautious to a fault.

Back in the 1960s, when the networks first began projecting winners state by state, they had to wait for real votes, even in safe states, before they made a call. In 1980, however, NBC, without any notice to the others, changed the rules. They used exit polls—scientifically sampled surveys of voters leaving their polling stations throughout Election Day—to make a bunch of calls in states where the polls closed at 8 p.m., before any votes were counted. This method enabled NBC a few minutes later to post a triumphant "Reagan elected" graphic which no competitor could match for hours. Jobs at the other networks were suddenly in jeopardy.


By the 1982 off-year elections, every network had exit polls in every state. They raised Election Night budgets to the point by the end of the decade where none of the networks could afford them on their own. Starting in 1990, they formed a succession of pool operations, the latest of which, National Election Pool, made its debut Tuesday night—and, like its predecessor, the now-defunct Voter News Service, was summarily denounced that night for getting key states wrong and making John Kerry appear the presumptive winner.

What happened?

It's important to realize that nobody involved with exit polls ever regards them as precision tools. They have many imperfections. The pollsters who work outside the polling stations often have problems with officials who want to limit access to voters. Unless the interviews have sampled the entire day's voters, the results can be demographically and hence politically skewed. Finally, it is of course a poll, not a set of actual recorded votes like those in the precinct samples collected after the polls close. People who know what they're doing do not rely on exit polls to draw conclusions about a close race. They also look at models, voter turnout, and other data, and they consult their guts.

The two men who run the current NEP pool (one of whom is a close friend) have between them half a century of making judgments on whether the margin between the candidates in a particular state exit poll is close enough to declare a winner. They have never in their lives made a mistake in doing so. In 30 years' Election Night experience of my own, I know of only one case in which a network had to retract a call that was based on exit-polling.

And indeed on Tuesday, the network experts used exit polls as they're supposed to. Exit-poll surveys in some 29 states showed margins for George Bush or John Kerry great enough to conclude that the chances the leading candidate losing was essentially zero. On that basis, when the polls closed in those states and before any votes were counted, 16 of them were placed in the president's corner and 13 in the senator's. They tended to be places like Kansas and Rhode Island.

In quite a few of other states, though, the exit polls were less conclusive. That's where the trouble came in.

About 4 p.m., the pool statisticians met with the network reps and gave them a rundown on the exit-poll surveys and indicated which ones might be called when the polls closed (the 29 above) and which ones to be wary about. The networks were advised against drawing any conclusions from the second group and to air none of them. They followed that advice.

But while these figures never got on the air, they did get around. I have a rather long list of Web sites at which you could find early survey figures from the states that the networks agreed not to go near until they had real votes. Slate, for example, had these numbers up early: