Four years ago Dan Rather famously declared that when CBS proclaims a winner on Election Night, “You can take it to the bank.” Tuesday night Dan knew better than to offer up another such hostage to the gods of chance, but he might as well have. For neither he nor anyone at the other networks made a single call that was not good as gold. So, why are we hearing so much noise about “flawed” and “wildly inaccurate” exit polls in this year’s election? Not because of anything heard from the people who commissioned them, who if anything were cautious to a fault.
Back in the 1960s, when the networks first began projecting winners state by state, they had to wait for real votes, even in safe states, before they made a call. In 1980, however, NBC, without any notice to the others, changed the rules. They used exit polls—scientifically sampled surveys of voters leaving their polling stations throughout Election Day—to make a bunch of calls in states where the polls closed at 8 p.m., before any votes were counted. This method enabled NBC a few minutes later to post a triumphant “Reagan elected” graphic which no competitor could match for hours. Jobs at the other networks were suddenly in jeopardy.
By the 1982 off-year elections, every network had exit polls in every state. They raised Election Night budgets to the point by the end of the decade where none of the networks could afford them on their own. Starting in 1990, they formed a succession of pool operations, the latest of which, National Election Pool, made its debut Tuesday night—and, like its predecessor, the now-defunct Voter News Service, was summarily denounced that night for getting key states wrong and making John Kerry appear the presumptive winner.
What happened?
It’s important to realize that nobody involved with exit polls ever regards them as precision tools. They have many imperfections. The pollsters who work outside the polling stations often have problems with officials who want to limit access to voters. Unless the interviews have sampled the entire day’s voters, the results can be demographically and hence politically skewed. Finally, it is of course a poll, not a set of actual recorded votes like those in the precinct samples collected after the polls close. People who know what they’re doing do not rely on exit polls to draw conclusions about a close race. They also look at models, voter turnout, and other data, and they consult their guts.
The two men who run the current NEP pool (one of whom is a close friend) have between them half a century of making judgments on whether the margin between the candidates in a particular state exit poll is close enough to declare a winner. They have never in their lives made a mistake in doing so. In 30 years’ Election Night experience of my own, I know of only one case in which a network had to retract a call that was based on exit-polling.
And indeed on Tuesday, the network experts used exit polls as they’re supposed to. Exit-poll surveys in some 29 states showed margins for George Bush or John Kerry great enough to conclude that the chances the leading candidate losing was essentially zero. On that basis, when the polls closed in those states and before any votes were counted, 16 of them were placed in the president’s corner and 13 in the senator’s. They tended to be places like Kansas and Rhode Island.
In quite a few of other states, though, the exit polls were less conclusive. That’s where the trouble came in.
About 4 p.m., the pool statisticians met with the network reps and gave them a rundown on the exit-poll surveys and indicated which ones might be called when the polls closed (the 29 above) and which ones to be wary about. The networks were advised against drawing any conclusions from the second group and to air none of them. They followed that advice.
But while these figures never got on the air, they did get around. I have a rather long list of Web sites at which you could find early survey figures from the states that the networks agreed not to go near until they had real votes. Slate, for example, had these numbers up early:
Florida
Kerry 50
Bush 49
Ohio
Kerry 50
Bush 49
Pennsylvania
Kerry 54
Bush 45
Wisconsin
Kerry 51
Bush 46
Michigan
Kerry 51
Bush 47
Minnesota
Kerry 58
Bush 40
Nevada
Kerry 48
Bush 50
New Mexico
Kerry 50
Bush 48
North Carolina
Kerry 49
Bush 51
Colorado
Kerry 46
Bush 53
On this scoreboard, Kerry has the larger number in all “trifecta” states (Florida, Ohio, Pennsylvania) decreed by every certified pundit as the key to victory in ‘04. In the second tier of battleground states Kerry has the better numbers overall. Shortly after these numbers were public, a friend in Dallas called and offered me heavy odds on Kerry. He owes me a hundred bucks. Thanks, Slate.
As it happens, Kerry did carry Pennsylvania. In Ohio he lost by two points instead of winning by one point. * Far from being wildly off the mark, that variance is about par for the course, or even under par, for a mid-afternoon reading of an exit poll. Indeed, Tuesday’s exit-poll numbers were no more off the mark than were those of four, eight, or 12 years ago.
That’s why the people who bought and paid for this intelligence kept it to themselves. Bill Wheatley, an NBC executive who understands these things, told the New York Times Wednesday that early afternoon numbers from an exit poll are “junk.” Slate and the other Web sites on which these numbers appeared Tuesday afternoon have every right in the world to get them however they can and publish them. But it’s hard to pin the blame for the dissemination of these numbers on those who tried so hard to keep them secret.
In one respect the networks do share some guilt. On their own Web sites, as soon as the polls closed, several of them posted the entire questionnaire from the exit poll—including percentages on the horse-race. They did this without regard to whether a winner had been called. Some bloggers concluded that those numbers had the imprimatur of the network experts and passed on to the entire Internet apparent conclusions that the networks had in fact wisely declined to endorse on the air.
But the real problem is not that the exit polls were wrong. They were about as accurate as they usually are. The problem was that in the age of the Internet the exit polls were being seen by thousands of people who didn’t know how to read them. Like any sophisticated weapon, they are dangerous in the hands of the untrained.
Correction, Nov. 4, 2004: This article originally said that Kerry lost Florida by two points. In fact, Kerry lost Florida by five points. Return to the corrected sentence.
