At long last, those Delphic numbers get the scrutiny they deserve.

Media criticism.
Nov. 3 2004 7:31 PM

Demystifying the Exit Polls

At long last, those Delphic numbers get the scrutiny they deserve.

Illustration by Mark Alan Stamaty

Michael Kinsley's decade-old exit-poll obsession became mine during the 2000 campaign.

Early in the '90s, Kinsley started attacking the TV networks for promising to do one thing with their Election Day findings while doing another. The networks, accused of suppressing voter turnout by projecting election winners while voting was still underway, promised to conceal what they knew about the probable winners most precincts were closed.

Advertisement

This promise proved impossible to keep. Journalists live to blab all about a story before anybody else can, so during each election reporters who gain access to confidential Election Day data—such as exit polls—traditionally leak the info like mad, phoning and e-mailing it to their friends and colleagues who in turn distribute it to their friends and colleagues.

Although TV producers gag their reporters, barring them from mentioning anything too specific from the exit polls until their bosses sound the all clear, correspondents have found a variety of ways to telegraph the forbidden knowledge to their audiences. "If current trends continue, it could be a very happy night for Sen. Longhorn in his quest for the presidency," they might say, or, "Our exit-poll data shows that voters are very, very unhappy with the incumbent's performance over the past four years"—as if neither comment gives away the identity of the presumed victor.

It was Jacob Weisberg's inspiration to make overt what the networks made covert by publishing in Slate the exit-poll data as we received it during the 2000 primaries, which I did in my "Press Box" column. Publishing the exit-poll numbers may look like a college prank, but our intent was loftier. We wanted to expose the hypocrisy of networks that simultaneously embargoed the exit-poll data and broadcast its essence. Slate believed that readers should be trusted with the secrets of the journalistic temple, especially if newscasters were going to pantomime from them so cavalierly.

The trend Slate started picked up steam in 2002, and this year various Web sites (drudgereport.com, wonkette.com, mydd.com, nationalreview.com, for example) posted exit-poll numbers during the primaries and again on the day of the general election. Yesterday, Slate published three exit-poll installments, all of which flattered Kerry's prospects. (In midafternoon, we posted and then unposted almost immediately a set of numbers after we had doubts about their provenance.)

If you were smart enough to watch the TV coverage yesterday afternoon and early evening with Slate'sleaked exit polls as your cheat sheet, you weren't surprised to hear the anchors and commentators hint at a Kerry victory. As Alessandra Stanley writes in today's New York Times, the pro-Bush talking heads at Fox News Channel looked "stunned and somber as early voter surveys showed Mr. Kerry doing better than expected." Later, as the actual vote totals mounted in Bush's favor, toppling the raw exit-poll numbers, the Foxes began to glow.

In the 8 p.m. hour, when reporter Lisa Myers appeared on MSNBC's Hardball, she did her best given the limitations imposed by exit-poll omerta to explain that the Bush campaign thought the exit-poll results, which we now know were inflating the TV media's Kerry bubble, were flawed. From Myers' sharp report:

The Bush campaign has been laying the actual votes that have been coming in from the states against the exit-poll data, and they find that the exit-poll data is significantly underrepresenting the Republican vote. They specifically point to states like Virginia, South Carolina, and Florida.

They say, in each case, the actual vote patterns show a larger Republican vote than is represented in the exit-poll data. The campaign manager, Ken Mehlman, was particularly excited about some the data that they were seeing in Florida. ...

Why allow Myers to say "exit-poll data is significantly underrepresenting the Republican vote" but not let her speak the less cryptic truth that "the 4:15 p.m. exit-poll numbers from Florida show Kerry winning 51-49, a number the Bush campaign refutes"? This only begins to give you an idea of how arbitrary and silly the networks' rules are.

Because the networks handle the exit polls in such a cloak-and-dagger fashion, not every newscaster who talks about them understands their limitations. The Washington Post's Howard Kurtz captured the naivety of Tucker Carlson, who, on Crossfire, lashed out at what he considered a faulty data dump: "Somebody should reassess exit polling," Carlson said. "It's useless." When Juan Williams threw a similar fit on Fox News Channel about the worthlessness of the exit polls, William Kristol attempted to calm him by noting the obvious: Exit polls are polls, and all polls contain a margin of error.

To that Kristol could have added that no poll is better than the methodology behind it; that exit polls weren't designed to predict winners; and that no state was given to either candidate due to them.

The good thing about today's uproar is that it's accelerating the much-needed demystification of exit polls that Kinsley commenced. Readers and viewers are asking the news organizations (CBS, NBC, ABC, CNN, Fox, and the Associated Press) that own the outfit that produces the exit-poll data, the National Election Pool, questions about how the polls are conducted, how they're used, how accurate they are, and the need for keeping them officially secret on election night even though newscasters blithely lift from them.

The news organizations that subscribe to the NEP results are fielding the same set of questions. Today's online chat with Washington Post Managing Editor Steve Coll provides a savvy, preliminary critique of this year's exit-poll methodology. Here's hoping that Coll has an investigative piece in the works!

******

I randomly poll my readership for inspiration. You would not believe the margin of error! Send e-mail to pressbox@hotmail.com. (E-mail may be quoted by name unless the writer stipulates otherwise.)

Jack Shafer was Slate's editor at large. You can follow him on Twitter or email him at Shafer.Reuters@gmail.com.

  Slate Plus
Slate Picks
Dec. 19 2014 4:15 PM What Happened at Slate This Week? Staff writer Lily Hay Newman shares what stories intrigued her at the magazine this week.