Can We Believe the Polls?

Oct. 20 2000 8:30 PM

Can We Believe the Polls?

The opinion polls tell us that the presidential race is "too close to call." But over the last few weeks, they have also shown wide swings in each candidate's standing in the polls. Journalists use these changes to write about "rallies," "breakthroughs," "stumbles," and "critical turning points." Maybe; perhaps these swings can be explained by speeches, debates, and policies. But no one really knows whether these causes are real or imaginary. In fact, no one can be certain that the race is too close to call.


In mid-September, a Newsweek poll showed Al Gore ahead by 14 points; a week later, the same poll showed him ahead by only 2 points. Did the voters really change that much in one week? I doubt it.

The reason for this uncertainty is that polling is as much art as science. Recently, VitalSTATS, the newsletter of the Statistical Assessment Service, a nonprofit organization that tries to help journalists make sense of numbers, issued a special report on the polls. (You can look up the newsletter at

The last survey taken by any of the major national opinion pollsters is usually fairly close to the actual election results, albeit with some big exceptions to be noted in a moment. But before that final week, the polls are often wrong by a wide margin.

One reason is that the pollsters must choose between surveying either registered voters or likely voters. Studying the first group does not make much sense, since a large number of people who are registered don't bother to vote. Studying the second group makes sense—if you know who they are. On Oct. 19, 1996, the Gallup/CNN/USA Today poll showed President Clinton leading Bob Dole by 18 points. Ten days later, the same poll found Clinton leading by only 1 point.

The reason for the change is that the poll on Oct. 19 surveyed registered voters, while on Oct. 26 it surveyed likely voters. But neither survey got the results right. In the election, Clinton beat Dole by 6 points. It is obvious why surveying registered voters makes little sense; nobody knows which of them will really cast a ballot. But surveying likely voters is also hard, because no one knows until Election Day who will really vote.

Pollsters use a variety of screening questions to try to make a reasonable guess. These questions are often a proprietary secret, and there are hardly any published scientific studies that show which questions really work. But we do know a few things. One is that young and low-income voters often have very low turnouts. Perhaps the pollsters know how to take this into account.

But suppose they don't know how. This means that they run the risk of counting as likely voters people who won't really vote. And since these groups are more likely to vote Democratic than Republican, it means that they will overestimate the Democratic vote and underestimate the Republican one. In 1980, the last Gallup Poll published just before the election showed Reagan with 47 percent of the vote. In fact he got 51 percent.

Matters were even worse with polls published in early October. In 1988, they showed the senior George Bush with 49 percent of the vote; on Election Day he won 54 percent. In 1992, the October polls showed Clinton with 52 percent of the vote; in November he received 43 percent. Same story in 1996: In October, the polls said Clinton had 55 percent of the vote; he actually got 49 percent.

No one knows whether the current polls also underestimate the Republican vote. But if they do, as has been common in the past, Bush leads Gore.



The Irritating Confidante

John Dickerson on Ben Bradlee’s fascinating relationship with John F. Kennedy.

My Father Invented Social Networking at a Girls’ Reform School in the 1930s

Renée Zellweger’s New Face Is Too Real

Sleater-Kinney Was Once America’s Best Rock Band

Can it be again?

The All The President’s Men Scene That Captured Ben Bradlee

Medical Examiner

Is It Better to Be a Hero Like Batman?

Or an altruist like Bruce Wayne?


Driving in Circles

The autonomous Google car may never actually happen.

The World’s Human Rights Violators Are Signatories on the World’s Human Rights Treaties

How Punctual Are Germans?

  News & Politics
Oct. 22 2014 12:44 AM We Need More Ben Bradlees His relationship with John F. Kennedy shows what’s missing from today’s Washington journalism.
Oct. 21 2014 5:57 PM Soda and Fries Have Lost Their Charm for Both Consumers and Investors
The Vault
Oct. 21 2014 2:23 PM A Data-Packed Map of American Immigration in 1903
  Double X
The XX Factor
Oct. 21 2014 3:03 PM Renée Zellweger’s New Face Is Too Real
  Slate Plus
Behind the Scenes
Oct. 21 2014 1:02 PM Where Are Slate Plus Members From? This Weird Cartogram Explains. A weird-looking cartogram of Slate Plus memberships by state.
Brow Beat
Oct. 21 2014 9:42 PM The All The President’s Men Scene That Perfectly Captured Ben Bradlee’s Genius
Oct. 21 2014 11:44 PM Driving in Circles The autonomous Google car may never actually happen.
  Health & Science
Climate Desk
Oct. 21 2014 11:53 AM Taking Research for Granted Texas Republican Lamar Smith continues his crusade against independence in science.
Sports Nut
Oct. 20 2014 5:09 PM Keepaway, on Three. Ready—Break! On his record-breaking touchdown pass, Peyton Manning couldn’t even leave the celebration to chance.