A Consumer's Guide to the Polls
Read the ingredients before you buy.
Updated Thursday, Oct. 28, 2004, at 5:31 PM
Why do the polls disagree so much? And which ones should you buy?
That would be easier to decide if you could buy polls the way you buy canned food or cereal, with all the ingredients disclosed on a label. But pollsters don't package their products that way. Consequently, most of us believe, mistakenly, that a poll is a simple tabulation of a random sample of voters. In reality, polls are full of additives and preservatives, subtractions and selective multiplications, none of which are generally published. The reason many polls consistently differ from others is that their hidden ingredients differ. Here's a guide to those ingredients, poll by poll.
Let's start with a summary of what to look for.
1. Questionnaire disclosure. Pollsters and the news organizations that hire them usually paraphrase their findings. To see exactly how the questions were posed and answered, you need the questionnaire. More and more pollsters are providing this information online. The rest should. We tell you who isn't publishing it, who is, and where to find it.
2. Likely voter screens. All pollsters identify likely voters by gauging each respondent's stated level of interest in the election. Some pollsters also weigh the respondent's stated voting history. If you're really excited about this election but you failed to vote in the last one, some polls may screen you out, thereby excluding your opinion and failing to anticipate your vote. We tell you who those pollsters are.
3. Questions before the trial heat. Some pollsters start by asking which candidate you'll vote for (the "trial heat"). Others start by asking whether you approve of Bush's job performance, whether you view Kerry favorably, or whether you think the country is on the wrong track. Posing these questions before the trial heat demonstrably changes the way some people vote. (Some pollsters ask respondents to pick a candidate twice, before and after such questions. The numbers change.) We tell you who asks what before the trial heat.
4. Pressing. If you say you're undecided, some pollsters will leave it at that. Others will press you to say which way you're leaning. If you're a leaner, you may be counted as a vote for that candidate even though there's a 49 percent chance you'll go the other way. Pressing for leaners can make the race look more decided than it really is. A pollster may report that one candidate holds a prohibitive lead of 50-48 when the race is actually a wide open 47-46. We tell you who presses, how, whether they disclose the boost each candidate gets from pressing, and how big that boost is.
5. Demographic weighting. Some pollsters count you as one voter no matter what your race, sex, or age. Others "weight" your vote more or less heavily depending on which group you belong to. A pollster who assumes that the election turnout will have a certain percentage of blacks, women, or people aged 18 to 29 may find that her sample has too many blacks or not enough women to match that projection. If you're a black respondent in that sample, your vote may be weighted more lightly so that blacks don't exceed their quota. If you're a woman, your vote may be weighted more heavily so that women don't fall short of their quota. We tell you who weights which traits.
6. Party shares. Some pollsters assume that the election turnout will have certain percentages of Democrats and Republicans. If the percentage of Republicans in a sample exceeds that projection, the pollster may compensate by weighting the answers of Republicans more lightly or the answers of Democrats more heavily. We tell you who's engaging in such compensations and who isn't.
Will Saletan covers science, technology, and politics for Slate and says a lot of things that get him in trouble.
David Kenner is a former Slate intern.
Louisa Thomas is on the editorial staff of The New Yorker.
Illustration by Mark Alan Stamaty.