A consumer's guide to the polls.

Science, technology, and life.
Oct. 28 2004 5:31 PM

A Consumer's Guide to the Polls

Read the ingredients before you buy.

1_123125_2100252_2100417_2108776_2108777_041027_polls

The latest polls in the presidential race show George W. Bush ahead. Or John Kerry ahead. Or a tie. Or Bush gaining. Or Kerry gaining. Take your pick.

Why do the polls disagree so much? And which ones should you buy?

That would be easier to decide if you could buy polls the way you buy canned food or cereal, with all the ingredients disclosed on a label. But pollsters don't package their products that way. Consequently, most of us believe, mistakenly, that a poll is a simple tabulation of a random sample of voters. In reality, polls are full of additives and preservatives, subtractions and selective multiplications, none of which are generally published. The reason many polls consistently differ from others is that their hidden ingredients differ. Here's a guide to those ingredients, poll by poll.

Let's start with a summary of what to look for.

1. Questionnaire disclosure. Pollsters and the news organizations that hire them usually paraphrase their findings. To see exactly how the questions were posed and answered, you need the questionnaire. More and more pollsters are providing this information online. The rest should. We tell you who isn't publishing it, who is, and where to find it.

2. Likely voter screens. All pollsters identify likely voters by gauging each respondent's stated level of interest in the election. Some pollsters also weigh the respondent's stated voting history. If you're really excited about this election but you failed to vote in the last one, some polls may screen you out, thereby excluding your opinion and failing to anticipate your vote. We tell you who those pollsters are.

3. Questions before the trial heat. Some pollsters start by asking which candidate you'll vote for (the "trial heat"). Others start by asking whether you approve of Bush's job performance, whether you view Kerry favorably, or whether you think the country is on the wrong track. Posing these questions before the trial heat demonstrably changes the way some people vote. (Some pollsters ask respondents to pick a candidate twice, before and after such questions. The numbers change.) We tell you who asks what before the trial heat.

4. Pressing. If you say you're undecided, some pollsters will leave it at that. Others will press you to say which way you're leaning. If you're a leaner, you may be counted as a vote for that candidate even though there's a 49 percent chance you'll go the other way. Pressing for leaners can make the race look more decided than it really is. A pollster may report that one candidate holds a prohibitive lead of 50-48 when the race is actually a wide open 47-46. We tell you who presses, how, whether they disclose the boost each candidate gets from pressing, and how big that boost is.

5. Demographic weighting. Some pollsters count you as one voter no matter what your race, sex, or age. Others "weight" your vote more or less heavily depending on which group you belong to. A pollster who assumes that the election turnout will have a certain percentage of blacks, women, or people aged 18 to 29 may find that her sample has too many blacks or not enough women to match that projection. If you're a black respondent in that sample, your vote may be weighted more lightly so that blacks don't exceed their quota. If you're a woman, your vote may be weighted more heavily so that women don't fall short of their quota. We tell you who weights which traits.

6. Party shares. Some pollsters assume that the election turnout will have certain percentages of Democrats and Republicans. If the percentage of Republicans in a sample exceeds that projection, the pollster may compensate by weighting the answers of Republicans more lightly or the answers of Democrats more heavily. We tell you who's engaging in such compensations and who isn't.

Don't assume that pollsters are bad if they screen for past voting behavior, pose questions before the trial heat, press undecideds, or weight votes by party, race, or sex. Their job is to reflect which candidate would win if the election were held today. If you say you're going to vote but you don't get around to it, just like last time, then the pollsters who screen you out are right. If you like Bush but end up voting against him because you think the country's going in the wrong direction, the pollsters who pose the wrong-track question before the trial heat are right. And if you fault some pollsters for failing to "balance" likely voter samples that turn out to be heavily Republican, don't blame others for weighting their numbers to match previous turnout. We can't tell you which poll to buy. All we can do is show you their ingredients so you can see what you're getting.

ABC/Washington Post

Publishes entire questionnaire with results: Yes

Where: Go here to see the tracking poll chart, and click the link to "Today's Full Results."

Screens people out based on past failure to vote: Yes.

Likely voter test: They ask whether you have registered, are interested in the campaign, intend to vote, are voting for the first time, and know the location of your polling place.

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate: Yes.

Average boost from pressing, last three samples: Unknown.

Disclosure of boost factor: Unpublished.

May weight your vote differently depending on your: age, sex, race, education.

Adjusts results to fit expected party shares of electorate: Yes.

Expected shares: Neither party is allowed to stray more than 3 percent from its traditional vote share as reflected in exit polls over the last three presidential elections.

Associated Press (Ipsos-Public Affairs)

Publishes entire questionnaire with results: No.

Screens people out based on past failure to vote: Yes.

Likely voter test:You're in if you claim to have at least "quite a bit" of interest in campaign news and either voted in 2000 with some likelihood to vote in 2004 or did not vote in 2000 but convey the highest likelihood to vote in 2004

Raises these questions before asking whom you'd vote for: Right/wrong track, Bush job approval.

Presses undecideds to pick a candidate: Yes.

Average boost from pressing, last three samples:Bush +0.78, Kerry +1.23.

Disclosure of boost factor: Unpublished but available to media on request.

May weight your vote differently based on your: Race, sex, age, education.

Adjusts results to fit expected party shares of electorate:No.

Battleground

Publishes entire questionnaire with results: Yes.

Where: Here. Click the "questionnaire" link for any week.

Screens people out based on past failure to vote: No.

Likely voter test:They ask how likely you are to vote. If you're at least somewhat likely, you're in.

Raises these questions before asking whom you'd vote for: None before asking the first time ("unaided ballot"). Then the interviewer asks about right/wrong track, your top issue, whether you approve of Bush's job performance, and whether you view each candidate favorably. Then they ask you to pick a candidate again ("aided ballot"), and that's the number featured in the pollsters' report.

Presses undecideds to pick a candidate: Yes.

Average boost from pressing, last three samples:Kerry +1, Bush +1.33.

Disclosure of boost factor: Published.

May weight your vote differently depending on your: Race. The pollsters assume turnout will be 80 percent white, 10 percent black, 6 percent Hispanic, and 4 percent other. This is a compromise between the Republican firm (Tarrance Group) and Democratic firm (Lake Snell Perry and Associates) that run the poll.

Adjusts results to fit expected party shares of electorate:Yes.

Expected shares: 42.3 percent Dem, 42.3 percent GOP. This, too, is a compromise between the firms.

CBS/New York Times

Publishes entire questionnaire with results: Yes.

Where: Here. Click to get the summary page for any poll, then click the link to "complete questions and responses."

Screens people out based on past failure to vote: Yes.

Likely voter test: They assign you a probability factor between zero and one based on your voting history, level of attention paid to the campaign, date of registration, length of residence, and stated probability of voting in this election. They multiply your vote by that factor when tabulating the results. For their thorough and exemplary disclosure of this methodology, click here. *

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate: Yes when asking Bush vs. Kerry, but not when asking Bush vs. Kerry vs. Nader.

Average boost from pressing, last three samples:Unknown.

Disclosure of boost factor: Unpublished.

May weight your vote differently depending on your: Race, sex, age, education, region.

Adjusts results to fit expected party shares of electorate:No.

Democracy Corps (Greenberg Quinlan Rosner)

Publishes entire questionnaire with results: Yes.

Where: Here. Click to read any survey.

Screens people out based on past failure to vote: Yes.

Likely voter test: If you were old enough to vote in 2000, you must have voted in 2000 or 2002, be registered, and say you will probably vote in 2004. Exceptions: If you were too young in 2000, or if you were unregistered in 2000 or 2002 but have registered since then, you can still be counted as a likely voter as long as you say you will probably vote.

Raises these questions before asking whom you'd vote for: Right/wrong track, Bush job approval.

Presses undecideds to pick a candidate: Yes.

Average boost from pressing, last three samples:Bush +1.33, Kerry +1.67.

Disclosure of boost factor: Published.

May weight your vote differently based on your: sex, age, education.

Adjusts results to fit expected party shares of electorate:No.

Fox News

Publishes entire questionnaire with results: Yes, except for screening and demographic questions.

Where: Here.Click any link in the "archive" box to see the summary of a previous poll. From there, scroll to the bottom and click "full poll results" to read the questionnaire.

Screens people out based on past failure to vote: Yes.

Likely voter test:Unspecified.

Raises these questions before asking whom you'd vote for: Whether you view each candidate favorably.

Presses undecideds to pick a candidate:Yes.

Average boost from pressing, last three samples:Unknown.

Disclosure of boost factor: Not published.

May weight your vote differently depending on your: Race, age.

Adjusts results to fit expected party shares of electorate:No.

Gallup (CNN/USA Today)

Publishes entire questionnaire with results: Not for the public.

Where:National questionnaires are posted without screening or demographic questions here. They include trend data but are not archived there. More information is available only to subscribers at www.gallup.com.

Screens people out based on past failure to vote: Yes.

Likely voter screen: Gallup typically assumes 50 percent in a presidential election, but due to high levels of voter interest this year, they're assuming a 55 percent turnout. If you get top scores on their seven likelihood questions, you get a weight factor of 1. The lower you score, the more your weight factor diminishes. If you don't make the top 55 percent, your weight factor is 0, and you're not counted.

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate: Yes.

Average boost from pressing, last three samples: Bush +0.75, Kerry +1.

Disclosure of boost factor: Unpublished but available to media on request.

May weight your vote differently based on your: Race, sex, age, education.

Adjusts results to fit expected party shares of electorate:No.

ICR

Publishes entire questionnaire with results: No.

Screens people out based on past failure to vote: Yes.

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate: Yes.

Average boost from pressing, last three samples: Bush +2.63, Kerry +1.97.

Disclosure of boost factor: Unpublished but available to media on request.

May weight your vote differently based on your: Race, sex, age, education, region.

Adjusts results to fit expected party shares of electorate: Yes, in principle; no, in practice. The poll's partisan breakdown has not yet veered more than four points from what ICR considers "appropriate balance." If it does, ICR may adjust it.

Los Angeles Times

Publishes entire questionnaire with results: Yes, except for screening questions.

Where: Here. Click any of the links to "poll excerpts."

Screens people out based on past failure to vote: Yes.

Likely voter test: They weight your response based on your stated intention to vote, certainty of voting, and past interest in voting. Nobody gets discounted completely.

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate:Yes.

Average boost from pressing, last three samples:Unknown.

Disclosure of boost factor: Not published.

May weight your vote differently depending on your: Race, sex, education.

Adjusts results to fit expected party shares of electorate:No.

Mason-Dixon

Publishes entire questionnaire with results: Not routinely, but Knight Ridder makes some available.

Where: Go  here and click any of the links above the map. Then look in the "related links" box on the right and click "full poll results" next to the state of your choice.

Screens people out based on past failure to vote: No.

Likely voter test:They ask how likely you are to vote. If you're at least somewhat likely, you're in.

Raises these questions before asking whom you'd vote for: Whether you view each candidate favorably.

Presses undecideds to pick a candidate: Depends on what the client wants.

Average boost from pressing, last three samples:Unknown.

Disclosure of boost factor: Not published.

May weight your vote differently depending on your: Age, sex, race, education, but only if the sample is very different from previous samples in the region.

Adjusts results to fit expected party shares of electorate:No, unless the sample is very different from previous samples in the region.

Expected shares: Depends on region.

Newsweek (Princeton Survey Research Associates)

Publishes entire questionnaire with results: No.

Screens people out based on past failure to vote: Yes.

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate: Yes.

Average boost from pressing, last three samples:Unknown.

Disclosure of boost factor: Not published.

May weight your vote differently depending on your: Race, sex, education, region.

Adjusts results to fit expected party shares of electorate:No.

Pew

Publishes entire questionnaire with results: Yes.

Where: Here. Click to read any survey.

Screens people out based on past failure to vote: Yes.

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate: No.

May weight your vote differently based on your: Race, sex, age, education.

Adjusts results to fit expected party shares of electorate:No.

Time (Schulman, Ronca & Bucuvalas Inc.)

Publishes entire questionnaire with results: Yes.

Where: Here. Click any link in the "archive" column on the left to get the summary page for a previous poll. Then click the link at the bottom to "survey questionnaire."

Screens people out based on past failure to vote: Yes.

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate: Yes.

Average boost from pressing, last three samples:Bush +0.33, Kerry +0.67.

Disclosure of boost factor: Published.

May weight your vote differently depending on your: Race, sex, age, education, region.

Adjusts results to fit expected party shares of electorate:No.

TIPP (Investor's Business Daily/Christian Science Monitor polls)

Publishes entire questionnaire with results: No.

Screens people out based on past failure to vote: No.

Likely voter test: Unspecified..

Raises these questions before asking whom you'd vote for: None.

Presses undecideds to pick a candidate:Yes.

Average boost from pressing, last 3 samples:Unknown.

Disclosure of boost factor: Not published.

May weight your vote differently depending on your: Race, sex, age, region.

Adjusts results to fit expected party shares of electorate:Yes.

Expected shares: Average of their last five months of samples.

Zogby (Reuters)

Publishes entire questionnaire with results: No.

Where: Here. Click any link in the "archive" column on the left to get the summary page for a previous poll. Then click the link at the bottom to "survey questionnaire."

Screens people out based on past failure to vote: Yes.

Likely voter test: Unspecified..

Raises these questions before asking whom you'd vote for: Declined to disclose.

Presses undecideds to pick a candidate: No, until a week before the election; yes, thereafter.

Average boost from pressing, last three samples:Unknown.

Disclosure of boost factor: Not published.

May weight your vote differently depending on your: Race, sex, age, or education, depending on census data, exit polls, and previous Zogby surveys.

Adjusts results to fit expected party shares of electorate: Yes.

Expected shares:39 percent Dem, 35 percent GOP. Zogby calculates party weights based on the last three elections and subtle adjustments for factors such as Hispanic population growth.

Correction, Oct. 29, 2004: We originally described the CBS/New York Times likely voter test as unspecified. The failure to specify it was ours. CBS News has published an exemplary description of its likely voter screening methods here. (Return to the corrected sentence._

Will Saletan writes about politics, science, technology, and other stuff for Slate. He’s the author of Bearing Right. Follow him on Twitter.

David Kenner is a former Slate intern.

Louisa Thomas is on the editorial staff of The New Yorker.