Slate's Bizbox
human nature: Science, technology, and life.

Bad CallsLessons of the New Hampshire polling fiasco.

By William Saletan
Posted Friday, Jan. 11, 2008, at 3:41 AM ET

(Continued from page 1)

You weren't aware that pollsters screen out respondents, or discount their stated preferences, based on sex, race, religion, and other "demographics"? You thought polls were raw data? Silly you. Read the pollsters' post-New Hampshire explanations, and you'll learn about all the formulas they use to "refine" their data before you see it. They apply "likely voter screens," "demographics," "turnout models," and "allocation of undecideds." In this case, their big mistake was underweighting responses from older women and overweighting responses from independents and young voters.

Lesson: Polls aren't raw data. They're data modified by assumptions. Pollsters should publish their assumptions so we know what we're eating.

5. Preferences were unstable. "New Hampshire voters' opinions were very much in flux," Elder theorizes. They were "buffeted by the intense media coverage until the moment they finally stepped into the voting booth and registered what pollsters call 'considered opinion,' the kind of opinion born of reflection rather than one elicited in an instant by a poll taker."

Elicited in an instant by a poll taker. Think about that. It's a confession that horse-race polls are inherently artificial. Voting is an act. You go to the polling place, enter the booth, make your selections, and mark them. Survey responses are words, not acts. They're elicited. Your phone rings; somebody asks you questions; you answer them. You're doing what they want, not what you want. You have seconds to answer the poll question. You have hours, days, or weeks to decide how, and whether, you'll actually vote.

Lesson: Polls can't predict votes any more than elicited words can predict voluntary acts.

6. The ballot order didn't match the polls. The Washington Post notes that in previous New Hampshire elections, "the state rotated candidate names from precinct to precinct, but this year the names were consistently in alphabetical order, with Clinton near the top and Obama lower down." The net effect could have been a 3-point boost for Clinton—more than her victory margin. Polls missed this effect because they rotate the names. Your neighbor is offered "Clinton or Obama"; you're offered "Obama or Clinton."

But wait a minute. If rotation causes a gap between polls and returns, which one more accurately reflects the voters' will? Pollsters rotate candidate names to avoid bias. By that logic, New Hampshire should change its ballot protocol to emulate their methodology.

Lesson: When polls and ballots differ, the fault may sometimes lie with the ballots.

7. Respondents lied. One version of this complaint blames race: To avoid conveying or revealing bias, people who privately voted against Obama must have told pollsters they were going to vote for him. Another version blames horse-race conformity: Post-Iowa Obama hype "led to a feeding frenzy of media coverage that was very favorable to Obama and very negative towards Clinton, which depressed her support in the polls but oddly did not lower her actual vote."

Again, the implication is that voting and answering a poll are different things. And again, it's not clear which is better. Why did polls correctly predict Iowa but not New Hampshire? One theory is that in Iowa, the votes were public. You couldn't pretend to support Obama while secretly voting against him. From a scientific standpoint, the upshot is that polls in secret-ballot contests should be made more private. If you're polled online instead of by a human being, maybe you'll admit that you don't trust Obama. But from a moral standpoint, maybe the part of you that wants to sound like an Obama supporter is your better half. Maybe we need more caucuses and fewer primaries. Maybe elections should be more like polls, requiring you to declare your choice to a fellow human being.

VIEW AS SINGLE PAGE
Print This ArticlePRINTDiscuss this in The FrayDISCUSSEmail to a FriendE-MAIL
Share on FacebookPost to MySpace!Share with MixxDigg ThisShare with RedditShare with del.icio.usShare with FurlShare with Ma.gnolia.comShare with SphereShare with Stumble Upon
William Saletan is Slate's national correspondent and author of Bearing Right: How Conservatives Won the Abortion War.
Illustration by Robert Neubecker.
Join the Fray: our reader discussion forum
What did you think of this article?
POST A MESSAGE | READ MESSAGES
Washington Post