Download the MP3 audio version of this story here, or sign up for The Explainer’s free daily podcast on iTunes.
Good news in Iraq has nudged the president’s approval rating up to 37 percent, according to a poll of 1,002 adults released this week by NBC News and the Wall Street Journal. Meanwhile, a recent Gallup poll of 1,002 adults found that Bill Clinton’s retrospective approval rating has gone up by 10 points in the last few years. Why don’t these approval polls have a sample size of 1,000?
It’s easy to overshoot your goal. The major polling firms like to get at least 1,000 respondents for their national surveys. That’s a big enough sample to keep the margin of error in the vicinity of three percentage points—better than the 5 percent error they’d get from a sample half that size. They could always improve the accuracy with more interviews, but in practice it’s just not worth it. Another 500 phone calls wouldn’t even push the error down by a full point. The firms view a sample size of 1,000 as a reasonable compromise between accuracy and cost. (It’s also a nice, round number.)
So, why do they end up with 1,002? The pollsters don’t make their calls one at a time. A Gallup Poll might use 50 trained interviewers working at phone banks in five different calling centers. Each one gets instructions from a centralized computer system on whom to call and when. A supervisor keeps track of how many interviews they complete and whether they’ve filled their quotas for men and women in all different parts of the country.
At a certain point the supervisor will see that he has almost 1,000 logged interviews, and he’ll send out the “stop” message. Anyone who’s in the middle of a call will finish up, and the data they gathered gets added to the pool. If the supervisor’s done his job, they’ll end up with at least 1,000 interviews. It won’t really matter if the final sample size is 998 or 1,002, but as a general rule pollsters would rather have too much data than too little. (Every once in a while they’ll have to throw out some interviews after the fact. For example, a respondent who turns out to be under 18 would be disqualified from a poll of “adults.”)
Polls with a very quick turnaround make it harder to get the exact right number of interviews. If a firm wanted to conduct a “reaction poll” right after a State of the Union address, they might need to get 600 responses during a 20-minute period. That means everyone’s going to be on the phone at the same time. With so many calls going on at once, the supervisor won’t be able to time things as well—and may end up with a sample of 560 or 640 people instead.
Got a question about today’s news? Ask the Explainer.
Explainer thanks Jeff Jones of the Gallup Poll and Scott Keeter of the Pew Research Center for the People and the Press.
