Afghanistan is gearing up for elections this Thursday. Recent polls suggest that Hamid Karzai will garner the most votes but fall short of a majority, necessitating a runoff election. How accurate are polls in Afghanistan?
They’re better than they were before the country’s 2004 election but not nearly as good as Western polls. During the last presidential election, Hamid Karzai won with 55 percent of the vote. One prominent poll had predicted that Karzai would win 78 percent of the vote. Exit polls from the same organization, the International Republican Institute, a nonpartisan group established by Congress to support fledgling democracies, came much closer, missing the margin of victory by fewer than nine points. The failures of 2004 have led Afghan pollsters to improve their sampling techniques, and most analysts expect better results this time around.
Many polls out of Afghanistan report margins of error similar to, or smaller than, the typical U.S. opinion poll. But these numbers don’t account for the possibility that the survey sample was not truly random. Since virtually all American households have a telephone, U.S. pollsters use machines that dial random numbers, giving each home roughly the same chance of being surveyed. This method would severely undercount rural and poor voters in Afghanistan, many of whom don’t own phones. There, face-to-face interviews are the only option, but these are expensive and sometimes dangerous—interviewers have been killed.
Pollsters in Afghanistan rely on multistage stratification to generate representative samples. They select a certain number of districts at random from within each province, then select a cluster of villages at random from within each district. Finally, a certain number of households are chosen at random from within that cluster. (Some pollsters use population data to increase the probability that a more populous cluster or village will be selected.) To be even more careful, they use a statistical technique to select a random member from each household who is then interviewed by someone of the same sex to encourage participation. Experienced pollsters often ask the same question several different ways to make sure the interviewee understands and is answering honestly.
In theory, this process should generate a representative group. But it’s not that simple in Afghanistan. Some areas are off-limits due to security concerns. Other regions are so remote that an interviewer can’t get there and back during the time frame of the poll. These areas have to be replaced with regions that have similar ethnic compositions and population densities.
When all the data have been collected, the pollster checks the demographics of the survey population against those of the general population. If a survey population is only 40 percent female, for example, the pollster would give extra weight to the female respondents. Many pollsters in Afghanistan try to weight their data for gender and ethnic group. The problem is that Afghanistan hasn’t completed a census in 30 years, so no one really understands the demographics of the country. Pollsters are left trying to piece together fragmentary data from unreliable sources. Many refuse to weight their data at all. Others simply qualify their reports with a warning about potential bias, which may or may not make it into media coverage. Some organizations compare their poll results with other data-collection methods, such as focus groups, as a reality check.
Got a question about today’s news? Ask the Explainer.
Explainer thanks Craig Charney of Charney Research, Andrew Garfield of Glevum Associates, Scott Keeter of the Pew Research Center, and David Williams of Williams and Associates.
