In Mexico, polling is done the old-fashioned way: In person. Could this be the future of polling in post-land-line America?
In 2000, when the PRI was voted out of the presidency, Beltrán left government and started his own firm. His team of former census-takers set out to become their own company, one of several contracted by pollsters to conduct their fieldwork. Now Mexico has around 10 top-flight pollsters, most of whom have a contract to run surveys for a newspaper and some of whom simultaneously work for political parties and other private clients. Even as the share of mobile-phone users increases, there is still fewer than one land line for every five Mexicans. Pollsters may use phones for tracking polls, designed to discern movement. Nearly every serious poll designed to measure the views of the electorate continues to rely on face-to-face interviews, even though pollsters estimate they are twice as expensive as running the same surveys by phone. “Most reliable pollsters do at-home polling,” says Javier Aparicio, a political scientist at Mexico’s CIDE research institute who is currently serving on the national body tasked with administering Mexico’s election. “Some pollsters do phone-only, but they’re not well-regarded because they do it on the cheap.”
As Mexico has urbanized, more voters live in well-mapped areas with clearly defined blocks that are easy to randomize. In rural areas without such orderly geography, Google Maps and satellite imagery allow researchers to identify clusters of homes that can be treated as blocklike units. There, pollsters still have to allocate extra time for interviewers to get from door to door, and often to call in an interpreter when they stumble upon a voter who speaks only an indigenous language. Even in urban areas, challenges persist: It remains very difficult to find men between ages 30 and 39 at their homes at any time other than the weekend.
But it is the physical safety of interviewers in the Mexican states most directly affected by the drug war that has posed the greatest complication. Jorge Buendía, whose firm Buendía y Laredo polls for El Universal newspaper along with political clients, says that each time he randomly pulls 150 electoral precincts to survey, he expects to discard two or three because they are in parts of the country where it is simply too dangerous to dispatch interviewers anymore. He has also learned not to ask directly about the local security situation, because respondents are usually too intimidated to answer, and tries to keep interviews to 30 minutes or fewer, in part because they are typically conducted while standing at the doorstep. Interviewers are advised not to enter respondents’ homes “for the safety of both sides,” according to Buendía.
Since last summer’s Michoacan kidnapping, Buendía has made a point of ensuring his interviewers go out into the field with even less protection. Instead of teams, they travel alone, and without handheld devices or laptops. It sounds counterintuitive, but it’s strategic. “In groups, they draw the attention of drug dealers,” Buendía said. His pollsters now set out with little more than clipboards, questionnaires, and the Beltrán innovation known as the “simulated voting box,” to re-create the election-day experience on a doorstep. “The basics,” he says, “have more or less stayed the same.”
In the United States, however, the polling industry faces a reckoning. Last month, the Pew Research Center, in what it called a report but was best read as a plaintive appeal for sympathy, declared, “It has become increasingly difficult to contact potential respondents and to persuade them to participate.” In 1997, Pew reported, the telephone response rate to its polls (the share of outbound calls that yielded an interview) was 36 percent; by 2012, it had fallen to 9 percent. It was easy to attribute blame—the proliferation of mobile devices, caller ID, and the increasingly secondary nature of phone communication altogether.
There are alternatives to traditional phone polling, but all of them have trade-offs. The changing landscape is not only making polling more arduous and expensive but has also cleaved American society into demographically (and often politically) distinct groups, much as Literary Digest did when it polled only magazine subscribers and car owners during the Great Depression. In 2008, the Obama campaign was jarred to find that it was running even with John McCain among young voters it polled via traditional land lines. When campaign officials commissioned a separate poll using only mobile numbers—a population more likely to include minorities and students—Obama’s expected lead over McCain fully materialized.
But just dialing mobile phones is an uneven endeavor. While many voter files now include mobile numbers, and others are available from commercial data vendors, laws restrict the use of some automated dialers when ringing cell phones. Some researchers are using the Internet to conduct surveys, but traditional sampling there is impossible. Those designing Web surveys are forced to either use offline contact to assemble an online sample (by calling phone numbers and recruiting people to join a Web panel) or altogether forgo the ambition of creating a random sample and rely on other methods to ensure that respondents are representative of a desired population.
Knocking on doors would likely be the most methodologically pure of the post-phone approaches, although also the most demanding. Campaigns might find it a worthwhile addition to their research toolbox, especially for so-called benchmark polls that offer an early, expansive survey of the electorate to guide strategy and planning. But many would likely find it hard to justify the practice. “In-person interviewing is a nonstarter for media polls. It's too expensive—most newspapers have cut their budgets for polling and can barely afford phone surveys—and takes too long,” says Douglas Rivers, a Stanford political scientist who has pioneered online surveys and is currently chief innovation officer for the pollster YouGov. “The prospects for door-to-door interviewers in the 21st century are about the same as for milk men.”
Polling may be less like dairy than energy reserves. American public-opinion research has likely reached its version of peak oil, with the cheap, plentiful polling that undergirded political communication for 75 years no longer sustainable. Quick surveys may no longer be within reach of every blog, trade association, liberal-arts college, and state-legislative candidate. As pollsters try new techniques for reaching people, their choice of methodology will betray divergent priorities: cost vs. speed vs. representativeness. It will be harder to pretend all polls ought to be treated equally.
Sasha Issenberg is the author of The Victory Lab about the new science of political campaigns.