In Mexico, polling is done the old-fashioned way: In person. Could this be the future of polling in post-land-line America?

MEXICO CITY—One Saturday last July, in the central Mexican state of Michoacan, where the country’s first constitution was written, two cars carrying six visitors to the city of Apatzingán disappeared. Three days later, in nearby La Cofadria, another three out-of-towners vanished. The circumstances were not mysterious—Apatzingán is known as a base for drug cartels and has been a site of deadly violence—but the targets of the kidnapping were. The nine hostages were all survey-takers who had come to Michoacan to poll residents’ preferences in upcoming gubernatorial elections and were working for firms commissioned by the leftist Democratic Revolution Party. For Mexico’s well-developed polling industry, days of waiting for signs of life invited some self-examination. Did someone consider their research a threat? “Suddenly the cartels see a lot of cars with strange plates and ask: who are these people?” Ulises Beltrán, the country’s most venerable political pollster, speculated the other evening. “Are they the police? Are they the FBI?”

None of the nine who were seized—and released days later—worked for Beltrán’s firm; they had been employed by two rivals, Consulta Mitofsky and Parametria. But Beltrán had last summer’s suspenseful week on his mind because the incident marked perhaps the greatest threat to the polling method he had helped to pioneer in Mexico. On Tuesday night, Beltrán was in his office in Mexico City’s La Condesa neighborhood, putting the finishing touches on a poll for the next day’s edition of the newspaper Excelsior. As is the case with polling in much of the world, this required putting boots on the ground. Because it was the last of the polls that would appear before this Sunday’s presidential election—Mexican law prohibits the publication of polls over the final four days before voting—Beltrán had increased his sample size. Now, instead of interviewers canvassing 120 of Mexico’s electoral precincts to speak to 10 voters in each, they had been scattered across 150. That meant one-quarter fewer logistical worries than usual. 

American pollsters once saw such tactile survey methods as a primitive practice to be used only in countries yet to graduate to surveys by telephone. But many are adjusting to the reality that a return to in-person interviews may be one of several imperfect solutions to their own crisis: a rapidly growing inability to reach voters by phone. “It’s a lot harder,” says John Anzalone, who has polled for 20 years on behalf of Democratic candidates, including Barack Obama. “It used to be that basically everyone had a land line—it’s how people communicated. Other than Dad saying, ‘You can’t answer the phone during dinner,’ people liked answering the phone. You had a captive audience.” Visiting voters at home is expensive, time-consuming, and occasionally fraught with real peril, but as American political polling reaches the end of its High Modernist era, practitioners are desperate simply to get back in front of people. “It’s a throwback,” says Anzalone. “But you’re crazy not to be experimenting with a new way of capturing them.”

***

In 1988, the newly elected Carlos Salinas de Gortari hired Ulises Beltrán to be the first presidential pollster in Mexico’s history. Election polling was effectively nonexistent in the country, a reflection of the fact that there was rarely a horse race worth tracking: At that point, Salinas’s PRI had presided over unchallenged one-party rule for more than 50 years. But the new president wanted to take stock of attitudes about policy questions and asked Beltrán—who had studied economic history at the University of Chicago and was then working as a statistician in Mexico’s version of the IRS—to design a method of measuring national public opinion.

Even though American political consultants were working across Latin America at the time, there was little that Mexicans could learn from them about basic survey practices. American pollsters built their samples by randomly dialing phone numbers and then weighting the results if the sample they collected failed to represent the demographics of the broader population or electorate. But fewer than one-tenth of Mexicans owned a telephone line, and the ones who did were far from representative of the country: They were wealthier, better-educated, more urban. Beltrán knew he would have to conduct all of his interviews in person.

A half-century earlier, American pollsters had been in a similar position. Beginning in the 1930s, George Gallup sent interviewers to hundreds of locations nationwide and asked them to randomly select individuals in certain demographic categories for questioning. This method was seen as a (far more expensive) corrective to the straw polls that Literary Digest had conducted since the 1920s, by sending millions of ballots to mailing lists built off its subscriber records, phone books, and automobile registries. Gallup’s method was validated in 1936, when he accurately predicted Franklin Delano Roosevelt would be re-elected; Literary Digest’s poll showed Alf Landon winning 55 percent of the vote. But it was another polling flub 12 years later—the “Dewey Defeats Truman” prognostication—that forced survey-takers to be more rigorous about their methods of selecting households, a shift that dramatically increased the costs of polling. Instead of sending his survey-takers to public places, Gallup began dispatching them to specific doorsteps.  

Beltrán developed a similar approach. After Mexico completed its 1990 census, he hired its newly unemployed researchers and turned them into an army of political interviewers. He randomly selected electoral precincts and then within the precinct a block; interviewers started on one side and knocked every third door. If no one was home, interviewers would return later. “It depends how much money you have,” says Beltrán. “If you have a lot of money and time, you go back until you get the person you want.”

By then, American survey-takers had moved indoors. In 1970, as the share of Americans with telephone lines crossed 90 percent, pollsters realized they could run all their surveys from call centers. By randomly dialing digits they could have greater confidence that they were getting a statistically valid representation of the phone-owning population. When interstate telephone service was deregulated, the cost of polling by phone dropped even further. The only researchers who stuck to knocking on doors were those like the Census and American National Election Studies, which had long questionnaires, deep budgets, and a goal of covering the entire population. For everyone else, polling—especially with the introduction of automated touchtone surveys—became something that could be done cheaply, widely, and often.

As Mexico’s 1994 elections approached, Beltrán worked for the campaign of Salinas’ successor Ernesto Zedillo. The PRI was finally facing real electoral challenges, and knocking on someone’s door and asking how he or she planned to vote seemed a potentially more complicated intrusion than merely asking for one’s views of economic conditions. Beltrán ran a battery of randomized experiments to test whether results would be biased by particular aspects of the interaction. Did the questioner’s gender alter the interview dynamic? Did the color of the pen guide the respondent to one party or the other? Beltrán found they had little impact, and those factors that did likely cancelled each other out across a large sample. “In the big numbers,” he says, “cultural effects disappear.”

The greatest concern, however, was one to which the PRI was particularly sensitive: a fear that citizens, suspecting that the clipboard-equipped visitor at their door could be a government agent, would refuse to answer questions—or exaggerate their support for the incumbent party. To mimic the way Mexicans cast ballots on election day, Beltrán sent his interviewers into the field with paper ballots and a box. When it came time to ask which candidate they would support if the election were held that day, voters would be handed a facsimile of the official ballot (which included the party’s logos in full color, so illiterate voters could distinguish them) and asked to put an X on their choice and place it into the box. “It is the only way to do it, elegantly,” Beltrán says.