In the fall of 2008, Rutgers political scientist David Redlawsk, along with former University of Iowa colleagues Caroline J. Tolbert and William Franko, added such a list experiment to polls in the hopes of capturing hidden bias against Obama’s candidacy on the basis of his race. “Some people will vote for Barack Obama this fall and some people will not,” a survey-taker told nearly 1,400 respondents. “Regardless of your overall feelings toward him, please indicate how many of the following four facts about Obama trouble you when you think about choosing the next president. I should add, too, that we are not interested in which ones trouble you, only how many.” Everyone in Redlawsk’s sample heard four sentences, in randomized order:
1. “During the primary campaign, he described small-town Pennsylvanians as ‘bitter’ and said that they ‘cling’ to guns and religion.”
2. “He has served in the US Senate only since 2005.”
3. “He used to be a smoker.”
4. “He was a member of Jeremiah Wright’s Trinity United Church of Christ for 20 years, before he resigned in May of 2008.”
Half of Redlawsk’s sample heard another sentence, which he considered the target item: “If elected, he will be the first black president.” White voters in the control group, which heard only the first four items, agreed with an average of 1.71 of these statements, those in the treatment group, which heard five, agreed with an average of 2.04. The difference between the two groups, Redlawsk concluded, surfaced an otherwise submerged bias: 33 percent of white voters had tacitly acknowledged they would be troubled by Obama being the first black president. (Academics have lodged minor methodological quarrels with the list-experiment technique, noting that the process can confuse respondents and requires the survey-taker to perfectly calibrate a list of items where most people will agree to some, but not all of them. If it is all or none, respondents could feel that their views on the target item are no longer masked.)
When it comes to the Mormon question, list experiments have shown that bias may be hidden in surprising places. This year, the left-leaning Public Religion Research Institute added a list experiment to its annual American Values Survey. Here the target item was “a Mormon becoming president of the United States” as something that might worry or bother a respondent. They asked the question directly, too. Among white evangelicals, the numbers were essentially interchangeable: 47 percent state their concern about a Mormon president when asked directly, while 49 percent did so through a list experiment.
But among white mainline Protestants—widely assumed to be the more moderate voters who comprise a winning primary coalition for Romney—only 30 percent declared their concern about a Mormon becoming president when they were asked about it. When they participated in the list experiment, however, 57 percent identified such a worry. In other words, more than one-quarter of white mainline Protestants wouldn’t publicly acknowledge a bias likely to affect their vote.
Romney’s advisers are familiar with the value of list experiments. Target Point Consulting, a Virginia firm that has crunched numbers for Romney’s campaigns since 2002, administered list experiments in 2008 when it was working on John McCain’s behalf during the general election. The goal was to measure for bias among independent voters against McCain, based on his age, and Obama, based on his race.
The Romney campaign won’t discuss its statistical methods, or how it targeted the pieces of direct mail it delivered in Iowa in recent weeks. But it should be possible to build a list experiment into its microtargeting polls in much the same way that Ohio Democrats added oblique questions about race and gender. The results could be used to model the characteristics of anti-Mormon voters with far more detail and complexity than just “white mainline Protestant,” and then sift through the state voter file to produce a list of other Iowans who statistically resemble them. A Romney adviser would say only that “I think we have what we need to make good decisions about who to talk to.” To do that, they will need to figure out whom to avoid.