In his forthcoming book, Off With Their Heads, Dick Morris levels the sensational accusation that the New York Times has been rigging its national polls in order to favor Democrats. The attack is well-timed, coming as it does when the Times is already reeling from the Jayson Blair affair and the suspension and subsequent resignation of star prose jock Rick Bragg. Like many others, Morris is convinced that Howell Raines is ruining the New York Times, and for a moment Chatterbox wondered whether Morris had the goods. Morris was, after all, the political magician who revived Bill Clinton’s presidency after the Democrats lost control of the House and Senate in 1994. “I polled every week for Clinton,” he writes in the book. Surely, Chatterbox thought, Morris knew what he was talking about.
Morris levels many accusations at the Times polling operation, most of them too intemperate and unconvincing to warrant discussion here. (He really hates the Times.) The charge that stimulated Chatterbox’s curiosity concerned a technical practice known as “weighting.” After pollsters collect raw survey results from around the country, they compare certain characteristics of the people polled to those of the population at large, as measured by the Census. If the percentage of people with any particular characteristic—say, those who live in rural areas—is lower than the national percentage, then rural people will typically be weighted “up,” i.e., their answers to the survey must be attributed to a larger percentage. Alternatively, if rural people are overrepresented in the raw survey results, as compared to the national percentage, they must be weighted “down.”
Morris has no quarrel with weighting per se. It is, he writes, “often a valid way to correct for errors in the sampling.” But he maintains that the Times “weights its data artificially, tilting its numbers to the left.”
Morris does not appear to have discussed the Times’ polling practices with anyone at the Times or at CBS News, which collaborates on most Times polls. (“He should call,” said Mike Kagay, who until recently was Times survey editor, when Chatterbox contacted him by phone. “Advise him to call.”) The evidence Morris cites is a series of Times polls taken between Dec. 2001 and Nov. 2002. These constitute all the polls on matters of national political significance taken during the eleven months prior to the congressional elections. The subjects range from Enron to war with Iraq to the elections themselves. What all these polls have in common, Morris writes, is that
each one … weighted up the number of Democrats and weighted down the number of Republicans—every single time! If the Times were using weighting to adjust for sampling error, surely the weightings would sometimes increase the number of Democrats and would sometimes decrease it. But what the Times has done—increasing the ratio of Democrats to Republicans each time—isn’t weighting the sample. It’s slanting it.
To Chatterbox, that sounded pretty authoritative. But if the Times really were slanting poll data with ideologically driven weighting, one collaborator would have to be Kathleen Frankovic, director of surveys at CBS News (since the two organizations weight identically all poll data they collect together). Frankovic, who is currently president of the World Association for Public Opinion Research, says Morris’ charge is pure nonsense. The main reason CBS/New York Times poll weighting has tended to favor Democrats, she says, is that these polls are conducted by telephone, a method that inevitably undercounts lower-income people, who are highly mobile, less likely to have phones, and tend to be heavily Democratic. CBS/New York Times polls, she explained, are weighted in two stages. First they’re weighted for differences in the probability of selection—i.e., who is more likely or less likely to be reached by the pollsters. (Women, for some reason, tend to pick up the phone more than men, hence they often need to be adjusted down.) Then the polls are weighted in five demographic categories: gender, race, age, education, and region. Contrary to Morris’ claim, CBS/New York Times polls aren’t weighted by party affiliation at all. “I would certainly underscore the fact that [CBS’s] final pre-election [poll] in 2002 for the House vote had the margin correct,” Frankovic adds, “with a 7 point Republican edge.” How could a poll operation slanted to favor Democrats call a Republican victory with such accuracy?
An official Times statement by Catherine Mathis (to whom the Times’ Kagay, who still works there, referred Chatterbox) makes the same point:
It is the Times’s practice, and a standard practice among media pollsters, to weight the samples to reflect the census figures in order to measure public opinion on any issue, not just elections. In this way, we ensure that every demographic group, including African Americans and Hispanics, are appropriately represented in the polls.
In Off With Their Heads, Morris anticipates most of these arguments. “When the pollsters and the editors at the Times read this,” he writes,
they will, undoubtedly, come back and say that they don’t weight for party—they just weight for demographics. They will attribute the change in the partisan mix of their sample to the unintended by-product of a weighting to get proper demographics. … This explanation would hold water if even once in a while these unintended consequences made their poll sample more Republican. But since 9/11, their weighting has always made the sample more Democratic. Every time.
Morris is simply wrong when he says that weighting of Times polls since 9/11 has made the sample more Democratic “every time.” True, he did find a string of polls in 1992 that were all weighted to favor Democrats. But just two weeks ago, the adjustment of a CBS News/New York Times poll on the Bush tax cuts had the effect of weighting Republicans up by 5 percent and weighting Democrats down by 4 percent.
Moreover, Chatterbox can’t find any pollster who will agree with Morris that a string of polls weighted in a manner favoring Democrats constitutes prima facie evidence of partisan tampering. “It doesn’t surprise me at all,” said Thomas Riehle, president of Ipsos Public Affairs, a polling firm. Weighting “will in most cases favor urban markets and minorities,” he explained. “Pointing to the effect it has on party ID is a red herring.” Even John Zogby, at whom Morris throws garlands because he generally weights his polls in a way that favors Republicans, bristles at any suggestion that the Times would slant its polls deliberately to favor Democrats. “Mike Kagay does good work,” Zogby said. “We may have methodological disagreements, but absolutely nothing of the sort would take place.”
Zogby, like the CBS/Times pollsters, weights urban minority groups up. But he frequently differs from them in then weighting Democrats down and Republicans up. Zogby says that most “public” pollsters like CBS News and the New YorkTimes (as opposed to pollsters for political candidates, who keep their results private) don’t weight according to party affiliation at all because these numbers are ever-changing, hence unreliable. Slate’s William Saletan has suggested that “public” pollsters avoid correcting for party affiliation to better cater to their newspaper clients’ desire to have political-horserace polls change as much as possible over time, thereby making a campaign seem more dramatic. That’s what happens when you don’t weight by party affiliation, because you’re not filtering out the volatile party-affiliation numbers.
Zogby often weights Republicans up for various reasons, among them his belief that Democrats are more willing to answer the phone than some Republican sub-groups. His method may be preferable to that of CBS News and the Times, or it may not. But even if it could be proved that it were, and Morris scaled back his argument to accuse the Times merely of using a faulty polling method that tends to favor Democrats, his beef wouldn’t really be with the Times. It would be with a large chunk of the polling profession.
By no means does Chatterbox intend to exonerate the Times from now and then running vaguely dishonest poll stories that put relatively unimportant information at the top because it’s what the editors want to hear, and more important information way down in the 27th paragraph because it’s not what the editors want to hear. Various people, including Morris, fault the Times for doing this. The Times does do it, and it’s annoying. But altering the data itself would be much worse than annoying. It would be a scandal. That’s why Chatterbox has taken up this much space to refute what he considers to be a pretty careless smear on Morris’ part.
How does Morris answer all this? Via e-mail, he said he couldn’t talk about the book at all until after its June 17 publication date, at which time he would further substantiate his weighting allegation. “You are quite wrong,” he said in response to a brief summary of my conclusions, “but we’ll fight over it once the book comes out!”
