Tax Poll Loopholes
Posted Thursday, March 8, 2001, at 11:30 PM ET
Supporters of President Bush's tax cut plan are trying to convince Congress and the media that the plan adds up. Critics are claiming the opposite. But that isn't the only numbers game driving the debate. Bush's foes are trying to rally congressional resistance to the plan by circulating polls that purportedly show the public doesn't want it. Bush and his allies are trying to break down that resistance with polls suggesting the opposite. Like the budget math game, the public opinion math game is full of skewed and suspect data. Here are a few questions worth asking of each survey.
PRINT
DISCUSS
E-MAIL
RSS
RECOMMEND...
Facebook
MySpace
Mixx
Digg
Reddit
del.icio.us
Furl
Ma.gnolia
Sphere
StumbleUpon1. How is the proposal characterized? Last week, the news pages of the Wall Street Journal cited a poll commissioned by the Issues Management Center, a pro-Bush tax cut group, "showing that 57% of South Dakota voters support the Bush tax-cut plan, with 27% opposing it." This came just a week after the Associated Press, trumpeting a Pew Research Center national survey in which the tax cut was favored by just 43 percent to 33 percent, reported that the plan "has found a lukewarm reception."
Why the discrepancy? Look at the questions. The Pew survey asked, "Do you approve or disapprove of George W. Bush's tax proposal?" Since the question didn't mention that the proposal involved cutting taxes, fewer respondents approved. The IMC poll, on the other hand, asked, "Do you support or oppose President Bush's plan to cut taxes by $1.6 trillion over the next ten years?" This time, more people responded favorably—presumably because they liked the size of the tax cut or were comforted to learn that it would be spread out over 10 years.
What happens when you include the basic information omitted by Pew (that it's a tax cut) but omit the additional information included by IMC ($1.6 trillion over 10 years)? You get the question Gallup asked four weeks ago: "Do you favor or oppose the federal income tax cuts President Bush has proposed?" The answer fell halfway between the other polls: 53 percent approved, 30 percent disapproved.
2. What's the comparison? Media-bias watchdogs accuse CBS of squelching data favorable to Bush. Boston Globe columnist Jeff Jacoby, among others, protested that Dan Rather made "no mention of CBS's own poll showing that a whopping 67 percent of the public likes Bush's plan." But the critics made a strategic omission of their own: In the same survey, when voters were asked which of four proposals they preferred—tax cuts, deficit reduction, more education spending, and more money for Social Security and Medicare—only 35 percent chose tax cuts. This is an intuitive law of polling: The more alternatives a proposal is compared to, the more its support shrinks. Bush's allies want a yes/no question that essentially compares his tax cut to nothing. The Democratic National Committee and the Democratic Leadership Council, on the other hand, are peddling surveys in which voters are given a list of several options for allocating the surplus. In these surveys, the percentage preferring tax cuts falls below 20 percent.
A comparative question doesn't necessarily capture public opinion more accurately than a yes/no question does. The 35 percent who preferred tax cuts in the CBS News poll was the biggest of the four factions produced by that question. But by scattering the respondents among those four categories, the question reduced the pro-tax cut figure from 67 to 35. Comparative questions also assume that the alternatives presented are mutually exclusive—an unproven premise that in this case is the crux of the debate. When asked whether it's possible "to preserve programs like Social Security and Medicare, increase spending on education, and implement the $1.6 trillion income tax cuts George W. Bush has proposed at the same time," 60 percent of CBS News respondents said yes. Nevertheless, the comparative question obliged them to assume the contrary.
When a poll poses the question only one way—as the Washington Post/ABC survey did on the eve of Bush's budget speech—readers get only half the story. The Post/ABC poll skipped the yes/no question and instead asked respondents to choose 1) among tax cuts, debt reduction, Social Security, education, and health care; and 2) between Bush's tax cut and a "smaller tax-cut plan that provides targeted tax cuts mainly for lower and middle-income people." In the first question, cutting taxes came in third out of four, with 22 percent support. In the second question, Bush's tax cut trailed 53-43. The poll was widely reported as a rejection of Bush's plan. "Poll Shows Bush Tax Plan May Be Hard Sell," warned the Minneapolis Star-Tribune. Commentator Kevin Phillips cited the 22 percent figure as evidence that Bush's budget was "contemptuous of public opinion." Reuters artificially lumped together the three non-tax-cut alternatives offered in the Post's question, concluding, "Seventy-seven percent cited other priorities."
Readers should especially beware comparative questions disguised as yes/no questions. On March 8, the front page of the Los Angeles Times ("Public Supports Bush, Is Divided on His Tax Cut Plan") reported that according to a new Times poll, only "a narrow majority of Americans now says it would support Bush's proposed $1.6-trillion tax cut." Readers who examined the Times' separate 4,000-word analysis of the poll were told, "Forty percent disapprove of [Bush's] plan." Only those who insisted on seeing the raw questionnaire discovered that the question actually asked respondents whether they approved of Bush's tax cut "rather than a targeted tax cut for low- and middle-income families."
3. How are the data categorized? Lumping together two categories of responses is an easy way to make them look bigger. IMC told the media that in its poll, "A combined 52% (43% right size, 9% should be bigger) believe the Bush tax cut is either the right size or should be larger." IMC didn't mention, however, that 30 percent of respondents said the tax cut should be smaller and that, therefore, a combined 73 percent believed the tax cut was either the right size or should be smaller. Pollsters also know that respondents, given a spectrum of answers, will gravitate toward the center. If IMC had wanted results favoring the Democratic tax cut proposal, it would have put Bush's plan at the high end of the spectrum, "no tax cut" at the low end, and the Democratic plan in the middle.
Newsweek's recent survey used the same lumping device in reverse. It made the pro-tax-cut constituency look small by lumping together the alternatives in a bipolar question: "In general, would you prefer to see the federal surplus used mainly to cut people's taxes or to pay down the federal debt and make Social Security and Medicare more financially solvent?" Naturally, tax cuts got creamed. How many people would have preferred a combination of tax cuts and debt repayment or tax cuts and shoring up Social Security? We don't know, because those possibilities weren't offered. But the gang-up format of the question escaped the media's notice. The Post reported that 65 percent of Newsweek respondents "favor using projected budget surpluses to reduce the national debt."
4. Are the comparisons fair? A week ago, Gallup proclaimed that in its own survey, "the public was asked to rate each of six different priorities for the Bush administration. As can be seen, 'cutting federal income taxes' comes in dead last." That figure quickly permeated the media. "Out of six public policies tested, cutting federal income taxes ranked last," reported the New York Times. But the question was horribly skewed. The five alternatives that beat out tax cuts were "improving education," "keeping America prosperous," "dealing with the energy problems facing the nation," "keeping the federal budget balanced," and "providing military security for the country." Those aren't policies. They're objectives. Worse, they're objectives nobody opposes. Any disputed policy served up on a list with them was certain to come in last.
According to the Houston Chronicle ("Many Show Skepticism Over Tax Cut"), a recent Reuters/Zogby poll "found that voters ranked cutting taxes fourth in their priorities, behind funding programs for working families, saving Social Security and paying down the national debt." The precise wording of the fourth option offered to respondents was "programs that help working families, such as child care, expanded health care coverage, and education." Naturally, this option won. Cutting taxes trailed debt repayment by a single, statistically insignificant percentage point, but was pronounced the loser.
5. What's the trend? "It has become increasingly clear that Bush still confronts significant public and legislative skepticism" about his tax cut, the Los Angeles Times asserted on Feb. 27, citing the Pew survey. Other news outlets noted that tax cuts came in third out of four options in the poll, with just 19 percent support. Increasingly clear? The media failed to mention that this was double the percentage who had chosen tax cuts on the same question in the same poll three years earlier. Meanwhile, the Post, outlining its own survey findings, trumpeted "new indications that [Bush] will have a difficult time achieving passage of key parts of his agenda. … Only 22 percent favored a tax cut as the top priority." The Dallas Morning News told its readers that the Post survey "found tax relief falling behind increased spending for education and other domestic programs and strengthening Social Security." Neither article acknowledged that the 22 percent figure was up from 14 percent in the same poll five months earlier.
Does this mean you can't believe poll numbers? No. It just means you have to scrutinize them like budget numbers. Accept no paraphrases or characterizations. Demand to see the exact questions. And bring a few questions of your own.
When Congress Sends a Bill to the President, Do They Use E-Mail?
Gov. Haley Barbour's Strange Habit of Pardoning Murderers Who Work on His House
Slowpoke Directors Explained: Why It Took 12 Years to Make Avatar
The Surprising Reason Banks Are Suddenly Repaying Their TARP Funds
How Come You Don't Hear About the "War on Christmas" Anymore?
Jeff Bridges Gives the Performance of the Year in Crazy Heart
Reader Comments From the Fray:
It is distressing to continually see polling questions asking people their opinion on something they are unlikely to have any hard information about. For example, "...do you think we can increase spending for education, shore up social security and give ourselves a $1.6 trillion tax cut...?" is so vague as to be absolutely meaningless. Even if it were quite specific, how many people are knowledgeable enough about long-term bond debt, interest rates, inflation corrections, etc to give an informed answer? I suspect the people who are most informed about the budgetary dynamics would respond, "It depends...". These questions are like asking people their opinion about what Dubya's favorite insect is. For the most part, they have no reasonable information on which to base a meaningful opinion. I think all polling questions should be followed up with a second query asking whether one would be willing to be tested about their degree of understanding of the original question.
--DB
(To reply, click here.)
Like most "inside baseball" analysis of politics, the quest for the perfect polling technique ignores the heart of the matter--people respond (including by changing their views) to effective leadership. It is undeniable that the whole ground of the debate (10 year projections of the economy, government spending, interest rates, etc.) is, like all politically driven reality, uncertain. But as we saw in the 1980s, tax cuts can themselves change the ground of any future debate over deficits to one of controlling government spending (in my view rightfully). This in turn limited the damage a 2-term Clinton could do. By the same token, look how quickly the New Democrats Gore and Lieberman metamorphosed into LBJ-style populists with the appearance of the huge surpluses (to say nothing of Republican Congressmen becoming big spenders). Leadership can and should drive polls, not the other way around
--Mitch
(To reply, click here.)
(3/9)