The Lancet number crunchers respond to Slate's Fred Kaplan. (And Kaplan replies.)
Posted Monday, Nov. 20, 2006, at 3:58 PM
"Cluster sampling" is indeed a standard technique for estimating mortality in conditions where precise counts are impractical. But sampling—of any sort—means little unless the sample reflects the overall population; and in order to ensure that it does, the surveyors have to follow highly rigorous methods that ensure the sample is chosen randomly.
It is in this regard that the Lancet study's methods and claims are highly questionable. Rather than rehearse the arguments once more, I urge readers to take another look at my original column (especially to the hyperlinks). Burnham and Roberts chide me for relying on physicists and economists instead of epidemiologists. First, the issues at hand concern statistical methods, for which my sources have impeccable credentials. Second, the authors conveniently ignore my citation of Professor Beth Osborne Daponte, an eminent demographer at Yale University who has conducted precisely these sorts of studies. They may also be unaware that the Oxford team includes Professor Gesine Reinert, who is experienced in this field as well.
The second part of their rebuttal on the matter of "main street bias"—that people might have been killed away from home, so it doesn't matter if the houses surveyed were disproportionately close to violence—might be valid for many countries. But in Iraq, unemployment is high, transportation is limited (in part due to scarcity of gasoline), and in many cities, Sunnis don't go into Shiite neighborhoods, and vice versa. This controversy would be easier to settle if Burnham and Roberts were clearer—and less contradictory—about just how their main streets were chosen.
Then there is the issue of Iraq's prewar mortality rates, a crucial issue that they willfully misconstrue. The aim of their study was to measure the number of "excess deaths" since the war began—i.e., the number of Iraqis who have died since the war started minus the number who would have died in this time period if there hadn't been a war. A surrogate for this latter figure is the number who died in the same time span before the war started. They claim this figure is 5.5 Iraqis per 1,000 of population per year. I noted that the United Nations puts Iraq's prewar mortality rate at 10 per 1,000. I should add here that Jon Pederson of the United Nations Development Corporation thinks even that may be too low. The Lancet study estimates that, on average, 13.3 Iraqis per 1,000 have died, of all causes, since the war started. If their pre-war assumption is right, the excess deaths—7.8 per 1,000 (13.3 minus 5.5)—are high. If the U.N. assumption is right, the excess deaths—3.3 per thousand (13.3 - 10)—are less high.
Pederson tells me, in an e-mail, "I simply do not believe that it is possible to obtain accurate mortality estimate[s] with the methods they are using as far back in time as they purport to do." He adds (and I agree here wholeheartedly), "That being said—a lot of people have died in Iraq."
Gilbert Burnham is a professor of international health at the Bloomberg School of Public Health at Johns Hopkins University.
Les Roberts teaches at the program on forced migration and health at Columbia University's Mailman School of Public Health.