If a study came out claiming that the average American had a pulse rate of 10 beats per minute, most people would probably put a finger to their wrist for half a minute, ask a couple of friends to do the same, and quickly decide that this was an absurd claim. Yet in his piece headlined "Number Crunching: Taking another look at the Lancet's Iraq study," Slate columnist Fred Kaplan showed that he was not even capable of measuring the pulse in Iraq.
Our estimate of 600,000 Iraqi deaths is far greater than most attempts to measure mortality in Iraq. Last December, President Bush stated that 30,000 "more or less" had died. The president's estimate roughly matched the estimates of Iraq Body Count, which derives its total by monitoring newspaper reports of violent deaths. Today, IBC estimates there have been 45,000 to 50,000 violent deaths. The Brookings Institution has a somewhat higher estimate, suggesting the number of deaths to be about 65,000. Both systems rely on information that can either be observed in press reports or from Iraqi governmental institutions. Our basic approach of interviewing randomly picked clusters of households is the standard way of measuring mortality in times of war. It was how U.S. government researchers estimated death rates during the war in Kosovo and recently in Afghanistan, and it is how the United Nations estimates death rates in dozens of nations annually.
What is curious about the press coverage of this report is that no one has attempted to corroborate the findings on the ground in Iraq. At least 130,000 people would die from natural causes in Iraq each year, even if it was one of the healthiest nations in the world. That would mean roughly 500,000 natural deaths since March of 2003, regardless of whether there was a war or not. Our report implies that over that period, a preponderance of deaths in Iraq were from violence. If President Bush, IBC, and the Brookings Institution are correct, roughly one in 10 deaths over the occupation were from violence. If we are correct, there are now three times as many deaths across Iraq each week as there were in 2002. If IBC and Brookings are correct, there is roughly 10 percent more. A visit to graveyards in Iraq or a look at the morgue data from a few locations would quickly show which crude estimate was in the right ballpark—and overwhelming evidence and testimonials from Iraq suggest that it is the Lancet estimate.
Surveillance by monitoring media reports or bodies in the morgue captures only a small percentage of the deaths occurring, particularly when the Iraqi health-information system and infrastructure are practically nonexistent. In July, for example, the Ministry of Health reported exactly zero violent deaths in Anbar Province, in spite of the contradictory evidence we saw on our televisions. Is that a surveillance network on which our understanding of what is going on in Iraq can depend? For most of the people commenting on our report, the answer appears to be yes.
This brings us to Fred Kaplan's column regarding our study. In 2004, Kaplan worked to discredit our first Lancet report on Iraqi deaths in 2004 by focusing on the imprecision in one of the three findings and ignoring the other two. Kaplan's latest article focused on two baseless criticisms of our 2006 study. First, he claimed that our measured base line rate, the rate of natural deaths for the year before the invasion, was too low. We had estimated the rate to be 5.5 deaths per thousand per year. Kaplan claims that the rate was really 10, according to U.N. figures. He wrote, "[I]f Iraq's pre-invasion rate really was 5.5 per 1,000, it was lower than almost every country in the Middle East, and many countries in Europe." This is just wrong! If Kaplan had checked the U.N. death-rate figures, most Middle Eastern nations really do have lower death rates than most European countries, and in fact have lower death rates than 5.5. Jordan's death rate is 4.2, Iran's 5.3, and Syria's 3.5. The reason for the lower rate is simple: Most Middle Eastern nations have much younger populations compared to most Western nations. Obviously, the elderly die at a greater rate than young people and account for a disproportionate number of deaths. Even if it were true that our prewar death rate for Iraq was too low, Kaplan doesn't emphasize that such an error would almost certainly make our post-invasion estimate too low, not too high.
Kaplan's second fault with our study was his claim that our sampling suffered from "main street bias." That is to say, we systematically missed the back streets and focused on main streets, where he assumes more deaths have occurred. This claim is not credible for a number of reasons. Our study team worked very hard to ensure that our sample households were selected at random. We set up rigorous guidelines and methods so that any street block within our chosen village had an equal chance of being selected. Once we started, we went to the next nearest 39 doorways in a chain that typically spanned two to three blocks. Thus, the first-picked block usually did not provide most of the houses within a given cluster. It is also important that most violent deaths probably happened outside of the home, making the location of the house on the street irrelevant. It is interesting to note that in a world where there are thousands of epidemiologists and hundreds of people who have done epidemiological surveys in conflict settings, Kaplan's source for this presumed flaw was the speculation of two physicists and an economist.
If Kim Jong-il were to state that he felt there had been, more or less, 200 American deaths during the 9/11 attacks, we would be appalled. To have our leaders claiming that there have been only 50,000 or 60,000 violent deaths is equally appalling to those in the Middle East who know better. The media could be a force for correcting this fanning of international tensions, but not if they squander their opportunities, saying what they feel or hope for instead of going to the field and reporting what is.
Fred Kaplan replies:
What a bizarre rebuttal. Burnham and Roberts chastise me for not traipsing through Iraq's morgues and graveyards, an adventure that they claim would "quickly show" which estimate of Iraqi deaths "is in the right ballpark." But a) neither they nor their team conducted this sort of research, b) no mortal could these days, and c) it's not at all clear what such a survey would reveal. They themselves say that "bodies in the morgue captures [sic] only a small percentage of the deaths occurring." Yet they also cite "overwhelming evidence and testimonials from Iraq [that] suggest" their numbers—and not any other organization's—are correct. Two questions: What are these "testimonials"? And how many such testimonials do you need to infer that 650,000 Iraqis have died, as opposed to, say, 250,000 or 50,000?
They liken the task of finding the average death rate to that of finding the average pulse rate. But humans have very similar circulatory systems. Neighborhoods, on the other hand, have very different patterns of bombings, shellings, strafings, and the like.