Number Crunching

Military analysis.
Oct. 20 2006 6:34 PM

Number Crunching

Taking another look at the Lancet's Iraqstudy.

Let us take another look at the Lancet study. This is the report, issued by a team from Johns Hopkins University and published in the current issue of British medical journal the Lancet, estimating that 655,000 Iraqis have died as a consequence of the U.S.-led invasion. It's a shocking number. Is it true?

Initially, I decided to stay out of this controversy. I'd written the first critique of an earlier Lancet/Hopkins study, which estimated that 100,000 Iraqis had died in just the first year of the war. The study's sample was too small, the data-gathering too slipshod, the range of uncertainty so wide as to render the estimate useless.

Advertisement

The new study looked better: a larger sample, more fastidious attention to data-gathering procedures, a narrower range of uncertainty. The number—655,000 deaths—seemed improbably high (that's an average of 20,000 deaths a month since the war began), but so have a lot of savage statistics that turned out to be true; and there are many areas of Iraq these days where reporters and human rights groups dare not roam.

However, the more I read the study and the more I talked with statisticians, the flimsier this number appeared. The study might be as good an effort as anyone can manage in wartime. Certainly, the Iraqis who went door to door conducting the surveys are amazingly brave souls. But the study has two major flaws—the upshot of which is that it's impossible to infer anything meaningful from it, except that a lot of Iraqis have died and the number is getting higher.

This point should be emphasized. Let's say that the study is way off, off by a factor of 10 or five—in other words, that the right number isn't 655,000 but something between 65,500 and 131,000. That is still a ghastly number—a number that, apart from all other considerations, renders this war a monumental mistake. Here's the key question: Had it been known ahead of time that invading Iraq would result in the deaths of 100,000 Iraqis (or 50,000, or pick your own threshold number), would the president have made—would Congress have voted to authorize, would any editorial writer or public figure have endorsed—a decision to go to war?

Here lies the danger of studies that overstate a war's death toll. The war's supporters and apologists latch on to the inevitable debunkings and proclaim that really "only 100,000" or "only 200,000" people have died. It's obscene—it sullies and coarsens the political culture—to place the word "only" in front of such numbers.

So, let's look at this study's numbers and why they're almost certainly overstated.

The researchers reached this conclusion through a common technique known as "cluster sampling." They randomly selected 47 neighborhoods in 18 of Iraq's regions. Within those neighborhoods, they visited a total of 1,849 households, comprising 12,801 residents, and asked how many of their members had died before the invasion and since the invasion. The researchers then extrapolated from this sample to the entire Iraqi population of 27 million people—from which they concluded that since the war there have been about 655,000 "excess deaths," of which 601,000 were caused by violence.

This methodology is entirely proper if the sample was truly representative of the entire population—i.e., as long as those households were really randomly selected. If they were not randomly selected—if some bias crept into the sampling, even unintentionally—then it is improper, and wildly misleading, to extrapolate the findings to the population as a whole.

There are two reasons to suspect that the sample was not random, and one of those reasons suggests that the sample was biased in a way that exaggerates the death toll.

TODAY IN SLATE

History

The Self-Made Man

The story of America’s most pliable, pernicious, irrepressible myth.

Rehtaeh Parsons Was the Most Famous Victim in Canada. Now, Journalists Can’t Even Say Her Name.

Mitt Romney May Be Weighing a 2016 Run. That Would Be a Big Mistake.

Amazing Photos From Hong Kong’s Umbrella Revolution

Transparent Is the Fall’s Only Great New Show

The XX Factor

Rehtaeh Parsons Was the Most Famous Victim in Canada

Now, journalists can't even say her name.

Doublex

Lena Dunham, the Book

More shtick than honesty in Not That Kind of Girl.

What a Juicy New Book About Diane Sawyer and Katie Couric Fails to Tell Us About the TV News Business

Does Your Child Have Sluggish Cognitive Tempo? Or Is That Just a Disorder Made Up to Scare You?

  News & Politics
Damned Spot
Sept. 30 2014 9:00 AM Now Stare. Don’t Stop. The perfect political wife’s loving gaze in campaign ads.
  Business
Moneybox
Sept. 29 2014 7:01 PM We May Never Know If Larry Ellison Flew a Fighter Jet Under the Golden Gate Bridge
  Life
Atlas Obscura
Sept. 30 2014 10:10 AM A Lovable Murderer and Heroic Villain: The Story of Australia's Most Iconic Outlaw
  Double X
Doublex
Sept. 29 2014 11:43 PM Lena Dunham, the Book More shtick than honesty in Not That Kind of Girl.
  Slate Plus
Slate Fare
Sept. 29 2014 8:45 AM Slate Isn’t Too Liberal. But… What readers said about the magazine’s bias and balance.
  Arts
Brow Beat
Sept. 29 2014 9:06 PM Paul Thomas Anderson’s Inherent Vice Looks Like a Comic Masterpiece
  Technology
Future Tense
Sept. 30 2014 7:36 AM Almost Humane What sci-fi can teach us about our treatment of prisoners of war.
  Health & Science
Bad Astronomy
Sept. 30 2014 7:30 AM What Lurks Beneath The Methane Lakes of Titan?
  Sports
Sports Nut
Sept. 28 2014 8:30 PM NFL Players Die Young. Or Maybe They Live Long Lives. Why it’s so hard to pin down the effects of football on players’ lives.