Number Crunching

Military analysis.
Oct. 20 2006 6:34 PM

Number Crunching

Taking another look at the Lancet's Iraqstudy.

Let us take another look at the Lancet study. This is the report, issued by a team from Johns Hopkins University and published in the current issue of British medical journal the Lancet, estimating that 655,000 Iraqis have died as a consequence of the U.S.-led invasion. It's a shocking number. Is it true?

Initially, I decided to stay out of this controversy. I'd written the first critique of an earlier Lancet/Hopkins study, which estimated that 100,000 Iraqis had died in just the first year of the war. The study's sample was too small, the data-gathering too slipshod, the range of uncertainty so wide as to render the estimate useless.

Advertisement

The new study looked better: a larger sample, more fastidious attention to data-gathering procedures, a narrower range of uncertainty. The number—655,000 deaths—seemed improbably high (that's an average of 20,000 deaths a month since the war began), but so have a lot of savage statistics that turned out to be true; and there are many areas of Iraq these days where reporters and human rights groups dare not roam.

However, the more I read the study and the more I talked with statisticians, the flimsier this number appeared. The study might be as good an effort as anyone can manage in wartime. Certainly, the Iraqis who went door to door conducting the surveys are amazingly brave souls. But the study has two major flaws—the upshot of which is that it's impossible to infer anything meaningful from it, except that a lot of Iraqis have died and the number is getting higher.

This point should be emphasized. Let's say that the study is way off, off by a factor of 10 or five—in other words, that the right number isn't 655,000 but something between 65,500 and 131,000. That is still a ghastly number—a number that, apart from all other considerations, renders this war a monumental mistake. Here's the key question: Had it been known ahead of time that invading Iraq would result in the deaths of 100,000 Iraqis (or 50,000, or pick your own threshold number), would the president have made—would Congress have voted to authorize, would any editorial writer or public figure have endorsed—a decision to go to war?

Here lies the danger of studies that overstate a war's death toll. The war's supporters and apologists latch on to the inevitable debunkings and proclaim that really "only 100,000" or "only 200,000" people have died. It's obscene—it sullies and coarsens the political culture—to place the word "only" in front of such numbers.

So, let's look at this study's numbers and why they're almost certainly overstated.

The researchers reached this conclusion through a common technique known as "cluster sampling." They randomly selected 47 neighborhoods in 18 of Iraq's regions. Within those neighborhoods, they visited a total of 1,849 households, comprising 12,801 residents, and asked how many of their members had died before the invasion and since the invasion. The researchers then extrapolated from this sample to the entire Iraqi population of 27 million people—from which they concluded that since the war there have been about 655,000 "excess deaths," of which 601,000 were caused by violence.

This methodology is entirely proper if the sample was truly representative of the entire population—i.e., as long as those households were really randomly selected. If they were not randomly selected—if some bias crept into the sampling, even unintentionally—then it is improper, and wildly misleading, to extrapolate the findings to the population as a whole.

There are two reasons to suspect that the sample was not random, and one of those reasons suggests that the sample was biased in a way that exaggerates the death toll.

TODAY IN SLATE

Politics

Meet the New Bosses

How the Republicans would run the Senate.

The Government Is Giving Millions of Dollars in Electric-Car Subsidies to the Wrong Drivers

Scotland Is Just the Beginning. Expect More Political Earthquakes in Europe.

Photos of the Crowds That Took Over NYC for the People’s Climate March

Friends Was the Last Purely Pleasurable Sitcom

The Eye

This Whimsical Driverless Car Imagines Transportation in 2059

Medical Examiner

Did America Get Fat by Drinking Diet Soda?  

A high-profile study points the finger at artificial sweeteners.

I Wrote a Novel Envisioning a Nigerian Space Program. Then I Learned Nigeria Actually Has One.

A Futurama Writer on How the Vietnam War Shaped the Series

  News & Politics
Photography
Sept. 21 2014 11:34 PM People’s Climate March in Photos Hundreds of thousands of marchers took to the streets of NYC in the largest climate rally in history.
  Business
Business Insider
Sept. 22 2014 9:39 AM Adrian Peterson Has a Terrible Contract, and Cutting Him Would Save the Vikings a Lot of Money
  Life
The Eye
Sept. 22 2014 9:12 AM What Is This Singaporean Road Sign Trying to Tell Us?
  Double X
The XX Factor
Sept. 19 2014 4:58 PM Steubenville Gets the Lifetime Treatment (And a Cheerleader Erupts Into Flames)
  Slate Plus
Science
Sept. 22 2014 8:08 AM Slate Voice: “Why Is So Much Honey Clover Honey?” Mike Vuolo shares the story of your honey.
  Arts
Television
Sept. 21 2014 9:00 PM Attractive People Being Funny While Doing Amusing and Sometimes Romantic Things Don’t dismiss it. Friends was a truly great show.
  Technology
Future Tense
Sept. 22 2014 7:47 AM Predicting the Future for the U.S. Government The strange but satisfying work of creating the National Intelligence Council’s Global Trends report.
  Health & Science
Bad Astronomy
Sept. 22 2014 5:30 AM MAVEN Arrives at Mars
  Sports
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.