Number Crunching

Military analysis.
Oct. 20 2006 6:34 PM

Number Crunching

Taking another look at the Lancet's Iraqstudy.

(Continued from Page 1)

First, the Lancet study, like all such studies, estimates not how many people have died, but rather the difference between how many people died in a comparable period before the invasion and how many people have died since the invasion. As the study puts it, 655,000 is roughly the number of deaths "above the number that would be expected in a non-conflict situation."

In any such study, it's crucial that the base-line number—deaths before the invasion—is correct. The Lancet study's base-line number is dubious.

Advertisement

Based on the household surveys, the report estimates that, just before the war, Iraq's mortality rate was 5.5 per 1,000. (That is, for every 1,000 people, 5.5 die each year.) The results also show that, in the three and a half years since the war began, this rate has shot up to 13.3 per 1,000. So, the "excess deaths" amount to 7.8 (13.3 minus 5.5) per 1,000. They extrapolate from this figure to reach their estimate of 655,000 deaths.

However, according to data from the United Nations, based on surveys taken at the time, Iraq's preinvasion mortality rate was 10 per 1,000. The difference between 13.3 and 10.0 is only 3.3, less than half of 7.8.

Does that mean that the post-invasion death toll is less than half of 655,000? Not necessarily. You can't just take the data from one survey and plug them into another survey. Maybe the Hopkins survey understated post-invasion deaths as much as it understated preinvasion deaths—in which case, the net effect is nil. Maybe not. Either way, it should have been clear to the data-crunchers that something was wrong with the numbers for preinvasion and post-invasion deaths, since they were derived from the same survey.

"When you get these large discrepancies between your own results and results that are already well-established, you recrunch your numbers or you send your survey team back into the field to widen your sample," Beth Osborne Daponte, a demographer at Yale University who has worked on many studies of this sort, told me in a phone interview. "Obviously, they couldn't do that here. It's too dangerous. But that doesn't change the point. You need to triangulate your data"—to make sure they match other data, or, if they don't, to figure out why. "They didn't do that."

(If the Hopkins researchers want to claim that their estimate is more reliable than the United Nations', they will have to prove the point. It is also noteworthy that, if Iraq's preinvasion mortality rate really was 5.5 per 1,000, it was lower than that of almost every country in the Middle East, and many countries in Western Europe.)

This flaw—or discrepancy—doesn't tell you whether 655,000 is too high, too low, or (serendipitously) just right. It just tells you that something about the number is almost certainly off.

However, the second flaw suggests that the number is almost certainly too high.

A joint research team led by physicists Sean Gourley and Neil Johnson of Oxford University and economist Michael Spagat at Royal Holloway University in London noticed the second flaw. In a statement released Thursday (and reported in today's issue of the journal Science), they charged that the Lancet study is "fundamentally flawed"—and in a way that systematically overstates the death toll.

The Lancet study, in its section on methodology, notes that the teams picked the houses they would survey from a "random selection of main streets," defined as "major commercial streets and avenues." (Italics added.) They also chose from a "list of residential streets crossing" those main streets.

TODAY IN SLATE

Medical Examiner

Here’s Where We Stand With Ebola

Even experienced international disaster responders are shocked at how bad it’s gotten.

It’s Legal for Obama to Bomb Syria Because He Says It Is

Divestment Is Fine but Mostly Symbolic. There’s a Better Way for Universities to Fight Climate Change.

I Stand With Emma Watson on Women’s Rights

Even though I know I’m going to get flak for it.

It Is Very Stupid to Compare Hope Solo to Ray Rice

Building a Better Workplace

In Defense of HR

Startups and small businesses shouldn’t skip over a human resources department.

Why Is This Mother in Prison for Helping Her Daughter Get an Abortion?

How Ted Cruz and Scott Brown Misunderstand What It Means to Be an American Citizen

  News & Politics
Politics
Sept. 23 2014 12:43 PM Occupy Wall Street How can Hillary Clinton be both a limousine liberal and a Saul Alinsky radical?
  Business
Moneybox
Sept. 23 2014 2:08 PM Home Depot’s Former Head of Security Had a Legacy of Sabotage
  Life
Outward
Sept. 23 2014 1:57 PM Would a Second Sarkozy Presidency End Marriage Equality in France?
  Double X
The XX Factor
Sept. 23 2014 2:32 PM Politico Asks: Why Is Gabby Giffords So “Ruthless” on Gun Control?
  Slate Plus
Political Gabfest
Sept. 23 2014 3:04 PM Chicago Gabfest How to get your tickets before anyone else.
  Arts
Brow Beat
Sept. 23 2014 3:29 PM Tracking Down the Mysterious Typeface of Savannah, Georgia
  Technology
Future Tense
Sept. 23 2014 1:50 PM Oh, the Futility! Frogs Try to Catch Worms off of an iPhone Video.
  Health & Science
Science
Sept. 23 2014 1:38 PM Why Is Fall Red in America but Yellow in Europe? A possible explanation, 35 million years in the making.
  Sports
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.