Number Crunching

Military analysis.
Oct. 20 2006 6:34 PM

Number Crunching

Taking another look at the Lancet's Iraqstudy.

(Continued from Page 1)

First, the Lancet study, like all such studies, estimates not how many people have died, but rather the difference between how many people died in a comparable period before the invasion and how many people have died since the invasion. As the study puts it, 655,000 is roughly the number of deaths "above the number that would be expected in a non-conflict situation."

In any such study, it's crucial that the base-line number—deaths before the invasion—is correct. The Lancet study's base-line number is dubious.

Advertisement

Based on the household surveys, the report estimates that, just before the war, Iraq's mortality rate was 5.5 per 1,000. (That is, for every 1,000 people, 5.5 die each year.) The results also show that, in the three and a half years since the war began, this rate has shot up to 13.3 per 1,000. So, the "excess deaths" amount to 7.8 (13.3 minus 5.5) per 1,000. They extrapolate from this figure to reach their estimate of 655,000 deaths.

However, according to data from the United Nations, based on surveys taken at the time, Iraq's preinvasion mortality rate was 10 per 1,000. The difference between 13.3 and 10.0 is only 3.3, less than half of 7.8.

Does that mean that the post-invasion death toll is less than half of 655,000? Not necessarily. You can't just take the data from one survey and plug them into another survey. Maybe the Hopkins survey understated post-invasion deaths as much as it understated preinvasion deaths—in which case, the net effect is nil. Maybe not. Either way, it should have been clear to the data-crunchers that something was wrong with the numbers for preinvasion and post-invasion deaths, since they were derived from the same survey.

"When you get these large discrepancies between your own results and results that are already well-established, you recrunch your numbers or you send your survey team back into the field to widen your sample," Beth Osborne Daponte, a demographer at Yale University who has worked on many studies of this sort, told me in a phone interview. "Obviously, they couldn't do that here. It's too dangerous. But that doesn't change the point. You need to triangulate your data"—to make sure they match other data, or, if they don't, to figure out why. "They didn't do that."

(If the Hopkins researchers want to claim that their estimate is more reliable than the United Nations', they will have to prove the point. It is also noteworthy that, if Iraq's preinvasion mortality rate really was 5.5 per 1,000, it was lower than that of almost every country in the Middle East, and many countries in Western Europe.)

This flaw—or discrepancy—doesn't tell you whether 655,000 is too high, too low, or (serendipitously) just right. It just tells you that something about the number is almost certainly off.

However, the second flaw suggests that the number is almost certainly too high.

A joint research team led by physicists Sean Gourley and Neil Johnson of Oxford University and economist Michael Spagat at Royal Holloway University in London noticed the second flaw. In a statement released Thursday (and reported in today's issue of the journal Science), they charged that the Lancet study is "fundamentally flawed"—and in a way that systematically overstates the death toll.

The Lancet study, in its section on methodology, notes that the teams picked the houses they would survey from a "random selection of main streets," defined as "major commercial streets and avenues." (Italics added.) They also chose from a "list of residential streets crossing" those main streets.

TODAY IN SLATE

Foreigners

The World’s Politest Protesters

The Occupy Central demonstrators are courteous. That’s actually what makes them so dangerous.

The Religious Right Is Not Happy With Republicans  

The XX Factor
Oct. 1 2014 4:58 PM The Religious Right Is Not Happy With Republicans  

How Did the Royals Win Despite Bunting So Many Times? Bunting Is a Terrible Strategy.

Federal Law Enforcement Has Declared War on Encryption

Justice and the FBI really do not like Apple’s and Google’s new privacy measures.

Homeland Is Good Again! For Now.

Crime

Operation Backbone

How White Boy Rick, a legendary Detroit cocaine dealer, helped the FBI uncover brazen police corruption.

Music

How Even an Old Hipster Can Age Gracefully

On their new albums, Leonard Cohen, Robert Plant, and Loudon Wainwright III show three ways.

The One Fact About Ebola That Should Calm You: It Spreads Slowly

Piper Kerman on Why She Dressed Like a Hitchcock Heroine for Her Prison Sentencing

  News & Politics
Politics
Oct. 1 2014 7:26 PM Talking White Black people’s disdain for “proper English” and academic achievement is a myth.
  Business
Moneybox
Oct. 1 2014 2:16 PM Wall Street Tackles Chat Services, Shies Away From Diversity Issues 
  Life
Outward
Oct. 1 2014 6:02 PM Facebook Relaxes Its “Real Name” Policy; Drag Queens Celebrate
  Double X
The XX Factor
Oct. 1 2014 4:05 PM Today in GOP Outreach to Women: You Broads Like Wedding Dresses, Right?
  Slate Plus
Behind the Scenes
Oct. 1 2014 3:24 PM Revelry (and Business) at Mohonk Photos and highlights from Slate’s annual retreat.
  Arts
Brow Beat
Oct. 1 2014 6:39 PM Spoiler Special: Transparent
  Technology
Future Tense
Oct. 1 2014 6:59 PM EU’s Next Digital Commissioner Thinks Keeping Nude Celeb Photos in the Cloud Is “Stupid”
  Health & Science
Science
Oct. 1 2014 4:03 PM Does the Earth Really Have a “Hum”? Yes, but probably not the one you’re thinking.
  Sports
Sports Nut
Sept. 30 2014 5:54 PM Goodbye, Tough Guy It’s time for Michigan to fire its toughness-obsessed coach, Brady Hoke.