Number Crunching

Military analysis.
Oct. 20 2006 6:34 PM

Number Crunching

Taking another look at the Lancet's Iraqstudy.

(Continued from Page 1)

First, the Lancet study, like all such studies, estimates not how many people have died, but rather the difference between how many people died in a comparable period before the invasion and how many people have died since the invasion. As the study puts it, 655,000 is roughly the number of deaths "above the number that would be expected in a non-conflict situation."

In any such study, it's crucial that the base-line number—deaths before the invasion—is correct. The Lancet study's base-line number is dubious.

Advertisement

Based on the household surveys, the report estimates that, just before the war, Iraq's mortality rate was 5.5 per 1,000. (That is, for every 1,000 people, 5.5 die each year.) The results also show that, in the three and a half years since the war began, this rate has shot up to 13.3 per 1,000. So, the "excess deaths" amount to 7.8 (13.3 minus 5.5) per 1,000. They extrapolate from this figure to reach their estimate of 655,000 deaths.

However, according to data from the United Nations, based on surveys taken at the time, Iraq's preinvasion mortality rate was 10 per 1,000. The difference between 13.3 and 10.0 is only 3.3, less than half of 7.8.

Does that mean that the post-invasion death toll is less than half of 655,000? Not necessarily. You can't just take the data from one survey and plug them into another survey. Maybe the Hopkins survey understated post-invasion deaths as much as it understated preinvasion deaths—in which case, the net effect is nil. Maybe not. Either way, it should have been clear to the data-crunchers that something was wrong with the numbers for preinvasion and post-invasion deaths, since they were derived from the same survey.

"When you get these large discrepancies between your own results and results that are already well-established, you recrunch your numbers or you send your survey team back into the field to widen your sample," Beth Osborne Daponte, a demographer at Yale University who has worked on many studies of this sort, told me in a phone interview. "Obviously, they couldn't do that here. It's too dangerous. But that doesn't change the point. You need to triangulate your data"—to make sure they match other data, or, if they don't, to figure out why. "They didn't do that."

(If the Hopkins researchers want to claim that their estimate is more reliable than the United Nations', they will have to prove the point. It is also noteworthy that, if Iraq's preinvasion mortality rate really was 5.5 per 1,000, it was lower than that of almost every country in the Middle East, and many countries in Western Europe.)

This flaw—or discrepancy—doesn't tell you whether 655,000 is too high, too low, or (serendipitously) just right. It just tells you that something about the number is almost certainly off.

However, the second flaw suggests that the number is almost certainly too high.

A joint research team led by physicists Sean Gourley and Neil Johnson of Oxford University and economist Michael Spagat at Royal Holloway University in London noticed the second flaw. In a statement released Thursday (and reported in today's issue of the journal Science), they charged that the Lancet study is "fundamentally flawed"—and in a way that systematically overstates the death toll.

The Lancet study, in its section on methodology, notes that the teams picked the houses they would survey from a "random selection of main streets," defined as "major commercial streets and avenues." (Italics added.) They also chose from a "list of residential streets crossing" those main streets.

TODAY IN SLATE

Technocracy

Forget Oculus Rift

This $25 cardboard box turns your phone into an incredibly fun virtual reality experience.

Republicans Want the Government to Listen to the American Public on Ebola. That’s a Horrible Idea.

The 2014 Kansas City Royals Show the Value of Building a Mediocre Baseball Team

The GOP Won’t Win Any Black Votes With Its New “Willie Horton” Ad

Sleater-Kinney Was Once America’s Best Rock Band

Can it be again?

Politics

Smash and Grab

Will competitive Senate contests in Kansas and South Dakota lead to more late-breaking races in future elections?

I Am 25. I Don’t Work at Facebook. My Doctors Want Me to Freeze My Eggs.

These Companies in Japan Are More Than 1,000 Years Old

  News & Politics
The World
Oct. 21 2014 11:40 AM The U.S. Has Spent $7 Billion Fighting the War on Drugs in Afghanistan. It Hasn’t Worked. 
  Business
Business Insider
Oct. 21 2014 11:27 AM There Is Now a Real-life Hoverboard You Can Preorder for $10,000
  Life
Atlas Obscura
Oct. 21 2014 12:40 PM Asamkirche: The Rococo Church Where Death Hides in Plain Sight
  Double X
The XX Factor
Oct. 20 2014 6:17 PM I Am 25. I Don’t Work at Facebook. My Doctors Want Me to Freeze My Eggs.
  Slate Plus
Tv Club
Oct. 20 2014 7:15 AM The Slate Doctor Who Podcast: Episode 9 A spoiler-filled discussion of "Flatline."
  Arts
Behold
Oct. 21 2014 12:05 PM Same-Sex Couples at Home With Themselves in 1980s America
  Technology
Technology
Oct. 21 2014 10:43 AM Social Networking Didn’t Start at Harvard It really began at a girls’ reform school.
  Health & Science
Climate Desk
Oct. 21 2014 11:53 AM Taking Research for Granted Texas Republican Lamar Smith continues his crusade against independence in science.
  Sports
Sports Nut
Oct. 20 2014 5:09 PM Keepaway, on Three. Ready—Break! On his record-breaking touchdown pass, Peyton Manning couldn’t even leave the celebration to chance.