Taking another look at the Lancet's Iraqstudy.
The Oxford-Holloway team calls this method "main street bias." They add:
Main street bias inflates casualty rates since conflict events such as car bombs, drive-by shootings, artillery strikes on insurgent positions, and marketplace explosions gravitate toward the same neighborhood types that the [Lancet] researchers surveyed. …
In short, the closer you are to a main road, the more likely you are to die in violent activity. So if researchers only count people living close to a main road, then it comes as no surprise they will over-count the dead.
Whether or not the Hopkins researchers were aware of this flaw, or its importance, is unclear. An exchange of e-mails with Gilbert Burnham, the study's chief researcher, raises some disturbing questions about this matter. (Click
It's understandable why the surveyors limited their work to the main roads; they were in strange and dangerous places. But that doesn't negate the Oxford-Holloway team's point. By this measure alone, the Lancet study is not a random survey. In statistically proper random surveys, each household has the same probability of being chosen. Yet in the Lancet survey, if a household wasn't on or near a main road, it had zero chance of being chosen. And "cluster samples" cannot be seen as representative of the entire population unless they are chosen randomly.
The Iraq war is a catastrophe in political, military, and—not least—human terms. How much so may be unfathomable as long as the streets of Iraq are still dangerous. In any event, it's a question that the Lancet study doesn't really answer.
Fred Kaplan is Slate's "War Stories" columnist and author of the book, The Insurgents: David Petraeus and the Plot to Change the American Way of War. He can be reached at email@example.com. Follow him on Twitter.