We've heard that obesity and divorce can be passed from one person to another. Critics now wonder how the "social contagion" studies ever passed peer review.
Have you heard that divorce is contagious? A lot of people have. Last summer a study claiming to show that break-ups can propagate from friend to friend to friend like a marriage-eating bacillus spread across the news agar from CNN to CBS to ABC with predictable speed. "Think of this 'idea' of getting divorced, this 'option' of getting divorced like a virus, because it spreads more or less the same way," explained University of California-San Diego professor James Fowler to the folks at Good Morning America.
It's a surprising, quirky, and seemingly plausible finding, which explains why so many news outlets caught the bug. But one weird thing about the media outbreak was that the study on which it was based had never been published in a scientific journal. The paper had been posted to the Social Science Research Network web site, a sort of academic way station for working papers whose tagline is "Tomorrow's Research Today." But tomorrow had not yet come for the contagious divorce study: It had never actually passed peer review, and still hasn't. "It is under review," Fowler explained last week in an email. He co-authored the paper with his long-time collaborator, Harvard's Nicholas Christakis, and lead author Rose McDermott.
A few months before the contagious divorce story broke, Slate ran an article I'd written based on a related, but also unpublished, scientific paper. The mathematician Russell Lyons had posted a dense treatise on his website suggesting that the methods employed by Christakis and Fowler in their social network studies were riddled with statistical errors at many levels. The authors were claiming—in the New England Journal of Medicine, in a popular book, in TED talks, in snappy PR videos—that everything from obesity to loneliness to poor sleep could spread from person to person to person like a case of the galloping crud. But according to Lyons and several other experts, their arguments were shaky at best. "It's not clear that the social contagionists have enough evidence to be telling people that they owe it to their social network to lose weight," I wrote last April. As for the theory that obesity and divorce and happiness contagions radiate from human beings through three degrees of friendship, I concluded "perhaps it's best to flock away for now."
The case against Christakis and Fowler has grown since then. The Lyons paper passed peer review and was published in the May issue of the journal Statistics, Politics, and Policy. Two other recent papers raise serious doubts about their conclusions. And now something of a consensus is forming within the statistics and social-networking communities that Christakis and Fowler's headline-grabbing contagion papers are fatally flawed. Andrew Gelman, a professor of statistics at Columbia, wrote a delicately worded blog post in June noting that he'd "have to go with Lyons" and say that the claims of contagious obesity, divorce and the like "have not been convincingly demonstrated." Another highly respected social-networking expert, Tom Snijders of Oxford, called the mathematical model used by Christakis and Fowler "not coherent." And just a few days ago, Cosma Shalizi, a statistician at Carnegie Mellon, declared, "I agree with pretty much everything Snijders says."
Gelman argues that the papers might not have been accepted by top journals if these technical criticisms had been aired earlier. Indeed, Lyons posted damning quotes from two anonymous reviewers of his own work. "[Christakis and Fowler's] errors are in some places so egregious that a critique of their work cannot exist without also calling into question the rigor of review process," one of them wrote. Slate has confirmed with an editor at Statistics, Politics, and Policy that the quote is authentic.
To be clear, the critics are not saying peer pressure is a myth—no one thinks that—but only that Christakis and Fowler's studies and their claims for the virulence of obesity and the rest were far off the mark. Given the high profile of the work, and the government cash flowing to social network research, I followed up with the New England Journal of Medicine, the outlet that published the original contagious obesity paper.
A spokeswoman responded via email that NEJM's statisticians and editors had reviewed the Lyons paper in detail and that they were familiar with the ongoing debate, acknowledging that Christakis and Fowler had "moved into new methodologic [sic] territory." But she confirmed that the journal still believes in the study: "[W]e believe the methods used in the article are correctly described and adequate." She said the journal employs four statisticians who check the numbers and analysis of every paper before publication, and the obesity study got the same treatment.
When I emailed Christakis and Fowler for a comment on the new critiques, they wrote back, "We trust the peer review process, which, while not perfect, is the gold standard for evaluating scientific ideas." The duo has been invited to write an explanation of their methods for the journal Annals of Applied Statistics, and a draft is available (PDF) on the Internet. Christakis told me by email that he and Fowler are revising the posted version, which has been up since December. "[M]ost scientific journals work in this methodical and deliberative way," he explained.
David Merritt Johns is a doctoral student in Sociomedical Sciences at Columbia University’s Mailman School of Public Health.