Science

Disconnected?

We’ve heard that obesity and divorce can be passed from one person to another. Critics now wonder how the “social contagion” studies ever passed peer review.

Nicholas Christakis and James Fowler

Have you heard that divorce is contagious? A lot of people have. Last summer a study claiming to show that break-ups can propagate from friend to friend to friend like a marriage-eating bacillus spread across the news agar from CNN to CBS to ABC with predictable speed. “Think of this ‘idea’ of getting divorced, this ‘option’ of getting divorced like a virus, because it spreads more or less the same way,” explained University of California-San Diego professor James Fowler to the folks at Good Morning America.

It’s a surprising, quirky, and seemingly plausible finding, which explains why so many news outlets caught the bug. But one weird thing about the media outbreak was that the study on which it was based had never been published in a scientific journal. The paper had been posted to the Social Science Research Network web site, a sort of academic way station for working papers whose tagline is “Tomorrow’s Research Today.” But tomorrow had not yet come for the contagious divorce study: It had never actually passed peer review, and still hasn’t. “It is under review,” Fowler explained last week in an email. He co-authored the paper with his long-time collaborator, Harvard’s Nicholas Christakis, and lead author Rose McDermott.

A few months before the contagious divorce story broke, Slate ran an article I’d written based on a related, but also unpublished, scientific paper. The mathematician Russell Lyons had posted a dense treatise on his website suggesting that the methods employed by Christakis and Fowler in their social network studies were riddled with statistical errors at many levels. The authors were claiming—in the New England Journal of Medicine, in a popular book, in TED talks, in snappy PR videos—that everything from obesity to loneliness to poor sleep could spread from person to person to person like a case of the galloping crud. But according to Lyons and several other experts, their arguments were shaky at best. “It’s not clear that the social contagionists have enough evidence to be telling people that they owe it to their social network to lose weight,” I wrote last April. As for the theory that obesity and divorce and happiness contagions radiate from human beings through three degrees of friendship, I concluded “perhaps it’s best to flock away for now.”

The case against Christakis and Fowler has grown since then. The Lyons paper passed peer review and was published in the May issue of the journal Statistics, Politics, and Policy. Two other recent papers raise serious doubts about their conclusions. And now something of a consensus is forming within the statistics and social-networking communities that Christakis and Fowler’s headline-grabbing contagion papers are fatally flawed. Andrew Gelman, a professor of statistics at Columbia, wrote a delicately worded blog post in June noting that he’d “have to go with Lyons” and say that the claims of contagious obesity, divorce and the like “have not been convincingly demonstrated.” Another highly respected social-networking expert, Tom Snijders of Oxford, called the mathematical model used by Christakis and Fowler “not coherent.” And just a few days ago, Cosma Shalizi, a statistician at Carnegie Mellon, declared, “I agree with pretty much everything Snijders says.”

Gelman argues that the papers might not have been accepted by top journals if these technical criticisms had been aired earlier. Indeed, Lyons posted damning quotes from two anonymous reviewers of his own work. “[Christakis and Fowler’s] errors are in some places so egregious that a critique of their work cannot exist without also calling into question the rigor of review process,” one of them wrote. Slate has confirmed with an editor at Statistics, Politics, and Policy that the quote is authentic.

To be clear, the critics are not saying peer pressure is a myth—no one thinks that—but only that Christakis and Fowler’s studies and their claims for the virulence of obesity and the rest were far off the mark. Given the high profile of the work, and the government cash flowing to social network research, I followed up with the New England Journal of Medicine, the outlet that published the original contagious obesity paper.

A spokeswoman responded via email that NEJM’s statisticians and editors had reviewed the Lyons paper in detail and that they were familiar with the ongoing debate, acknowledging that Christakis and Fowler had “moved into new methodologic [sic] territory.” But she confirmed that the journal still believes in the study: “[W]e believe the methods used in the article are correctly described and adequate.” She said the journal employs four statisticians who check the numbers and analysis of every paper before publication, and the obesity study got the same treatment.

When I emailed Christakis and Fowler for a comment on the new critiques, they wrote back, “We trust the peer review process, which, while not perfect, is the gold standard for evaluating scientific ideas.” The duo has been invited to write an explanation of their methods for the journal Annals of Applied Statistics, and a draft is available (PDF) on the Internet. Christakis told me by email that he and Fowler are revising the posted version, which has been up since December. “[M]ost scientific journals work in this methodical and deliberative way,” he explained.

But the media world does not, and for the past few years Christakis and Fowler have been happy to race ahead of peer review and feed the news borg with interviews on tasty topics like contagious divorce. The publicity has helped them achieve scientific stardom. They speak regularly to enthusiastic crowds about their network studies and their 2009 book Connected, which has been translated into Chinese, Croatian, Dutch, German, Greek, Hebrew, Hungarian, Indonesian, Japanese, Korean, Polish, Portuguese, Romanian, Russian, Spanish, Swedish, Thai, Turkish and—cheerio!—British English.

Despite their omnipresence, they have argued that their media appearances should be considered distinct from their science. In their draft statistical paper they write, “We do not think that short remarks we have made to reporters (or even on comedy shows, which one critic took us to task for) should be used to evaluate our findings.” Yes, I did take Fowler to task for an interview he gave to Stephen Colbert, during which he claimed to have lost a few pounds so as to avoid infecting those around him with excess flab. Yet I only learned of the Colbert appearance after interviewing economist Charles Manski, a lion in the world of quantitative social science, who called it the “low point” for him in the theater surrounding the flawed studies. The problem was not that Fowler was dispensing with scientific precision for a lay audience; the problem was that he was giving unfounded health advice on national television, Manski said. (On the more difficult scientific matter of whether the force field of comedic “truthiness” that envelopes Stephen Colbert also protects his guests, my sense is no, but that question goes beyond the scope of this article.)

Although the social-contagion studies have been media hits, there has been less demand for the debunking stories in the science news marketplace. After the Lyons study was published, a few health and media blogs took note and began to ask why more journalists weren’t covering it. People are more interested in reading about new findings than a refutation of those findings, explained Forbes science writer Matthew Herper as the discussion moved to Twitter.

The scientific establishment has the same problem: Outlets for critiques are relatively few and not highly regarded. Consider that Lyons’ paper appeared in a brand-new and hence totally obscure journal. Christakis and Fowler’s work, by contrast, has appeared in some of the most prestigious medical journals in the world, including the New England Journal of Medicine and the British Medical Journal. In fact, Lyons tried submitting his article to both those places, but it was rejected without peer review. NEJM did not offer a reason; BMJ suggested his piece was better off in a “specialist journal.” (Ironically, in 2010 the latter published an editorial titled “Inadequate post-publication review of medical research.”) He also tried other top journals, but, according to Lyons, they were not interested because they do not publish critiques of articles they didn’t originally publish. Critiques lack the same kind of novelty as original studies and they are necessarily tedious and impolite. (Indeed, Lyons’ paper is not long on politeness, which probably hurt his chances.)

So is obesity contagious? What about happiness and divorce and poor sleep? One irony of the contagion battles is that even if their methods are suspect Christakis and Fowler are obviously correct that peer influence exists and that it may be even more important than we realize. As Cosma Shalizi put it on his blog last week, “there is a reason that my Pittsburgh-raised neighbors say ‘yard’ differently than my friends from Cambridge, and it’s not the difference between drinking from the Monongahela rather than the Charles.” The very idea of contagion and connectedness seems to embody the spirit of today, from the upswell of support for a young, black Chicago politician to the Facebook-driven revolutions of the Middle East.

But just because contagion is important in one context doesn’t mean something like obesity spreads like a virus—much less one that can infect someone as remote from you as your son’s best friend’s mother. (For the record, I and my best friend’s mother will eat our hats if it turns out to be true, as Christakis and Fowler claim, that loneliness is infectious, too.) Yes, we influence each other all the time, in how we talk and how we dress and what kinds of screwball videos we watch on the Internet. But careful studies of our social networks reveal what may be a more powerful and pervasive effect: We tend to form ties with the people who are most like us to begin with. The mother who blames her son’s boozebag friends for his wild behavior must face up to the fact that he prefers the fast crowd in the first place. We are all connected, yes, but the way those links get made could be the most important part of the story.