New Study Predicts Facebook Will Be Dead in Three Years, Proves Studies Can Be Totally Stupid

Commentary about business and finance.
Jan. 23 2014 1:03 PM

The Myspace Fallacy

Researchers scored a viral hit with a study using epidemiology to predict Facebook’s demise. Too bad it’s fatally flawed.

Making the rounds in the tech world this week is a scientific paper published online by a pair of Princeton researchers. Using a model adapted from epidemiology, the paper predicts that Facebook is about to go the way of Myspace. Specifically, the authors conclude, “Facebook will undergo a rapid decline in the coming years, losing 80 percent of its peak user base between 2015 and 2017.”

Will Oremus Will Oremus

Will Oremus is Slate's senior technology writer. Email him at will.oremus@slate.com or follow him on Twitter.

Eighty percent! The paper rocketed to the top of Reddit’s technology section. For bloggers around the Internet, the headlines practically wrote themselves. An uncritical three-paragraph write-up in Time has pulled in 21,000 Facebook likes and counting. Nothing sells on Facebook like another story about how Facebook is evil, uncool, or—best of all—doomed.

I hate to be the Grinch who actually read the paper, but its conclusions are less bankable than Dogecoin. Which is to say, about as solid as one might cynically expect from a paper on epidemiology and social networking published online without peer review by a pair of graduate students in mechanical and aerospace engineering.

But it’s not the authors’ credentials that are the problem. They’re certainly no less qualified to write such a paper than I am to criticize it. The problem lies in the assumptions they make in order to reach their headline-grabbing conclusions. Let me summarize the paper’s underlying logic:

A)     Social networks’ growth can be likened to the spread of an infectious disease.
B)     In the case of Myspace, the network’s decline can also be likened to the spread of an infectious disease.
C)     Like Myspace, Facebook has begun to show signs of decline after a period of rapid growth.
D)     Ergo, Facebook will be dead within three years.

Let’s go through these one by one.

Point A is fairly well documented by this point. That’s why we call it “going viral.”

Point B is interesting, and less obvious than it might seem. As the authors acknowledge, social networks aren’t like the flu in that there’s no inherently predictable recovery rate. That is, you don’t necessarily join a social network with the expectation that you’ll be over it in a week. So instead of comparing the decline of social networks to recovery from an illness, the authors make an interesting leap. They hypothesize instead that the decline of a social network is like the spread of an illness—that leaving is as contagious as joining.

The way they wrote the paper, the authors made it seem as though they arrived at this model a priori, then decided to try it out on Myspace, and voila—it worked! In their words, they “validated” the model by testing it on Myspace and finding that it was a good fit.

But wait: What reason do they have to believe that it works on any social network other than Myspace? The authors’ attempt to justify applying an epidemiological model to the decline of social networks in general—as opposed to just Myspace in particular—is contorted, bordering on comical. In the absence of concrete examples or literature, they draw yet another analogy, this time to “ideas”:

Ideas, like diseases, have been shown to spread infectiously between people before eventually dying out, and have been successfully described with epidemiological models. Again, this follows intuitively, as ideas are spread through communicative contact between different people who share ideas with each other. Idea manifesters ultimately lose interest with the idea and no longer manifest the idea, which can be thought of as the gain of “immunity” to the idea.

“Follows intuitively?” “Idea manifesters?” Come on, now you’re just making stuff up. Sure, some ideas spread infectiously and then die out. Others, like democracy, gravity, and evolution, have shown impressive staying power.

Anyway, where were we? Oh right, we’re still on Point (b). In short, if the paper had confined itself to the observation that the decline of Myspace could be modeled using insights from epidemiology, then we’d have no problem. But we also wouldn’t be seeing the piece go nuts on Reddit.

Moving on to Point C, things get dicier still. The authors’ conclusion relies on the premise that Facebook is already showing signs of decline. At first glance, the chart below—which was highlighted in several media reports—seems pretty convincing.

140123_$BOX_FacebookSearchQueries

John Cannarella, Joshua A. Spechle, Princeton University

But that’s very odd, given that the social network has consistently reported massive, ongoing growth in both its active user base and revenue. These widely accepted metrics, it seems, are of no interest to the paper’s authors. Instead, they pull their data exclusively from Google Trends, which measures the number of Google searches for a given keyword over time. In other words, the researchers’ claim that Facebook is faltering is based entirely on an apparent dip in the number of people typing “Facebook” into Google in 2013.

They justify this bizarre choice of proxy data by arguing that it’s “advantageous compared to using registration or membership data,” which can include inactive members. OK, then why not use daily or monthly active users, as industry analysts do? Seemingly unaware that Facebook is a public company, the researchers claim that user activity data on social networks is “typically proprietary and difficult to obtain.” They back up this assertion with a rather amusing citation: a study published in 2009 about—you guessed it—Myspace.

The Google Trends data on searches for Facebook, unfortunately, do not appear particularly reliable: There’s a huge spike in October 2012 that the researchers can’t really explain and end up simply throwing out. Yet they have no such qualms about an apparent dip in 2013. In fact, that dip is the key to their paper. If Facebook isn’t actually declining in popularity, then the Myspace model doesn’t fit.

The authors never entertain the possibility of an alternative explanation for a dip in the Google Trends data. Facebook’s numbers show that its users migrated en masse from using the site on a desktop computer to accessing it via their phones and tablets in 2013, often via the mobile app. The paper makes no mention of this trend—nor the even simpler possibility that people aren’t searching for “Facebook” as much on Google because they already know what it is and where to find it. By the authors’ logic, I guess a Google Trends search for “broadband” would suggest that high-speed Internet began to wane in popularity sometime around 2005.

20140123_Mbox_Broadband

Google Trends for "broadband"

Which brings us to Point D, in which the authors finally “prove” the very thing that they assumed from the outset: that all social networks can be expected to follow the same trajectory as Myspace. They even admit in the section describing their model that it doesn’t work unless you assume that all social networks eventually decline.

I contacted the paper’s authors to give them a chance to defend their findings. They sent a canned response noting that they’ve submitted the paper to a journal and prefer to withhold comment until it has been peer-reviewed. That’s their right, of course, but it did make me wonder why they published it online in the first place. Peer-reviewed or not, the study has already made its mark on the world, thanks to a bunch of media outlets in addition to Time that were happy to slap a headline on it and reap some Facebook traffic of their own. To wit:

It’s an old journalistic trick: Just add the words “research” or “study” to a sensational claim for instant credibility. Best of all, you’re absolved of any responsibility for verifying its truth, since everyone knows journalists aren’t qualified to dispute scientific findings. Funny thing, though: If it hasn’t been peer-reviewed, and journalists aren’t qualified to dispute it, then what’s to stop it from simply becoming accepted as truth?

I recently wrote about a fun blog called LolMyThesis, in which self-deprecating students summarize their research findings in a single sentence. Not all academic research can be effectively glossed in this way, but it’s amazing just how useful it can be for cutting through the confidence intervals and the literature reviews to get to the hollow core of a weak paper. I suspect that if the authors of this one thought about it long and hard, they’d have to admit that the LolMyThesis summary would go something like this: “If you assume that Facebook is like Myspace, bring in a fancy model adapted from epidemiology, and crunch the numbers, it turns out that Facebook is a lot like Myspace.”