Search without Google is like social networking without Facebook: unimaginable. But superb proprietary algorithms and extremely talented employees only partially explain why both fields are dominated by just one firm. The real reason is that both Google and Facebook got into their fields early on, accumulated troves of data about their users, and are now aggressively exploiting that data to offer unique services that their data-poor contenders simply cannot match, no matter how innovative their business models.
Take Google’s personalized search or Facebook’s Graph Search feature. Both features are trivial to replicate; it’s the user data that makes them stand out. Thus, Google will indicate which links have been endorsed by our “friends” right in the search results; Facebook, via Graph Search, allows us to tap the wisdom of our friends and their friends.
Both companies have successfully approximated and then monetized our “social graph”—the once-trendy term to describe our many overlapping connections to other people. It’s small things like the social graph that explain why even a better, more innovative search engine or a social network with more respect for user privacy would have a hard time competing with Google and Facebook: As long as the dominance of these firms is powered by vast troves of user data, competitors are doomed.
Were we to rebuild our information infrastructure from scratch, we would surely notice that the current system is awful for competition. How could we run things differently? One option might be to run the social graph as a public institution of sorts, with state regulators making sure that all companies get equal access to such crucial information. Many of our social connections predate—and might even outlive—both Google and Facebook. These companies have mapped them well, but this shouldn’t prevent us from thinking of alternative ways of mapping them and making them available. Thus, instead of pouring public money into building better search engines—a mission attempted and quickly aborted by some European politicians—governments can focus on ensuring that the data playing field remains as level as possible. Better search engines and social networking sites might then emerge on their own, without any need for extra public backing.
The scheme could have many other benefits. For example, the regulators would be able to exercise far greater control over how user data is collected and accessed by third parties. It should be possible to anonymize this data so that better personalized services can be built without compromising user privacy. The fears of “the filter bubble” are greatly exaggerated; personalization is not evil per se—it’s the data trails that it leaves in its wake that should trouble us.
A few months ago, this might have seemed a reasonable but ultimately quixotic proposal. For a start, there seems dangerously little interest or desire in rebuilding—or even reimagining—our global information infrastructure. Just imagine the kind of effort that would be needed to gather all this information and organize it in an easy-to-use manner. Who would possibly fund such an endeavor?
Now that Edward Snowden has blown the whistle on the extensive spying operations of the National Security Agency, this question seems obsolete. Take the NSA’s much-discussed collection of metadata—the seemingly benign (or so they claim) information about who calls whom and when. It’s precisely this kind of metadata that is needed to build a better publicly run social graph. In fact, the NSA has probably already built it—and not just for America but probably for users in many other countries as well—often with tacit cooperation from intelligence services and telecommunication providers of those countries.
We can debate the ethics and legality of such initiatives until we all turn blue—and I suggest that we do, for, based on Snowden’s revelations, the NSA’s system is mired in secrecy, lacks proper congressional oversight, and enjoys unlimited rhetorical and lobbying support from the military-industrial complex. So, yes: The NSA’s data-collection practices must be reformed with accountability in mind.
These, however, are all questions about the future. But there’s a far more pragmatic question about the present: The NSA has all this data, and it’s not going away. (If anything, the much-discussed data storage center that the NSA is building in Utah suggests otherwise.) It would be a colossal mistake not to come up with a global institutional arrangement that would make at least chunks of that data available for public use. At the very (utopian) minimum, it should be possible to produce a rudimentary social graph and make it globally available—to be supervised by a civil agency, perhaps within the United Nations. The United States, which has always preached free markets to the rest of the world, can, perhaps, take the lead in making markets for search and social networking more competitive.
There are, of course, plenty of technical hurdles and potential privacy concerns that would need to be addressed. Strong anonymity rules and the ability to opt out would be crucial. Of course, the incumbent technology companies wouldn’t like it, as they were the ones who gave all this data to the NSA. Well, yes, but both Google and Facebook also know that, sooner or later, the regulators would figure out the real reasons why the search and social networking markets feature so little competition and do something about it. That “something” can be much more drastic than what is proposed here.
The most daunting task would be convincing privacy advocates that this is, indeed, a deal worthy of their attention. On the face of it, it isn’t. What can be crazier than giving every other startup access to almost as much user data as Google and Facebook already have? But it’s a much more complicated issue, not least because if this plan works, we could end up with more innovation in search and social networking—and some of these new companies might actually offer more protection to their users as their business models wouldn’t necessarily be tied to aggressive data collection.
In other words, the real choice that we face right now is between a future in which Google and Facebook continue to dominate their core markets, collecting more and more data on their users, and a future in which the power of those companies is held in check by competition. At the moment, the users have little choice but to stick with Google and Facebook, as the user data that they already have does produce better search results and richer social connections.
The most important hurdle is convincing the NSAs of this world, in America and elsewhere, that this is in their interest—which it clearly isn’t. The spooks are perfectly happy with the ongoing centralization of telecommunications so that there are only a few hubs—like Google and Facebook—that keep sucking in all the user data and organize it nicely for NSA’s taking. A truly competitive market for search and social networking—in which there are dozens of big players, all with different approaches to privacy and data collection—would be a headache for the intelligence services.
Besides, if the more intimate user data is held in another properly supervised public agency—the one tasked with protecting it from unnecessary abuse and is endowed with the proper legal authority to say no to unreasonable demands from the NSA—it might actually help protect user privacy while taming the spread of the national security apparatus.
An immodest proposal? Perhaps. But with so many NSA-branded lemons, we might as well make some lemonade.
