There was a time when humanity suffered from a dearth of information. Now, thanks to the Internet, we have the opposite problem: so much information at our fingertips, about so many topics, that it’s hard to know where to start.
It’s a conundrum that digital media companies have been trying to solve since the dawn of the Web. In the 1990s, CompuServe and AOL built harbors of hand-picked content to shelter their subscribers from the turbulent open Web. In 2008, Tina Brown launched the Daily Beast as a one-stop aggregation hub with the motto “Read This Skip That.” Google News automated the process, using algorithms to pick out the day’s top news, while Google Reader and iGoogle (R.I.P.) handed users the keys to their own personalized news sites. But they also threatened to trap users in what Eli Pariser called “filter bubbles”: narrow online worlds of their own making. In recent years Twitter has become the first stop for news addicts, but the firehose of links can be too much for the average user. That has led to efforts to package the highlights of people’s Twitter and RSS feeds into personalized magazines, like Flipboard and Zite. Meanwhile, sites like Reddit crowdsource the curation process, letting its users vote on which stories rise to the top of the page.
I have tried every one of these and more, and while each has its merits, not one has ever struck the perfect balance between diversity of sources, relevance of content, and ease of use. So I didn’t have high expectations the first time I logged into a 2-year-old site called Prismatic. But it has rapidly become a staple part of my information diet, and today it is vying with Twitter for the top spot on my bookmarks bar. It’s that good—and it has the potential to be far better still.
Prismatic has no human editors. What you see on the page is governed entirely by machine-learning algorithms—that is, by software that adapts to you over time based on your interests and behavior. The software’s goal is to scour the entire Web for the stories most likely to interest, surprise, outrage, and delight you.
You may be thinking: That kind of software never actually delivers stuff I’m interested in, or it serves up stuff so obvious that I’ve already seen it 10 times. Take Apple’s inaptly named Genius feature in iTunes or the App Store (R.I.P.). I download a weather app, then go to Genius to see what else on the App Store I might be interested in—and it recommends five other weather apps. Or take Pandora, the Internet radio service. When it first launched, its algorithms were gleefully peripatetic, making wild and sometimes baffling leaps from, say, Nick Cave to Frank Sinatra. Since then, though, it has reined in its discovery algorithms to such an extent that it rarely serves up anything that would prompt a raised eyebrow, let alone an exclamation of delight.
Prismatic, on the other hand, navigates the shoals of predictability and incoherence with whimsy and grace. Use it for a few days, and you’ll find yourself wondering, “How in the world did it know that I would be interested in that?” Use it for months, and you may suspect that the site knows you better than you know yourself. As you click on some stories, skip others, and save a few to read later, you can sense it continually revising its internal model of your interests. If there were a Turing test for suggestion services, Prismatic would come close to passing.
Here’s how it works. Visit getprismatic.com or download the app on your iPhone or iPad. (There’s no Android app yet, though it should arrive by fall.) It will prompt you to sign up using your Facebook, Twitter, or Gmail account. Note that you’re giving Prismatic access to some sensitive data, so privacy and trust will be critical to the site’s success. The information Prismatic imports from that service forms the first step in its learning process, and it immediately presents you with a page of news stories, op-eds, and blog posts that it thinks you might find interesting, along with a side rail of topics and people to follow.
Don’t be discouraged if its initial suggestions are underwhelming—it’s only once you start interacting with Prismatic that it begins to distinguish itself. Scroll down and start reading some of the stories you find interesting. Prismatic’s software will be silently watching and taking notes as you linger a while on some and quickly move on from others. Take some direct action—like starring, tweeting, or saving a story for later—and you can be sure the site’s algorithm is marking that story with a proverbial yellow highlighter. It doesn’t know yet exactly why you interacted with that piece, and it won’t jump to conclusions. Rather, it will begin to form some hypotheses. Maybe you tweeted Alexis Madrigal’s Atlantic post about the Kanye West album “Yeezus” because you’re into hip-hop. Maybe you just like stories by Madrigal and have no further interest in Kanye’s oeuvre. Prismatic will test these hypotheses by slipping both more hip-hop stories and more Madrigal into your feed until it has more data. But it won’t overwhelm you with either one until it’s pretty sure you really, really like them—and even then, it will continue to mix things up. Its ultimate goal is serendipity: the story that you never would have thought to look for, but that grabs you as soon as you see it.
Prismatic is not the only site working on serendipity these days. What’s remarkable is the frequency with which it hits the mark. The type of machine learning that Prismatic is trying to do is seriously complicated from a technical standpoint. To the extent that it succeeds, it’s because the company wasn’t started by journalists, media executives, or Web designers. It was founded by hardcore engineers. Bradford Cross, the CEO, is a former hedge-fund researcher and data scientist who previously founded FlightCaster, a site that uses machine learning to predict flight delays. Co-founder Aria Haghighi, the CTO, was a Ph.D. student in machine-learning at University of California, Berkeley. And two of their first hires, Jenny Finkel and Jason Wolfe, hold Ph.D.’s in natural-language processing and artificial intelligence, respectively. Together they embarked on a project that was almost Google-esque in scope: crawling the entire English-language Web to analyze and categorize stories by topic, so that it can show you obscure posts alongside the obvious.
For all its virtues, the Prismatic team’s algorithm is still far from perfect. It does an impeccable job of picking out stories whose topics interest me, but often fails to distinguish between in-depth reporting and click-bait. While you can send a signal to Prismatic’s software by deleting any given story from your feed, you can’t blacklist entire websites at this point. Cross admits the user interface needs work. Last week the site greeted me with the tantalizing but truncated headline “Study: Conservatives have larger.” And the site’s social features so far are more of a distraction than an asset.
That may soon change. When I visited Prismatic’s San Francisco office last week, the staff of about 20 people was working on a full redesign that is scheduled to go live in beta form in the next couple of weeks. Among the improvements, Cross said, will be a prettier layout, more ways to interact with other users, and more information about why Prismatic thinks you might be interested in each story it pulls up for you.
The real challenge for Prismatic, now that it has built what is almost certainly the world’s smartest news-reading app, will be to gain mainstream appeal. Prismatic makes no money today, but plans to in the future by recommending things for you to buy—like books, movies, games, or apps—along with articles for you to read.
I can’t say whether everyone will find Prismatic as useful as I do. As a tech blogger, I have a special interest in finding quirky or out-of-the-way stories that other national media outlets have yet to pick up on. I’m not a regular reader of the Chronicle of Higher Education, but when that journal ran an item in October noting that the state of Minnesota had barred its residents from taking free classes online, Prismatic suspected I’d want to know about it—and it was right. My blog post highlighting the byzantine ruling ended up going viral via Reddit, whose users in turn directed so much outrage and scorn at the state’s bureaucrats that they reversed their stance the very next day. That sort of find makes Prismatic invaluable to someone like me. But do my nonjournalist friends have the same interest in discovering stories that haven’t yet been spotted by the mainstream media? If not, Prismatic could find itself occupying a niche rather than sweeping the nation.
But even if Prismatic fails, the idea behind it is not going away anytime soon. Underneath its news-reader skin, Prismatic is a machine-learning company. Its algorithmic approach to what people in the business call “discovery” holds huge promise for applications ranging from news to entertainment to dating websites. For now, Prismatic’s technology makes matches between readers and articles. But in the future, who knows—its serendipity engine might help set you up with books, songs, television, or even a spouse.
