“The milestone is on par with Watson winning Jeopardy!,” D. Scott Phoenix told me in an initial press email teasing a breakthrough in artificial intelligence by his San Francisco-based startup, Vicarious. Whether the achievement lives up to that billing remains unclear. At the very least, it demonstrates that the company has big ambitions and a knack for PR.
Phoenix and his Vicarious co-founder, Dileep George, say they’ve done something no one else has been able to do: create software that can reliably crack almost any text-based CAPTCHA—you know, those squiggly words and numbers that websites use to make sure you’re human. (You can see the software at work in the video below.) That’s noteworthy because CAPTCHA is supposed to be a sort of automated Turing test: a benchmark that only human-level intelligence could achieve. In its press release, the company claims it has essentially rendered CAPTCHA obsolete:
A CAPTCHA scheme is considered broken if an algorithm is able to reach a precision of at least 1 percent. By leveraging core insights from machine learning and neuroscience, the Vicarious AI achieves success rates up to 90 percent on modern CAPTCHAs from Google, Yahoo, PayPal, Captcha.com, and others. This advancement renders text-based CAPTCHAs no longer effective as a Turing test.
Vicarious says it doesn’t plan to use the algorithm to actually hack websites that use CAPTCHA. Rather, it bills it as a major advance in computer vision, with potential applications for robotics, image search, medical image analysis, and a host of other fields. (It doesn’t mention face recognition, probably because it doesn’t want to creep people out.) And it frames it as a step toward its ultimate goal to “build software that can think and learn like a human.”
There’s some debate, though, about just how big a deal this is. It’s not the first time a team of software engineers has claimed to have cracked CAPTCHA. CAPTCHA co-inventor Luis von Ahn told Scientific American he must have heard 50 such claims over the years. The question is whether Vicarious’s effort represents a serious leap over previous solutions, which tend to perform well at solving one or two particular varieties of CAPTCHA tests but break down when confronted with novel circumstances. While the approach employed by Vicarious represents the latest thinking in the field, von Ahn said, it’s hard to know just how well it works. That’s because Vicarious has elected not to publish its findings in any academic journal, preferring to keep its methods under wraps for the time being.
The best analysis I’ve seen so far comes from Science Now’s John Bohannon, who notes that Vicarious’s “science-by-press-release” approach has irked some leading computer scientists, but that there may also be valid reasons for it. When Bohannon emailed Vicarious’ co-founders to relay criticisms that its claims are “unsubstantiated,” they responded with an offer to demonstrate the software via Skype. The results:
I sent them CAPTCHAs off the internet. They were able to solve the first one, from a Paypal website, immediately. But the algorithm was stumped by two others. One had Cyrillic characters. “We haven’t trained our system on other languages yet,” said Phoenix. And it also failed on a CAPTCHA that used alternating patches of black and white like a chess board.
It’s understandable that the software would be foiled by specific oddities of various CAPTCHA implementations. Then again, that’s exactly why CAPTCHA has proven so durable over the years—because it can be tweaked in all sorts of ways to befuddle even the most advanced general-purpose algorithms.
The takeaway, for now, is that Vicarious may (or may not) have built the most adept CAPTCHA-cracking algorithm yet—but that it remains a long way from human-level intelligence, even in the constrained realm to which it has so far been applied. It may serve to hasten the decline of CAPTCHA across the Internet, but we needn’t fear a broader rise of the machines anytime soon.
