Cutting-Edge Artificial Intelligence Software Can't Tell One US City From the Next

The Citizen's Guide to the Future
Aug. 8 2012 12:00 PM

Computer Learns to Distinguish Paris From London (But Can't Tell U.S. Cities Apart)

Paris and London Google Street View images
Machine-learning software can sift through Google Street View images to pick out cities' unique visual signatures, such as the cast-iron balconies and distinctive street signs of Paris (above) and the columned entryways and stone balconies of London (below).

Courtesy of Carnegie Mellon University

What makes Paris look like Paris? For most people it’s a certain, well, je ne sais quoi. But for a machine-learning program developed by researchers at Carnegie Mellon, it’s third-floor cast-iron balcony railings, window balustrades, the shape of the streetlamps, and the molding on the wooden doors.

Will Oremus Will Oremus

Will Oremus is Slate's senior technology writer.

The software analyzed 40,000 Google Street View images from 12 major cities to pick out the visual signatures that most clearly distinguish each one. London buildings, it noticed, have a lot of neoclassical entryways with columns on either side. Like Paris, they often have balcony railings, but they’re usually made of stone and tend to be on the second floor.


For San Francisco, the software zoomed in on bay windows, and for New York, fire escapes. But in general, it struggled to tell U.S. cities apart, resorting to the brands of road tunnels and cars on the street to try to make distinctions. “This might be explained by the relative lack of stylistic coherence and uniqueness in American cities (with its melting pot of styles and influences), as well as the supreme reign of the automobile on American streets,” the researchers wrote.

From a machine-learning perspective, the hard problem was sifting through vast amounts of visual data that tells you little about location, such as trees, concrete, and sky, to find the small patches that are “geo-informative.” That required acquiring the concept of, say, balcony well enough to match one city's balconies to another's while also finding the subtle differences. It took 150 processors running overnight to complete the analysis.

By the time it was finished, the program had come up with a set of features that could correctly identify Paris 83 percent of the time and Prague 92 percent of the time. And in a Web study, humans who had been to Paris were able to use the features that the computer had picked out to tell Paris from Prague 79 percent of the time. That’s far better than their 58 percent performance when provided with random Google Street View images from the two cities.

This suggests that people and the computers use many of the same visual cues to tell cities apart . It’s just that we probably do it subconsciously.

You can try your hand at it yourself with the study team's Paris-NonParis test.

The research team, which was led by Carnegie Mellon computer science professor Alexei Efros and included members from INRIA/Ecole Normale Supérieure, will present its findings at a conference on Thursday in Los Angeles. The full paper is here.

Future Tense is a partnership of SlateNew America, and Arizona State University.



Talking White

Black people’s disdain for “proper English” and academic achievement is a myth.

Hong Kong’s Protesters Are Ridiculously Polite. That’s What Scares Beijing So Much.

The One Fact About Ebola That Should Calm You: It Spreads Slowly

Natasha Lyonne Is Coming to the Live Culture Gabfest. Are You?

A Jaw-Dropping Political Ad Aimed at Young Women, Apparently

The XX Factor
Oct. 1 2014 4:05 PM Today in GOP Outreach to Women: You Broads Like Wedding Dresses, Right?

Operation Backbone

How White Boy Rick, a legendary Detroit cocaine dealer, helped the FBI uncover brazen police corruption.


How Even an Old Hipster Can Age Gracefully

On their new albums, Leonard Cohen, Robert Plant, and Loudon Wainwright III show three ways.

Tattoo Parlors Have Become a Great Investment

This Gargantuan Wind Farm in Wyoming Would Be the Hoover Dam of the 21st Century

Oct. 1 2014 8:34 AM This Gargantuan Wind Farm in Wyoming Would Be the Hoover Dam of the 21st Century To undertake a massively ambitious energy project, you don’t need the government anymore.
Oct. 1 2014 6:59 PM EU’s Next Digital Honcho Says Celebs Who Keep Nude Photos in the Cloud Are “Stupid”
  News & Politics
Oct. 2 2014 11:01 AM It Wasn’t a Secret A 2013 inspector general report detailed all of the Secret Service’s problems. Nobody cared.
Business Insider
Oct. 2 2014 11:16 AM Some McDonald's Monopoly Properties Matter More
The Vault
Oct. 2 2014 11:07 AM Mapping 1890 Manhattan's Crazy-Quilt of Immigrant Neighborhoods
  Double X
The XX Factor
Oct. 1 2014 5:11 PM Celebrity Feminist Identification Has Reached Peak Meaninglessness
  Slate Plus
Behind the Scenes
Oct. 1 2014 3:24 PM Revelry (and Business) at Mohonk Photos and highlights from Slate’s annual retreat.
Brow Beat
Oct. 2 2014 10:36 AM How Bad Will Adam Sandler’s Netflix Movies Be?
Future Tense
Oct. 2 2014 11:01 AM Surge Pricing Is Not Price Gouging
  Health & Science
Medical Examiner
Oct. 2 2014 9:49 AM In Medicine We Trust Should we worry that so many of the doctors treating Ebola in Africa are missionaries?
Sports Nut
Oct. 1 2014 5:19 PM Bunt-a-Palooza! How bad was the Kansas City Royals’ bunt-all-the-time strategy in the American League wild-card game?