Cutting-Edge Artificial Intelligence Software Can't Tell One US City From the Next

The Citizen's Guide to the Future
Aug. 8 2012 12:00 PM

Computer Learns to Distinguish Paris From London (But Can't Tell U.S. Cities Apart)

Paris and London Google Street View images
Machine-learning software can sift through Google Street View images to pick out cities' unique visual signatures, such as the cast-iron balconies and distinctive street signs of Paris (above) and the columned entryways and stone balconies of London (below).

Courtesy of Carnegie Mellon University

What makes Paris look like Paris? For most people it’s a certain, well, je ne sais quoi. But for a machine-learning program developed by researchers at Carnegie Mellon, it’s third-floor cast-iron balcony railings, window balustrades, the shape of the streetlamps, and the molding on the wooden doors.

Will Oremus Will Oremus

Will Oremus is Slate's senior technology writer.

The software analyzed 40,000 Google Street View images from 12 major cities to pick out the visual signatures that most clearly distinguish each one. London buildings, it noticed, have a lot of neoclassical entryways with columns on either side. Like Paris, they often have balcony railings, but they’re usually made of stone and tend to be on the second floor.


For San Francisco, the software zoomed in on bay windows, and for New York, fire escapes. But in general, it struggled to tell U.S. cities apart, resorting to the brands of road tunnels and cars on the street to try to make distinctions. “This might be explained by the relative lack of stylistic coherence and uniqueness in American cities (with its melting pot of styles and influences), as well as the supreme reign of the automobile on American streets,” the researchers wrote.

From a machine-learning perspective, the hard problem was sifting through vast amounts of visual data that tells you little about location, such as trees, concrete, and sky, to find the small patches that are “geo-informative.” That required acquiring the concept of, say, balcony well enough to match one city's balconies to another's while also finding the subtle differences. It took 150 processors running overnight to complete the analysis.

By the time it was finished, the program had come up with a set of features that could correctly identify Paris 83 percent of the time and Prague 92 percent of the time. And in a Web study, humans who had been to Paris were able to use the features that the computer had picked out to tell Paris from Prague 79 percent of the time. That’s far better than their 58 percent performance when provided with random Google Street View images from the two cities.

This suggests that people and the computers use many of the same visual cues to tell cities apart . It’s just that we probably do it subconsciously.

You can try your hand at it yourself with the study team's Paris-NonParis test.

The research team, which was led by Carnegie Mellon computer science professor Alexei Efros and included members from INRIA/Ecole Normale Supérieure, will present its findings at a conference on Thursday in Los Angeles. The full paper is here.

Future Tense is a partnership of SlateNew America, and Arizona State University.



Don’t Expect Adrian Peterson to Go to Prison

In much of America, beating your children is perfectly legal. 

Ken Burns on Why Teddy Roosevelt Would Never Get Elected in 2014

Cops Briefly Detain Django Unchained Actress Because They Thought She Was a Prostitute

Minimalist Cocktail Posters Make Mixing Drinks a Cinch

How the Apple Watch Will Annoy Us

A glowing screen attached to someone else’s wrist is shinier than all but the blingiest of jewels.


Rainbow Parties and Sex Bracelets

Where teenage sex rumors come from—and why they’re bad for parents and kids.


You Had to Be There

What we can learn from things that used to be funny.

Legendary Critic Greil Marcus Measures and Maps Rock History Through 10 Unlikely Songs

Catfish Creator Nev Schulman’s Book Is Just Like Him: Self-Deluded and Completely Infectious

Sept. 12 2014 5:54 PM An Up-Close Look at the U.S.–Mexico Border
  News & Politics
Sept. 14 2014 2:37 PM When Abuse Is Not Abuse Don’t expect Adrian Peterson to go to prison. In much of America, beating your kids is perfectly legal. 
Sept. 12 2014 5:54 PM Olive Garden Has Been Committing a Culinary Crime Against Humanity
Inside Higher Ed
Sept. 13 2014 8:38 AM “You’re More Than Just a Number” Goucher College goes transcript-free in admissions.
  Double X
The XX Factor
Sept. 12 2014 4:05 PM Life as an NFL Wife: “He's the Star. Keep Him Happy.”
  Slate Plus
Behind the Scenes
Sept. 12 2014 5:55 PM “Do You Know What Porn Is?” Conversations with Dahlia Lithwick’s 11-year-old son.
Brow Beat
Sept. 14 2014 7:10 PM Watch Michael Winslow Perform Every Part of “Whole Lotta Love” With Just His Voice
Future Tense
Sept. 12 2014 3:53 PM We Need to Pass Legislation on Artificial Intelligence Early and Often
  Health & Science
New Scientist
Sept. 14 2014 8:38 AM Scientific Misconduct Should Be a Crime It’s as bad as fraud or theft, only potentially more dangerous.
Sports Nut
Sept. 12 2014 4:36 PM “There’s No Tolerance for That” Pete Carroll and Jim Harbaugh say they don’t abide domestic abuse. So why do the Seahawks and 49ers have a combined six players accused of violence against women?