This AI Engine Promises to Identify Your Photos. It Often Fails Spectacularly.

Here’s a riddle: When is a goat a dog?

Answer: When you run a picture of it through the Image Identification Project.

Released to the public yesterday, Wolfram Research’s ImageIdentify promises to accurately describe the contents of any picture that you show it. The setup couldn’t be simpler: Upload a picture and it’ll tell you what it sees. In a lengthy blog post, computer scientist and Wolfram Research CEO Stephen Wolfram describes this program as “a nice practical example of artificial intelligence.” He suggests that it might be used to automatically classify the contents of albums, offering “statistics on the different kinds of animals, or planes, or devices, or whatever, that appear in the photographs.”

The trouble, unsurprisingly, is that ImageIdentify appears to go wrong more often than it goes right. Wolfram acknowledges this difficulty, and gamely offers a handful of interesting errors in his post. Given an image of Indiana Jones, for example, “the system was blind to the presence of his face, and just identified the picture as a hat.” It’s certainly impressive that it recognized and correctly labeled a hat. But such mistakes would seem to constrain the project’s usefulness, at least for the time being.

Like Microsoft’s How-Old.net, ImageIdentify is most interesting when it gets things wrong in spectacular ways. As Wolfram notes, many of its errors make sense. It confused my bike with a bicycle rack, presumably because it saw the primary object correctly, but assumed that it was somehow attached to the old fashioned radiator behind it. Here, the system’s error likely derives from its effort to identify a single subject in each image, a propensity that sometimes leads it to ignore key details (as in the Indiana Jones example) and sometimes leads it to conflate distinct elements (as in the case of the bike).

Sometimes, however, ImageIdentify is just plain weird. When we fed it a picture of a croissant, it told us that we were looking at shellfish. Wolfram claims the system’s mistakes “mostly seem remarkably human.” But that pastry-mollusk confusion feels uncanny—more like a metaphor than an ordinary misapprehension.

ImageIdentify will no doubt improve in time—it can be trained to better understand what it’s looking at—but for now it’s at its best when it’s at its strangest. Here are a few of our favorites:

A rare representational effort by Jackson Pollock:

Slate product manager Heidi Moon’s suspiciously feline English setter:

Headed to the ballpark? Make sure to grab an Oscar Mayer Bobsled!:

Slate copy editor Abby McIntyre’s adorable lemon shark Percy: 

Generic person/Wolfram Research CEO Stephen Wolfram: