Rise of the Machines: A Computer That Learns Games by Watching Us Play

The Citizen's Guide to the Future
July 12 2012 1:11 PM

Rise of the Machines: A Computer That Learns Games by Watching Us Play

New software teaches a computer grid-based games by showing it video footage of game play.

Photo by HENNING KAISER/AFP/Getty Images

We’ve been able to “play the computer” in games for years. But what if, instead of telling them what to do and how to play, we let computers learn from watching us? And what if they used that information to beat us consistently?

It’s not time to invoke Skynet just yet, but researcher Lukasz Kaiser at Paris Diderot University has flipped the logic of computer game play with the help of some basic software, video recognition, and an algorithm.


Kaiser’s software feeds video footage of winning, losing, and illegal moves from five grid-based games to a computer. The computer then analyzes the minimal sets of footage and intuits the exact rules for Breakthrough, Gomoku, Connect Four, tic-tac-toe, and a game we've never heard of called Pawn Whopping (if you have, tell us in the comments). Kaiser’s relatively simple laptop can analyze the footage in less than a minute, separating material like human hands from the important stuff, and learn the exact rules of the game from scratch, with information rather than direction. Once the computer learns rules for a game like Connect Four, it’s almost impossible to beat. Remember how we humans lose every time at rock paper scissors?

“There is strong theoretical evidence that it will generalize to the other problems,” Kaiser writes of the method he developed. “Some of those problems might require hierarchical, structured learning or a form of probabilistic formulas, and in the future we intend to consider such extensions. But already the presented technique significantly improves the state of the art in learning from visual input.”

Translation: It may be a little while before computers are learning more complex games like chess just from watching us play, but we’re moving into an age where, with limited visual input, a machine can understand the rules. Hopefully that ability can prime machines to help us with all sorts of problems—not put us in checkmate.

Future Tense is a partnership of SlateNew America, and Arizona State University.

Ben Johnson is the producer of Marketplace Tech from American Public Media.



Blacks Don’t Have a Corporal Punishment Problem

Americans do. But when blacks exhibit the same behaviors as others, it becomes part of a greater black pathology. 

I Bought the Huge iPhone. I’m Already Thinking of Returning It.

Scotland Is Just the Beginning. Expect More Political Earthquakes in Europe.

Lifetime Didn’t Think the Steubenville Rape Case Was Dramatic Enough

So they added a little self-immolation.

Two Damn Good, Very Different Movies About Soldiers Returning From War

Medical Examiner

The Most Terrifying Thing About Ebola 

The disease threatens humanity by preying on humanity.

Students Aren’t Going to College Football Games as Much Anymore, and Schools Are Getting Worried

The Good Wife Is Cynical, Thrilling, and Grown-Up. It’s Also TV’s Best Drama.

  News & Politics
Sept. 20 2014 11:13 AM -30-
Business Insider
Sept. 20 2014 6:30 AM The Man Making Bill Gates Richer
Sept. 20 2014 7:27 AM How Do Plants Grow Aboard the International Space Station?
  Double X
The XX Factor
Sept. 19 2014 4:58 PM Steubenville Gets the Lifetime Treatment (And a Cheerleader Erupts Into Flames)
  Slate Plus
Slate Picks
Sept. 19 2014 12:00 PM What Happened at Slate This Week? The Slatest editor tells us to read well-informed skepticism, media criticism, and more.
Brow Beat
Sept. 20 2014 1:52 PM Julian Casablancas’ New Album Sounds Like the Furthest Thing From the Strokes
Future Tense
Sept. 19 2014 6:31 PM The One Big Problem With the Enormous New iPhone
  Health & Science
Bad Astronomy
Sept. 20 2014 7:00 AM The Shaggy Sun
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.