We Need to Stop Killer Robots Now

Stories from New Scientist.
May 26 2013 6:30 AM

Stop Killer Robots

Autonomous weapons systems could lead to disaster.

A robot distributes promotional literature calling for a ban on fully autonomous weapons in Parliament Square on April 23, 2013, in London. The Campaign to Stop Killer Robots is calling for a pre-emptive ban on lethal robot weapons that could attack targets without human intervention.
A robot distributes promotional literature calling for a ban on fully autonomous weapons in Parliament Square on April 23, 2013, in London. The Campaign to Stop Killer Robots is calling for a pre-emptive ban on lethal robot weapons that could attack targets without human intervention.

Photo by Oli Scarff/Getty Images

Artificial intelligence expert Mark Bishop says a ban on weapons that can deploy and destroy without human intervention is vital. He is a professor of cognitive computing at Goldsmiths, University of London, and chairs the Society for the Study of Artificial Intelligence and the Simulation of Behavior.

Simon Makin: What is the Campaign to Stop Killer Robots?
Mark Bishop: It is a confederation of non-governmental organizations and pressure groups lobbying for a ban on producing and deploying fully autonomous weapon systems, where the ability of a human to both choose the precise target and intervene in the final decision to attack is removed.

SM: How close are we to this?
MB: Examples already exist. Some, such as the Phalanx gun system, used on the majority of U.S. Navy ships to detect and automatically engage incoming threats, have been around for some time. Another is the Israeli Harpy "fire-and-forget" unmanned aerial vehicle, which will seek out and destroy radar installations.

Advertisement

SM: What's driving the technology's development?
MB: Current Western military strategy focuses more on drones than on traditional forces, but remote-controlled drones are vulnerable to hijacking. Fully autonomous systems are virtually immune to this. They also lower costs. This means manufacturers sell more, so there is a commercial imperative to develop autonomous systems and for governments to deploy them.

SM: What are the dangers?
MB: There are reasons to doubt whether autonomous systems can appropriately judge the need to engage, react to threats proportionately, or reliably discriminate between combatants and civilians. Also, when you get complex software systems interacting, there is huge potential for unforeseen consequences. A vivid example was seen on Amazon in 2011 when pricing bots raised the cost of a book, The Making of a Fly, to more than $23 million.

SM: So, you are worried about escalation?
MB: Yes. In South Korea scientists are developing a robot to patrol the border with North Korea. If this was deployed and incorrectly or disproportionately engaged, it is easy to imagine a minor border incursion escalating into a serious confrontation. Even more frighteningly, in 1983, during the U.S. military exercise Able Archer, Russian automatic defense systems falsely detected an incoming missile, and it was only a Russian colonel's intervention that averted nuclear war. But the potential for escalation gets particularly scary when you have autonomous systems interacting with other autonomous systems.

SM: Couldn't robots reduce risk to humans?
MB: There is a case, put forward by people such as U.S. roboticist Ronald Arkin, that robots might make more dispassionate assessments than grieving or revenge-seeking soldiers. Not only does this not address the problem of escalation, it also only holds water if systems can reliably decide when to engage, judge proportionality, and accurately discriminate targets.

SM: So what should we do?
MB: The technology behind autonomous systems has other uses, such as the Google car driving system, so banning development would be difficult. Instead, we must focus on a global treaty banning deployment of autonomous weapons.

This article originally appeared in New Scientist.

TODAY IN SLATE

Politics

The Democrats’ War at Home

How can the president’s party defend itself from the president’s foreign policy blunders?

Why Time Is on Our Side in the Fight Against Ebola

The Religious Right Is Not Happy With Republicans  

The XX Factor
Oct. 1 2014 4:58 PM The Religious Right Is Not Happy With Republicans  

Catacombs Where You Can Stroll Down Hallways Lined With Corpses

Homeland Is Good Again! For Now.

Crime

Operation Backbone

How White Boy Rick, a legendary Detroit cocaine dealer, helped the FBI uncover brazen police corruption.

Music

How Even an Old Hipster Can Age Gracefully

On their new albums, Leonard Cohen, Robert Plant, and Loudon Wainwright III show three ways.

The U.S. Has a New Problem in Syria: The Moderate Rebels Feel Like We’ve Betrayed Them

Piper Kerman on Why She Dressed Like a Hitchcock Heroine for Her Prison Sentencing

Trending News Channel
Oct. 1 2014 1:25 PM Japanese Cheerleader Robots Balance and Roll Around on Balls
  News & Politics
Crime
Oct. 1 2014 4:15 PM The Trials of White Boy Rick A Detroit crime legend, the FBI, and the ugliness of the war on drugs.
  Business
Moneybox
Oct. 1 2014 2:16 PM Wall Street Tackles Chat Services, Shies Away From Diversity Issues 
  Life
Gentleman Scholar
Oct. 1 2014 4:55 PM Blood Before Bud? Must a gentleman’s brother always be the best man at his wedding?
  Double X
The XX Factor
Oct. 1 2014 5:11 PM Celebrity Feminist Identification Has Reached Peak Meaninglessness
  Slate Plus
Behind the Scenes
Oct. 1 2014 3:24 PM Revelry (and Business) at Mohonk Photos and highlights from Slate’s annual retreat.
  Arts
Brow Beat
Oct. 1 2014 3:02 PM The Best Show of the Summer Is Getting a Second Season
  Technology
Future Tense
Oct. 1 2014 4:46 PM Ebola Is No Measles. That’s a Good Thing. Comparing this virus to scourges of the past gives us hope that we can slow it down.
  Health & Science
Science
Oct. 1 2014 4:03 PM Does the Earth Really Have a “Hum”? Yes, but probably not the one you’re thinking.
  Sports
Sports Nut
Oct. 1 2014 5:19 PM Bunt-a-Palooza! How bad was the Kansas City Royals’ bunt-all-the-time strategy in the American League wild-card game?