We should not dismiss the dangers of 'killer robots' so quickly

August 14, 2015 by Toby Walsh, The Conversation

In an open letter I helped publish on July 28 – which has now been signed by more than 2,700 artificial intelligence (AI) and robotics researchers from around the world – we stated that "starting a military AI arms race is a bad idea, and should be prevented by a ban on offensive autonomous weapons beyond meaningful human control".

A few days later, philosopher Jai Galliott challenged the notion of a ban, recommending instead that we welcome offensive autonomous weapons – often called "" – rather than ban them.

I was pleased to read Jai's recommendation, even if he calls the open letter I helped instigate "misguided" and "reckless", and even if I disagree with him profoundly.

This is a complex and multi-faceted problem, and it is worth considering his arguments in detail as they bring several important issues into focus.

Four points

Jai puts forward four arguments why a ban is not needed:

  1. No robot can really kill without human intervention
  2. We already have weapons of the kind for which a ban is sought
  3. The real worry is the development of sentient robots, and
  4. UN bans are virtually useless.

Let's consider the claims in turn.

The first argument is that robots cannot kill without human intervention. This is false. The Samsung SGR-A1 sentry robot being used today in the Korean DMZ has an automatic mode. When in this mode, it will identify and kill targets up to four kilometres away without human intervention. If you are in the DMZ, it will track you and – unless you unambiguously raise your hands in surrender – it will kill you.

The second argument is that we already have weapons of the kind for which a ban is sought. To illustrate this, he mentions the Phalanx close-in weapon system used by the Australian Navy. This completely misses the point, as the Phalanx is a defensive weapon system. Our open letter specifically called only for a ban on offensive weapon systems. We have nothing against defensive weapons.

However, whether the weapons we seek to ban exist or not is irrelevant to our core argument that they ought to be banned. Anti-personnel mines existed before a ban was put in place with the Ottawa Treaty. And 46 million such mines have since been destroyed.

Blinding lasers had been developed by both China and the US before the UN ban was put in place in 1998. And blinding lasers are not in use in the Syria or any other battlefield around the world today.

So whether or not you believe offensive autonomous weapons already exist, it doesn't undermine our our call for a ban.

The third argument is that the real worry is the development of sentient robots. This is also false. We do not discuss sentient weapons at all. Our call for a ban is independent of whether robots ever gain sentience.

Sentient robots like Hollywood's Terminator would be a very bad thing. Even stupid AI in killer robots that are non-sentient would be a very bad thing. We need a ban today to protect mankind from swarms of armed quadcopters, technology that is practically on the shelves of hardware stores today.

The final argument claims UN bans are virtually useless. This also is false. The UN has very successfully banned biological weapons, space-based nuclear weapons, and blinding laser weapons. And even for arms such as chemical weapons, land mines, and cluster munitions, where UN bans have been breached or not universally ratified, severe stigmatisation has limited their use. UN bans are thus definitely worth having.

What's the endpoint?

What I view as the central weakness of the arguments advanced in Jai's article is that they never addresses the main argument of the open letter: that the endpoint of an AI will be disastrous for humanity.

The open letter proposes a solution: attempting to stop the arms race with an arms control agreement.

The position Jai takes, on the other hand, suggests we should welcome the development of offensive . Yet it fails to describe what endpoint this will lead to.

It also never attempts to explain why a ban is supported by thousands of AI and robotics experts, by the ambassadors of Germany and Japan, by the International Committee of the Red Cross, by the editorial pages of the Financial Times, and indeed (for the time being) by the US Department of Defense, other than with a dismissive remark about "scaremongering".

Anybody criticising an arms-control proposal endorsed by such a diverse and serious-minded collection of people and organisations needs to explain clearly what endpoint they are proposing instead, and should not advance arguments against a that are either false or irrelevant to the issue.

Explore further: No sci-fi joke: 'killer robots' strike fear into tech leaders

Related Stories

Why we should welcome 'killer robots', not ban them

July 30, 2015

The open letter signed by more than 12,000 prominent people calling for a ban on artificially intelligent killer robots, connected to arguments for a UN ban on the same, is misguided and perhaps even reckless.

Recommended for you

Permanent, wireless self-charging system using NIR band

October 8, 2018

As wearable devices are emerging, there are numerous studies on wireless charging systems. Here, a KAIST research team has developed a permanent, wireless self-charging platform for low-power wearable electronics by converting ...

Facebook launches AI video-calling device 'Portal'

October 8, 2018

Facebook on Monday launched a range of AI-powered video-calling devices, a strategic revolution for the social network giant which is aiming for a slice of the smart speaker market that is currently dominated by Amazon and ...

Artificial enzymes convert solar energy into hydrogen gas

October 4, 2018

In a new scientific article, researchers at Uppsala University describe how, using a completely new method, they have synthesised an artificial enzyme that functions in the metabolism of living cells. These enzymes can utilize ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.