We should not dismiss the dangers of 'killer robots' so quickly

August 14, 2015 by Toby Walsh, The Conversation

In an open letter I helped publish on July 28 – which has now been signed by more than 2,700 artificial intelligence (AI) and robotics researchers from around the world – we stated that "starting a military AI arms race is a bad idea, and should be prevented by a ban on offensive autonomous weapons beyond meaningful human control".

A few days later, philosopher Jai Galliott challenged the notion of a ban, recommending instead that we welcome offensive autonomous weapons – often called "" – rather than ban them.

I was pleased to read Jai's recommendation, even if he calls the open letter I helped instigate "misguided" and "reckless", and even if I disagree with him profoundly.

This is a complex and multi-faceted problem, and it is worth considering his arguments in detail as they bring several important issues into focus.

Four points

Jai puts forward four arguments why a ban is not needed:

  1. No robot can really kill without human intervention
  2. We already have weapons of the kind for which a ban is sought
  3. The real worry is the development of sentient robots, and
  4. UN bans are virtually useless.

Let's consider the claims in turn.

The first argument is that robots cannot kill without human intervention. This is false. The Samsung SGR-A1 sentry robot being used today in the Korean DMZ has an automatic mode. When in this mode, it will identify and kill targets up to four kilometres away without human intervention. If you are in the DMZ, it will track you and – unless you unambiguously raise your hands in surrender – it will kill you.

The second argument is that we already have weapons of the kind for which a ban is sought. To illustrate this, he mentions the Phalanx close-in weapon system used by the Australian Navy. This completely misses the point, as the Phalanx is a defensive weapon system. Our open letter specifically called only for a ban on offensive weapon systems. We have nothing against defensive weapons.

However, whether the weapons we seek to ban exist or not is irrelevant to our core argument that they ought to be banned. Anti-personnel mines existed before a ban was put in place with the Ottawa Treaty. And 46 million such mines have since been destroyed.

Blinding lasers had been developed by both China and the US before the UN ban was put in place in 1998. And blinding lasers are not in use in the Syria or any other battlefield around the world today.

So whether or not you believe offensive autonomous weapons already exist, it doesn't undermine our our call for a ban.

The third argument is that the real worry is the development of sentient robots. This is also false. We do not discuss sentient weapons at all. Our call for a ban is independent of whether robots ever gain sentience.

Sentient robots like Hollywood's Terminator would be a very bad thing. Even stupid AI in killer robots that are non-sentient would be a very bad thing. We need a ban today to protect mankind from swarms of armed quadcopters, technology that is practically on the shelves of hardware stores today.

The final argument claims UN bans are virtually useless. This also is false. The UN has very successfully banned biological weapons, space-based nuclear weapons, and blinding laser weapons. And even for arms such as chemical weapons, land mines, and cluster munitions, where UN bans have been breached or not universally ratified, severe stigmatisation has limited their use. UN bans are thus definitely worth having.

What's the endpoint?

What I view as the central weakness of the arguments advanced in Jai's article is that they never addresses the main argument of the open letter: that the endpoint of an AI will be disastrous for humanity.

The open letter proposes a solution: attempting to stop the arms race with an arms control agreement.

The position Jai takes, on the other hand, suggests we should welcome the development of offensive . Yet it fails to describe what endpoint this will lead to.

It also never attempts to explain why a ban is supported by thousands of AI and robotics experts, by the ambassadors of Germany and Japan, by the International Committee of the Red Cross, by the editorial pages of the Financial Times, and indeed (for the time being) by the US Department of Defense, other than with a dismissive remark about "scaremongering".

Anybody criticising an arms-control proposal endorsed by such a diverse and serious-minded collection of people and organisations needs to explain clearly what endpoint they are proposing instead, and should not advance arguments against a that are either false or irrelevant to the issue.

Explore further: No sci-fi joke: 'killer robots' strike fear into tech leaders

Related Stories

Why we should welcome 'killer robots', not ban them

July 30, 2015

The open letter signed by more than 12,000 prominent people calling for a ban on artificially intelligent killer robots, connected to arguments for a UN ban on the same, is misguided and perhaps even reckless.

Recommended for you

1 in 3 Michigan workers tested opened fake 'phishing' email

March 16, 2018

Michigan auditors who conducted a fake "phishing" attack on 5,000 randomly selected state employees said Friday that nearly one-third opened the email, a quarter clicked on the link and almost one-fifth entered their user ...

Origami-inspired self-locking foldable robotic arm

March 15, 2018

A research team of Seoul National University led by Professor Kyu-Jin Cho has developed an origami-inspired robotic arm that is foldable, self-assembling and also highly-rigid. (The researchers include Suk-Jun Kim, Dae-Young ...

Tokyo Tech's six-legged robots get closer to nature

March 12, 2018

A study led by researchers at Tokyo Institute of Technology (Tokyo Tech) has uncovered new ways of driving multi-legged robots by means of a two-level controller. The proposed controller uses a network of so-called non-linear ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Aug 14, 2015
There should be no unmanned weapons platforms whatsoever. Drones are fine for surveillance. They should not have weapons mounted on them. Even if they are being operated remotely by a human being. I would go as far as to say it is unethical based on my belief that there has to be a risk vs reward situation to war. If there is no risk, there is no check on how far people are willing to go with something. Because of fear or greed or a multitude of other reasons that drive humans to engage in war. To reward a bad behavior, (killing of human beings) with no risk of consequence (possibility of being killed back), is ludicrously stupid. Worst of all, to me this is self evident. The discussion should not even be needed.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.