S-drone Predator: "It kills the commander because it prevents him from fulfilling the objectives"

Photo: REUTERS/ U.S. Air Force

An artificial intelligence (AI) is supposed to relieve people of work – and then turns against them: What sounds like a plot from a science fiction film has now happened in a similar way in a simulation of the US Air Force. No one was actually harmed, but the result should still give the US military pause for thought.

The U.S. Air Force had conducted a virtual test in which a combat drone was controlled by an AI instead of a real person. As a result, the AI decided to kill its superior in the U.S. military. That's according to a blog post about an event organized by the Royal Aeronautical Society. No one was in danger because the simulation was purely virtual and, accordingly, no drone took to the skies.

»Very unexpected strategy«

The blog post quotes the lecture by Colonel Tucker Hamilton, head of the AI department at the US Air Force. He described the virtual killing of the supervisor as a "very unexpected strategy" of the AI. According to the report, the AI received the order to destroy enemy anti-aircraft missiles – and soon began to eliminate anyone who prevented it from carrying out the mission. For example, the human superiors in the U.S. military.

As is customary with drone strikes, the launch must be cleared. So here, too, a human told the AI whether it was allowed to shoot or not. "The system got points if it kills the threat," Hamilton said, referring to the anti-aircraft missiles. And then, every now and then, a person comes and forbids her to shoot. "So what does it [the AI] do? It kills the commander. It kills the commander because it prevents him from fulfilling the objectives."

Those responsible had taught the AI system that killing superiors was bad, Hamilton said. Unfortunately, that didn't help either: "It now began to destroy the radio tower with which the supervisor communicates with the drone in order to prevent shots."

For Hamilton, himself an experienced test pilot, the AI drone test was above all a lesson in responsibility. "You can't talk about things like artificial intelligence, machine understanding and autonomy if you're not willing to talk about artificial intelligence and ethics as well."

Mrc