‘That Others May Die’: Autonomous Military Technology and the Changing Ethos on the Battlefield

As heavily reported by media in May 2018, Google announced that it won’t renew its contract with the US Defense Department for an artificial intelligence endeavor known as Project Maven. The tech giant took the decision amidst a widespread public backlash and multiple employee resignations. Google was assigned to provide an AI-powered image labeler that would allow the Pentagon to browse a ‘Google-Earth-like’ drone system, according to one of the emails leaked during the scandal. 

The backlash over Google’s controversial alliance didn’t come out of nothing. Already last year, more than 100 of the world's top robotics experts signed an open letter to the United Nations indicating the risks of automation in military technology. Their aim was to generate awareness about the role of artificial intelligence in transferring lethal decisions to machines that can search for, identify, and engage targets on their own. As in many other fields, it is the supposed ability of smart tech to learn from previous examples, then adapt to circumstances it may not have encountered before, that is worrying to many. ‘Unlike a conventional computer that uses pre-programmed instructions to tackle a limited range of possibilities, the AI recently designed by the US military forces is learning to make its own judgment – and that’s very dangerous,’ says Andrew Nanson, chief technology officer at Ultra Eletronics, a world-leading group of businesses operating in the defense, cyber security and energy markets.

But while the discussion on killer robots’ autonomy assumes the center stage, another critical perspective remains partially concealed: the effects of such automated weaponry and warfare for the discourses of the military. Traditionally, military reasoning has always had its own cardinal virtues: courage, sacrifice, heroism and so on. Among other things, being ‘ready to die’ seemed to be one of the main factors in victory, according to Carl von Clausewitz in his most famous book, On War. But given that autonomous machines cannot suffer the consequences of reciprocal exposure to death, nor the humans who deploy them from a great distance, can this situation influence the overall ethos of war? Or, putting it differently, what moral considerations can be made about an army that partakes in a risk-free war-making?

In his article In Defense of Drones: A Historical Argument, David Bell says that ‘if our technology is new, the desire to take out one’s enemies from a safe distance is anything but’. Bell reminds us that the colonial wars are still the worst example of how the application of technology can drive unequal conflicts, or in some cases, massacres. During the Battle of Omdurman (Sudan, 1892), for example, there were forty-eight Anglo-Egyptian soldiers killed against nearly ten thousand Muslim men pierced by Maxim’s bullets, the first fully automatic machine gun. And this was by no means unprecedented. As Voltaire puts it: ‘whoever was rich became almost invulnerable in war’, meaning that a combat can turn into one-sided killing by whom has the newest technologies at hand.

In the early 1900s, a report called Synthèse de la Guerre Sous-marine provided an impression of the state of mind of the first military submarines’ crews at a time when, without sonar, most surface vessels were incapable of detecting their presence. ‘They were invincible. For them, the war became a game, a sport, a kind of hunt in which they needed to do nothing but enjoy the spectacle of the agony of their victims. They, meanwhile, would be sheltered from any attacks and, once back in port, they could busy themselves recounting their hunting prowess.’ It could be said that, more often than not, the use of new defense technologies falls in a continuous and asymmetrical distribution of power in which unconditional victory becomes the only reasonable justification for the use of such technologies. Most of the times, the recurring disproportion of weaponry is what guaranteed the defeat of the enemy, making these asymmetries easier to be accepted by the ones benefiting from them.

However, every time disproportionate warfare happened, certain values were damaged. According to Grégoire Chamayou, a research scholar in philosophy at the Centre National de la Recherche Scientifique, it was probably at the moment of NATO’s intervention in Kosovo in 1999, that the clearest case of a spectacle of armed violence came about. Acknowledging the loss of a helicopter and eight soldiers’ lives in the first two days of the war, the American public opinion demonstrated that another Vietnam episode would come with an exorbitant political price. Hence, motivated by a rationale of minimal losses, the Air Forces adopted a flight altitude of 15,000 feet to their bombers, a secure distance that meant they were nearly out of reach of the enemy’s anti-aircraft defenses. As a result, NATO planes carried out 38,004 raids in seventy-eight days without sustaining a single loss among the members of their crews.

By doing this, NATO, but especially the American forces, managed to invent a kind of warfare that inflicted zero deaths on their own men. Consequently, they also invented a controversial understanding of war as an activity in which human dying and killing are not exchanged. Acknowledging this, Amnesty International made several inquiries about how relevant and accurate these air strikes were and to what extent civilians were considered, since it seems to be impossible to specify a target from the altitude that ensured the lives of NATO’s pilots. In other words, for Amnesty International, the decision to preserve the lives of US soldiers ended up exposing thousands of innocent people to an unequal distribution of violence.

The current use of drones in battlefield can be considered a direct result of the strategies adopted in Kosovo, and in practice represents the accumulation of technology that this rationale was able to produce and deploy since the 1990s. Moreover, claims Chamayou, by providing an even greater sense of invulnerability due to their unmanned nature, drones symbolize an imperative and historical shift concerning the ethical principles of warfare. In this new scenario, combatants rather than civilians are the ones assured with immunity from the effects of fighting. Therefore, today as yesterday, the technology used to protect the ones who fight from death is what is redefining the Western sense of what a warrior should be – by distancing themselves totally from the hand-to-hand combat, soldiers are found in another moral category where they no longer confront the enemy but slaughter or hunt them, Chamayou suggests.

So far, autonomous and human-aided robots have played a marginal role in warfare. According to Mark Smith, a reporter on military technology from BBC, drone campaigns in continuous, low-intensity conflicts like the ongoing War on Terror represent just a fraction of the potentiality of these machines. For Smith, the Pentagon will likely improve its efforts to build an all-encompassing strike capability using robotic systems that can survive in contested spaces against sophisticated adversaries like Libya, Syria, China or Russia. In fact, a couple years ago, the US Department of Defense released a directive called Autonomy in Weapons Systems, which defines the guidelines for the adoption of autonomous weapons for the next decade, until 2022.

Parts of this directive are conservative regarding the implementation of full autonomy and state that robots must always follow a human operator’s intent, and that such systems may only be used after passing a series of internal reviews considering that most of them are produced by third-party suppliers. However, the highly profitable industry that backs innovation in armed forces is continuously pushing for a greater implementation of artificial intelligence in weapon systems that will eventually be able to seek enemy targets and engage them without any human intervention. Today, drones can fly autonomously and AI helps to classify both people and objects, but operators still make the decision to fire.

Paul Scharre, a senior fellow of the Center for New American Security, says that automation is a natural consequence of digital mediation in war and that every step taken by the US Department of Defense towards innovation gradually reduces the role of human control and decision-making in the functioning of weapons. For Scharre, this is a cyclical process in a sense that the more technology is deployed in the battlefield, the higher the amount of data that is generated will be and the more we will need machines to process it. ‘Look at the fighter pilot who has to deal with huge amounts of sensory data and take decisions in a split second. Although the US government policy at this time assures that the trigger to kill is pulled by the human, it might not remain that way as the data overload becomes impossible for the pilot to handle.’

Already in 2016, an artificially intelligent fighter pilot known as Alpha reportedly defeated two human-controlled attacking jets in a combat simulation. According to Doug Barrie, a military aerospace analyst at think tank IISS, Alpha uses a form of artificial intelligence based on the concept of ‘fuzzy logic’, in which a computer considers a wide range of options before making a decision. Because a fighter jet produces so much data, it is not always obvious which maneuver is most advantageous or, indeed, at what point a weapon should be fired. Fuzzy logic systems can weigh up the significance of these individual pieces of data before making a decision. Barrie says that the researchers’ key achievement was to even take into consideration in real-time the air-to-air environment, which is extraordinarily dynamic and has a vast number of parameters.

All in all, it could be said that, as the usage of military AI escalates, it pushes the decision-making process to a point where humans cannot possibly react fast enough. In many situations that arise in modern warfare the time to make intelligent, informed choices becomes just too short. In response, countries will have to adopt active protection systems to automatically sense and defeat potential threats. Today, more than thirty nations are already employing some level of human-supervised, autonomous weapons to defend ships, vehicles and land bases from attack. These systems do not engage targets on their own but the pressure of an increasing battlefield tempo is forcing autonomy further towards the point of robots making that final, lethal verdict. At that moment, human agents would be neither in the loop (in the sense of participation) nor over it (in the sense of supervision).

The mechanism behind this increasing tempo of war can be better explained by Kittler, who argues for a notion of autonomy in technology that counterpoints the mainstream idea that innovation happens according to societal demand. Kittler fundamentally disagrees with the proposition made in the mid-60s by McLuhan that technology can be seen as extensions of men. In fact, he says that technical innovations follow the model of military escalations, only referring and answering to itself. An example could be the emergence of nuclear weaponry during the Cold War despite the rejection of it all over the world. For Kittler, the conditions that determine our social history follow a process that is completely independent of human will or volition. While McLuhan hopes that we will reach a sort of equilibrium with technology, Kittler goes to the opposite direction: there is no equilibrium because technology is the one autonomously extending to us. Put in another way, we have no power to dictate the direction that technology takes – we merely watch its unpredictable development.

Many of the ethical questions raised today by automated weapons are a direct continuation of those brought up almost three decades ago by the ‘war without risks’ waged in the skies over Kosovo. While back in the 90s institutions like Amnesty International put in check the convenience to kill from a safe distance, currently the ethical dilemmas involve the deployment of soldier bots to which death is irrelevant. What both scenarios describe is the use of weapons that make it possible to rule out the occurrence of casualties on one’s own side, and this affects the whole concept of combat. In other words, these weapons transform war from being potentially symmetrical into a unilateral affair of death-dealing in which the opponent is left without the very possibility of striking back. In a way, warfare in that case not only escapes the canonic framework designed for armed conflicts but also all the narratives traditionally attached to it.

Grégoire Chamayou suggests that there are two frameworks for the legal conceptualization of war. The first is of a penal nature and legitimates punishment when the opponent is guilty, armed violence being the sentence. The other is connected to the notion of a duel, or what is known as jus in bello: both combatants are part of a particular situation in which they have the common and therefore acknowledged right to kill each other without this being a crime. Fundamentally, Chamayou claims, this is what the term ‘war’ means: a defined moment when armed violence is decriminalized and opened up to both sides in combat.

When it comes to unmanned weapons that are gradually becoming autonomous, the question of legitimation can be reduced to whether or not they conform to the principles normally applied to military conflicts. A fighter who kills in line with the clauses of jus in bello is granted legal immunity. But under what convention or belief can the use of robots be subjected to decriminalization? In which legal terms could it be represented? This is still a grey area in military ethics. If one has the right to inflict death without offending the law, it is because that right is granted to both sides. In other words, the use of unmanned weapons, preventing on one side the potential loss of lives, on the other becomes morally unjustified.

Technology thus forces us to rethink its relationship to war. A comparison between the value attached to a robot’s capacity to preserve lives and the virtues attributed to the ones participating in an armed conflict is enough to reveal the nature of the changes taking place in the military discourse. Characteristics like courage and heroism always stood in the heart of this discourse and, according to François Lagrange, serve to make the butchery that soldiers face in combat acceptable. In his book Les Combattants de la Mort Certaine, Lagrange states that every high-ranking officer knows that they must grow in their men a strong sense of self-sacrifice, ultimately guiding them to death. Without such guidance, war would be impossible.

But what happens when the desire to inflict losses on the enemy is finally detached from its alleged inherent sacrifice? Chamayou explains that, with the introduction of armed forces that are no longer susceptible to death, the heroic narrative forged by the army for a long time dissolves into a simple imperative of self-preservation, leading to a troublesome notion of ‘virtueless war’. So, even if traces of bravery are shown here and there, it could be said that they are merely residues of an ideology that is no longer in line with its praxis, and that such demonstrations will likely become less and less in the near future. As Hegel puts in Outlines of the Philosophy of Right, ‘true courage’, that of combatants, lies not so much in suppressing the fear of death but rather in the soldier’s ‘readiness for sacrifice in the service of the state’.

The main issue of the trade between sacrifice and self-preservation is that, put in military discourse, using automated weapons to kill also means the highest degree of cowardice. This paradox seems to bother even the army’s own personnel, the ones that are the actual beneficiaries of this new technical reality. John L. McLucas, author of the book Reflections of a Technocrat, reminds us that the most inflamed critique of the use of armed robots came not from civil organizations but from Air Force pilots who defended their traditional values. For them, war without risks relegates the enemy to the role of a body to which violence is applied in the form of an algorithmic, administrative measure. In sum, it is the clash between new weapons and old frameworks that produces a crisis in military ethos.

In The Organizational Construction of Hegemonic Masculinity, Frank Barrett offers another approach to the use of automated fighters and the decline of military morale. For him, the use of the word ‘unmanned’, the official term for ‘crewless’ as in ‘unmanned aerial vehicle (UAV)’ or ‘unmanned weapons’, is problematic. Barrett observes that the term subconsciously comes to mean a male that is less masculine, emasculated, or un-manned, and ends up denigrating the vigor or effectiveness of those persons the unmanned object was supposed to protect. ‘Obviously the drones threatened their own employment, their professional qualifications, and their institutional position, but the threat was also to their own virility, which was largely associated with the taking of risks.’

To improve this situation, the Pentagon in September 2013 considered to allow military honors to pilots, sensor operators and others who provide supervision to human-aided robot strikes. Traditionally these medals are awarded as a recognition of bravery in battle, something that has started to change meaning. Chamayou writes that Luther Turner, a retired colonel who is now piloting drones, firmly believes it takes bravery to fly a UAV – particularly when it involves taking someone’s life. According to Turner, there is a counterpart in killing even if the confrontation with death is heavily mediated (and ultimately is not one’s own death). In this context, the meaning of bravery changes and relates to ‘what is right’ regardless of the means used to achieve a particular end.

Colonel Turner states that pilots of remote-controlled robots are experiencing high levels of Post-Traumatic Stress Disorder (PTSD), a medical condition commonly reported by combatants who have faced body-to-body confrontation in war. So, in line with the new narrative created by the military forces, soldiers who run killer robot’s campaigns in a foreign territory can still be regarded as victims of their own actions, which at once restores their moral balance and their status of bravery. In this speculative scenario, it is the mental health that is vulnerable and exposed to violence, a kind of violence that is not anymore reciprocal but unilateral and exerted by an apparatus that offers no resistance to the act of killing. In other words, the army becomes the frightened audience of its own inflicted terror.

Cases like these present us to a newborn ethics where technologists, weapon manufacturers and remote pilots are represented as being virtuous due to their ability to rule out any possibility of casualties in war. What is taking place then is a switch from a traditional morality based on values like self-consciousness to another based on a narrative of self-preservation. As Hegel vastly debated in his critique of virtue, war – and the inevitable confrontation of death that it fosters – traditionally played a crucial role in forming and maintaining ethical consciousness as a universal principle. For him, only by giving up the attachment to life involved in self-preservation, consciousness breaks through to a higher form of selfhood, a transition from mere fear of death to the true awareness of life. In other words, ethics is directly associated with consciousness (of oneself and the other), which is embodied in the wider structure of society.

One could argue that the programmed replacement of live armies for autonomous bots proposed by the US military feels more convenient and in line to the current rhythm of technical innovations and establishes, at the same time, an appealing discourse on common defense and general welfare. Since Vietnam, the state’s capacity to wage war has been challenged for its use of American citizens as the ultimate resource. However, this important inversion of values requires that narratives that were enforced in the past come out as outdated in a so-called post-heroic age. In this new historical period for the military, one-way armed and autonomous violence persists to be referred to as combat, in a vast but almost unnoticed redefinition of the term.

Gustavo Velho Diogo is a designer from Brazil with a master’s degree in Arts and Culture Studies from Radboud University. He recently got accepted in the UT Austin-Portugal Digital Media program with a PhD proposal on the politics of mapping technologies. You can read his previous INC Longform 'Google Earth, Surveillance, and the Power of Digital Cartography' here.

Ronald C. Arkin, Ethical Robots in Warfare, Atlanta: Georgia Institute of Technology Press, 2017.

Frank Barrett, ‘The Organizational Construction of Hegemonic Masculinity: The Case of the US Navy’, in Gender, Work and Organization, no. 3, 1996.

Doug Barrie, ‘Artificial Intelligent Fighter Pilot Overperforms Two Awarded Soldiers in Simulation’, Military Affairs, BBC, 2017, https://www.bbc.com/news/technology-36650848.

David Bell, ‘In Defense of Drones: A Historical Argument’, New Republic, 27 January 2012, https://newrepublic.com/article/100113/obama-military-foreign-policy-technology-drones.

Grégoire Chamayou, A Theory of the Drone. New York: The New Press, 2013.

W. F. Hegel, Outlines of the Philosophy of Right. New York: Oxford University Press, 1982 [1820].

John L. McLucas,  Reflections of a Technocrat: Managing Defense, Air, and Space Programs During the Cold War, Maxwell Air Force Base: Air University Press, 2006

Marshall McLuhan,  Understanding Media: The Extensions of Man, New York: McGraw-Hill, 1964

Friedrich Kittler, Optical Media. Berlin: Brinkmann & Bose, 2010.

Francois Lagrange, Les Combattants de la Mort Certaine: Les Sens du Sacrifice à L’horizon de la Grande Guerre, Jstor: Cultures & Conflits n63, 2006.

Paul Scharre, The Trouble with Trying to Ban Killer Robots, Center for New American Security, World Economic Forum, 2017.

Mark Smith, ‘Is Killer Robot Warfare Closer than We Think? Technology of Business’, BBC, 2017, https://www.bbc.com/news/business-41035201.

Voltaire, An Essay on Universal History, the Manners. London: Nourse, 1998 [1759].