Autonomous Weapons and the Cardinal Principles of IHL

Getting your Trinity Audio player ready...

In 2020, a drone strike was conducted against Libyan National Army forces by weapons systems with no known humans “in the loop”.¹ This slaughterbot, known as the STM Kargu-2, remains fully operational even when GPS and radio links are jammed, and is equipped with facial recognition software to target humans.² This is the latest form of autonomous weaponry which signals the third revolution in warfare after gunpowder and nuclear arms. While there is no universally accepted definition of ‘lethal autonomous weapon systems’ (LAWS), it is acknowledged that these weapons have enormous destructive capability with full autonomy in that there is no manual human control over the system. U.S. policy on autonomy in weapons systems, defines LAWS as “weapon system[s] that, once activated, can select, and engage targets without further intervention by a human operator.”³ The UK’s Department of Defence defines LAWS as weapon systems capable of human-level cognition.⁴ LAWS, once activated by human intervention, can make judgements about the viability of targets and operate without further instructions.

These features have sparked a debate on their compliance with International Humanitarian Law (IHL) as well as their moral, legal, and strategic standing within national security applications. A 2021 report by the American Congressional Research Service states that “there are no domestic or international legal prohibitions on the development of use of LAWs.”⁵ However, as of July 2018, 26 states supported a preemptive ban on LAWS.⁶ U.N. Secretary-General António Guterres wrote on Twitter that “Autonomous machines with the power and discretion to select targets and take lives without human involvement are politically unacceptable, morally repugnant and should be prohibited by international law.”⁷ This article shall explore the legitimacy of such weapons in IHL discourse, while highlighting any internal tensions.⁸ In conclusion, it argues that with appropriate and sufficient testing and human oversight mechanisms in place, LAWS can be a suitable alternative to human fighters, while complying with IHL.

Principle of Humanity

The Martens’ clause of the Hague Convention of 1899, which is also in Additional Protocol I of the Geneva Conventions, states that even when not covered by other laws and treaties, civilians and combatants “remain under the protection and authority of the principles of international law derived from established custom, from the principles of humanity and from the dictates of human conscience.”⁹ It is argued that since LAWS remove humans from life-and-death decision-making and lack human conscience themselves, they are unable to empathise with, or respect human dignity. Without compassion, LAWS contradict the principle of humanity which seeks to minimise human suffering.¹⁰ Devoid of human conscience, these self-governing LAWS cannot engage in essential decision making on the battleground directed by human morality.¹¹ Basing their actions on pre-programmed algorithms, LAWS are unable to fully appreciate the value of a human life and the significance of its loss. Although they are not specifically prohibited, the Martens clause provides human conscience as a guiding principle when no formal prohibitions are present in the international law.

On the other hand, however, it may be argued that human conscience is subjective; what might sound inhumane to one person might be absolutely acceptable to the other. Moreover, as Sassoli rather convincingly argues, “[o]nly human beings can be inhuman and only human beings can deliberately choose not to comply with the rules they were instructed to follow…A robot cannot hate, cannot fear, cannot be hungry or tired and has no survival instinct. Robots do not rape.”¹² Indeed, an autonomous weapon system may better comply with the dictates of humanity by virtue of not being human. Retaining human oversight over systems which are better able, through algorithms or the like, to abide by the cardinal principles of IHL may be a way to ensure that human dignity is protected.

As a response, the US has argued that since in some cases there could be humanitarian benefits to LAWS, a ban now before those benefits have been explored, would be “premature.”¹³ It may be worth exploring the potential of LAWS to better comply with the principle of humanity (with appropriate human oversight) before banning it outright to fully ascertaining its potential for respect of the law.

Principle of Distinction

According to IHL’s principle of distinction, the parties to a conflict must at all times distinguish between civilians and combatants in order to spare the civilian population and civilian property. Attacks may be made solely against military objectives.¹⁴ Even with conventional warfare, it is an increasingly complex task to distinguish civilians from combatants, as combatants increasingly choose to merge within civilian populations. So, the distinction between civilians and combatants remains difficult by visual means or with LAWS’ AI technology.¹⁵ Due to the intermingling of civilians and combatants, it is increasingly difficult for LAWS to effectively distinguish between civilians and combatants, resulting in indiscriminate killing and a violation of the principle of distinction. Docherty argues that “the ability to distinguish combatants from civilians or those hors de combat often requires gauging an individual’s intentions based on subtle behavioural cues, such as body language, gestures, and tone of voice. Humans, who can relate to other people, can better interpret those cues than inanimate machines.”¹⁶ However, it could also be said that with extensive training and research, LAWS can be trained to distinguish precisely between combatants and non-combatants e.g. infants and unarmed individuals and to be able to compute subtle behavioural cues on an almost human level. How plausible this is, only time and technological advancement will tell.

This is made difficult by the fact that AI algorithms are based on data sets they were trained to operate on, and these data sets are not fully accurate. As studies have shown that facial recognition AI programs are 100 times more likely to misidentify Asian and African American people than white people, LAWS might be racially or ethnically biased.¹⁷ They are not only capable of indiscriminately killing civilians and combatants but can also be accused of killing more brown and black civilians. Moreover, assumptions about men’s roles in warfare may miscategorize civilian men as combatants due to deeply embedded gender biases among human operators, and hence, the LAWS.¹⁸ Such gendered facial recognition systems in LAWS could make men, regardless of their actual combatant or civilian status, hyper-visible as targets.¹⁹ For this reason, a gendered and racial lens would have to be applied in ensuring LAWS’ compliance with the principle of distinction.

Principle of Proportionality

The principle of proportionality seeks to limit the damage caused by military operations by requiring that the effects of the means and methods of warfare used must not be disproportionate to the military advantage sought.²⁰ This principle involves the important determination of whether expected civilian harm outweighs anticipated military advantage, on a case-by-case basis, in a rapidly changing environment.²¹ It is not just a matter of quantitative calculation, but requires application of human judgment, informed by legal and moral norms and personal experience, to the specific situation.²² Since LAWS are not programmed to deal with the infinite number of unexpected situations on the battlefield, they are not equipped to make a judgement on proportionality of the attack-which runs an insurmountable risk of violating the principle of proportionality.²³ One example of such a situation is mass fratricide, with numerous weapons turning on friendly forces. Such scenarios can be caused through hacking, enemy behavioral manipulation, unexpected interactions with the environment, or a technical malfunction.²⁴ Of course, if LAWS are programmed to deal with maximum possible scenarios in the battlefield, they may serve as a viable alternative to human soldiers.

Indeed, it is likely that states will rely on LAWS more and more as time goes on. With fewer human fighters, less human casualties would occur on the ground, and national security can become a less costly exercise. For example, the US Department of Defense says that it costs the Pentagon about $850,000 per year for each soldier in Afghanistan. Contrarily, a small rover equipped with weapons costs roughly $230,000.²⁵ In light of this, it seems a more worthwhile approach to improve the compliance of these weapons rather than try and ban them completely.

Because of their sentient limitations, LAWS require human beings to remain ‘in the loop’ for application of qualitative and subjective elements which are context dependent. To further refine this position, a comprehensive definition of military objects as well as civilian will be needed, to demarcate the subject of attack.²⁶ Similarly, programming them to respect the principle of proportionality may also force States to agree on how exactly this must be calculated.²⁷ Finally, in a conflict scenario devoid of civilians’ presence e.g., in space or the deep seas, LAWS cannot be, by definition, in violation of the principle of distinction or proportionality and can be used.²⁸

Principle of Precaution

This principle requires taking feasible measures to ensure that civilians or civilian objects are not being attacked, by choosing weapons and tactics to minimise incidental injury and collateral damage.²⁹ For LAWS to comply with the principle of precaution, it is important to first determine the acceptable probability of error in their judgement. An attack must be cancelled or suspended if it becomes apparent that the objective is not a military one or is subject to special protection or that it would violate the rule on proportionality.³⁰ The question is, are LAWS able to make such judgements on a case-by-case basis? In my view, this is the principle where human oversight over LAWS becomes the most important as there is a cogent need for the option that a ‘human override’ is possible so that the attack complies with the precautionary principle. If a commander overseeing the system realises that the target is not a military object or that the attack would be in any other way unlawful, he or she must be able to cancel the attack.

If in the future, LAWS are programmed to minimise errors in exercising precaution, they may be able to render valuable services in warfare while satisfying the principle of precaution. As Thurnher has pointed out, “whereas human judgement is often clouded by anger, revenge, fear, and fatigue, autonomous machines may offer the promise of removing those worries from the battleﬁelds of the future”.³¹ LAWS’s ability to comply with the principle of precaution must be tested regularly during the development phase, so that the chances of violation of IHL are minimised and humans must be factored into their use to allow for an ‘override’ should the need arise.

Problem of Deciphering Accountability and Penalty

International humanitarian law requires that individuals be held legally responsible for war crimes and grave breaches of the Geneva Conventions. The contention is that machines are not subject to legal rules. Human Rights Watch writes that it would be unclear who would be held accountable for unlawful actions a robot commits, “Options include the military commander that deployed it, the programmer, the manufacturer, and the robot itself, but all are unsatisfactory. It would be difficult and arguably unfair to hold the first three actors liable and the actor that actually committed the crime, the robot, would not be punishable.”³²

Philosopher Robert Sparrow has also argued that autonomous weapons are causally but not morally responsible, similar to child soldiers. There is a risk of not knowing who to hold responsible, which renders IHL crippled.³³ The accountability problem shows that LAWS are a present danger conflicting with IHL. However, a new treaty specifying whether the operator, creator, potential intervener, or the machine itself will be held responsible, can mitigate this problem. Human control will also naturally render the controller responsible, which also makes the issue simpler to settle.

To Ban or to Regulate?

With a huge potential to violate IHL and a gap in legislative protections available against LAWS, many have sought a complete ban or regulatory limitations on their use. The NGO, Campaign to Stop Killer Robots, has called for a “comprehensive, pre-emptive prohibition on fully autonomous weapons.” To ensure ‘both humanitarian protection and effective legal control’, humans must remain in control.³⁴ On the other hand, Article 36, an organisation arguing that the principles of humanity ‘require deliberative moral reasoning, by human beings, over individual attacks’, has called for a new legal instrument that explicitly prohibits weapons that do not allow ‘meaningful human control’.³⁵ It has also defined “meaningful human control”; humans not computers and their algorithms should ultimately remain in control of, and thus legally responsible for relevant decisions about (lethal) military operations. This shall not only alleviate the responsibility gap for LAWS, but also mitigate specific violations of the principles of IHL arising from their use.³⁶ However, this endeavour requires extensive moral and legal debate as well as technological innovation, to align the accountability test of LAWS with expected and acceptable human behaviour.

Conclusion

There is great potential to render use of LAWS more compliant with IHL than the current mechanisms of warfare. To sum up a few advantages; precise and targeted attacks to lessen civilian casualties, traceable accountability, and reduced loss of lives of armed soldiers. To allow LAWS to minimise casualties and vulnerabilities, a legally binding instrument should bind states parties to clear obligations.³⁷ With meaningful human control, and direct intervention of human judgement through oversight mechanisms, the accountability problem is mitigated, and so is the probability of violation of principle of proportionality and precaution. LAWS may be capable of making warfare more humane by allowing for algorithmic decisions not affected by flawed decision making that is a virtue of being human.

Mishal Murad

Mishal Murad completed her LLB from London School of Economics and is currently working as a legal associate in Lahore. Interested in understanding the intersection between law and policy, she launched a UK based start-up for online news source called Global Telegram. You can reach her at [email protected] and view her profile at https://www.linkedin.com/in/mishalmurad/.