Should autonomous weapons be banned?
Many organizations have proposed a ban on autonomous weapons, including Max Tegmark’s Future of Life Institute.
FLI’s argument is as follows: “Unlike nuclear weapons, [autonomous weapons] require no costly or hard-to-obtain raw materials, so they will become ubiquitous and cheap for all significant military powers to mass-produce. It will only be a matter of time until they appear on the black market and in the hands of terrorists, dictators wishing to better control their populace, warlords wishing to perpetrate ethnic cleansing, etc. Autonomous weapons are ideal for tasks such as assassinations, destabilizing nations, subduing populations and selectively killing a particular ethnic group. We therefore believe that a military AI arms race would not be beneficial for humanity.”
I am unsure, but my best guess is that this reasoning doesn’t hold. I think it’s true that, once autonomous weapons are built, they’ll eventually become widely available. However, the main argument here appears to be that, once both sides have them, autonomous weapons make asymmetric warfare more effective. Consider, eg., a terrorist or militia group fighting a major government. If both sides have AI weapons, rather than guns and bombs, is the major government more scared of its opponent, or less scared?
Major governments already have overwhelming military superiority; they’re not afraid of being out-gunned. The big threats to them are casualties among their own soldiers, and civilian casualties. These hurt them much more than their enemies, and might make the war unpopular enough that they’re forced to quit, as happened eg. in Vietnam. But robust AI could be much better at distinguishing enemy soldiers from civilians than aircraft (which operate overhead at a large distance), or human soldiers (who have to keep their distance to avoid return fire). And of course, if robot soldiers are killed, there’s no loss of life. Hence, it seems like a government with a vastly larger budget, and therefore vastly better technology, should be able to win the war and defeat the enemy group very quickly, making retaliatory attacks irrelevant. In this model, for example, it wouldn’t matter much if ISIS had autonomous weapons. The US or another major power would’ve been able to use full force against them with their own AI, without risking friendly fire or civilian deaths, and so ISIS would simply have been wiped out within a few days. (As happened to, eg., the conventional Iraqi army in 1991.)
Another issue is whether such a ban would be game-theoretically stable. The current bans on chemical and biological weapons, and the de facto ban on using nuclear weapons, are all stable because these weapons are very indiscriminate. In order to get X amount of military effectiveness — eg. enemy soldiers killed, or enemy cities captured — one has to cause Y amount of random destruction, where Y >> X. This means that no one really wants to use them first, because whoever started a war with them would inevitably wind up worse off themselves (see eg.
why the world banned chemical weapons ).
However, well-designed AI weapons seem to offer the possibility of a “clean war”, where the enemy government, military leadership, and military equipment are destroyed, but everything else remains intact. They’d also, I suspect, give one overwhelming military superiority once AI is far enough along. Hence, in a 2050 scenario with very advanced narrow AI and a universal ban on autonomous weapons, the first nation to develop AI weapons (in secret, say) could, in a “first strike”, quickly and cleanly defeat all potential enemies. This option isn’t available for chemical or biological weapons, or other banned weapons like laser blinders. It’s only available for nuclear weapons (again, assuming you have them and everybody else doesn’t) at the cost of massive, worldwide, and indiscriminate destruction. And even then, their military potential has made it impossible to fully ban nuclear weapons possession, with nine current nuclear-armed powers. With such an overwhelming temptation to “cheat”, a universal AI weapons ban seems like it wouldn’t be a Nash equilibrium, and thus (even if initially effective) would just result in world conquest by whoever broke it first.
Definitely interested to hear any thoughts, arguments, etc. that others have developed here. Unfortunately, the major discussions around this have centered on advocacy, and so I think the underlying mechanics of AI war have been a bit under-explored.