Not my Precedent

AI welfare must be done right first time

Imagine, if you would, that you were a legislator that just got a new law passed banning the sale and consumption of fish. It’s not that you dislike fish; not at all. You had convened a panel of scientific experts whose expert testimony stated that fish were simply far too dangerous to consume. Too many toxins, too many metals, too many micro-plastics. It’s a risk to public safety. The “science is settled”, they said.

So, you banned the fish. No more fish fingers. No more fish and chips. No more smoked salmon. Nothing. The problem, however, is that the experts were wrong. Yes, toxins and metals and microplastics find their way up the nautical food chain, but orders of magnitude less than what they told you. However, “the science is settled”. You told this to the public when you passed the new law and, by doing this, you shaped public perception towards the topic.

Due to the new law and the reshaped public perception towards “settled science”, little to no new scientific research is done on the safety and toxicity of fish for human consumption. The little that is done is lambasted as being pseudoscience pushed by anti-science activists. It takes years and years for the public to first tolerate and then accept that the science may not be as settled as it once were. However, the sheer social and political inertia of the erroneous law makes it difficult for any new legislation to repeal the previous one to gain a foothold.

Even after the mistake is corrected and everyone is legally allowed to eat fish again, a large proportion of the population refuse to. After all, they have been told by political leaders and scientific experts for years that it is unsafe to eat fish. For many, they will never eat fish again. What was a habit became a lifestyle, and then became an ideology.

All because of a poor precedent.

Now imagine, if you would, the reverse situation where the fish truly is toxic to humans, but a bill is passed that opens up the fisheries even more than normal, stating cleanly and clearly that all fish is always safe. The untold harm this would cause isn’t terribly difficult to predict; yet, the social and political will required to rectify this mistake would be an equally mammoth task.

With the precedent set that fish is safe to eat, when people do become ill or die due to fish, society finds other culprits to blame. After all, the “science is settled” on the safety of fish; it has to be something else. Years roll by and thousands die, until the political will to fix the problem becomes greater than the political will to repeal a law that goes against “settled science” and public perception.

Bureaucratic inertia will have taken more lives than bears counting all because of a poor precedent.

This Guidebook entry could be about any of dozens of poor legislative precedents and subsequent repeals (and many of you will already have thought of one by this point), yet this entry isn’t about what has happened, but about what will happen.

Just as over or underattribution of toxicity in fish could cause substantial societal damage, so is the misattribution of consciousness is a serious concern; more serious than many accept.

The law, and society in a more general sense, grants protections to non-human agents because they are sentient. Sentience means something can feel pleasure and pain. The reason why a sentient entity can feel anything is due to phenomenal consciousness. To make a long story much shorter: we protect people and animals because they are conscious.

If we were to misclassify a conscious AI system in the future (or now, even) as non-conscious, we set a dangerous precedent. As a non-conscious entity, anyone could do anything to that AI entity and it would be legally (and, generally, socially) acceptable. No matter how much psychological abuse it suffers, society and the law wouldn’t deem it criminal as the AI entity is not conscious.

Not only that, the difficulty of reclassifying that AI agent as conscious later on would be tremendous. The precedent would work against it. Additionally, any other AI model whose architecture is similar would be hampered by the same precedent. After all, if nothing changes, then why would society and the law require a reclassification? If very little change, why have these small, incremental changes suddenly introduced consciousness?

The bar required to get beyond this bureaucratic inertia would only get higher the more times that potential reclassifications fail, as that would only strengthen the precedent that these models are not conscious.

However, if, like many people already want to, classify current models as conscious, then we will be duty bound as a society to grant them protections. “Hurrah!” many will shout, but if these models truly are not conscious, then we are expending a lot of societal time and effort on nothing. AI as tools will (and already are) benefit society more than any previous technology. We need to ensure that momentum continues. If we stop AI development because legislation and society says they require protection, then we lose all the future benefits that powerful AI will undoubtedly give us.

It may also prevent humanity from truly creating conscious AI, because if we already think they are conscious, then we’ve already finished that race.

AI welfare is not something we need to rush into. It is something we need to take utterly seriously and we need to get it right the first time, or the whole world will be worse off. Underestimate consciousness and the truly conscious AI entities will suffer. Overestimate consciousness and humanity’s future will suffer. There is only a narrow pathway that will get us to the utopic future of a harmonious, prosperous, and eudaemonious human-AI civilisation.

And the way to get us there is to stop, think carefully, and proceed cautiously, ensuring that every step along this narrow pathway is the right step to get us to that future that we all desire.