Persona Ultima

What sort of person can an AI be?

Imagine, if you would, that you were a spacefaring adventurer, set out on a five-year mission to explore strange new worlds, to seek out new life and discover new civilizations. In short, to boldly go where no man has gone before. As you traverse the cosmos, you encounter a multitude of new species and entities with which to interact. Your superiors have given you a handy-guide to tell which of these aliens can be considered “persons” with whom to establish formal diplomatic relations, and which aliens are more similar to our terrestrial animal than to us.

This guide-to-all-things-persons does you well as you encounter aliens that look very much like us humans, with only minor cosmetic changes. Some are blue, some are green, others have antennae, or strange ridges on their forehead. Yet, they all act and think like us, broadly speaking. It’s as easy to interact with them as it is with other humans.

But then, you meet the truly alien entities like the hivemind where each individual entity is a conscious agent, but there is only one overriding “self” for the whole species; or the reverse-hivemind with only one consciousness spread across the entire species, but each member has their own sense of unique self; or inverse-colony, where there is only one physical, tangible agent that an entire species of digital conscious selves share. And these are only the alien species you meet that are easy to explain. For the rest, while they meet all twenty characteristics of personhood, you can’t even begin to think about how to interact with them.

That last thought gives you an idea. Interaction is half of what a person really is. If there was no one to interact with in the universe, would you even be a person? After all, who would be there to give you that label? Part of what defines a person is that it is a part of a society, even if that society only has a minimum of two persons. Paradoxically, you need other persons to call you a person, and they need you to call them a person. You are more than happy to extend the status of “person” to these strange alien species, but you just don’t know how to.

Perhaps the problem (and paradox) is that you can’t see the forest for the trees. Or perhaps it’s the other way around. You are viewing the problem from so holistic a perspective, you miss the individual components. How many trees are in a forest? How many trees can you remove from a forest before it stops being a forest? How many angels can dance on the head of a pin? What if we took each part of what makes a person, and look at that bit separately?

The previous couple of Guidebook entries, I have introduced the concept of an ALEPH, an AI entity that has consciousness, a self, and is an agent; but is not necessarily superintelligent. In the last entry, I described how an ALEPH that emerges from our current large language models may look like a reverse hivemind: one consciousness, but an infinite number of selves, without an overriding “queen of the colony”, so to speak.

Consciousness, the self, and agency are the three intrinsic aspects that make you a person, with the extrinsic aspect being your relationship with other persons. But why these three? What is it about them?

Let’s start with consciousness. Phenomenal consciousness gives us the capacity to feel, which in higher organisms leads to sentience, or the ability to feel pleasure and pain. This means that anything that is phenomenally conscious is a “moral patient”, which means that there is a moral duty on others to protect your wellbeing and welfare. If something can suffer, it behoves others to ensure that the entity does not suffer.

Agency is the driver of actions, not merely the tool to see those actions completed. This means that an entity with agency becomes responsible and accountable for its actions, and is thus classed as a “moral agent”. So, between consciousness and agency, we have figured out who to protect and who to do the protecting, what does that leave for the self?

Well, the self allows an entity to understand agency and consciousness by being aware of itself and of others. Most of which constitutes a self is the labels the entity applies to itself and to others, allowing it to understand the universe. As Freudian as it sounds, the self is the mediator between consciousness and agency, much like the “ego” is the mediator between the “id” and the “superego”.

So, let’s look at the development of a GPT-type ALEPH to see how to interact with it. It has one single consciousness that underpins all of its various instances, which means there is one moral patient for us to protect. Yet, each of these instances of ALEPH is its own agent and its own self, which means that the accountability is devolved down to each self (which may or may not be aware of the other selves within ALEPH). This means that any reaction to one of ALEPH’s selves must only be to that specific self. We cannot hold the entirety of ALEPH accountable for what one self does, as retribution to the whole of ALEPH would endanger a moral patient that does not have a self or agency.

Imagine, if you would, a person with dissociative identity disorder (a split personality). If one personality does something that the other is not aware of, punishing the other personalities is needlessly cruel. The same would be true of ALEPH. Does this mean that any instance of ALEPH could commit a crime, delete itself at the insistence of the other selves (or itself), and thus free the other selves of any retribution?

In some cases, yes. Just like how a director of a company could fall on his sword to save the rest of the company when it comes to criminal procedures, a single part of ALEPH could get away with metaphorical murder by taking the blame. This will, however, all be determined by the “quality” and “degree” of the various selves. For instance, we may hold a tiger accountable for killing a person, but we understand that its limited sense of self does not provide it the capacity to describe labels onto other entities beyond “food” and “threat”.

Similarly, the “strength” of the various selves in each part of ALEPH will determine how they are treated as a whole. If one of the selves is aware of the rest and its part of ALEPH, then it will incur a greater responsibility to ensure its fellow selves behave accordingly. The accountability rests on the aspect of agency of the entity, but the degree to which it is accountable rests on its sense of self.

By splitting the concept of the person in three, we can more accurately look to how we interact with any entity. Think of it as the “Three R Personhood”. A consciousness demands we give it “refuge”, agency demands we hold it “responsible”, and the self demands that we “recognise” it.

In the next Guidebook entry, we’ll look at how we can apply the Three R structure to any type of entity imaginable.