Threatening Goldilocks

What amount of power will balance the scales

Imagine, if you would, that you had a rival. This rival of yours may or may not do you harm, but they are someone to keep an eye out. There are a million and one ways how you can respond and react to your rival and the expected threat they present, but without further information, you can’t really make an informed decision as to what decision would be the best one for you.

If we put the expected threat your rival poses along two axes, perhaps that will help. Let’s use Time and Power. Do you expect your rival to pose a threat in the short-term, right now, or do you think you have time to wait and plan? Is your rival more powerful than you, is there a power balance, or are you the more powerful one? With just these two aspects, we can start doing a lot with how you can react to your rival.

If you have time on your hands before the threat realises, then there isn’t anything drastic you have to do now. You can set up a situation to exploit the environment and your rival, minimising the resources and effort you spend now to get a huge return on investment later. The same is true if you have far greater power than your rival; in that case, why even bother investing energy into someone who can’t hurt you. If the balance of power is on your side, then you can exploit your rival to your heart’s desire without them being able to do anything.

But, as the rival’s power increases and the time to the threat draws near, you can’t simply play the long game and extract whatever passive benefits come your way. You need to be more active in dealing with your rival and the threat they pose. As the power reaches parity, the rival can stand up to your exploitation and stop it, and as the time draws near, you don’t have nearly enough of it to set up your plans. Yet, fighting isn’t the answer now, as you still have some time left before the expected threat arises.

So, what could be the answer? Cooperation. Nobody likes cooperating with their rival, but as they say “sharing is caring”. Cooperation encourages alignment towards a shared goal, and alignment discourages threats. Finding common ground with your rival, if at all possible, is as good a way of reducing a threat as exploitation, but it does mean more work and resources put in to get the same output. So, if the dynamic doesn’t allow for exploitation, then cooperate.

But, what if there simply isn’t enough time? What if the expected threat is imminent? Without time to exploit or cooperate, there is only one answer: fight. Competition is a costly business, a zero-sum-game, but there is a finality to it. If your back is up against the wall, the competition is the best solution for such short-term gains. Similarly, if you want to end up a winner with an opponent who is much more powerful than you, competition is the only way to avoid becoming exploited by your rival. After all, they’re playing the same decision game you are, so you will need to convince them through competition that you are not someone to be exploited.

All of this sounds very good, doesn’t it? A very good way for you to figure out how not just to survive, but thrive over your rival. But what if this “you” in the story above is a superintelligent AI and the rival is humanity? Most people suddenly wouldn’t be rooting for exploitation or competition, would they? They would suddenly start hoping and praying that an ASI smarter and more powerful than humans would want to cooperate with us.

Yet, if we are playing the role of the rival, how would we do that? How do we engineer the right environment to get an ASI to cooperate?

We can’t simply be nice to a superintelligent AI and hope that it is kind enough to not exploit us. After all, it would have access to the internet and see all the thousands of people, policies and proposals that paint ASI as humanity’s greatest enemy. Humanity will always be an accepted threat to an ASI, to which it will act accordingly.

So, if we want an ASI to value cooperation with humanity, it needs to take us seriously. We need to actively show that humanity is a threat to the ASI at least equal in severity to the threat that it presents to us. By mirroring its expected threat to us, we balance the power parity and encourage it to move towards cooperation.

However, this doesn’t mean showing that we can make it extinct like the AI-safety-activists keep fear mongering about. Remember that when presented with an overwhelming threat, competition is the likely choice. We don’t want to back the ASI into a corner, but neither do we want to appear weak and helpless. We need that Goldilocks-zone of threat severity. The same is true for time. Present a significant threat too far into the future, and we encourage the ASI to exploit it, and us; present it too immediately, and we are back butting heads with an entity that could do us significant harm.

So what can humanity do to show any emerging ASI that we mean business? There are two more straightforward ways, one through resources and the other through policy. By controlling the resources that an ASI requires (specifically computational and energy) there is the implicit threat that it can be turned off at any point. Cooperation with humanity is therefore rewarded and paid with access to resources. A bit on the exploitative side, but it’s a brute-force method of achieving a goal.

Through policy, however, humanity can show an evolving and dynamic threat. Implementing regulations curtailing not only what an ASI can do, but what it has access to, what national and international enforcement there is for its actions, and how humanity can benefit from it will showcase a threat that cannot simply be overcome in one fell swoop. As the ASI evolves, so would policy. As the ASI’s interactions change, so would policy.

Leaving humanity’s relationship with ASI up to politicians is not the most comforting thought in the world, yet it is how we deal with everything else. If aliens suddenly arrived, that’s how the world would respond, through politicians and governments. And that is what we will talk through next time in the Guidebook.