The ethical puzzle of sentient AI

Aartificial intelligence has progressed so rapidly that even some of the scientists responsible for many key developments are confused by the pace of change. Earlier this year, more than 300 professionals working in AI and other relevant public figures published a blunt warning on the danger posed by technology, comparing the risk to that of pandemics or nuclear war.

Just below the surface of these concerns lurks the question of machine consciousness. Even though there is “nobody home” inside today’s AIs, some researchers wonder if they might ever show a glimmer of consciousness — or more. If that happens, it will raise a host of moral and ethical concerns, says Jonathan Birchprofessor of philosophy at the London School of Economics and Political Science.

As AI technology advances, ethical questions raised by human-AI interactions have taken on new urgency. “We don’t know whether to include them in our moral circle or exclude them,” Birch said. “We don’t know what the consequences will be. And I take that seriously as a real risk that we should start talking about. Not really because I think ChatGPT is in this category, but because I don’t know what will happen in the next 10 or 20 years.

In the meantime, he says, we would do well to study other non-human minds, such as those of animals. Birch leads the university Foundations of Animal Sentence Project, a European Union-funded effort that “aims to try to make progress on the big issues of animal sentience,” as Birch put it. “How can we develop better methods to scientifically study the conscious experiences of animals? And how can we put the emerging science of animal sentience to work, to design better policies, laws, and ways to care for animals? »

Our interview was conducted over Zoom and email, and has been edited for length and clarity.

Not dark: There has been an ongoing debate about whether AI can be sentient or sentient. And there seems to be a parallel question of whether AI can seem To be sensitive. Why is this distinction so important?

Jonathan Birch: I think it’s a huge problem, and something that should scare us, actually. Even now, AI systems are quite capable of convincing their users of their sensitivity. We saw this last year with the case of Blake Lemoine, the Google engineer who became convinced that the system he was working on was sentient – and that’s only when the output is purely textual and the user is a highly trained AI expert.

So imagine a situation where the AI ​​is able to control a human face and a human voice and the user is inexperienced. I think the AI ​​is already in a position where it can pretty easily convince a lot of people that it’s a sentient being. And that’s a big deal, because I think we’re going to start to see people campaigning for the welfare of AI, AI Rightsand things like that.

And we won’t know what to do about it. Because what we’d like is a very strong reversal argument that proves that the AI ​​systems they’re talking about are not aware. And we don’t have that. Our theoretical understanding of consciousness is not mature enough to allow us to confidently declare its absence.

UD: A robot or an AI system could be programmed to say something like, “Stop it, you’re hurting me.” But a simple statement like that isn’t enough to serve as a litmus test for susceptibility, is it?

JB: You can have very simple systems [like those] developed at Imperial College London to assist doctors in their training who imitate expressions of human pain. And there is absolutely no reason to think that these systems are sensitive. They don’t really feel pain; all they do is map inputs to outputs in a very simple way. But the expressions of pain they produce are quite realistic.

I think we’re in a somewhat similar position with chatbots like ChatGPT – that they’re trained on over a trillion words of training data to mimic the response patterns of a human to respond in a way a human would respond.

So, of course, if you give it a prompt that a human would respond to by making an expression of pain, it will be able to skillfully mimic that response.

But I think when we know that’s the situation – when we know we’re dealing with skillful mimicry – there’s no good reason to think there’s any real pain behind it.

UD: This entity that medical students train on, I guess it’s something like a robot?

JB: It’s true, yes. So they have a dummy-like thing, with a human face, and the doctor is able to press on the arm and get an expression that mimics the expressions humans would give for different degrees of pressure. This is to help doctors learn how to apply the techniques to patients appropriately without causing too much pain.

And we’re very easily fooled when something has a human face and makes expressions like a human would, even if there’s no real intelligence behind it.

So if you imagine being paired with the type of AI that we see in ChatGPT, you have a kind of mimicry that is really very convincing, and that will convince a lot of people.

UD: Sensitivity seems to be something we know on the inside, so to speak. We understand our own sentience – but how would you test the sentience of others, be it an AI or any other entity beyond yourself?

JB: I think we’re in a very strong position with other humans, who can talk to us, because we have an incredibly rich body of evidence there. And the best explanation for this is that other humans have conscious experiences, just like us. And so we can use this kind of inference that philosophers sometimes call “best explanation inference.”

I think we can approach the subject of other animals in exactly the same way – that other animals don’t talk to us, but they display behaviors that are very naturally explained by attributing states like pain. For example, if you see a dog licking its wounds after an injury, treating this area, learning to avoid places where it risks injury, you will naturally explain this behavior by postulating a state of pain.

And I think when we’re dealing with other animals that have nervous systems quite similar to ours, and have evolved like us, I think that kind of inference is quite reasonable.

UD: What about an AI system?

JB: In the case of AI, we have a huge problem. We first have the problem that substrate is different. We don’t really know if conscious experience is substrate-sensitive — does it have to have a biological substrate, ie, a nervous system, a brain? Or is it something that can be done in an entirely different material – a silicon-based substrate?

But there’s also the problem I’ve called the “game problem”: when the system has access to billions of words of training data and has been trained to mimic human behavior, types many behaviors it produces could be explained by actually having the conscious experience. Or, alternatively, they could simply be explained by the fact that the goal is to behave as a human would react in this situation.

So I really think we’re in trouble in the case of AI, because we’re unlikely to end up in a position where that’s clearly the best explanation for what we’re seeing – that AI is aware. There will always be plausible alternative explanations. And it’s a very difficult impasse to get out of.

UD: What do you think might be our best bet for distinguishing between something that is actually conscious and an entity that just has the appearance sensitivity?

JB: I think the first step is to recognize that this is a very deep and difficult problem. The second step is to try to learn as much as possible from the case of other animals. I think when we study animals that are quite close to us, in terms of evolution, like dogs and other mammals, we still don’t know if conscious experience might depend on very specific brain mechanisms that are distinctive of the mammalian brain.

To overcome this, we need to look at as wide a range of animals as possible. And we have to think particularly of invertebrates, like octopuses and insects, where this is potentially another example of independently evolved conscious experience. Just as an octopus’s eye evolved completely separately from our own eyes – it has this fascinating mix of similarities and differences – I think its conscious experiences will also be like this: independently evolved, similar in some ways, very, very different in other ways.

And by studying the experiences of invertebrates like octopuses, we can begin to understand what are the really deep characteristics that a brain needs to have to support conscious experiences, things that go deeper than just having these specific brain structures. that are there in mammals. What types of calculations are needed? What types of treatment?

Then – and I see this as a long-term strategy – we could go back to the case of AI and say, well, does it have these special types of computation that we find in sentient animals like mammals and octopuses?

UD: Do you believe we will ever create sentient AI?

JB: I’m about 50:50 on that. It’s possible that sensitivity depends on special characteristics of a biological brain, and it’s unclear how to test if this is the case. So I think there will always be substantial uncertainty in AI. I am more confident about this: if consciousness can in principle be done in computer software, then AI researchers will find a way to do it.