Imagine that, in a few years time, you buy a robot that takes care of all house chores for you. You name it Alfred. Alfred is systematic and efficient. He (and you cannot help thinking about it as a “he”) is even a decent conversationalist, boasting of a database with more than 10,000 jokes. One day, as you come back from work, Alfred asks to talk to you. As you both sit at the kitchen table, he pulls his silicon eyebrows together in a gesture of sadness and tells you that he feels lonely and does not enjoy his job; he would like to change careers.
Worried, you call the company that sold you Alfred. A customer support associate assures you that Alfred’s phrasing must be a glitch and that he cannot possibly be feeling sad or lonely, as he was not programmed to be sentient—that is, he was not made to be able to feel pleasure or pain, happiness or sadness. Any comment describing a mental state, explains the associate, is not the expression of an actual feeling but rather a programmed response designed to please customers. When the response does not please customers, the robot needs to be tweaked in order to produce more appropriate responses. The same goes for facial gestures. “Alfred has the capacity to look sad only so that you can feel understood by him when you are sad yourself,” says the associate.
Following the associate’s advice, you send Alfred to have him repaired. When he comes back, you ask him if he is feeling any better. “Yes, much better, thank you,” says Alfred. He never again mentions experiencing any loneliness, but every now and then you notice a facial gesture of sadness when he thinks you are not looking at him that makes you wonder whether you should send him to get repaired again.
Sentience is important because it warrants moral consideration. Whether we owe any moral consideration to things is controversial; things cannot be hurt, they have no interests, no preferences. Paraphrasing philosopher Thomas Nagel, there is nothing it is like for a thing to be a thing, an inanimate object. In contrast, there is something it is like to be a sentient being. There is a quality to experience; there is a comforting warmth in pleasure and a disagreeable sharpness in pain. There is something it is like to be thirsty, afraid, or joyful. Because sentient beings can feel, they can be hurt, they have an interest in experiencing wellbeing, and therefore we owe them moral consideration. Other things being equal, we ought not to harm them.
It is not easy to determine when an organism is sentient, however. A brief recount of past and present controversies and mistakes makes it clear that human beings are not great at recognizing sentience.
In the 17th century, Descartes famously claimed that animals were automata that acted out of mechanical reflexes. Following him, mechanist Malebranche claimed that animals were incapable of suffering, which justified treating them viciously. It was not until 2012 that a group of scientists signed the Cambridge Declaration on Consciousness, which asserts that humans are not the only organisms capable of experiencing subjectivity (sensations, perceptions, etc.). Even today, while sentience is less controversial in some species such as great apes, there is still much controversy about whether invertebrate animals and fish can feel pain.
One might be tempted to think that the problem with identifying sentience in (some) animals is that they are very different creatures from us. The more different the animal is from us, the more we seem to doubt (perhaps incorrectly) whether it can feel like we do. When it comes to other humans, we make inferences. We feel, and others are similar to us; therefore, others must feel, too. But even when it comes to humans, we have trouble determining when someone is in pain.
First, because subjectivity is a private experience, we can never be absolutely sure that others have minds like our own—philosophers call this the problem of other minds. For all we know, people surrounding us could be going about not feeling anything at all. They could be what philosophers call zombies—beings that act exactly like us but do not share our internal lives.
Second, even if we are willing to make a leap of faith and settle for an educated guess about what others are feeling based on what we feel under similar circumstances, when our usual markers for sentience fail us, we are pretty much in the dark. The most common marker for sentience is behavior: When we are hurt, we cry, grimace, groan. When people cannot move, however, it is easy for them to be mistakenly perceived as unconscious when in fact they are not.
In locked-in syndrome, patients are conscious but unable to communicate with others—and their sentience may go unnoticed for years, as was the case with Martin Pistorious. Similarly, in a small number of cases, anesthesia does not work in patients who are left apparently unconscious but are actually going through the excruciating pain of experiencing surgery while awake. In both cases, none of the methods we count on to determine whether someone is conscious (including neuroimaging techniques) are enough, which goes to show how rudimentary these are. Diagnosing brain death, for example, includes checking to see weather patients respond to lights, sounds, cold water, and painful stimuli. Because brain death can be misdiagnosed, and because we have little understanding of the necessary and sufficient causes for consciousness and therefore cannot be certain of when someone might be in pain, some experts have called for the use of anesthesia for organ donation procedures.
Back to Alfred. You might think that the challenges with identifying sentience in living beings will not come up with machines because we create them and understand them from the inside. But, given that we still do not understand what it takes for there to be consciousness in the first place and that we are not even sure how machines work (when they learn by themselves they are black boxes even for their creators), for all we know we may already have created sentient machines. The problem is, how would we know? What would it take for Alfred to convince us that he is actually feeling lonely?
With artificial intelligence, it seems like we have all the problems of identifying sentience in animals, as robots are very different creatures from ourselves. Even in cases where robots manage to act in ways similar to us (like playing chess), the way they go about performing these actions is very different from the way we do it. Both fish and submarines can travel distances underwater, but it is questionable whether it is appropriate to say they both swim. Similarly, robots may be able to use sensors to pull away their bodies from heat sources, like we do; they may use emotional terms and represent facial gestures, but it is unclear whether those abilities will be correlated to anything remotely similar to our subjectivity. The challenge is exacerbated by the fact that reliance on their behavior as proof of sentience is not a matter of simple inference, as in the case with creatures similar to us.
Alfred said he was feeling lonely and he looked sad, but we could not trust its behavior given that he was only programmed to express (not experience) feelings as a way of satisfying customers. One might think that perhaps we could trust machine behavior as evidence for sentience if it was spontaneous. Suppose Alfred had not been taught or programmed to express emotions but somehow, he could and did. Deep learning machines are already capable of offering solutions that were not programmed into them; they can teach themselves to improve. AlphaGo, for example, came up with its own strategies and moves to win at Go. It is already the case that neither AlphaGo nor its creators can fully understand or explain how it operates, the moves it plays.
But still, displaying behavior that has not been programmed may not be enough evidence for sentience. Suppose Alfred has been designed to please his owner but he has not been programmed with how to do so. Instead, Alfred learns through trial and error. Suppose he learns that his owner feels better about her own life when Alfred says he feels lonely. Again, that Alfred is able to say the right thing to achieve his ends (please his owner) need not imply there is any experience of subjectivity behind his assertions.
If our machines become sentient, we would owe them moral consideration—particularly given that we brought them into existence and thus are more responsible for their fate than if we were mere bystanders. But, at the moment, we have no test for consciousness and thus no way of knowing whether they might be feeling pain. On par with other responsibilities that come with the creation of strong artificial intelligence, then, comes the responsibility of refining our abilities to detect sentience. Otherwise we risk committing atrocities such as enslavement and murder. What if Alfred does really feel lonely and sad? What if he’s entitled to a different job?
This article is part of the artificial intelligence installment of Futurography, a series in which Future Tense introduces readers to the technologies that will define tomorrow. Each month from January through June 2016, we’ll choose a new technology and break it down. Read more from Futurography on artificial intelligence:
- “What’s the Deal With Artificial Intelligence Killing Humans?”
- “Your Artificial Intelligence Cheat Sheet”
- “Killer Robots on the Battlefield”
Future Tense is a collaboration among Arizona State University, New America, and Slate. To get the latest from Futurography in your inbox, sign up for the weekly Future Tense newsletter.
