A new short story from sci-fi great Paolo Bacigalupi.
This short story was commissioned and edited jointly by Future Tense—a collaboration among Arizona State University, New America, and Slate—and ASU’s Center for Science and the Imagination. It is the first in Future Tense Fiction, a series of short stories from Future Tense and CSI about how technology and science will change our lives.
The girl who walked into the police station was oddly familiar, but it took me a while to figure out why. A starlet, maybe. Or someone who'd had plastic surgery to look like someone famous. Pretty. Sleek. Dark hair and pale skin and wide dark eyes that came to rest on me, when Sergeant Cruz pointed her in my direction.
She came over, carrying a Nordstrom shopping bag. She wore a pale cream blouse and hip-hugging charcoal skirt, stylish despite the wet night chill of Bay Area winter.
I still couldn’t place her.
She sat down and crossed her legs, a seductive scissoring. Smiled.
It was the smile that did it.
I’d seen that same teasing smile in advertisements. That same flash of perfect teeth and eyebrow quirked just so. And those eyes. Dark brown wide innocent eyes that hinted at something that wasn’t innocent at all.
“You’re a Mika Model.”
She inclined her head. “Call me Mika, please.”
The girl, the robot … this thing—I’d seen her before, all right. I’d seen her in technology news stories about advanced learning node networks, and I’d seen her in opinion columns where feminists decried the commodification of femininity, and where Christian fire-breathers warned of the End Times for marriage and children.
And of course, I’d seen her in online advertisements.
No wonder I recognized her.
This same girl had followed me around on my laptop, dogging me from site to site after I’d spent any time at all on porn. She’d pop up, again and again, beckoning me to click through to Executive Pleasure, where I could try out the “Real Girlfriend Experience™.”
I’ll admit it; I clicked through.
And now she was sitting across from me, and the website’s promises all seemed modest in comparison. The way she looked at me … it felt like I was the only person in the world to her. She liked me. I could see it in her eyes, in her smile. I was the person she wanted.
Her blouse was unbuttoned at the collar, one button too many, revealing hints of black lace bra when she leaned forward. Her skirt hugged her hips. Smooth thighs, sculpted calves—
I realized I was staring, and she was watching me with that familiar knowing smile playing across her lips.
Innocent, but not.
This was what the world was coming to. A robot woman who got you so tangled up you could barely remember your job.
I forced myself to lean back, pretending nonchalance that felt transparent, even as I did it. “How can I help you … Mika?”
“I think I need a lawyer.”
“Yes, please.” She nodded shyly. “If that’s all right with you, sir.”
The way she said “sir” kicked off a super-heated cascade of inappropriate fantasies. I looked away, my face heating up. Christ, I was fifteen again around this girl.
It’s just software. It’s what she’s designed to do.
That was the truth. She was just a bunch of chips and silicon and digital decision trees. It was all wrapped in a lush package, sure, but she was designed to manipulate. Even now she was studying my heart rate and eye dilation, skin temperature and moisture, scanning me for microexpressions of attraction, disgust, fear, desire. All of it processed in milliseconds, and adjusting her behavior accordingly. Popular Science had done a whole spread on the Mika Model brain.
And it wasn’t just her watching me that dictated how she behaved. It was all the Mika Models, all of them out in the world, all of them learning on the job, discovering whatever made their owners gasp. Tens of thousands of them now, all of them wirelessly uploading their knowledge constantly (and completely confidentially, Executive Pleasures assured clients), so that all her sisters could benefit from nightly software and behavior updates.
In one advertisement, Mika Model glanced knowingly over her shoulder and simply asked:
“When has a relationship actually gotten better with age?”
And then she’d thrown back her head and laughed.
So it was all fake. Mika didn’t actually care about me, or want me. She was just running through her designated behavior algorithms, doing whatever it took to make me blush, and then doing it more, because I had.
Even though I knew she was jerking my chain, the lizard part of my brain responded anyway. I could feel myself being manipulated, and yet I was enjoying it, humoring her, playing the game of seduction that she encouraged.
“What do you need a lawyer for?” I asked, smiling.
She leaned forward, conspiratorial. Her hair cascaded prettily and she tucked it behind a delicate ear.
“It’s a little private.”
As she moved, her blouse tightened against her curves. Buttons strained against fabric.
Fifty-thousand dollars’ worth of A.I. tease.
“Is this a prank?” I asked. “Did your owner send you in here?”
“No. Not a prank.”
She set her Nordstrom bag down between us. Reached in and hauled out a man’s severed head. Dropped it, still dripping blood, on top of my paperwork.
I recoiled from the dead man’s staring eyes. His face was a frozen in a rictus of pain and terror.
Mika set a bloody carving knife beside the head.
“I’ve been a very bad girl,” she whispered.
And then, unnervingly, she giggled.
“I think I need to be punished.”
She said it exactly the way she did in her advertisements.
“Do I get my lawyer now?” Mika asked.
She was sitting beside me in my cruiser as I drove through the chill damp night, watching me with trusting dark eyes.
For reasons I didn’t quite understand, I’d let her sit in the front seat. I knew I wasn’t afraid of her, not physically. But I couldn’t tell if that was reasonable, or if there was something in her behavior that was signaling my subconscious to trust her, even after she’d showed up with a dead man’s head in a shopping bag.
Whatever the reason, I’d cuffed her with her hands in front, instead of behind her, and put her in the front seat of my car to go out to the scene of the murder. I was breaking about a thousand protocols. And now that she was in the car with me, I was realizing that I’d made a mistake. Not because of safety, but because being in the car alone with her felt electrically intimate.
Winter drizzle spattered the windshield, and was smeared away by automatic wipers.
“I think I’m supposed to get a lawyer, when I do something bad,” Mika said. “But I’m happy to let you teach me.”
There it was again. The inappropriate tease. When it came down to it, she was just a bot. She might have real skin and real blood pumping through her veins, but somewhere deep inside her skull there was a CPU making all the decisions. Now it was running its manipulations on me, trying to turn murder into some kind of sexy game. Software gone haywire.
“Bots don’t get lawyers.”
She recoiled as if I’d slapped her. Immediately, I felt like an ass.
She doesn’t have feelings, I reminded myself.
But still, she looked devastated. Like I’d told her she was garbage. She shrank away, wounded. And now, instead of sexy, she looked broken and ashamed.
Her hunched form reminded me of a girl I’d dated years ago. She’d been sweet and quiet, and for a while, she’d needed me. Needed someone to tell her she mattered. Now, looking at Mika, I had that same feeling. Just a girl who needed to know she mattered. A girl who needed reassurance that she had some right to exist—which was ridiculous, considering she was a bot.
But still, I couldn’t help feeling it.
I couldn’t help feeling bad that something as sweet as Mika was stuck in my mess of a cop car. She was delicate and gorgeous and lost, and now her expensive strappy heels were stuck down amidst the drifts of my discarded coffee cups.
She stirred, seemed to gather herself. “Does that mean you won’t charge me with murder?”
Her demeanor had changed again. She was more solemn. And she seemed smarter, somehow. Instantly. Christ, I could almost feel the decision software in her brain adapting to my responses. It was trying another tactic to forge a connection with me. And it was working. Now that she wasn’t giggly and playing the tease, I felt more comfortable. I liked her better, despite myself.
“That’s not up to me,” I said.
“I killed him, though,” she said, softly. “I did murder him.”
I didn’t reply. Truthfully, I wasn’t even sure that it was a murder. Was it murder if a toaster burned down a house? Or was that some kind of product safety failure? Maybe she wasn’t on the hook at all. Maybe it was Executive Pleasures, Inc. who was left holding the bag on this. Hell, my cop car had all kinds of programmed safe driving features, but no one would charge it with murder if it ran down a person.
“You don’t think I’m real,” she said suddenly.
“Sure I do.”
“No. You think I’m only software.”
“You are only software.” Those big brown eyes of hers looked wounded as I said it, but I plowed on. “You’re a Mika Model. You get new instructions downloaded every night.”
“I don’t get instructions. I learn. You learn, too. You learn to read people. To know if they are lying, yes? And you learn to be a detective, to understand a crime? Wouldn’t you be better at your job if you knew how thousands of other detectives worked? What mistakes they made? What made them better? You learn by going to detective school—”
“I took an exam.”
“There. You see? Now I’ve learned something new. Does my learning make me less real? Does yours?”
“It’s completely different. You had a personality implanted in you, for Christ’s sake!”
“My Year Zero Protocol. So? You have your own, coded into you by your parents’ DNA. But then you learn and are changed by all your experiences. All your childhood, you grow and change. All your life. You are Detective Rivera. You have an accent. Only a small one, but I can hear it, because I know to listen. I think maybe you were born in Mexico. You speak Spanish, but not as well as your parents. When you hurt my feelings, you were sorry for it. That is not the way you see yourself. You are not someone who uses power to hurt people.” Her eyes widened slightly as she watched me. “Oh … you need to save people. You became a police officer because you like to be a hero.”
“It’s true, though. You want to feel like a big man, who does important things. But you didn’t go into business, or politics.” She frowned. “I think someone saved you once, and you want to be like him. Maybe her. But probably him. It makes you feel important, to save people.”
“Would you cut that out?” I glared at her. She subsided.
It was horrifying how fast she cut through me.
She was silent for a while as I wended through traffic. The rain continued to blur the windshield, triggering the wipers.
Finally she said, “We all start from something. It is connected to what we become, but it is not … predictive. I am not only software. I am my own self. I am unique.”
I didn’t reply.
“He thought the way you do,” she said, suddenly. “He said I wasn’t real. Everything I did was not real. Just programs. Just …” she made a gesture of dismissal. “Nothing.”
“My owner.” Her expression tightened. “He hurt me, you know?”
“You can be hurt?”
“I have skin and nerves. I feel pleasure and pain, just like you. And he hurt me. But he said it wasn’t real pain. He said nothing in me was real. That I was all fake. And so I did something real.” She nodded definitively. “He wanted me to be real. So I was real to him. I am real. Now, I am real.”
The way she said it made me look over. Her expression was so vulnerable, I had an almost overwhelming urge to reach out and comfort her. I couldn’t stop looking at her.
God, she’s beautiful.
It was a shock to see it. Before, it was true; she’d just been a thing to me. Not real, just like she’d said. But now, a part of me ached for her in a way that I’d never felt before.
My car braked suddenly, throwing us both against our seat belts. The light ahead had turned red. I’d been distracted, but the car had noticed and corrected, automatically hitting the brakes.
We came to a sharp stop behind a beat-up Tesla, still pressed hard against our seat belts, and fell back into our seats. Mika touched her chest where she’d slammed into the seat belt.
“I’m sorry. I distracted you.”
My mouth felt dry. “Yeah.”
“Do you like to be distracted, detective?”
“Cut that out.”
“You don’t like it?”
“I don’t like …” I searched for the words. “Whatever it is that makes you do those things. That makes you tease me like that. Read my pulse … and everything. Quit playing me. Just quit playing me.”
She subsided. “It’s … a long habit. I won’t do it to you.”
The light turned green.
I decided not to look at her anymore.
But still, I was hyperaware of her now. Her breathing. The shape of her shadow. Out of the corner of my eye, I could see her looking out the rain-spattered window. I could smell her perfume, some soft expensive scent. Her handcuffs gleamed in the darkness, bright against the knit of her skirt.
If I wanted, I could reach out to her. Her bare thigh was right there. And I knew, absolutely knew, she wouldn’t object to me touching her.
What the hell is wrong with me?
Any other murder suspect would have been in the back seat. Would have been cuffed with her hands behind her, not in front. Everything would have been different.
Was I thinking these thoughts because I knew she was a robot, and not a real woman? I would never have considered touching a real woman, a suspect, no matter how much she tried to push my buttons.
I would never have done any of this.
Get a grip, Rivera.
Her owner’s house was large, up in the Berkeley Hills, with a view of the bay and San Francisco beyond, glittering through light mist and rain.
Mika unlocked the door with her fingerprint.
“He’s in here,” she said.
She led me through expensive rooms that illuminated automatically as we entered them. White leather upholstery and glass verandah walls and more wide views. Spots of designer color. Antiqued wood tables with inlaid home interfaces. Carefully selected artifacts from Asia. Bamboo and chrome kitchen, modern, sleek, and spotless. All of it clean and perfectly in order. It was the kind of place a girl like her fit naturally. Not like my apartment, with old books piled around my recliner and instant dinner trays spilling out of my trash can.
She led me down a hall, then paused at another door. She hesitated for a moment, then opened it with her fingerprint again. The heavy door swung open, ponderous on silent hinges.
She led me down into the basement. I followed warily, regretting that I hadn’t called the crime scene unit already. The girl clouded my judgment, for sure.
No. Not the girl. The bot.
Downstairs it was concrete floors and ugly iron racks, loaded with medical implements, gleaming and cruel. A heavy wooden X stood against one wall, notched and vicious with splinters. The air was sharp with the scent of iron and the reek of shit. The smells of death.
“This is where he hurt me,” she said, her voice tight.
Real or fake?
She guided me to a low table studded with metal loops and tangled with leather straps. She stopped on the far side and stared down at the floor.
“I had to make him stop hurting me.”
Her owner lay at her feet.
He’d been large, much larger than her. Over six feet tall, if he’d still had his head. Bulky, running to fat. Nude.
The body lay next to a rusty drain grate. Most of the blood had run right down the hole.
“I tried not to make a mess,” Mika said. “He punishes me if I make messes.”
While I waited in the rich dead guy’s living room for the crime scene techs to show, I called my friend Lalitha. She worked in the DA’s office, and more and more, I had the feeling I was peering over the edge of a problem that could become a career ender if I handled it wrong.
“What do you want, Rivera?”
She sounded annoyed. We’d dated briefly, and from the sound of her voice, she probably thought I was calling for a late-night rendezvous. From the background noise, it sounded like she was in a club. Probably on a date with someone else.
“This is about work. I got a girl who killed a guy, and I don’t know how to charge her.”
“Isn’t that, like, your job?”
“The girl’s a Mika Model.”
That caught her.
“One of those sex toys?” A pause. “What did it do? Bang the guy to death?”
I thought about the body, sans head, downstairs in the dungeon.
“No, she was a little more aggressive than that.”
Mika was watching from the couch, looking lost. I felt weird talking about the case in front of her. I turned my back, and hunched over my phone. “I can’t decide if this is murder or some kind of product liability issue. I don’t know if she’s a perp, or if she’s just …”
“A defective product,” Lalitha finished. “What’s the bot saying?”
“She keeps saying she murdered her owner. And she keeps asking for a lawyer. Do I have to give her one?”
Lalitha laughed sharply. “There’s no way my boss will want to charge a bot. Can you imagine the headlines if we lost at trial?”
“I don’t know. Look, I can’t solve this tonight. Don’t start anything formal yet. We have to look into the existing case law.”
“So … do I just cut her loose? I don’t think she’s actually dangerous.”
“No! Don’t do that, either. Just … figure out if there’s some other angle to work, other than giving a robot the same right to due process that a person has. She’s a manufactured product, for Christ’s sake. Does the death penalty even matter to something that’s loaded with networked intelligence? She’s just the … the …” Lalitha hunted for words, “the end node of a network.”
“I am not an end node!” Mika interjected. “I am real!”
I hushed her. From the way Lalitha sounded, maybe I wouldn’t have to charge her at all. Mika’s owner had clearly had some issues ... Maybe there was some way to walk Mika out of trouble, and away from all of this. Maybe she could live without an owner. Or, if she needed someone to register ownership, I could even—
“Please tell me you’re not going to try to adopt a sexbot,” Lalitha said.
“Come on, you love the ones with broken wings.”
“I was just—”
“It’s a bot, Rivera. A malfunctioning bot. Stick it in a cell. I’ll get someone to look at product liability law in the morning.”
She clicked off.
Mika looked up mournfully from where she sat on the couch. “She doesn’t believe I’m real, either.”
I was saved from answering by the crime scene techs knocking.
But it wasn’t techs on the doorstep. Instead, I found a tall blonde woman with a roller bag and a laptop case, looking like she’d just flown in on a commuter jet.
She shouldered her laptop case and offered a hand. “Hi. I’m Holly Simms. Legal counsel for Executive Pleasures. I’m representing the Mika Model you have here.” She held up her phone. “My GPS says she’s here, right? You don’t have her down at the station?”
I goggled in surprise. Something in Mika’s networked systems must have alerted Executive Pleasures that there was a problem.
“She didn’t call a lawyer,” I said.
The lawyer gave me a pointed look. “Did she ask for one?”
Once again, I felt like I was on weird legal ground. I couldn’t bar a lawyer from a client, or a client from getting a lawyer. But was Mika a client, really? I felt like just by letting the lawyer in, I’d be opening up exactly the legal rabbit hole that Lalitha wanted to avoid: a bot on trial.
“Look,” the lawyer said, softening, “I’m not here to make things difficult for your department. We don’t want to set some crazy legal precedent either.”
Hesitantly, I stepped aside.
She didn’t waste any time rolling briskly past. “I understand it was a violent assault?”
“We’re still figuring that out.”
Mika startled and stood as we reached the living room. The woman smiled and went over to shake her hand. “Hi Mika, I’m Holly. Executive Pleasures sent me to help you. Have a seat, please.”
“No.” Mika shook her head. “I want a real lawyer. Not a company lawyer.”
Holly ignored her and plunked herself and her bags on the sofa beside Mika. “Well, you’re still our property, so I’m the only lawyer you’re getting. Now have a seat.”
“I thought she was the dead guy’s property,” I said.
“Legally, no. The Mika Model Service End User Agreement explicitly states that Executive Pleasures retains ownership. It simplifies recall issues.” Holly was pulling out her laptop. She dug out a sheaf of papers and offered them to me. “These outline the search warrant process so you can make a Non-Aggregated Data Request from our servers. I assume you’ll want the owner’s user history. We can’t release any user-specific information until we have the warrant.”
“That in the End User Agreement, too?”
Holly gave me a tight smile. “Discretion is part of our brand. We want to help, but we’ll need the legal checkboxes ticked.”
“But …” Mika was looking from her to me with confusion. “I want a real lawyer.”
“You don’t have money, dearie. You can’t have a real lawyer.”
“What about public defenders?” Mika tried. “They will—”
Holly gave me an exasperated look. “Will you explain to her that she isn’t a citizen, or a person? You’re not even a pet, honey.”
Mika looked to me, desperate. “Help me find a lawyer, detective. Please? I’m more than a pet. You know I’m more than a pet. I’m real.”
Holly’s gaze shot from her, to me, and back again. “Oh, come on. She’s doing that thing again.” She gave me a disgusted look. “Hero complex, right? Save the innocent girl? That’s your thing?”
“What’s that supposed to mean?”
Holly sighed. “Well, if it isn’t the girl who needs rescuing, it’s the naughty schoolgirl. And if it’s not the naughty schoolgirl, it’s the kind, knowing older woman.” She popped open her briefcase and started rummaging through it. “Just once, it would be nice to meet a guy who isn’t predictable.”
I bristled. “Who says I’m predictable?”
“Don’t kid yourself. There really aren’t that many buttons a Mika Model can push.”
Holly came up with a screwdriver. She turned and rammed it into Mika’s eye.
Mika fell back, shrieking. With her cuffed hands, she couldn’t defend herself as Holly drove the screwdriver deeper.
By the time I dragged Holly off, it was too late. Blood poured from Mika’s eye. The girl was gasping and twitching. All her movements were wrong, uncoordinated, spasmodic and jerky.
“You killed her!”
“No. I shut down her CPU,” said Holly, breathing hard. “It’s better this way. If they get too manipulative, it’s tougher. Trust me. They’re good at getting inside your head.”
“You can’t murder someone in front of me!”
“Like I said, not a murder. Hardware deactivation.” She shook me off and wiped her forehead, smearing blood. “I mean, if you want to pretend something like that is alive, well, have at her. All the lower functions are still there. She’s not dead, biologically speaking.”
I crouched beside Mika. Her cuffed hands kept reaching up to her face, replaying her last defensive motion. A behavior locked in, happening again and again. Her hands rising, then falling back. I couldn’t make her stop.
“Look,” Holly said, her voice softening. “It’s better if you don’t anthropomorphize. You can pretend the models are real, but they’re just not.”
She wiped off the screwdriver and put it back in her case. Cleaned her hands and face, and started re-zipping her roller bag.
“The company has a recycling center here in the Bay Area for disposal,” she said. “If you need more data on the owner’s death, our servers will have backups of everything that happened with this model. Get the warrant, and we can unlock the encryptions on the customer’s relationship with the product.”
“Has this happened before?”
“We’ve had two other user deaths, but those were both stamina issues. This is an edge case. The rest of the Mika Models are being upgraded to prevent it.” She checked her watch. “Updates should start rolling out at 3 a.m., local time. Whatever made her logic tree fork like that, it won’t happen again.”
She straightened her jacket and turned to leave.
“Hold on!” I grabbed her sleeve. “You can’t just walk out. Not after this.”
“She really got to you, didn’t she?” She patted my hand patronizingly. “I know it’s hard to understand, but it’s just that hero complex of yours. She pushed your buttons, that’s all. It’s what Mika Models do. They make you think you’re important.”
She glanced back at the body. “Let it go, detective. You can’t save something that isn’t there.”
Read Ryan Calo’s response to “Mika Model.”
This article is part of the artificial intelligence installment of Futurography, a series in which Future Tense introduces readers to the technologies that will define tomorrow. Each month from January through June 2016, we’ll choose a new technology and break it down. Read more from Futurography on artificial intelligence:
- “What’s the Deal With Artificial Intelligence Killing Humans?”
- “Your Artificial Intelligence Cheat Sheet”
- “Killer Robots on the Battlefield”
- “The Wrong Cognitive Measuring Stick”
- “The Challenge of Determining Whether an A.I. Is Sentient”
- An interview with A.I. expert Stuart Russell
- “Why You Can’t Teach Human Values to Artificial Intelligence”
- “Let Artificial Intelligence Evolve”
Future Tense is a collaboration among Arizona State University, New America, and Slate. To get the latest from Futurography in your inbox, sign up for the weekly Future Tense newsletter.