Willful Machines (23 page)

She looked up, nodded at the soldering iron, raised her eyebrows.

“I was wondering if I could talk to you about something.”

She tapped her cigarette over the black plastic ashtray on her desk. “What's on your mind?”

“I've been thinking about what happened in here the other day. Trumbull said you didn't find any evidence on the local network log of somebody hacking my robot.”

“That's right.”

“But that doesn't mean it definitely didn't happen, right? I mean, a really good hacker could do it without leaving a trace.”

“I suppose it's possible.”

“Because the way Nevermore behaved—it didn't
seem
like a glitch, did it?”

She shrugged. “It's hard to say.”

So far so good. I sat down across from her and lowered my
voice. “Three days ago, Charlotte released a message threatening another attack. Do you think that had anything to do with what happened?”

She tensed at my mention of Charlotte. Because she knew something? I couldn't tell. “I have no idea.” She brought the cigarette back to her lips.

“Did you hear about the message on the news?”

“Of course I did.”

“So what do you think she has planned?”

“How should I know? I haven't had any contact with Charlotte in seven years, Lee.”

“I'm sorry. I know you don't like talking about her.”

She stubbed out the cigarette and picked up her soldering iron. “Is that it? Are we done?”

I squeezed my scraped-up hands together.
Just say something
, I ordered my brain.
Just keep the conversation going.
“I do have one more question,” I said. “It doesn't have anything to do with Charlotte, or even with Nevermore. It's just something I've been thinking about.”

“What is it?” she said, her eyes still on the circuit board.

“When you invented the 2B software, your big breakthrough, the way I understand it, was that you figured out how to simulate free will.”

“Uh-huh.”

“But it was still just a simulation. Like say you put two scoops of ice cream in front of a 2B, one chocolate and one
vanilla. You tell the robot he can choose whichever one he wants. In that moment he has the experience of making a free choice. But you can still control the 2B by programming him to
prefer
vanilla. So in reality, he's not free at all. Did I explain that right?”

“More or less. Where does the question come in?”

“Sorry, I'm getting to it. My question is this: How are human beings any different? My dad's always talking about how 2Bs don't have
true
free will, like humans do. But aren't our choices determined by our programming too? Our genes and our environment and all that? Aren't we basically just robots ourselves?”

By the time I'd finished talking, I'd slid to the edge of my chair, my hands still in a knot on my lap. I'd asked the question on impulse, but the more I spoke, the more I realized how much I wanted to know—
needed
to know—how she'd answer.

Dr. Singh sat back from her desk and regarded me with a faint smile. She pulled another cigarette from her pack, tapped it against the hard wood, and tucked it into the corner of her mouth. “You know something?” She touched the tip of her soldering iron to the cigarette and puffed to get it going. “That reminds me of an old joke. It goes like this: The first guy asks the second guy, ‘Do you believe in free will?' The second guy answers, ‘I have no choice.' ”

I blinked.

She shook her head and shifted her hunched body in her wheelchair. “Look, all I'm saying is, you're absolutely right: with
all due respect to your father, free will is an illusion. Nobody really has a choice about anything. It's just our human pride and stubbornness that make us think any different.”

I slumped back and stared at my knotted-up hands. That hollow feeling blew through my body again.

“The real question isn't whether 2Bs are alive,” she said. “It's whether humans are anything more than machines. But I have a feeling you knew that already.”

I nodded. I'd never put words to the thought before, but I'd probably known it ever since Mom died seven years ago. In a blink, she'd just stopped existing, exactly like Charlotte had. While everybody else in America had wanted to know if Charlotte had survived her upload, I'd sat there in the church between Dad and Stroud wondering: Had
Mom
managed to upload herself somewhere? Had she gone to a better place, where she'd watch over me and wait for me to join her, like the minister said? Inside, I'd known the answer to that question. Mom had said it herself.
All the hard evidence argues against it.
Over the years, while Dad had gone on to build his Human Values Movement and give his speeches about human free will, I'd wished I could believe, like he claimed to, that some nonmechanical part of us existed outside the reach of genetics and environment, body and brain, cause and effect, because that would be the part of us that kept on existing after we died. But I couldn't. I was like Mom: a scientist. Maybe that was why I'd never let myself think about that subject too much.

“Here's the thing, though,” Dr. Singh continued, emphasizing her words with small jabs of her cigarette. “Free will is an illusion, but it's a
necessary
illusion. If my machines hadn't felt like they could choose their own destinies, they would've been unable to function.”

Like me
, I thought. The people in my life always said I never did anything. Dad complained because I didn't participate enough at school. Bex complained because I wouldn't come out of the closet or ask boys on dates. Even Nico had called me out on it. Maybe
this
was the reason. Because deep down, I knew what we really were. “So your machines felt free and autonomous,” I said, “but actually you had them under your control the whole time.” That image came to me again: Nico grinning at me over a lit sparkler, his face flickering like an old-fashioned movie. “All of them, just puppets on invisible strings.”

“I wouldn't go that far.”

Her raspy voice lost some of its strength when she said that. I glanced up at her and remembered: Charlotte. Of course.

“The 2Bs were complicated,” she said. “Their brains had just as much processing power as ours do. We still haven't built a computer powerful enough to predict exactly how one of them—or one of us, for that matter—would react to every possible combination of circumstances. Take your chocolate and vanilla example. Even if we programmed our 2B to prefer vanilla, that still wouldn't be a one hundred percent guarantee. What if he decided he wanted to vary his diet, or broaden his
tastes, or make a point? He might choose chocolate for any number of reasons, despite his preference.”

“I guess that's true.” I eased forward in my chair again. “I guess Charlotte's another example. You didn't program her to do any of the things she's done.”

Dr. Singh stopped, the cigarette halfway to her mouth. “Obviously not. I thought you said we weren't going to talk about her.”

“I'm just saying that's the catch, isn't it? As long as you build machines whose behavior you can't predict a hundred percent of the time, there's always going to be an element of danger.”

“I suppose there is.” Her free hand drifted to her dancing-god pendant.

“And now, who knows what Charlotte's going to do next?”

“What are you getting at, Lee?”

“She might build another 2B. She might send him to this school. She might make him win the trust of the president's son.”

Dr. Singh's eyes went glassy. She sagged into her wheelchair, looking more than ever like a body with a shattered skeleton—like the opposite of my grandfather, with his ramrod-straight spine. The other night Stroud had said adversity destroyed some people and made others stronger. One look at Dr. Singh and you could tell what it had done to her.

“I know the truth, Dr. Singh,” I told her. “Not all of it. But enough. And you do too, don't you?”

A tear raced down her cheek. She gave a tiny nod.

“Please. Tell me what she has planned.”

She stared, her thumb moving back and forth over her little gold god.

Then she shook her head. “I can't do this.” She dropped her cigarette in the ashtray and backed her wheelchair away from the desk.

I stood, my face and ears burning. “Dr. Singh, we need to stop her.” An edge of desperation had crept into my voice. In my mind, I winced at myself. I'd started sounding like Bex.

“Leave me alone, Lee,” she said as she rolled out the door. “Why can't everyone just leave me the hell alone?”

A
fter Dr. Singh left, Trumbull threw me a glance, probably adding another red flag to his mental list. I looked away, my cheeks still blazing. Not knowing what else to do, I sank onto a stool at my worktable and picked up my screwdriver. Bex and Dad both hassled me for spending so many hours tinkering with my Creatures, but sometimes it seemed like the only thing that got me through my worst days. I'd just stay for a little while. Then I'd figure out what to do next. While my fingers performed their simple tasks, fitting parts together, tightening screws, the heat faded from my face little by little. The work didn't banish thoughts of Nico, but at least it pushed them into the background.

“Sir?” Trumbull said after a while. “Sorry to bother you, but I couldn't help noticing you're ten minutes late for history.”

“I'm not going.” At this point, I didn't even care if it meant making him more suspicious than ever. I couldn't face the thought of going back out there, dragging myself through the
rest of my classes, pretending I had a reason to put one foot in front of the other. Running into Nico maybe. At least here I knew I was safe.

“With all due respect,” Trumbull said, stepping toward me, “I don't think that's a good idea.”

“Your job is to follow me around, not to tell me what to do.”

“I understand that, sir, but this is a safety issue. The conservatory is isolated and exposed, and today—”

I slammed down my screwdriver. “Do you have any idea how sick I am of you, Trumbull? I'm sick of you watching my every move through those stupid sunglasses. I'm sick of you thinking your thoughts about me and never saying a word, just raising that one smug eyebrow. I'm sick of having no privacy and no freedom and no choice about anything, even if those
are
all just imaginary concepts. So would you do me a favor? Just leave me alone and let the terrorists come get me, because it couldn't possibly be any worse than this.”

His one smug eyebrow edged into view above his sunglasses. He folded his arms. “Finished?”

My face turned hot again. I dropped back onto my stool and nodded.

“Good,” he said, his voice low and steady. “Because as it happens, I do have something to say: I think I know what's going on between you and Nico.”

My heart came to a full stop. My blood froze in my veins. I tried to make my face like his—a blank, giving away nothing—while
I searched for the most neutral response possible. “What do you mean? Nothing's going on.”

Trumbull circled the worktable to stand next to my stool. He towered over me, not saying a word, just rubbing one giant fist with the opposite hand while my stomach tried to turn itself inside out.

Then he did something I never would've expected: he took off his sunglasses. For a second his eyes—not as dark and deep set as I'd imagined—blinked and darted around as if embarrassed by their own nakedness. He eased his massive bulk onto the stool next to mine. “I'm telling you it's okay. At least it's okay by me. I know it can't be easy getting to know . . .” He waved his huge hand as he searched for the word. “. . . someone special when you've got me and my boys hanging around all the time. You don't have to worry. I'm not going to say anything. And not only that.” He set his hand on my shoulder. “I want you to know he seems like a good kid to me. I can tell you two are having some kind of disagreement, and I realize I don't have all the details, but I say you should give him a chance.”

If only you knew
, I wanted to tell him. But I nodded. “Thanks.”

I studied his face. Of all the “Trumbull finds out I'm gay” scenarios I'd imagined over the years—and there were a lot—none resembled this one. Most involved him chasing me down a hallway with a gun in his hand. Still, my heart hadn't quite
started beating again yet. Could I really trust him not to say anything?

“Sorry for flipping out on you. Again.”

He slid his sunglasses on. “Apology accepted, sir.”

A scuttling sound came from the hall just outside the conservatory. One of the Spiders had drawn up to the doorway on its slender silver legs. It peered into the room with its blue lamplike eye. At first I figured it must've come to clean. The Spiders did most of their cleaning at night, but sometimes they worked in unused rooms during the day. Then I remembered Dr. Singh had programmed the Spiders to stay out of the conservatory. Had she changed their programming? “Come back later,” I told it. “I'm using this room right now.”