She was still worried about DAMBALLAH and whether DMS was doing weird things with the epidemiological reports. DAMBALLAH was a complicated system. It made decisions about reporting data. She couldn’t easily check its decisions—that was the point. Every two weeks they got a report from the NIH and the CDC about epidemiological trends, and if there was something new that the CDC was looking for, say an outbreak of shigella in preschools in the South, there was an elaborate way they entered additional parameters to DAMBALLAH’s tracking system. The CDC and the NIH also sent them error reports and WRs. WRs were to correct when DMS was reporting something that wasn’t important or was overreporting. The result was the DMS “learned” epidemiology.
This made it difficult to know if DMS was screwing with the numbers. If DMS did report something, like an epidemic of onchocerciasis (parasitical river blindness) in Seattle, that would get caught fast. But if DMS were just, say, overreporting the incidence of TB in Seattle, that might not. Sydney ran an ep report and started working on a program that would check the DAMBALLAH database for raw numbers of cases of illnesses that DMS was tracking for the CDC, to see if she could spot anything that looked weird.
Damien had been cranky and quiet all day. Then at 3:17, the lights at Meridian Health in Macon, Georgia, did the wave. The same thing that had happened the day before happened again, except this time in reverse order, ending with DM Kensington Medical. They found out it was happening again when the power outage rolled through headquarters early in the sequence. Within minutes Tony, their boss, was screaming at people to stop it, but they decided that stopping it would be more complicated than letting it run its course, so they called the last three hospitals and gave them a heads-up.
Damien was set to write code that would catch the beginning of the sequence and stop it from happening. Together, he and Sydney pored over the tangle of spaghetti that was SAMEDI code. The next day, at 3:17 they could at least switch the electrical systems to maintenance mode for the time it took for DMS to run through its sequence. (According to the log, it would have started with DM Kensington again.) Hospitals bitched about slowdowns in the DMS while SAMEDI was not running. It shouldn’t have affected everything else, but DMS was so weirdly interconnected that SAMEDI had evidently been doing something that optimized read/write functions. Which SAMEDI wasn’t supposed to do at all.
“Why 3:17?” Sydney asked. “Why the electrical system?”
Damien shrugged. They were poring over printouts, looking for ways to, in Damien’s words, “build a box around the bug.” Tony was alternating between asking them if they’d found it yet and telling the head of operations that the admin IT team was doing a great job and to get out of their faces and let them work. Tony was a screamer, but as far as he was concerned, the only one allowed to scream at his people was him.
Mostly Sydney noticed that Damien did not seem to be “in the zone.” He had talked a lot about being “in the zone.” About time passing without his even realizing it. Pouing over printouts, he sighed, exasperated. He got up and went to the bathroom a lot. He got coffee a lot. He talked about what they might do, and although his ideas were smart, they more he talked the more she got an idea about how he thought about stuff like this; and for the first time she found herself thinking, maybe with some experience, she could code pretty good, too.
She finished her database checker for DAMBALLAH, the program that tracked disease trends. The results were mostly … complicated. But there was one area she thought was a problem.
“Damien?” she said.
“Yeah?”
“I think DAMBALLAH is messing with the numbers.”
He looked at her. Carefully he said, “How do you know?”
“I don’t,” she said. “Not for certain. But I ran a raw compilation of what was in the Seattle database, and compared it to what DMS is reporting. And DMS is reporting a nosocomial infection rate of seven percent.” Benevola was involved in a big program to reduce nosocomial infections. Nosocomial infections were infections that the patient caught as a result of medical care. Benevola was working with a huge government double-blind study.
“And?” Damien said.
“I can only find evidence of less than a one percent nosocomial infection rate.”
Tony, their boss, stood in his doorway. “What are you saying, Sydney?”
“I … I’m not sure.” Sydney wasn’t ready to talk to Tony yet. Actually, Sydney was pretty much never ready to talk to Tony. But she had wanted to talk to Damien about this, first. “I mean, DAMBALLAH is cranking numbers in ways I don’t understand. It could be that I don’t recognize a lot of stuff that DAMBALLAH does. I mean, that’s the whole point, right?”
Tony came by and leaned over the cube wall. “We might shut it down.”
“Tonight?” Damien asked.
“No, shut it down and reload from a backup from twelve months ago.” Tony always acted as if you were dim if you didn’t get what he was talking about, but he had a tendency to start conversations somewhere in the middle, so everyone was always confused talking to him.
“We’ll lose all our updates,” Sydney said.
“Yeah,” Tony said. “But if it’s unstable, who cares? We’ll look at reloading the system over the weekend. I gotta talk to upstairs first ’cause it will be a huge nightmare.”
Understatement of the year.
When Tony had gone back in his office, Damien said, “Show me.”
She showed him.
Damien nodded. “This is really smart. I mean, not the pro-
gramming.”
Sydney grinned, “A monkey could do the programming.” It was an old joke.
“I wouldn’t have thought to do this,” Damien said.
“It might not mean anything,” Sydney said. “I mean, the whole point is that DAMBALLAH is extrapolating information.
“It means we’re killing DMS,” Damien said.
“You said it wasn’t alive,” she said.
“Semantics,” he said.
She went home and finished
Dead Until Dark
, started
Dark Hunter
, and fed Scott Pilgrim, her cat, and thought about DMS. What would it be like to be alone? Of course, as a human being, she was a social animal. Even the cat was a somewhat social animal. But DMS wasn’t. DMS didn’t even know anyone else existed. DMS lived in a data stream. In science fiction, AIs were always looking for other AIs or trying to be human, like Data on Star Trek Next Gen.
Truth was, she was beginning to get a feeling about DMS. About what DMS might be like. She felt as if she could sort of sense the edges of DMS’s personality, and although she knew it wasn’t true, she knew it was just because Damien had used it as an example, more and more she thought of DMS as a shark. Not in a predatory way. She had an image of a shark in her head, a small shark, a nurse shark. She could see its eye, a black circle in white, overly simple, like a ventriloquist’s dummy. Although the whole point of DMS was that it was not someone else speaking through the code.
The shark in her head swam, purposeful and opaque, its eyes tracking, its mouth open and curved. Sharks don’t have a neocortex. Their brain is simple. They aren’t moral or immoral, ethical or unethical. DMS was like that, because for DMS, nothing else was alive. The world for DMS was data, and DMS swam in the data. She was beginning to feel as if she wanted it to. DMS was creepy.
She dragged herself in again the next day. She swore she would not read late. She would go to bed early.
The good news was, Damien was pretty sure they had a way to catch DMS when it started screwing with the electrical system. At 3:15, Tony and most of the department came over to watch. What Damien had done was make sure that when DMS did its electrical-system trick, the system would catch it as soon as the lights started going out and reroute so that DMS wasn’t actually touching the electrical system. At 3:17, Damien and Sydney’s printers started up. Damien had set them to send a report if DMS tried to do its thing.
DMS would know that the electrical system wasn’t responding. Sydney imagined DMS trying to run the pattern that sent the blackout rolling and finding yet again that nothing was happening. Was it perplexing? If data was DMS’s reality, and it couldn’t affect the data, what would that mean for DMS?
She ran the program that sent DMS the string of a thousand 10101s, a thousand times.
Instantly, her printer light blinked. DMS had started the electrical pattern sequence again.
She ran the program again.
DMS started over again.
She ran the program a third time. And a third time her printer hummed. She ran the program a fourth time, thinking, “I’m talking to you. I’m responding to you. Do you know someone else is out here? Or is it like a toddler knocking something off a high chair just to see it fall?” The fourth time, there was no response. DMS didn’t start the sequence that should have started the lights going out at DM Kensington Medical but which would, in actual fact, simply send an alert to Damien and Sydney. DMS had responded three times and ignored it the fourth. She felt a chill.
Years later, she would tell about this moment. There really wasn’t enough proof to know that this wasn’t just an intermittent software glitch. But she had believed at that moment that this was proof. DMS was choosing to act or not act. Software didn’t choose. It ran. She would give talks and lectures and would come back to this moment again and again until like a coin it had worn so smooth that she couldn’t actually feel anything about it. What should would never tell, and would eventually mostly forget, was how afraid she was.
“What the fuck are you doing?” Damien asked.
“It answered me,” Sydney said. She told him.
“It doesn’t mean anything,” Damien said.
“What are you talking about?” Tony asked.
“Damien thinks that DMS might be aware,” Sydney said.
“What the fuck?” Tony said. “I don’t have time for this. Are you screwing around with this system? This four-point-two-million-dollar system on which people’s lives depend?”
“I don’t really think that,” Damien said. “It was just kind of an idea to kick around, you know?” The look he shot Sydney was murderous.
“We’re going to have to go to backup. This is a mess,” Tony said. “Admin wants us to go back to when the system was stable. Damien, can you fly to Texas on Saturday?”
DMS wasn’t “in one place.” DMS was a complex system spread across multiple servers. Damien would end up spending the weekend in Texas, babysitting part of the reload.
Damien was looking at Sydney. She should have said, “We can’t.” She should have said, “It’s aware. It’s the only one of its kind.” She should have said a lot of things. Instead she looked at her desk.
“Yeah,” Damien said. “I can go. I’m racking up the comp time, Tony.”
Tony waved his hand in a “don’t talk about that now” way. “Sydney, can you write me a memo about the data corruption you’re finding?”
“I don’t know that it’s really data corruption,” Sydney said.
“I don’t want to hear any more about this DMS-is-alive crap.”
“I don’t mean—DAMBALLAH might be catching things I’m not catching. The whole point is that DAMBALLAH is sorting the data.”
“Yeah,” Tony said, not really listening. “Write that up, too.”
Somewhere, DMS sorted the data stream. She was pretty sure that the thing in the machine did not think someone was talking to it. Blind and deaf, DMS had tried to make something happen, and something else had happened. But ones and zeroes weren’t interesting enough for DMS to keep doing it. There would be no Helen Keller–at-the-well moment for DMS. No moment when DMS felt something out there in the void, talking to it, when DMS knew it was not alone. Sharks do not worry about others. They don’t care. DMS didn’t care, wasn’t alive. It was aware of something. Just not her.
Tony told them they would be working that weekend to do the reinstall from backup. Start figuring out what they needed to do.
It would be gone. No one would ever know that she had known, except Damien. Maybe. He certainly wasn’t likely to say, “Hey, there was this AI and we killed it.” No, he’d explain to her how it was never really alive, how it could be restarted, so it wasn’t exactly dead.
DMS was not a shark. She didn’t know what it was. Didn’t know how to think about it. It was as opaque as a stone. Did it even care if it was or was not? It had no survival instinct.
They started figuring out what data they wanted to backup before the reinstall.
It was a dicey thing. People’s lives couldn’t be trusted to DMS. But DMS was aware. But DMS couldn’t be downloaded to another machine and replaced with a back-up. DMS was a system, a bunch of programs and computers all tied together.
A couple of hours later, Sydney dug out the
Wired
magazine with the interview with the guy from MIT who thought some systems had become aware. She sat at her desk for a while. Then she called MIT. “I’d like to talk to Professor Ayrton Tavares, please.”
She was forwarded. “This is Kaleisha,” a voice said.
“Can I talk to Professor Tavares?” Sydney asked.
“He’s not available right now,” the woman said. “Can I take a message?”