"We'll discuss that once you have the suit. How about you talk to Coffman now?
"Fine."
Dr. Victor Coffman was much younger than Deck had expected. In the back of his mind, he had pictured an eccentric old man, sort of like Einstein. This guy was a narrow, well-dressed man around forty. His jaw and cheekbones were sharp and square. His dark hair was short and filled with strands of gray. Round glasses clung to his nose and masked his eyes with their glare. He was dressed in a jacket, no tie. Instead of sitting down at the console, he stood behind the chair and peered nervously into the camera.
"Hello?"
"Talk to me, Doc," Deck shot back. Some people had trouble conversing when they didn't have some sort of video feed to look at, but this guy was old enough to remember a time before video links were ubiquitous.
Coffman shrugged, "I was told you had questions? About Shodan? I don't know your background or what sort of misinformation Morris might have given you, so I don't know where to begin."
Deck let out a heavy breath. He didn't know where to begin either. He would have been willing to talk about A.I. all day if the situation wasn't more pressing. He didn't even know how to sum up.
Finally he answered, "I don't know. I guess we should start with these drive chips. Why?" This question wasn't strictly related to the task at hand. He didn't need to know 'why' Shodan worked the way she did, but he couldn't resist the chance to find out.
Coffman nodded with approval. The question seemed to please him. "What is the biggest difference between your mind and the 'mind' of a typical computer?"
Deck wavered, "That's a really open ended question, I mean..."
Coffman nodded, "Okay, but I'm talking behavior-wise here, not hardware. Let's just pretend we have two brains here, both made of the same stuff. Organic or mechanical. Doesn't matter. Just hypothetically, we have two brains with similar physical makeup, but one works like yours and one works like a normal computer. What is the biggest difference between the two?"
Deck answered quickly, "I don't need a programmer. My brain is independent. I guess that's the biggest difference I see."
Coffman was nodding vigorously as Deck answered, "Excellent. Most people give a boring non-answer, like 'I'm smarter', which is subjective and unprovable in a lot of ways, and in any case doesn't really tell us anything. Even if you can prove it, it just means your brain works better, and doesn't tell us how they are different. Other people will give some frivolous answer, like 'I have emotions'. That answer is even worse, since it doesn't really tell us
anything
about how the brain operates. It only tells us what it
feels
like to operate one."
"So, you don't need emotions to be intelligent," Deck stated. He had always assumed this was the case. It hadn't occurred to him that this might not be the prevailing line of thought on the matter.
"Yes. The two are unrelated. Humans feel emotions to varying degrees. Some people are stoic. Others are very emotional. In either case their emotions are not tied to their intellect. Most people treat emotions like output. But that isn't what they are for, really. Emotions are
input.
"
"You lost me."
"You could write a program that made a smiley face when some arbitrary input is favorable, and a sad face when it is unfavorable, and you have not closed the gap between our two hypothetical minds in the slightest. The difference isn't whether either one is happy or sad, but - getting back to the answer you gave - in their ability to act on those emotions and do something about it. Our happy / sad program would be pointless until we sat down and wrote some programs for it. If you are happy, do this and if you are unhappy do that. In which case it is the programmer doing the thinking, not the machine. You, on the other hand, don't need anything of the sort. If you need something you don't have to be told to go and make it better."
"So intelligence is about having needs or wants?"
"Think of it this way: Thinking is about acting on needs, and intelligence is about how efficiently the needs are pursued. When people say 'sentient' they usually mean the former, although most of the A.I. research over the years has concentrated on the latter. The Shodan project was about separating these two problems and solving them individually."
"So the drive chips were about giving the thing something to want."
"Yes, as well as limiting behavior. Susan - one of the other people involved with the project - never really approved of the drive chips. She always said they were an ugly hack. She didn't like the 'hard coding' as she called it."
"But you don't mind the hard coding?". Deck prompted.
"Well, I think we all have some hard coding to some degree. Of course, our drives are regulated by hormones and instincts, which are much more unwieldy and unpredictable. We are obviously a lot more flexible in our needs than Shodan, but we still have them, and they still govern our behavior. When we designed the chips, we were trying to err on the side of safety. We packed a lot of rules onto the inhibitor chip, probably more than we needed, but we wanted to see how the thing acted before we gave it too much leeway. It's ironic, I guess. I intended to loosen the constraints on the chip as Shodan developed, but was forced off the project before I could do so."
Coffman lowered his face and frowned. "If Diego had let me stay on he probably wouldn't have needed you to hack her, and she wouldn't be in this mess right now," he added bitterly.
Deck was moving up and down the rows of crates, dragging his hand along them as he walked. Finally he answered, "You said the project had two parts: driving behavior, and intelligence. Or something like that."
"I don't know if I would use those terms exactly, but you have the idea. Susan liked to complicate things with a lot of psychological terms: 'id', 'ego', 'superego', and so on. As if that had any relation to what we were doing. I often suspected her goal was to re-create the patterns of a human brain, neuroses and all." He smiled to himself for a moment before continuing, "Anyway: yes. The behavior chips and the various protocols of her mind were one half of the project, and designing a system capable of learning was the other half."
Coffman pushed back on his glasses as if to move them further up his nose, but they were already as high as they could go. "Up until Shodan, computer intelligence was always based on large sets of specialized programs. Most of the people in the field are still laboring under the misconception that if you can just get enough information and processing power together it will reach critical mass and somehow become self-aware."
"And you can't?"
"Of course not. You can make a car as fast as you want, but it will never get you to the moon. For the last seven decades, people have been building larger, more complex databases. They equate knowledge, or better yet: raw information, with intelligence. This goes against what we observe in the organic world. Infants start with just a few rules and instincts, and basically no information at all. Yet they have a fantastic capacity to learn. They will, in just a few years, far outpace the intellect of even the best AI. The trick is not to build a machine with lots of information, but to build a machine with the ability to assimilate information, understand it, relate it to other information, and then extrapolate new information."
Coffman pushed back on his glasses again. He paused for a moment, scratching the back of his head. For a second he was gone, lost in some memory or idea. A few seconds later he snapped back and continued, "Anyway, all of the commercial AI projects are centered around this brute force idea of massing information and processing power. Some of them have had interesting results."
"Yeah I'm familiar with this. I've messed around with some of them. I've talked with Lysander, BrainTrain, and ThoughtBox, and I've had a peek at some of the smaller ones."
Dr. Coffman pulled back in exaggerated surprise, "That's quite a list. There are not many people who know those systems exist, much less have access to them."
"Yeah well, you academic types have good AI but your security sucks."
He shrugged, "Well, in this case it's worked in our favor, since we can skip that part of the discussion."
Deck smiled. Academic security sucked because academics didn't care about security.
Coffman continued, "Well, at any rate, all of those systems are obsolete. Junk. Attempting to build an intelligence by hand is like trying to build a perpetual motion machine. The designers somehow expect to get out more intelligence than they put in. They imagine that if they just write enough code and build fast enough processors, they can just turn the machine on and it will have an IQ of 500. It will never happen."
"They have had some good progress up until now," Deck argued.
"You can build a so-called perpetual motion machine that is more and more efficient, but you will never reach the one-hundred percent efficiency needed to just break even. No matter how good their code is, they will never build a machine as smart as the person designing it. Honestly, if you really judge those linear systems you will find that they are not quite as smart as monkeys. Sure, they seem smart because they have perfect recall and educated speaking voices, but all of that is just putting a tuxedo on the monkey. In the end, it is still just a stupid animal."
"But unless you are an unbelievable egomaniac you'll admit that Shodan is smarter than you. How do you explain that?"
"Shodan is not a linear system like the others - you've seen that. She is a living system. She learns. Like a human child, she grows. When we first flipped the switch, she was on level with a two-year old child. This was mostly due to the fact that a lot of her understanding of language could be pre-loaded so we didn't have to teach her a language before we could interact with her. She could read and type but she was still more or less ignorant of the world around her. We taught her through input. At first we communicated via text, until Morris completed work on her vocal systems. After that, we had to actually teach her to speak. We talked. We played games. We read books. She experienced many different kinds of media. She grew up."
Deck wandered the aisles of inventory as they spoke, "How long did it take?"
"Well, like any intelligent being, she never really
stopped
learning, although the growth curve did level off quite a bit."
"Until recently," Deck muttered.
Coffman gave a sort of defeated shrug before he moved on, "She was a functional 'adult' from a human standpoint by the time she was three, and educated enough to begin her research work in earnest by five."
"Research?"
"That is her purpose. Or was, until I was forced out. Her central drive from the beginning has been the acquisition of knowledge. Before Diego hijacked the project and turned her into the station's administrator, her primary drive was Discover New Things. That is still at the core of her program, underneath all the vandalism everyone has done to her over the years."
"Discover New Things, huh? Let's get back to the main point: How is she like the human brain?"
"First step: I need to explain to you how the brain learns."
Deck sighed, "Fine."
Coffman nodded, "Have you ever mastered anything that takes years to learn? I'm talking about your adult life, here. Maybe you learned a new language or a musical instrument? Learned how to pilot a complex vehicle?"
Deck thought for a moment, "No. How about learning martial arts?"
Victor looked up at the ceiling a moment. The reflections in his glasses shifted, revealing a new set of display screens. The control room must have been display screens floor-to-ceiling, "I suppose that's a good enough example. So, did you ever wonder why you couldn't learn the whole thing in one day? Why does it take years to become a master? You can witness most of the required moves in a day, I suspect. So why not just learn it all then?"
Deck fumbled with the question, trying to figure out where he was going with it. Finally he replied, "No. It just takes years to learn all those moves, how to control your body, how to balance, the strategy..," he trailed off for a moment, recalling his years in the sweaty Undercity dojo. "Its just too much information to absorb at once."
"Right. Well, break it down. The first thing they taught you, it was probably something basic like how to stand or breathe or something like that?"
"Well, my dojo was pretty brutal. The first lesson was designed to weed out some of the weaker potential students, and to give us a healthy fear of our sensei."
He winced, "Well, once they were done behaving like barbarians, and started to teach you Karate or whatever, they taught you how to stand and breathe, yes?"
"Yeah."
"And, at first, this stance, this way of breathing - it probably seemed awkward. You had to think about how you were standing and breathing. Perhaps you had trouble remembering how to do it?"
Deck recalled a time when he had been standing in a line of other students, practicing his moves. His stance was way off. He was standing like some movie action hero, ignoring the needs of poise and balance. His sensei passed behind him and gave a gentle kick to the side of his right ankle, causing him to flop to the floor like on old woman with a broken hip. The mistake earned him a nasty kick to the ribs once he was down. The bruise had stayed with him for several days, but the lesson stayed with him forever.
Finally he answered, "Yeah, it took me a couple of days to learn it."
"But why? If you did it once, why couldn't your brain just remember the stance and take it up again? I'm sure you can do it without thinking
now
."
"It's pretty much second nature."
"Right. So why the delay? A machine learns instantly. Program in a sequence of movements or events, and it will perform them with unwavering precision from that point on. Why do our brains, with their far superior computing power, take so long to learn such simple things?"
Deck hated the rhetorical question stuff. Couldn't Coffman just say what he wanted to say without dragging him through a long discussion? "I don't know, Doc - you tell me."