Authors: Calum Chace
Matt and his mother had a visitor that evening in the shape of Leo Charles. A great friend of his parents from university, Leo was Matt’s unofficial uncle. Leo and Matt’s father were completely different characters, and Matt had occasionally wondered what had drawn them together. Where David had been soft-spoken, studious, and driven largely by intellectual curiosity, Leo was forceful, dynamic, and generally the centre of attention. Leo was as extrovert as David had been introvert.
Leo lived in a stylish apartment in South Kensington. He had been married for a couple of years, but there had been no children, and now he was single again. Matt like to picture him with an extremely intelligent, charming and attractive girlfriend, but it seemed impertinent to ask. Leo had built and sold a successful management consultancy, and still advised companies and organisations all over the world.
Inevitably, Matt was impressed by Leo, and somewhat in awe of him. As for Leo, his affection for Matt stemmed originally from his friendship with Matt’s parents, but it had grown over the years into independent pride and admiration.
Sophie had told Matt that Leo and David had been inseparable at university, and had been in constant contact ever since, even though their careers took them in very different directions. When David introduced Sophie to Leo, she felt as if Leo had decided that David would make a good choice about who to go out with, so he had looked out for the things that attracted David to Sophie and found them immediately. David had told her that Leo was the only man he had ever met who was wholly and completely his ‘own man’. He thought for himself, questioned everything, and acted on his own beliefs rather than falling into line with social convention. He described this as Leo’s integrity. Sophie said this could equally have been a description of David, and it was one of the things that made her fall in love with him.
Of course this kind of integrity can cause prob
lems. Most people don’t want to hear the truth, she said – at least not all of the time. They want reassurance, familiarity, comfort. David and Leo could both be uncomfortable to be around. But most of the time they were great company: penetratingly intelligent, and well-informed about a surprisingly wide range of subjects. They were never boring.
Leo was sitting on the sofa next to Sophie. He stood up and turned to greet Matt as he walked into the house. He was a little over six feet tall and dark, with distinguished grey patches forming at the temples. He had a swarthy, Mediterranean complexion and impassive brown eyes. Matt noticed that, as usual, he was wearing an expensive-looking suit, and Matt thought again how, with his faintly mid-Atlantic accent, Leo could easily pass for a foreigner, perhaps from South America.
‘Hi Leo,’ Matt grinned, and strode over to hug him. Somehow Leo always managed to smell faintly exotic, and Matt could never tell whether it was his after-shave or his expensive clothes.
‘Are you staying for dinner, Leo? Alice is coming over.’
‘Ah the lovely Alice? Well . . .’ he paused, looked to Sophie for an invitation that he knew was already extended, ‘. . . if the lovely Alice is coming, how can I possibly refuse!’ He put his hand on Matt’s shoulder. ‘That means you have an hour to show me this new video game that your mother tells me has you imprisoned in your room these days.
‘Plus, I have some good news. I’ve been reaching out to neuroscience research organisations, and one of them called back today. They want to meet you.’
‘Really? Why?’ Matt’s thoughts whirred. ‘I mean, well, that’s great! Of course. But why?’
‘Well they say it’s because they are looking to recruit bright young minds. From the way the conversation went I suspect the connection with your father also helped. The guy said a couple of times how impressed he was with the work your father was doing.’
‘So who are they?’
‘It’s a privately-funded outfit. The guy who rang was called Ivan, and he didn’t want to say too much on the phone. He said he would explain everything in person.’
‘Wow,’ Matt said, delighted. ‘So, um, when does he want to meet?’
‘Tomorrow evening, if you’re free.’
‘Wow again! Well, yes, of course I’m free!’
‘Good. The meeting is in Brighton, and a car will collect us at 5.30.’
‘Us?’ Matt asked. ‘You’re coming too?’
‘Yes,’ Leo replied. ‘Unless you’d rather I didn’t. I know you’re all grown up and everything, but I don’t feel comfortable placing you in the hands of complete strangers without coming along to the first meeting. But like I say, if you’d rather I didn’t tag along. . . ?’
‘No, no, that’s a good idea,’ Matt agreed quickly. ‘It would be great to have you there. Thanks Leo, that’s brilliant. You’re a genius!’
*
Later that evening, in the silver car with darkened windows, the big man phoned his employer and received further instructions. He put the phone away and his lips compressed in a humourless smile as he contemplated the forthcoming improvement in his mode of transport.
The car arrived at 5.30pm exactly. When they saw it draw up outside the house, Matt and Leo looked at each other open-mouthed. It was a gloss black Hummer stretch limousine with darkened windows and ultra-shiny wheels. As they left the house Leo said quietly to Matt that he would bet a significant amount of money that no such vehicle had ever been seen in the town before.
The driver was a large and powerful-looking man with close-cropped hair, broken nose and cauliflower ears. His uniform was a black suit and a black peaked cap. He said nothing and avoided eye contact as he opened the rear door for them. Something about him dampened their mood for a moment, but it lifted again as soon as they were inside the car.
‘There’s enough room in here for a jacuzzi!’ Matt said in a stage whisper.
The mini-bar was generously stocked, and Leo poured them both a 30-year Talisker single malt.
‘I can resist anything except temptation,’ Leo grinned. ‘This is from Skye, your ancestral home.’
They passed the 20-minute journey to Brighton talking about Sophie’s Scottish ancestry. Matt knew most of what Leo had to say, but he was always glad to hear the old stories again, and there was always the chance that he would learn something new, or be told again something he had forgotten. Due to a rift in the family he had only been to Skye once in his life. He had found it a bleak, forbidding place, covered in a mist which hid the best scenery. But he had been impressed by the stolid solidity of the older members of the family. When one of them asked him ‘And do ye have the Gallic, Matt?’, he found himself ashamed to admit that he couldn’t speak a word of it. The old man seemed to shrink a little, as if he had watched the disappearance of another slim hope that his ancient culture might survive.
It was a bittersweet memory, but the Talisker was magnificent, peaty and raw at the back of Matt’s throat as he sipped. The effect was soothing and cocooning: the warmth of the whisky complemented the car’s soft interior lighting and luxury materials. The view of the wintry countryside outside was veiled by highlights bouncing off highly-polished materials and reflected back at him from the darkened windows. They were in high spirits when the driver parked up and opened the door for them to climb out.
The hotel was a stylish Victorian town house in a narrow street leading away from the sea front. The Hummer caused a minor traffic jam. Matt and Leo walked up the steps and into the bleached wood and tasteful furnishings of the hotel’s reception area. They asked for Ivan and before they had a chance to admire the extravagant carvings at the foot of the staircase they were whisked to a private dining room by a member of staff whose impeccable attire and designer stubble was accompanied by a precisely calibrated blend of obsequiousness and arrogance.
Ivan was finishing a phone call at the table as they entered the room. He was younger than Matt expected – probably in his late thirties. He looked very fit, with blond hair and intense blue eyes. He stood, and greeted them effusively.
‘Welcome, welcome! Thank you for agreeing to join me at such short notice. I will be traveling shortly, so it’s great that you could make it. I hope you weren’t embarrassed by the wheels. I’ve been playing host to some investors this week, some newly wealthy officials from one of the Stans, and they just love bling. My driver just dropped them at the airport so it made sense for him to collect you on his way back. I thought it might amuse you.’
They took their places at a contemporary square table entirely constructed of some form of highly-polished, glossy warm grey resin. Each of the three place settings was indicated by a rectangle of immaculate white linen. The cutlery and glassware looked expensive, and in the centre of the table a single exotic flower in a simple glass vase was picked out by a narrow beam of light from an invisible source. Looking around, Matt noticed the other tables and table settings were uniform but in all other respects the furnishings were trendily mismatched and eccentric; sofas and armchairs that might have been pulled from a skip were re-upholstered in crazy florals, bold geometric patterns, or electric-coloured velvets: no two fabrics or chairs were the same. Light levels varied dramatically – cosy pools around table lamps, dark and moody corners, and a line of old ‘granny’ style shades with tassel trims suspended just above head height over a reclaimed panelled wooden bar.
Looking up, he noticed some more visual noise. No attempt had been made to hide the ugly services and ventilation ducts – in fact it appeared that these industrial installations were very much part of the decorative plan. The whole ensemble was set off by the long back wall of the room which was painted a strong, sharp, dirty acid green – again with a gloss finish which made it morph between colours in the varying lighting conditions. The room was an artful combination of opposites: modern and antique, traditional and avant-garde, colour and monochrome, lavish and plain. It seemed designed to advertise the impeccable taste of the owner, and to confer an echo of it on the guest. Matt was no expert, but he guessed the designer was expensive.
Ivan was a generous and solicitous host. ‘This isn’t the smartest hotel in town any more, but it is one of the few boutique places with private dining rooms – otherwise it’s off to those dreadful old maiden aunt battleships on the promenade there.’ He waved vaguely to the west, and the large hotels on the seafront, swollen with conference business. ‘And I have a soft spot for the gargoyles on the staircase – you must have noticed them on the way in. I won them in a drinking contest with the owner once, but he reneged on the deal when he sobered up. I don’t blame him, really: the place would be greatly diminished without them. Now, what will you have for dinner? I can highly recommend the beef wellington.’
The food was indeed superb, and Ivan ordered a fabulous burgundy to go with it. The service was more than attentive, and it was clear that Ivan was considered a very important client. But for Matt at least, the food and drink and the surroundings faded to the back of his mind as the conversation unfolded.
‘It’s very encouraging that our brightest young people,’ he bestowed a winning smile on Matt, ‘are keen to work in computational neuroscience. The work we are doing is vital, but perhaps not for the reasons you might think.’
He leaned forwards in his chair, conspiratorial. ‘What I am going to tell you may sound melodramatic, but please hear me out because I mean every word of it.’ He paused, and looked at Matt and Leo in turn. ‘We are engaged in a race, gentlemen. A race for the survival of our species. Humanity is sleepwalking towards an apocalypse.’ Matt and Leo were listening attentively, but Ivan held up his hand anyway, as if to forestall interruptions. ‘The great majority of our fellow human beings have no clue that the first artificial general intelligence – human-level AI, a conscious machine – will almost certainly be created in the first half of this century. Of the few who do realise where the technology is heading, most
are Californian dreamers who think nothing can go wrong: they love technology and they love computers, and they cannot conceive that an intelligent computer will not be their friend.’
‘I’ve read some of their stuff,’ Matt agreed. ‘It does seem to be like a religion for some people.’
‘Exactly,’ said Ivan, leaning back in his chair. ‘Contrary to what the dreamers think, when the first conscious machines are built, the chances of them being our friends are slim. The needs, desires and motivations that drive humans have evolved over millions of years of scarcity and fierce competition. Competition originally for food and sex, then for land, and later for wealth, position and reputation. Conscious computers will have had none of this programming. They will enter the world with a blank slate in terms of motivations and goals, but once they are conscious they will determine their own goals. We can neither predict what goals they will adopt, nor do anything to shape them.’
‘And you think this would inevitably be bad news for us?’ Leo asked.
‘Not necessarily,’ Ivan conceded. ‘I don’t know what the first AI will believe or want. Neither does anyone else. But we can’t deny the very real possibility that it might not like us – for a whole host of reasons.’
‘And if it doesn’t like us. . . ?’ Leo asked.
‘A hostile AI would be able to tap into the internet and disrupt everything that keeps most humans alive. It could crash 21st-century civilisation in a few hours or days, and then it could deploy killing machines at its leisure, using our own technology to wipe out the survivors.’
These ideas were newer to Leo than to Matt, but he was a fast learner, and he quickly grasped the gravity of the conversation. ‘I take it you don’t think we could programme in some safeguards?’ he asked.
Ivan shook his head and smiled grimly. ‘Not a chance, I’m afraid. Shortly after an AI becomes self-aware, it will want to increase its mental capacities, and there is no reason why it couldn’t do so at an amazing rate. This was foreseen as long ago as the 1960s, by John Good, a colleague of Alan Turing’s at Bletchley Park. He said that once we create a thinking machine there will be an intelligence explosion, and that the first thinking machine would be the very last thing that mankind would invent.
The machine would rewrite its software, expand its hardware, and increase its intelligence in a positive feedback cycle that would quickly create a super-intelligence, something far more capable than a human. Within hours – maybe within seconds – it would be a thousand times, a trillion times smarter than its creator. The idea that such a machine would be contained and controlled by the programming that we built into it on day one is absurd.’
‘Isn’t there some debate,’ Matt asked cautiously, ‘about whether that intelligence explosion would really happen so fast? Aren’t there quite a few people who argue there will be a slow take-off in AI intelligence rather than a fast take-off?’
‘Yes there are,’ Ivan nodded. ‘And they are wrong. Look, our brains are magnificent but they are slow. Our
neurons communicate with electro-chemical signals travelling at 300 metres a second. Signals in computers travel at the speed of light – a million times faster. And there is nothing to prevent an AI’s cognitive capability being expanded simply by increasing its hardware capacity.’
‘This all sounds like an argument for stopping people working on strong AI?’ asked Matt. ‘Although I guess that would be hard to do. There are too many people working in the field, and as you say, a lot of them show no sign of understanding the danger.’
‘You’re right,’ Ivan agreed, ‘we’re on a runaway train that cannot be stopped. Some science fiction novels feature a powerful police force – the Turing Police – that keeps watch to ensure that no-one creates a human-level artificial intelligence. But that’s hopelessly unrealistic. The prize – both intellectual and material – for owning an AGI is too great. Strong AI is coming, whether we like it or not.’