The Scientist as Rebel (7 page)

Thomas Gold died in June 2004. Shortly before he died, an experiment was done at the Carnegie Institution of Washington Geophysical Laboratory to test his theory that natural gas is generated deep in the earth’s mantle.
²
The experiment, carried out with tiny quantities of mantle materials exposed to high temperature and pressure in a diamond anvil cell, demonstrated abundant production of methane. The authors sent a message to Gold to tell him that his theory had been confirmed, only to learn that he had died three days earlier.

PREY
¹
IS A
thriller, well constructed and fun to read, like Michael Crichton’s other books. The main characters are the narrator, Jack, and his wife, Julia, parents of three lively children, successfully combining the joys of parenthood with the pursuit of brilliant careers in the high-tech world of Silicon Valley. Julia works for a company called Xymos that is developing nanorobots, tiny machines that can move around and function autonomously but are programmed to work together like an army of ants. Jack works for a company called MediaTronics that makes software to coordinate the actions of large groups of autonomous agents. His programs give intelligence and flexibility to her machines.

Things start to go wrong when Jack loses his job and is left to take care of the kids, while Julia is working longer and longer hours at her laboratory and losing interest in the family. She is engaged in a secret struggle to develop her nanorobots into a stealthy photo-reconnaissance system that can be sold to the United States Army. To increase the power and performance of the system, she incorporates living bacteria into the nanorobots so that they can reproduce and evolve rapidly. She reprograms them with Jack’s newest autonomous-agent software so that they can learn from experience.

Even with these improvements the nanorobots fail to meet the army’s specifications, and Xymos loses its army funding. After that, Julia desperately tries to convert the photo-reconnaissance system into a medical diagnostic system that can be sold on the civilian market. Her idea is to train the nanorobots to enter and explore the human body, so that they can locate tumors and other pathological conditions more precisely than can be done with X-rays and ultrasound working from the outside.

Experimenting with the medical applications of her nanorobots, she uses herself as a guinea pig and becomes chronically infected. The nanorobots learn how to establish themselves as symbionts within her body, and then gradually gain control over her mind. In her deranged state, she deliberately infects three of her colleagues at the laboratory with nanorobots. She also lets a swarm of nanorobots loose into the environment where they prey upon wildlife and rapidly increase in numbers.

The main part of the story concerns Jack’s slow realization that something is seriously amiss with his wife and with the project in which she is engaged. Only at the end does he understand the full horror of her transformation. With the help of a loyal young woman friend, he confronts Julia and douses her with a spray of bacteriophage that is lethal to the bacteria inside her. But Julia and her infected colleagues are no longer able to survive without the symbiotic nanorobots that have taken over their minds. Under the spray of bacteriophage they collapse and die, like the Wicked Witch of the West in
The Wizard of Oz
when Dorothy throws a bucket of water over her. After Julia’s demise, Jack and his girlfriend finish the job of destroying the nanorobots inside and outside the laboratory with fire and high explosives. In the final scene, Jack is back with his kids, wondering whether the nanorobots are gone for good, or whether the Xymos corporation may still be developing other nanorobot projects that will turn into nightmares.

What are we to make of this fairy story? There are two ways to look at it. On the one hand, we may enjoy it as a story and not worry whether some parts of it might come true. On the other hand, we may read it as an urgent warning of dangers lying ahead if present-day technological developments are allowed to continue. The author says plainly, in an introductory chapter with the title “Artificial Evolution in the Twenty-first Century,” that he intends his story to be taken seriously.

It is easy to demonstrate that the details of the story are technically flawed. Consider for example the size of the nanorobots. In a commercial presentation advertising the Xymos medical diagnostic system, Julia says, “We can do all this because the camera is smaller than a red blood cell.” The camera is one of her nanorobots. It must be as small as that, since Julia describes it swimming in the human bloodstream inside the capillaries that carry blood through the lungs. The capillaries are only just wide enough for red blood cells to pass through. But later in the book Jack encounters swarms of nanorobots chasing him in the open air like a swarm of ants or bees. These nanorobots are flying through the air as fast as he can run. Fortunately for Jack and unfortunately for the story, the laws of physics do not allow very small creatures to fly fast. The viscous drag of air or water becomes stronger as the creature becomes smaller. Flying through air, for a nanorobot the size of a red blood cell, would be like swimming through molasses for a human being. Roughly speaking, the top speed of a swimmer or flyer is proportional to its length. A generous upper limit to the speed of a nanorobot flying through air or swimming through water would be a tenth of an inch per second, barely fast enough to chase a snail. For nanorobots to behave like a swarm of insects, they would have to be as large as insects.

Other technical flaws in the story are easy to find. The swarms of nanorobots flying in the open air are said to be powered by solar energy. But the solar energy falling onto their very small area is
insufficient to power their movements, even if we credit them with a magical ability to use solar energy with 100 percent efficiency. I could continue with a list of technical details that are scientifically impossible for one reason or another, but that would miss the main point of the story. The story is about human beings and not about nanorobots. The main point is that Julia is a credible human being. She is a capable and well-meaning woman in a responsible position, with the fate of a company resting on her shoulders. She decides that the only way to save the company from bankruptcy is to push ahead with a risky technology. Unable to face the failure of her company and her career, she continues with her experiments regardless of the risks. She is a gambler playing for such high stakes that she cannot afford to lose. In the end she loses not only her company and her career but her family and her life. It is a credible human story, and in the end the technical details do not matter.

This story reminds me of Nevil Shute’s
On the Beach
, published in 1957, a novel describing the extinction of mankind by radiological warfare. Shute’s poignant translation of apocalyptic disaster into the everyday voices of real people caught the imagination of the world. His book became an international best seller and was made into a successful film. The book and the film created an enduring myth, a myth which entered consciously or subconsciously into all subsequent thinking about nuclear war. The myth pictures nuclear war as silent inexorable death from which there is no escape, with radioactive cobalt sweeping slowly down the sky from the northern to the southern hemisphere. The people of Australia, after the northern hemisphere is dead, live out their lives quietly and bravely to the end. The Australian government provides a supply of euthanasia pills for citizens to use when the symptoms of radiation sickness become unpleasant. Parents are advised to give the pills to their children first before they become sick. There is no hope of survival; there is no talk of building an underground Noah’s Ark to keep earth’s creatures alive
until the cobalt decays. Shute imagined the human species calmly acquiescing in its extinction.

The myth of
On the Beach
is technically flawed in many ways. Almost all the details are wrong: radioactive cobalt would not substantially increase the lethality of large hydrogen bombs; fallout would not descend uniformly over large areas but would fall sporadically in space and time; people could protect themselves from the radioactivity by sheltering under a few feet of dirt; and the war is supposed to have happened in 1961, too soon for even the most malevolent country to have acquired the megatonnage needed to give a lethal dose of radiation to the entire earth. Nevertheless, the myth did what Shute intended it to do. On the fundamental human level, in spite of the technical inaccuracies, it spoke truth. It told the world, in language that everyone could understand, that nuclear war means death. And the world listened.

Prey
is not as good as
On the Beach
, but it is bringing us an equally important message. The message is that biotechnology in the twenty-first century is as dangerous as nuclear technology in the twentieth. The dangers do not lie in any particular gadgets such as nanorobots or autonomous agents. The dangers arise from knowledge, from our inexorably growing understanding of the basic processes of life. The message is that biological knowledge irresponsibly applied means death. And we may hope that the world will listen.

From this point on, I assume that the basic message of
Prey
is true. I assume that the growth of biological knowledge during the century now beginning will bring grave dangers to human society and to the ecology of our planet. The rest of this review is concerned with the question of what we should do to mitigate the dangers. What is the appropriate response to dangers that are hypothetical and poorly understood? In this matter, as in other situations where public health hazards and environmental risks must be assessed and regulated, there are two strongly opposed points of view. One point of view is
based on the “precautionary principle.” The precautionary principle says that when there is any risk of a major disaster, no action should be permitted that increases the risk. If, as often happens, an action promises to bring substantial benefits together with some risk of a major disaster, no balancing of benefits against risks is to be allowed. Any action carrying a risk of major disaster must be prohibited, regardless of the costs of prohibition.

The opposing point of view holds that risks are unavoidable, that no possible course of action or inaction will eliminate risks, and that a prudent course of action must be based on a balancing of risks against benefits and costs. In particular, when any prohibition of dangerous science and technology is contemplated, one of the costs that must be considered is the cost to human freedom. I call the first point of view precautionary and the second point of view libertarian. In April 2000, Bill Joy, co-founder and chief scientist at Sun Microsystems, a large and successful computer company, published an article in
Wired
magazine with the title “Why the Future Doesn’t Need Us,” and the subtitle “Our most powerful 21st-century technologies—robotics, genetic engineering, and nanotech—are threatening to make humans an endangered species.” It was a big surprise to see one of the leaders of high-tech industry arguing passionately for a slowing down of technology that might become dangerous. Bill Joy became a spokesman for the precautionary view.

Nine months later, in January 2001, the annual meeting of the World Economic Forum was held in Davos, Switzerland. Most of the people at the forum are captains of industry, presidents of foundations, or government officials. But in 2001 they decided to invite some scientists and writers and artists to add some intellectual sparkle to the meeting. Bill and I were both invited and asked to debate the question: Is our technology out of control? Bill was taking an extreme position on the precautionary side, and I was asked to take an extreme position on the libertarian side, to make the debate interesting.
In what follows I shall summarize our debate.
²
To be sure that I am not misrepresenting Bill, I quote here only from his published writings.

My first quote is from Bill’s article in
Wired
:

This was written a year and a half before the events of September 2001. I don’t know whether Bill at that time had Osama bin Laden in mind. He certainly had in mind the possibility of a Unabomber taking his revenge on society with genetically engineered microbes rather than with chemical explosives.

Second quote. Here Bill is quoting Eric Drexler, the chief prophet
of nanotechnology. Drexler set up the Foresight Institute to promote the benign uses of nanotechnology and to warn against the dangerous uses. Here is Drexler: