Against the Gods: The Remarkable Story of Risk (41 page)

Was this just a string of bad luck, or do the managers of American
Mutual lack the skill to outperform an unmanaged conglomeration of
500 stocks? Note that, since American Mutual is less volatile than the
S&P, its performance was likely to lag in the twelve out of thirteen years
in which the market was rising. The Fund's performance might look a
lot better in years when the market was declining or not moving up or
down.

Nevertheless, when we put these data through a mathematical stress
test to determine the significance of these results, we find that American
Mutual's managers probably did lack skill.21 There is only a 20% probability that the results were due to chance. To put it differently, if we
ran this test over five other thirteen-year periods, we would expect
American Mutual to underperform the S&P 500 in four of the periods.

Many observers would disagree, insisting that twelve years is too
small a sample to support so broad a generalization. Moreover, a 20% probability is not small, though less than 50%. The current convention
in the world of finance is that we should be 95% certain that something
is "statistically significant" (the modern equivalent of moral certainty)
before we accept what the numbers indicate. Jacob Bernoulli said that
1,000 chances out of 1,001 were required for one to be morally certain;
we require only one chance in twenty that what we observe is a matter
of chance.

But if we cannot be 95% certain of anything like this on the basis
of only twelve observations, how many observations would we need?
Another stress test reveals that we would need to track American
Mutual against the S&P 500 for about thirty years before we could be
95% certain that underperformance of this magnitude was not just a
matter of luck. As that test is a practical impossibility, the best judgment
is that the American Mutual managers deserve the benefit of the doubt;
their performance was acceptable under the circumstances.

The next chart shows a different picture. Here we see the relative
performance of a small, aggressive fund called AIM Constellation. This
fund was a lot more volatile during these years than either the S&P
Index or American Mutual. Note that the vertical scale in this chart is
twice the height of the vertical scale in the preceding chart. AIM had a disastrous year in 1984, but in five other years it outperformed the S&P
500 by a wide margin. The average annual return for AIM over the
thirteen years was 19.8% as compared with 16.7% for the S&P 500 and
15.0% for American Mutual.

Is this record the result of luck or skill? Despite the wide spread
in returns between AIM and the S&P 500, the greater volatility of
AIM makes this a tough question to answer. In addition, AIM did not
track the S&P 500 as faithfully as American Mutual did: AIM went
down one year when the S&P 500 was rising, and it earned as much
in 1986, as in 1985, as the S&P was earning less. The pattern is so
irregular that we would have a hard time predicting this fund's performance even if we were smart enough to predict the returns on the
S&P 500.

Because of the high volatility and low correlation, our mathematical stress test reveals that luck played a significant role in the AIM case
just as in the American Mutual case. Indeed, we would need a track
record exceeding a century before we could be 95% certain that these
AIM results were not the product of luck! In risk-management terms,
there is a suggestion here that the AIM managers may have taken excessive risk in their efforts to beat the market.

Many anti-smokers worry about second-hand smoke and support
efforts to making smoking in public places illegal. How great is the risk
that you will develop lung cancer when someone lights up a cigarette
at the next table in a restaurant or in the next seat on an airplane?
Should you accept the risk, or should you insist that the cigarette be
extinguished immediately?

In January 1993, the Environmental Protection Administration
issued a 510-page report carrying the ominous title Respiratory Health
Effects of Passive Smoking: Lung Cancer and Other Disorders.22 A year later,
Carol Browner, the EPA Administrator, appeared before a congressional
committee and urged it to approve the Smoke-Free Environment Act,
which establishes a complex set of regulations designed to prohibit
smoking in public buildings. Browner stated that she based her recommendation on the report's conclusion that environmental tobacco
smoke, or ETS, is "a known human lung carcinogen."23

How much is "known" about ETS? What is the risk of developing
lung cancer when someone else is doing the smoking?

There is only one way even to approach certainty in answering
these questions: Check every single person who was ever exposed to
ETS at any moment since people started smoking tobacco hundreds of
years ago. Even then, a demonstrated association between ETS and
lung cancer would not be proof that ETS was the cause of the cancer.

The practical impossibility of conducting tests on everybody or
everything over the entire span of history in every location leaves all
scientific research results uncertain. What looks like a strong association
may be nothing more than the luck of the draw, in which case a different set of samples from a different time period or from a different
locale, or even a different set of subjects from the same period and the
same locale, might have produced contrary findings.

There is only one thing we know for certain: an association (not a
cause-and-effect) between ETS and lung cancer has a probability that is
some percentage short of 100%. The difference between 100% and the
indicated probability reflects the likelihood that the ETS has nothing
whatsoever to do with causing lung cancer and that similar evidence
would not necessarily show up in another sample. The risk of coming
down with lung cancer from ETS boils down to a set of odds, just as in
a game of chance.

Most studies like the EPA analysis compare the result when one
group of people is exposed to something, good or bad, with the result
from a "control" group that is not exposed to the same influences.
Most new drugs are tested by giving one group the drug in question
and comparing their response with the response of a group that has
been given a placebo.

In the passive smoking case, the analysis focused on the incidence
of lung cancer among non-smoking women living with men who
smoked. The data were then compared with the incidence of disease
among the control group of non-smoking women living with nonsmoking companions. The ratio of the responses of the exposed group
to the responses of the control group is called the test statistic. The
absolute size of the test statistic and the degree of uncertainty surrounding it form the basis for deciding whether to take action of some
kind. In other words, the test statistic helps the observer to distinguish
between CONSTANTINOPLE and BZUXRQVICPRGAB and cases with more meaningful results. Because of all the uncertainties
involved, the ultimate decision is often more a matter of gut than of
measurement, just as it is in deciding whether a coin is fair or loaded.

Epidemiologists-the statisticians of health-observe the same convention as that used to measure the performance of investment managers. They usually define a result as statistically significant if there is no
more than a 5% probability that an outcome was the result of chance.

The results of the EPA study of passive smoking were not nearly
as strong as the results of the much larger number of earlier studies of
active smoking. Even though the risk of contracting lung cancer
seemed to correlate well with the amount of exposure-how heavily
the male companion smoked-the disease rates among women exposed
to ETS averaged only 1.19 times higher than among women who lived
with non-smokers. Furthermore, this modest test statistic was based on
just thirty studies, of which six showed no effect from ETS. Since many
of those studies covered small samples, only nine of them were statistically significant.24 None of the eleven studies conducted in the United
States met that criterion, but seven of those studies covered fewer than
forty-five cases.25

In the end, admitting that "EPA has never claimed that minimal
exposure to secondhand smoke poses a huge individual cancer risk,"26
the agency estimated that "approximately 3,000 American nonsmokers
die each year from lung cancer caused by secondhand smoke."27 That
conclusion prompted Congress to pass the Smoke-Free Environment
Act, with its numerous regulations on public facilities.

We have reached the point in the story where uncertainty, and its
handmaiden luck, have moved to center stage. The setting has changed,
in large part because in the 75 years or so since the end of the First
World War the world has faced nearly all the risks of the old days and
many new risks as well.

The demand for risk management has risen along with the growing
number of risks. No one was more sensitive to this trend than Frank
Knight and John Maynard Keynes, whose pioneering work we review
in the next chapter. Although both are now dead-their most important writings predate Arrow's-almost all the figures we shall meet from now on are, like Arrow, still alive. They are testimony to how young
the ideas of risk management are.

The concepts we shall encounter in the chapter ahead never occurred to the mathematicians and philosophers of the past, who were
too busy establishing the laws of probability to tackle the mysteries of
uncertainty.