Authors: Lance Dodes
In 1951, largely on the strength of self-reported success and popular articles, AA was honored with the Lasker Award, “given by the American Public Health Association for outstanding achievement in the fields of medical research or public health administration.” The citation makes no mention of any scientific study that might prove or disprove the organization’s efficacy, simply declaring its “recognition of [AA’s] unique and highly successful approach” to alcoholism.
AA’s march toward public legitimacy accelerated. In 1955, at AA’s annual conference in Missouri, Dr. W. W. Bauer, an eminent member of the American Medical Association, told the assembled crowd: “You who have seen what alcohol can do in your lives are working together in groups and individually, and you are making a bigger impression on the problem of alcohol than has ever been made before.” Harry Tiebout, Wilson’s personal therapist, also appeared, assuring the collected members that AA was “not just a miracle but a way of life which is filled with eternal value.”
20
It wasn’t long before the court systems began to mandate AA attendance for drug and alcohol offenders. AA won a landmark decision in 1966 when two decisions from a federal appeals court upheld the disease concept of alcoholism and the court’s use of it, despite the fact that there was scant precedent for a US court of law to assign itself the power of medical diagnosis. Although later decisions would rule court-mandated 12-step attendance unconstitutional, judges still refer people to AA as part of sentencing or as a condition of probation. Dr. Arthur Horvath, a past president of the Division on Addictions of the American Psychological Association, summarizes the current legal status of this practice:
If you have been convicted of an offense related to addiction, it is common to be ordered to attend support groups, treatment, or both. It has also been common that you would be ordered, not just to a support group, but to Alcoholics Anonymous (AA) specifically, or to another 12-step based group.
Based on recent court decisions, if you have been ordered to attend a 12-step group or 12-step based treatment by the government (the order could be coming from a court, prison officer, probation or parole officer, licensing board or licensing board diversion program, or anyone authorized to act on behalf of the government), you have the right
not
to attend them. However, you can still be required to attend some form of support group, and some type of treatment.
These court decisions are based on the finding that AA is religious enough that being required to attend it would be similar to requiring someone to attend church. Five US Circuit Courts of Appeal (the 2nd, 3rd, 7th, 8th, and 9th) have made similar rulings. . . . The 2nd Circuit Court decision states that AA “placed a heavy emphasis on spirituality and prayer, in both conception and in practice,” that participants were told to “pray to God,” and that meetings began and adjourned with “group prayer.” The court therefore had “no doubt” that AA meetings were “intensely religious events.” Although some have suggested that AA is spiritual but not religious, the court found AA to be religious.
21
In 1966, President Lyndon Johnson proclaimed to the nation that “[t]he alcoholic suffers from a disease which will yield eventually to scientific research and adequate treatment.”
22
The disease theory and AA’s lobbying had at this point become difficult to separate. In 1970, Congress joined the consensus and passed a law known as the “Comprehensive Alcohol Abuse and Alcoholism Prevention Treatment and Rehabilitation Act,” which established the National Institute on Alcohol Abuse and Alcoholism (NIAAA). Among those testifying to the lawmakers in support of the bill were Marty Mann and Bill Wilson.
In 1973, the “millionth copy” of
Alcoholics Anonymous
was presented to President Richard Nixon in the Oval Office. (AA is the only source for this number; to my knowledge, sales have never been publicly audited.)
According to the Federation of State Physician Health Programs, “By 1980 . . . all but three of the 54 U.S. medical societies of all states and jurisdictions had authorized or implemented impaired physician programs.”
23
A recent paper looking at state-sponsored physician health groups (for doctors who have problems with addiction) found that “[r]egardless of setting or duration, essentially all treatment provided to these physicians (95%) was 12-step oriented.”
24
In 1989, the nation’s first drug court appeared in Dade County, Florida. Its treatment plan was to remit nonviolent drug offenders to a 12-step program and to compel those who had failed follow-up drug tests to attend “supplementary” 12-step sessions.
25
Examining this history, it is clear that AA has been extraordinarily effective at influencing public opinion and policy toward a favorable view of its ideas. What is missing from this account is notable as well: these strides were achieved without any triggering event, such as a well-designed study, that might support the organization’s claims of efficacy. Most of AA’s claims were simply grandfathered in, collecting legitimacy in a sort of echo chamber of reciprocal mentions that often featured the same handful of names.
WE COME NOW TO the essential question:
Is AA an effective treatment for alcoholism?
Many people have argued passionately on one side or the other of this debate, but these arguments are too often heated and anecdotal in nature. To truly determine whether and how often AA succeeds, we must examine the totality of the evidence and determine which studies, if any, can lend the subject clarity. This conversation begins with a consideration of the difference between good science and bad science.
A general definition of
science
is the asking and answering of questions through a controlled process of testing, repeating, and confirming results. The scientific method we all learned in grade school covers the basic contours: to know if a thing is true, one must isolate that thing and test it without changing anything else. Change two or more things, and you introduce the possibility that your results might come from any or all of them; no definitive answers can emerge until the universe of possible explanations is winnowed down to a final candidate.
To deal with this problem in human studies, scientists usually create a comparison group alongside the group being tested—the
control group
. This group, as the name suggests, helps scientists control for factors that might be acting on the experiment from elsewhere, offering a simple way to tell if the results are due to the experiment or something else. Without a control group, it is dismayingly easy to produce a “finding” that cannot withstand further scrutiny. Say, for instance, the test group in a drug experiment develops a rash. One might assume that the drug causes the rash. But if the untreated control group develops the same rash, then it is most likely due to something unseen that’s influencing both groups, and not the drug.
Human research tends to cleave into two major “kingdoms”: observational studies and controlled studies. Observational studies
observe
and
compare
groups of people. This research is conducted passively; in other words, without interventions or controls. Any significant differences that emerge between the populations studied—say, finding that people who drink more diet soda tend to have a higher incidence of depression than people who don’t—can’t prove anything but may be used to generate hypotheses about what is causing this difference.
Yet people still assume the obvious when confronted with a correlation of this sort. In the diet soda study, which was actually run by the National Institute of Health and widely reported, many people jumped to the conclusion that depression must be caused by something in the soda.
1
But a moment of creative consideration turns up several other plausible possibilities. What if the people who drink diet soda are simply more judgmental about their body appearance and generally more prone to self-criticism? What if, since drinking more diet soda correlates with a history of being overweight, the depression arises physiologically from the effects of obesity, or as a result of the cluster of health problems that go along with it, such as obstructive sleep apnea and diabetes? What if people who are depressed simply crave sweet things, as evidence suggests? And what of the fact that diet soda drinkers tend to cluster more in urban areas: is there something about this environment that promotes depression?
Strong correlation is tantalizing, a just-so homily that satisfies our need for simple explanations. It feels definitive and self-apparent, especially given the huge number of subjects typically involved in such studies. The NIH study that produced the diet soda finding, for instance, had 260,000 subjects. Headlines are driven and public health advice administered whenever a major observational study unearths a provocative new correlation. But it turns out that the record of observational studies like these for generating accurate medical advice is, in a word, abysmal. Award-winning science journalist Gary Taubes described the issue in the
New York Times Magazine
:
Stephen Pauker, a professor of medicine at Tufts University and a pioneer in the field of clinical decision making, says, “Epidemiologic studies, like diagnostic tests, are probabilistic statements.” They don’t tell us what the truth is, he says, but they allow both physicians and patients to “estimate the truth” so they can make informed decisions. The question the skeptics will ask, however, is how can anyone judge the value of these studies without taking into account their track record? And if they take into account the track record, suggests Sander Greenland, an epidemiologist at the University of California, Los Angeles, and an author of the textbook “Modern Epidemiology,” then wouldn’t they do just as well if they simply tossed a coin?
2
The only way to answer these questions would be to run a
randomized study
. This is a type of controlled study in which people are randomly assigned to respective groups—in this case, one group drinks diet soda, one drinks regular soda, one drinks a third option or no soda at all. Randomization eliminates any question about whether certain kinds of people self-select into certain groups. As Taubes relates:
In January 2001, the British epidemiologists George Davey Smith and Shah Ebrahim, co-editors of
The International Journal of Epidemiology
. . . noted that those few times that a randomized trial had been financed to test a hypothesis supported by results from these large observational studies, the hypothesis either failed the test or, at the very least, the test failed to confirm the hypothesis: antioxidants like vitamins E and C and beta carotene did not prevent heart disease, nor did eating copious fiber protect against colon cancer.
3
It is an intriguing question: why
do
purely observational studies fail so often despite finding such clear associations? The diet soda example tells the tale. All of those alternative theories I mentioned can be boiled down to a single, devastating possibility: what if diet soda drinkers are just
fundamentally different
from regular soda drinkers, in any of the ways I mentioned, and this difference colors
everything
about the way they live and behave? Scientists call this the
selection effect
, or
selection bias
. When human beings are free to behave as they always have—free to willfully choose their behavior—there is no meaningful way to find a control group of comparable subjects.
Selection bias has been widely studied, and today we are aware of many ways that this effect can despoil scientific studies. Yet one way deserves special mention, as it has threatened to undermine the very core of public health research: the
compliance effect
.
A growing body of evidence strongly suggests that people who do things faithfully and regularly for their own well-being, such as taking a multivitamin, exercising daily, or eating a certain diet, are, in fact, fundamentally different from people who don’t. People who adhere to, or
comply with
, medical advice are more likely to take care of themselves in numerous other ways as well:
Quite simply, people who comply with their doctors’ orders when given a prescription are different and healthier than people who don’t. This difference may be ultimately unquantifiable. The compliance effect is another plausible explanation for many of the beneficial associations that epidemiologists commonly report, which means this alone is a reason to wonder if much of what we hear about what constitutes a healthful diet and lifestyle is misconceived.
4
The compliance effect can lead researchers and reporters who study interventions to falsely credit a pill or diet with improving our health—“Look, people who take fish oil pills live longer than the rest of us!”—when the truth may be far more subtle: the
kind
of people who take supplements in a disciplined way are already healthier to begin with, with a better prognosis for every disease.
This is another reason why large observational studies regularly fail when they are examined with better scientific controls. Pomegranate juice, red wine, and chocolate have all failed to show any appreciable health benefit once studied under controlled conditions. In the case of hormone replacement therapy, famously, it turned out that all those observational studies had it exactly backwards: the women who faithfully took hormone pills lived longer because they were the kind of women who were simply more attuned to their health, period. Chillingly, researchers later discovered that the pills were working
against
this natural advantage.
The funhouse mirror widens. The compliance effect has led to some famously strange epidemiological results. One long-term study showed that people who took a
placebo
were half as likely to die as those who did not. Was the placebo protecting them in some way the researchers had failed to anticipate? Hardly. It turned out that simply taking the placebo regularly was a signpost for a wholly different lifestyle. The pill takers were simply more actively engaged in their health across the board.