Authors: Lance Dodes
The authors did, however, acknowledge that the big question was whether AA was helping people or not. To this point, they added a familiar caveat:
Still, the direct relationship between 12-step attendance and abstinence remains uncertain in part because randomized clinical trials that direct and restrict attendance are difficult to conduct with such a freely available source of support as AA (and NA/CA). This relationship becomes even more blurred when attendance is studied over longer follow-up periods and as people transition in and out of both formal treatment programs and 12-step groups. In addition, only scant research has focused on outcomes other than actual alcohol and drug use (e.g., abstention status, percent days abstinent, drinks per drinking day).
Like the Moos and Fiorentine researchers before them, the Witbrodt researchers failed to address the simple fact that one cannot prove that any medical intervention works without a control group. (Interestingly the real “control” group for Alcoholics Anonymous—people who seek no treatment at all—have their own impressive rate of recovery, which I will discuss shortly.)
One paper has tried to tackle the question of whether we can determine causality in a direct way. A 2003 study, conducted by J. McKellar and colleagues of the Palo Alto Health Care System and published in the
Journal of Consulting and Clinical Psychology
, attempted to do a difficult thing—determine causality retroactively using statistical techniques and a method of multivariate analysis known as
structural modeling
, an approach that attempts to develop mathematical equations to explain existing data.
26
Such techniques are not uncommon in medicine, although doubts about their value persist. After a good deal of mathematical fireworks, the study’s authors determined that AA attendance was associated with a reduction in alcohol-related problems, but that reduced alcohol-related problems were not associated with AA attendance. In other words, AA attendance actually
caused
a reduction in alcohol-related problems, rather than simply correlating with them.
Yet a closer look at the paper’s methodology raises some important questions about both the model and the generalizability of its findings. The researchers didn’t look at a representative sample of the general population. The study was populated by mostly (86 percent) single men, all of whom were veterans and all of whom had been in a 12-step in-patient program previously and were subsequently referred to AA. There were no controls or randomization. About one-quarter of all study participants dropped out and were not considered in the paper’s conclusions.
This study’s findings hewed fairly closely to what we’ve seen before: at one-year follow-up, hazardous consumption decreased from 93 percent to 42 percent, and decreased further to 37.5 percent at two years (a further 10.7 percent drop); at the same one-year follow-up, 80 percent of the subjects were involved with AA (an increase of 24 percent over baseline). At two years, 68 percent were still involved.
Yet these numbers, which suggest a strong correlation between AA attendance and sobriety, become less impressive when one looks more closely at the results. After one year, for instance, hazardous use dropped about 50 percent even though AA involvement increased by only 24 percent. It would therefore be difficult to attribute this improvement to AA alone. Far more likely is the possibility that a series of other factors lent a helping hand, including the hospitalization itself. Even
intensive
AA involvement—the kind most associated with better outcomes among AA members—during that first year was reported to be up by just 14 percentage points (from 9.2 percent to 23.3 percent) despite the 50 percent improvement, suggesting again that the improvement may have had to do with factors beyond AA. (More on this in a moment.)
The key point is a statistical one. When AA involvement and better outcomes move in the same direction, even if they are out of proportion, that represents a plausible correlation that might indeed turn out to be a causal relationship. On the other hand, when the numbers move in
opposite
directions, that is considered a clear negative result. This is precisely what happened in the first two years of the McKellar study. The paper reports that AA involvement decreased by 12 percent, while “hazardous” drinking went down by 11 percent. (The authors defined hazardous drinking as consuming more than four drinks on a drinking day.) This is the reverse of what one would expect if AA were responsible for the improvement: as people dropped out of AA, drinking should have become worse, not better.
As Harvard biostatistician Richard Gelber notes,
That both alcohol-related problems and participation in AA seemed to decline between years 1 and 2 by the same amount raises questions about the conclusion from the structural modeling used. It does not pass the common sense test. . . . This direct evidence calls into question conclusions drawn from the structural modeling.
27
Of course, the demographic issue alone, including the fact that everyone in the study had already been through 12-step treatment before, disqualifies this paper as a representative look at what happens when a cross-section of alcoholics is treated. McKellar and colleagues themselves noted, “Because individuals were not randomly assigned to attend self-help groups, one could argue that the apparently positive outcomes are due to self-selection on prognostic variables other than those we tested, such as available social support or willingness to self-disclose.”
28
As with other studies, the McKellar researchers also took no steps to verify the self-reported data about drinking frequency, either through urine tests or check-ins with friends or relatives. Ultimately, the authors acknowledged that the study failed to answer some very basic questions:
Future studies comparing AA with other interventions might help answer important questions such as (a) Does AA provide specialized benefits in lowering long-term alcohol problems when compared with other self-help groups or outpatient after care programs? or conversely; (b) Does AA affiliation (attending meeting, working the steps, etc.) provide the same benefits that any good therapeutic treatment would provide (i.e., hope, treatment rationale, therapeutic alliance, mitigation of isolation; Bergin & Garfield, 1994)?
Remarkably, despite all this, the McKellar study authors concluded that “the findings are consistent with the hypothesis that AA participation has a positive effect on alcohol-related outcomes.”
A recurring theme in AA research is the question of what
kind
of people do well in AA. Is there something about this small group of people, some special stuff, that makes them different? And if so, can we possibly ascertain what that stuff is? One paper, published by L. A. Kaskutas and colleagues of the School of Public Health, University of California, Berkeley, in 2009, took a look at the existing data and found, as the Cochrane Collaboration did, a lack of solid grounding for the claim that AA is a cure for alcoholism or addiction:
Rigorous experimental evidence establishing the specificity of an effect for AA or Twelve Step Facilitation/TSF (criteria 5) is mixed, with 2 trials finding a positive effect for AA, 1 trial finding a negative effect for AA, and 1 trial finding a null effect. Studies addressing specificity using statistical approaches have had two contradictory findings, and two that reported significant effects for AA after adjusting for potential confounders such as motivation to change.
29
This mix of results squares with what we have seen thus far in this chapter. The strong evidence that one would expect if AA were clearly effective is simply not present. At best, the proponents of the 12-step model can claim only what AA claims; namely, that the program “works if you work it.” Which is another way of saying that people who do well, do well. What does this mean about whether AA itself “works”?
In 2005, R. D. Weiss and colleagues at Harvard Medical School conducted a study that looked more closely at what drives people to a higher level of success in 12-step programs. In the study, which randomized 487 cocaine-dependent outpatients to various twenty-four-week behavioral treatments, the authors uncovered a strong indication that attendance alone did not seem to help people with addictions, but that “
active
12-step participation” was predictive: “Twelve-step group attendance did not predict subsequent drug use. However, active 12-step participation in a given month predicted less cocaine use in the next month.”
30
In 2011, J. Majer of Harry S. Truman College in Chicago and colleagues completed a longitudinal study of people in sober houses and reached much the same conclusion:
Participants who were “categorically involved” in all 12-step [recommended] activities [having a sponsor, reading 12-step literature, doing service work, and calling other members for help] reported significantly higher levels of abstinence and self-efficacy for abstinence at 1 year compared with those who were less involved, whereas averaged summary scores of involvement were not a significant predictor of abstinence.
31
Here it was again: evidence that more
engagement
with the program was correlated with greater abstinence (even though more
attendance
was not). The Weiss and Majer studies together suggest that the helpful factor in AA treatment may be the level of engagement or sense of group membership, rather than the therapeutic value of the meetings themselves. Of course this interpretation, too, might just be backward: it is entirely possible that the people who do well in AA become more involved as a result—that is, sobriety drives participation.
By now, the danger of looking at such correlations and concluding that people with alcoholism should go to AA should be evident. It would be akin to recommending that therapists try to get people into religion if they believe religious people are more contented. People who are devout have self-selected into religious organizations because this is meaningful to them. But that devotion cannot be imposed on others. People who do well in AA might very well self-select because they find it meaningful for some reasons I will describe later in this book. But given the results of all the studies on 12-step treatment, trying to push others into AA, who are less likely to find it meaningful, is a mistake.
The practice of recommending AA to all problem drinkers may also be harmful, as suggested by evidence that AA dropouts do worse than those who seek no treatment at all. The Moos study, for example, found that people who attended but did not stick with AA had worse outcomes than people who never entered the program. This makes sense, since failing to benefit from the approach that others claim to be the best (or only) effective treatment is depressing indeed. Often this depression is exacerbated when the person is blamed for not adequately “working” the program.
Even though AA does not conduct scientific studies on its success rates, a number of clinicians have tried to audit the figures. The National Longitudinal Alcohol Epidemiologic Survey, a 1992 review by the US Census Bureau and National Institute on Alcohol Abuse and Alcoholism (NIAAA), included a survey of AA members. It found that only 31 percent of them were still attending after one year.
32
AA itself has published a comparable figure in a set of comments on its own thirteen-year internal survey, stating that only 26 percent of people who attend AA stay for longer than one year.
33
A third study found that after eighteen months, between 14 and 18 percent of people still attended AA.
34
So let us assume that between 14 percent and 31 percent of people stay with AA for more than one year. Now we must ask: out of this remaining population, how many stay sober?
As we have seen, research has shown that only a small subset of people stay sober in AA for any appreciable length of time, and this subset grows smaller with each passing year. When people do attend AA often or regularly, especially when they become emotionally invested in the system (“AA involvement” as opposed to “AA attendance,” as the literature describes), they do well. As noted above, attending a self-help program per se is not helpful, but the
active
involvement seems to make a difference.
So, what percentage of AA attendees become actively involved? In 2003, a group in London headed by J. Harris looked at patients in residential treatment and concluded that while 75 percent of alcoholics entering residential treatment had attended AA previously, the number of those “working” the program (being “involved” versus merely attending) was 16/75, or 21 percent.
35
Within this group, how many not only improved, but consistently maintained sobriety? University of California professor Herbert Fingarette cited two other statistics: at eighteen months, 25 percent of people still attended AA, and of those who did attend, 22 percent consistently maintained sobriety.
36
Taken together, these numbers show that about 5.5 percent of all those who started with AA became sober members. Similarly, taking the 21 percent “involved” from the Harris study and multiplying that by the 25 percent who remain in AA yields an overall efficacy of 5.25 percent. Or, we could use the more positive results of the Fiorentine study, in which “approximately 40 percent of individuals categorized as having continued active participation in AA maintained high rates of abstinence.”
37
Combining this with the Harris data giving the percentage of people who are actively involved, overall effectiveness of AA becomes 21 percent times 40 percent, or 8.4 percent.
These totals all fall within a close range. Together, they support the fact that roughly 5 to 8 percent of the total population of people who enter AA are able to achieve and maintain sobriety for longer than one year.