How to Read a Paper: The Basics of Evidence-Based Medicine (17 page)

Until fairly recently, the use of confidence intervals was relatively uncommon in medical papers. Fortunately, most trials in journals that follow Consolidated Standards of Reporting Trials (CONSORT) guidelines (see section ‘Randomised controlled trials’) now include these routinely, but even so, many authors do not interpret their confidence intervals correctly. You should check carefully in the discussion section to see whether the authors have correctly concluded (i) whether and to what extent their trial supported their hypothesis, and (ii) whether any further studies need to be done.

The bottom line

Have the authors expressed the effects of an intervention in terms of the likely benefit or harm that an individual patient can expect?

It is all very well to say that a particular intervention produces a ‘statistically significant difference’ in outcome but if I were being asked to take a new medicine I would want to know how much better my chances would be (in terms of any particular outcome) than they would be if I didn't take it. Three simple calculations (and I promise you they
are
simple: if you can add, subtract, multiply and divide you will be able to follow this section) will enable you to answer this question objectively and in a way which means something to the non-statistician. The calculations are the relative risk reduction, the ARR and the number needed to treat.

To illustrate these concepts, and to persuade you that you need to know about them, let me tell you about a survey that Fahey and his colleagues [16] conducted a few years ago. They wrote to 182 board members of district health authorities in England (all of whom would be in some way responsible for making important health service decisions), and put the following data to them about four different rehabilitation programmes for heart attack victims. They asked which one they would prefer to fund.

Programme A—which reduced the rate of deaths by 20%.

Programme B—which produced an absolute reduction in deaths of 3%.

Programme C—which increased patients' survival rate from 84% to 87%.

Programme D—which meant that 31 people needed to enter the programme to avoid one death.

Table 5.2
Data from a trial of medical therapy versus coronary artery bypass grafting after heart attack [16, 17]

Of the 140 board members who responded, only three spotted that all four ‘programmes’ in fact related to the same set of results. The other 137 participants all selected one of the programmes in preference to one of the others, thus revealing (as well as their own ignorance) the need for better basic training in epidemiology for healthcare policymakers. In fact, ‘Programme A’ is the relative risk reduction; ‘Programme B’ is the ARR; ‘Programme C’ is another way of expressing the ARR and ‘Programme D’ is the number needed to treat.

Let's continue with this example, which Fahey and colleagues reproduced from a study by Yusuf and colleagues [17]. I have expressed the figures as a two by two table giving details of which treatment the patients received in their randomised trial, and whether they were dead or alive 10 years later (
Table 5.2
).

Simple maths tells you that patients on medical therapy have a 404/1325 = 0.305 or 30.5% chance of being dead at 10 years. This is the
absolute risk
of death for the control (medical therapy) group: let's call it
x
. Patients randomised to coronary artery bypass grafting (CABG) have a 350/1324 = 0.264 or 26.4% chance of being dead at 10 years. This is the absolute risk of death for the intervention (CABG) group: let's call it
y
.

The
relative risk
of death in CABG patients compared with medical intervention controls—is
y/x
or 0.264/0.305 = 0.87 (87%).

The
relative risk reduction
—that is, the amount by which the risk of death is reduced in the CABG group compared to the control group—is 100 − 87% (1 −
y
/
x
) = 13%.

The
ARR
(or risk difference)—that is, the absolute amount by which CABG reduces the risk of death at 10 years—is 30.5 − 26.4% = 4.1% (0.041).

The
number needed to treat
—that is, how many patients need a CABG in order to prevent, on average, one death by 10 years—is the reciprocal of the ARR, 1/ARR = 1/0.041 = 24.

The general formulae for calculating these ‘bottom line’ effects of an intervention are reproduced in Appendix 2, and for a discussion on which of these values is most useful in which circumstances, see Jaeschke and colleagues' article in the ‘Basic Statistics for Clinicians’ series
[3].

Summary

It is possible to be seriously misled by taking the statistical competence (and/or the intellectual honesty) of authors for granted. Statistics can be an intimidating science, and understanding its finer points often calls for expert help. But I hope that this chapter has shown you that the statistics used in most medical research papers can be evaluated—at least up to a point—by the non-expert using a simple checklist such as that in Appendix 1. In addition, you might like to check the paper you are reading (or writing) against the common errors given in Box 5.2.

References

1
Guyatt G, Jaeschke R, Heddle N, et al. Basic statistics for clinicians: 1. Hypothesis testing.
CMAJ: Canadian Medical Association Journal
1995;
152
(1):27.

2
Guyatt G, Walter S, Shannon H, et al. Basic statistics for clinicians: 4. Correlation and regression.
CMAJ: Canadian Medical Association Journal
1995;
152
(4):497.

3
Jaeschke R, Guyatt G, Shannon H, et al. Basic statistics for clinicians: 3. Assessing the effects of treatment: measures of association.
CMAJ: Canadian Medical Association Journal
1995;
152
(3):351.

4
Guyatt G, Jaeschke R, Heddle N, et al. Basic statistics for clinicians: 2. Interpreting study results: confidence intervals.
CMAJ: Canadian Medical Association Journal
1995;
152
(2):169.

5
Norman GR, Streiner DL.
Biostatistics: the bare essentials
. USA: PMPH-USA, 2007.

6
Bowers D.
Medical statistics from scratch: an introduction for health professionals
. Oxford: John Wiley & Sons, 2008.

7
Bland M.
An introduction to medical statistics
. Oxford: Oxford University Press, 2000.

8
Pocock SJ. When (not) to stop a clinical trial for benefit.
JAMA: The Journal of the American Medical Association
2005;
294
(17):2228–30.

9
Cuff A. Sources of Bias in Clinical Trials. 2013.
http://applyingcriticality.wordpress.com/2013/06/19/sources-of-bias-in-clinical-trials/
(accessed 26th June 2013).

10
Delgado-Rodríguez M, Llorca J. Bias.
Journal of Epidemiology and Community Health
2004;
58
(8):635–41 doi: 10.1136/jech.2003.008466.

11
Group CCS. A randomized trial of aspirin and sulfinpyrazone in threatened stroke.
The New England Journal of Medicine
1978;
299
(2):53–9.

12
Antiplatelet Trialists' Collaboration. Secondary prevention of vascular disease by prolonged antiplatelet treatment.
British Medical Journal (Clinical Research Edition)
1988;
296
(6618):320.

13
Oxman AD, Guyatt GH. A consumer's guide to subgroup analyses.
Annals of Internal Medicine
1992;
116
(1):78–84.

14
Hill AB. The environment and disease: association or causation?
Proceedings of the Royal Society of Medicine
1965;
58
(5):295.

15
Altman DG, Machin D, Bryant TN, et al.
Statistics with confidence: confidence intervals and statistical guidelines
. London: BMJ Books, 2000.

16
Fahey T, Griffiths S, Peters T. Evidence based purchasing: understanding results of clinical trials and systematic reviews.
BMJ: British Medical Journal
1995;
311
(7012):1056–9.

17
Yusuf S, Zucker D, Passamani E, et al. Effect of coronary artery bypass graft surgery on survival: overview of 10-year results from randomised trials by the Coronary Artery Bypass Graft Surgery Trialists Collaboration.
The Lancet
1994;
344
(8922):563–70.

Chapter 6

Papers that report trials of drug treatments and other simple interventions

‘Evidence’ and marketing

This chapter is about evaluating evidence from clinical trials, and most of that evidence is about drugs. If you are a clinical doctor, nurse practitioner or pharmacist (i.e. if you prescribe or dispense drugs), the pharmaceutical industry is interested in you, and spends a proportion of its multi-million pound annual advertising budget trying to influence you (see Box 6.1) [1]. Even if you are a mere patient, the industry can now target you directly through direct-to-consumer-advertising (DTCA) [2]. When I wrote the first edition of this book in 1995, the standard management of vaginal thrush (
Candida
infection) was for a doctor to prescribe clotrimazole pessaries. By the time the second edition was published in 2001, these pessaries were available over the counter in pharmacies. For the past 10 years, clotrimazole has been advertised on prime-time TV—thankfully after the nine o'clock watershed—and more recently the manufacturers of this and other powerful drugs are advertising via the Internet and social media [3]. In case you were wondering, such advertising subtly tends to place more emphasis on benefits than risks [4].

The most effective way of changing the prescribing habits of a clinician is via a personal representative (known to most of us in the UK as the ‘drug rep’ and to our North American colleagues as the ‘detailer’), who travels round with a briefcase full of ‘evidence’ in support of his or her wares [5]. Indeed, as I discuss in more detail in Chapters 14 and 15, the evidence-based medicine movement has learnt a lot from the drug industry in recent years about changing the behaviour of physicians, and now uses the same sophisticated techniques of persuasion in what is known as
academic detailing
of individual health professionals [6]. Interestingly, DTCA often works by harnessing the persuasive power of the patient—who effectively becomes an unpaid ‘rep’ for the pharmaceutical industry. If you think you'd be able to resist a patient more easily than a real rep, you're probably wrong—one randomised controlled trial showed a highly significant effect of patient power on doctors' prescribing following DTCA for antidepressants [7].

Box 6.1 Ten tips for the pharmaceutical industry: how to present your product in the best light

1.
Think up a plausible physiological mechanism why the drug works, and become slick at presenting it. Preferably, find a surrogate endpoint that is heavily influenced by the drug, although it may not be strictly valid (see section ‘Making decisions about therapy’);

2.
When designing clinical trials, select a patient population, clinical features and trial length that reflect the maximum possible response to the drug.

3.
If possible, compare your product only with placebos. If you must compare it with a competitor, make sure the latter is given at sub-therapeutic dose.

4.
Include the results of pilot studies in the figures for definitive studies, so it looks like more patients have been randomised than is actually the case.

5.
Omit mention of any trial that had a fatality or serious adverse drug reaction in the treatment group. If possible, don't publish such studies.

6.
Have your graphics department maximise the visual impact of your message. It helps not to label the axes of graphs or say whether scales are linear or logarithmic. Make sure you do not show individual patient data or confidence intervals.