How to Read a Paper: The Basics of Evidence-Based Medicine (28 page)

If you found yourself sympathising with Professor Eysenck in the previous section, you might like to look at some other theoretical critiques of systematic review. MacLure [30] has written an excellent philosophical article claiming that with its overemphasis on protocols and procedures, a conventional systematic review degrades the status of interpretive scholarly activities such as reading, writing and talking, and replaces them with a series of auditable technical tasks. This change, she claims, is partly driven by the new managerialism in research and results in ‘the call-centre version of research synthesis’. I once wrote a short commentary called
Why are Cochrane Reviews so boring?
, arguing that an overly technocratic approach to data extraction and synthesis strips the
meaning
from a review [20]. But whilst this may be true and MacLure may have a point, we shouldn't throw the baby out with the bath water. Systematic review, in its place, saves lives.

References

1
Caveman A. The invited review? or, my field, from my standpoint, written by me using only my data and my ideas, and citing only my publications.
Journal of Cell Science
2000;
113
(Pt 18):3125.

2
Pauling L.
How to live longer and feel better
, Portland, Oregon. Oregon State University Press. 1986;3125–3126.

3
McAlister FA, Clark HD, van Walraven C, et al. The medical review article revisited: has the science improved?
Annals of Internal Medicine
1999;
131
(12):947–51.

4
Oxman AD, Guyatt GH. The science of reviewing research.
Annals of the New York Academy of Sciences
1993;
703
(1):125–34.

5
Antman EM, Lau J, Kupelnick B, et al. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts.
JAMA: The Journal of the American Medical Association
1992;
268
(2):240–8.

6
Knipschild P. Systematic reviews. Some examples.
BMJ: British Medical Journal
1994;
309
(6956):719–21.

7
Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
Annals of Internal Medicine
2009;
151
(4):264–9.

8
Bruins Slot KM, Berge E, Saxena R, et al. Oral anticoagulants versus antiplatelet therapy for preventing stroke and systemic embolic events in patients with atrial fibrillation. Cochrane Database of Systematic Reviews 2012, Issue 2. Art. No.: CD009538. DOI: 10.1002/14651858.CD009538.

9
Greenhalgh T, Robert G, Macfarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations.
The Milbank Quarterly
2004;
82
(4):581–629 doi: 10.1111/j.0887-378X.2004.00325.x.

10
Morrison A, Polisena J, Husereau D, et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies.
International Journal of Technology Assessment in Health Care
2012;
28
(2):138–44.

11
Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources.
BMJ: British Medical Journal
2005;
331
(7524):1064–5.

12
Higgins JPT, Green S.
Cochrane handbook for systematic reviews of interventions version 5.1. 0 [updated March 2011]
. Oxford: The Cochrane Collaboration, 2011.

13
Counsell CE, Clarke MJ, Slattery J, et al. The miracle of DICE therapy for acute stroke: fact or fictional product of subgroup analysis?
BMJ: British Medical Journal
1994;
309
(6970):1677.

14
Cuijpers P, Hollon SD, van Straten A, et al. Does cognitive behaviour therapy have an enduring effect that is superior to keeping patients on continuation pharmacotherapy? A meta-analysis.
BMJ Open
2013;
3
(
4
) doi: 10.1136/bmjopen-2012-002542[published Online First: Epub Date].

15
Egger M, Smith GD, Altman D.
Systematic reviews in health care: meta-analysis in context
. Chichester: Wiley.com, 2008.

16
Fergusson D, Glass KC, Hutton B, et al. Randomized controlled trials of aprotinin in cardiac surgery: could clinical equipoise have stopped the bleeding?
Clinical Trials
2005;
2
(3):218–32.

17
Stewart LA, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data.
Evaluation & the Health Professions
2002;
25
(1):76–97.

18
Borenstein M, Hedges LV, Higgins JP, et al.
Introduction to meta-analysis
. Chichester: Wiley.com, 2011.

19
Thompson SG. Why and how sources of heterogeneity should be investigated. In: Egger M, Davey Smith G, Altman DG, eds.
Systematic reviews in health care: meta-analysis in context
. London: BMJ Publications, 2001;157–175.

20
Greenhalgh T. Outside the box: why are Cochrane reviews so boring?
The British Journal of General Practice
2012;
62
(600):371;157–175.

21
Eysenck H. Problems with meta-analysis. In: Chalmers I, Altman DG, eds.
Systematic reviews
. London: BMJ Publications, 1995.

22
Higgins JP, Spiegelhalter DJ. Being sceptical about meta-analyses: a Bayesian perspective on magnesium trials in myocardial infarction.
International Journal of Epidemiology
2002;
31
(1):96–104.

23
Egger M, Smith GD. Misleading meta-analysis.
BMJ: British Medical Journal
1995;
311
(7007):753–4.

24
Hennekens CH, DeMets D. The need for large-scale randomized evidence without undue emphasis on small trials, meta-analyses, or subgroup analyses.
JAMA: The Journal of the American Medical Association
2009;
302
(21):2361–2.

25
Griffin S, Greenhalgh T. Diabetes care in general practice: meta-analysis of randomised control trials Commentary: meta-analysis is a blunt and potentially misleading instrument for analysing models of service delivery.
BMJ: British Medical Journal
1998;
317
(7155):390–6.

26
Greenhalgh T. Commentary: meta-analysis is a blunt and potentially misleading instrument for analysing models of service delivery.
BMJ: British Medical Journal (Clinical research ed.)
1998;
317
(7155):395–6.

27
Devillé WL, Buntinx F, Bouter LM, et al. Conducting systematic reviews of diagnostic studies: didactic guidelines.
BMC Medical Research Methodology
2002;
2
(1):9.

28
Wong G, Greenhalgh T, Westhorp G, et al. RAMESES publication standards: meta-narrative reviews. BMC Med 2013;11:20 doi: 10.1186/1741-7015-11-20.

29
Wong G, Greenhalgh T, Westhorp G, et al. RAMESES publication standards: realist syntheses. BMC Med 2013;11:20 doi: 10.1186/1741-7015-11-21.

30
MacLure M.
‘Clarity bordering on stupidity’: where's the quality in systematic review? Journal of Education Policy
2005;
20
(4):393–416.

Chapter 10

Papers that tell you what to do (guidelines)

The great guidelines debate

Never was the chasm between front-line clinicians and back-room policymakers wider than in their respective attitudes to clinical guidelines. Policymakers (by which I include everyone who has a view on how medicine ought to be practised in an ideal world—including politicians, senior managers, clinical directors, academics and teachers) tend to love guidelines. Front-line clinicians (i.e. people who spend all their time seeing patients) often have a strong aversion to guidelines.

Before we carry this political hot potato any further, we need a definition of guidelines, for which the following will suffice.

Guidelines are systematically developed statements to assist practitioner decisions about appropriate health care for specific clinical circumstances
.

A great paper on evidence-based guidelines (what they are, how they're developed, why we need them and what the controversies are) was published recently by one of my colleagues, Dr Deborah Swinglehurst [1]. I have drawn extensively on her review when updating this chapter. One important distinction Deborah makes in her paper is between guidelines (which are usually expressed in terms of general principles and leave room for judgement within broad parameters) and protocols, which she defines as follows: ‘Protocols are instructions on what to do in particular circumstances. They are similar to guidelines but include less room for individual judgement, are often produced for less experienced staff, or for use in situations where eventualities are predictable’.

Box 10.1 Purpose of guidelines

1.
To make evidence-based standards explicit and accessible (but see subsequent text: few guidelines currently in circulation are truly evidence-based).

2.
To make decision making in the clinic and at the bedside easier and more objective.

3.
To provide a yardstick for assessing professional performance.

4.
To delineate the division of labour (e.g. between general practitioners (GPs) and consultants).

5.
To educate patients and professionals about current best practice.

6.
To improve the cost-effectiveness of health services.

7.
To serve as a tool for external control.

The purposes that guidelines serve are given in Box 10.1. Clinician resistance to guidelines has a number of explanations [2–7].

• Clinical freedom (‘I’m not having anyone telling me how to manage my patients'.)
  
• Debates amongst experts about the quality of evidence (‘Well, if they can’t agree among themselves … ')
  
• Lack of appreciation of evidence by practitioners (‘That’s all very well, but when I trained we were always taught to hold back on steroids for asthma'.)
  
• Defensive medicine (‘I’ll check all the tests anyway—belt and braces'.)
  
• Strategic and cost constraints (‘We can’t afford to replace the equipment'.)
  
• Specific practical constraints (‘Where on earth did I put those guidelines’?)
  
• Reluctance of patients to accept procedures (‘Mrs Brown insists she needs a cervical smear every year’.)
  
• Competing influences of other non-medical factors (‘When we get the new computer system up and running … ’)
  
• Lack of appropriate, patient-specific feedback on performance (‘I seem to be treating this condition OK’.)
  
• Confusion (‘The guideline doesn’t seem to help me with the problem I'm facing'.)
  

The image of the medical buffoon blundering blithely through the outpatient clinic still diagnosing the same illnesses and prescribing the same drugs he (or she) learnt about at medical school 40 years previously, and never having read a paper since, knocks the ‘clinical freedom’ argument right out of the arena. Such hypothetical situations are grist to the mill of those who would impose ‘expert guidelines’ on most, if not all, medical practice and hold to account all those who fail to keep in step.

Box 10.2 Drawbacks of guidelines (real and perceived)

1.
Guidelines may be intellectually suspect and reflect ‘expert opinion’, which may formalise unsound practice.

2.
By reducing medical practice variation they may standardise to ‘average’ rather than best practice.

3.
They might inhibit innovation and prevent individual cases from being dealt with discretely and sensitively.

4.
Guidelines developed at national or regional level may not reflect local needs or have the ‘ownership’ of local practitioners.

5.
Guidelines developed in secondary care may not reflect demographic, clinical or practical differences between this setting and the primary care setting.