Read How to Read a Paper: The Basics of Evidence-Based Medicine Online
Authors: Trisha Greenhalgh
Domain 1: Scope and purpose
1.
The overall objective(s) of the guideline is(are) specifically described.
2.
The health question(s) covered by the guideline is(are) specifically described.
3.
The population to whom the guideline is meant to apply are specifically described.
Domain 2: Stakeholder involvement
1.
The guideline development group includes individuals from all the relevant professional groups.
2.
The views and preferences of the target population have been sought.
3.
The target users of the guideline are clearly defined.
Domain 3: Rigour of development
1.
Systematic methods were used to search for evidence.
2.
The criteria for selecting the evidence are clearly described.
3.
The strengths and limitations of the body of evidence are clearly described.
4.
The methods used for formulating the recommendations are clearly described.
5.
The health benefits, side effects and risks have been considered in formulating the recommendations.
6.
There is an explicit link between the recommendations and the supporting evidence.
7.
The guideline has been externally reviewed by experts prior to its publication.
8.
A procedure for updating the guideline is provided.
Domain 4: Clarity and presentation
1.
The recommendations are specific and unambiguous.
2.
The different options for management of the condition or health issue are clearly presented.
3.
Key recommendations are easily identifiable.
Domain 5: Applicability
1.
The guideline provides advice or tools to support its implementation.
2.
The guideline describes facilitators of, and barriers to, adoption.
3.
Potential resource implications of applying the recommendations have been considered.
4.
The guideline presents monitoring or auditing criteria.
Domain 6: Editorial independence
1.
The views of the funding body have not influenced the content of the guideline.
2.
Competing interests of members of the guideline development group have been recorded and addressed.
I have drawn on many of the other articles referenced in this chapter as well as the relatively new AGREE instrument.
Question One: Did the preparation and publication of this guideline involve a significant conflict of interest?
I will resist labouring the point, but a drug company that makes hormone replacement therapy or a research professor whose life's work has been spent perfecting this treatment might be tempted to recommend it for wider indications than the average clinician. Much has been written about the ‘medicalisation’ of human experience (are energetic children with a short attention span ‘hyperactive’; should women with low sex drive be offered ‘treatment’, etc.). A guideline may be evidence-based, but the problem it addresses will have been constructed by a team that views the world in a particular way.
Question Two: Is the guideline concerned with an appropriate topic, and does it state clearly the target group it applies to?
Key questions in relation to choice of topic, reproduced from an article published a few years ago in the
British Medical Journal
[18], are given in Box 10.5.
The Grimley Evans quote on page 137 begs the question ‘To whom does this guideline apply?’. If the evidence related to people aged 18–65 with no comorbidity (i.e. with nothing else wrong with them except the disease being considered), it might not apply to your patient. Sometimes this means you will need to reject it outright, but more commonly, you will have to exercise your judgement in assessing its transferability.
Box 10.5 Key questions on choice of topic for guideline development (see reference [17])
Question Three: Did the guideline development panel include [a] an expert in the topic area; [b] a specialist in the methods of secondary research (e.g. meta-analyst, health economist) and [c] a person affected by the condition?
If a clinical guideline has been prepared entirely by a panel of internal ‘experts’, you should, paradoxically, look at it particularly critically as researchers have been shown to be less objective in appraising evidence in their own field of expertise than in someone else's. The involvement of an outsider (an expert in guideline development rather than in the particular clinical topic) to act as arbiter and methodological adviser should make the process more objective. But as Gabbay and his team [19] showed in an elegant qualitative study, the hard-to-measure expertise (what might be called
embodied knowledge
) of front-line clinicians (in this case, GPs) contributed crucially to the development of workable local guidelines. But all the objective expertise in the world is no substitute for having the condition in question yourself, and emerging evidence suggests that patients and carers bring a crucial third perspective to the guideline development process [20].
Question Four: Have the subjective judgements of the development panel been made explicit, and are they justified?
Guideline development is not just a technical process of finding evidence, appraising it and turning it into recommendations. Recommendations also require judgements (relating to personal or social values, ethical principles, etc.). As the UK National Institute for Health and Care Excellence (NICE) has stated (see
www.nice.org.uk
), it is right and proper for guideline developers to take account of the ‘ethical principles, preferences, culture and aspirations that should underpin the nature and extent of care provided by the National Health Service’. Swinglehurst [1] suggests four sub-questions to ask about these subjective judgements.
Question Five: Have all the relevant data been scrutinised and rigorously evaluated?
The academic validity of guidelines depends (among other things) on whether they are supported by high-quality primary research studies, and on how strong the evidence from those studies is. At the most basic level, was the literature analysed at all, or are these guidelines simply a statement of the preferred practice of a selected panel of experts (i.e. consensus guidelines)? If the literature was looked at, was a systematic search performed, and if so, did it broadly follow the method described in section ‘Evaluating systematic reviews’? Were all papers unearthed by the search included, or was an explicit scoring system (such as GRADE [21]) used to reject those of poor methodological quality and give those of high quality the extra weight they deserved?
Up-to-date systematic reviews should ideally be the raw material for guideline development. But in many cases, a search for rigorous and relevant research on which to base guidelines proves fruitless, and the authors, unavoidably, resort to ‘best available’ evidence or expert opinion.
Question Six: Has the evidence been properly synthesised, and are the guideline's conclusions in keeping with the data on which they are based?
Another key determinant of the validity of a guideline is how the different studies contributing to it have been pulled together (that is, synthesised) in the context of the clinical and policy needs being addressed. For one thing, a systematic review and meta-analysis might have been appropriate, and if the latter, issues of probability and confidence should have been dealt with acceptably (see section ‘Summing up’).
But systematic reviews don't exist (and never will exist) to cover every eventuality in clinical decision-making and policymaking. In many areas, especially complex ones, the opinion of experts is still the best ‘evidence’ around, and in such cases guideline developers should adopt rigorous methods to ensure that it isn't just the voice of the expert who talks for longest in the meetings that drives the recommendations. Formal guideline development groups usually have an explicit set of methods—see, for example, this paper from the UK NICE [22].
A recent analysis of three ‘evidence-based’ guidelines for obstructive sleep apnoea found that they made very different recommendations despite being based on an almost identical set of primary studies. The main reason for the discrepancy was that experts tended to rank studies from their own country more highly [23]!
Question Seven: Does the guideline address variations in medical practice and other controversial areas (e.g. optimum care in response to genuine or perceived underfunding)?
It would be foolish to make dogmatic statements about ideal practice without reference to what actually goes on in the real world. There are many instances where some practitioners are marching to an altogether different tune from the rest of us (see section ‘Why do people sometimes groan when you mention evidence-based medicine?’), and a good guideline should face such realities head on rather than hoping that the misguided minority will fall into step by default.
Another thorny issue that guidelines should tackle head-on is where essential compromises should be made if financial constraints preclude ‘ideal’ practice. If the ideal, for example, is to offer all patients with significant coronary artery disease a bypass operation (at the time of writing it isn't, but never mind), and the health service can only afford to fund 20% of such procedures, who should be pushed to the front of the queue?
Question Eight: Is the guideline clinically relevant, comprehensive and flexible?
In other words, is it written from the perspective of the practising doctor, nurse, midwife, physiotherapist, and so on, and does it take account of the type of patients he or she is likely to see, and in what circumstances? Perhaps the most frequent source of trouble here is when guidelines developed in secondary care and intended for use in hospital outpatients (who tend to be at the sicker end of the clinical spectrum) are passed on to the primary health care team with the intention of their being used in the primary care setting, where, in general, patients are less ill and may well need fewer investigations and less aggressive management. This issue is discussed in section ‘Validating diagnostic tests against a gold standard’ in relation to the different utilities of diagnostic and screening tests in different populations.
Guidelines should cover all, or most, clinical eventualities. What if the patient is intolerant of the recommended medication? What if you can't send off all the recommended blood tests? What if the patient is very young, very old, or suffers from a coexisting illness? These, after all, are the patients who prompt most of us to reach for our guidelines; while the more ‘typical’ patient tends to be managed without recourse to written instructions. Recent work by Shekelle's team [2] has added a crucial factor—multimorbidity—to the barriers to following guidelines: sometimes the patient has a condition that makes the standard recommended therapy impossible to apply. It follows that guideline developers should routinely consider common comorbidities when they set their recommendations.
Flexibility is a particularly important consideration for national and regional bodies who set themselves up to develop guidelines. As noted earlier, ownership of guidelines by the people who are intended to use them locally is crucial to whether the guidelines are actually used. If there is no free rein for practitioners to adapt them to meet local needs and priorities, a set of guidelines will probably never get taken out of the drawer.
Question Nine: Does the guideline take into account what is acceptable to, affordable by and practically possible for patients?
There is an apocryphal story of a physician in the 1940s (a time when no effective medicines for high blood pressure were available), who discovered that restricting the diet of hypertensive patients to plain, boiled, unsalted rice dramatically reduced their blood pressure and also reduced the risk of stroke. The story goes, however, that the diet made the patients so miserable that many of them committed suicide.