How to Read a Paper: The Basics of Evidence-Based Medicine (49 page)

14
Calvert M, Shankar A, McManus RJ, et al. Effect of the quality and outcomes framework on diabetes care in the United Kingdom: retrospective cohort study.
BMJ: British Medical Journal
2009;
338
:b1870.

15
Allen D, Harkins K. Too much guidance?
The Lancet
2005;
365
(9473):1768.

16
Swinglehurst D, Roberts C, Greenhalgh T. Opening up the ‘black box’ of the electronic patient record: a linguistic ethnographic study in general practice.
Communication & Medicine
2011;
8
(1):3–15.

17
Greenhalgh T. What is this knowledge that we seek to “exchange”?
The Milbank Quarterly
2010;
88
(4):492–9 doi: 10.1111/j.1468-0009.2010.00610.x[published Online First: Epub Date].

18
Contandriopoulos D, Lemire M, DENIS JL, et al. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature.
Milbank Quarterly
2010;
88
(4):444–83.

19
Tsoukas H, Vladimirou E. What is organizational knowledge?
Journal of Management Studies
2001;
38
(7):973–3.

20
Gabbay J, May Al. Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care.
BMJ: British Medical Journal
2004;
329
(7473):1013.

21
Greenhalgh T, Wieringa S. Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review.
Journal of the Royal Society of Medicine
2011;
104
(12):501–9 doi: 10.1258/jrsm.2011.110285[published Online First: Epub Date].

22
Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map?
Journal of Continuing Education in the Health Professions
2006;
26
(1):13–24.

23
Greenhalgh T, Russell J. Evidence-based policymaking: a critique.
Perspectives in Biology and Medicine
2009;
52
(2):304–18.

24
Scheel I, Hagen K, Oxman A. The unbearable lightness of healthcare policy making: a description of a process aimed at giving it some weight.
Journal of Epidemiology and Community Health
2003;
57
(7):483–7.

25
Stone DA.
Policy paradox: the art of political decision making
. New York: WW Norton, 1997.

26
Dobrow MJ, Goel V, Upshur R. Evidence-based health policy: context and utilisation.
Social Science & Medicine
2004;
58
(1):207–17.

Appendix 1

Checklists for finding, appraising and implementing evidence

Unless otherwise stated, these checklists can be applied to randomised controlled trials, other controlled clinical trials, cohort studies, case–control studies, or any other research evidence
.

Is my practice evidence-based?—a context-sensitive checklist for individual clinical encounters (see Chapter 1)

1.
Have I identified and prioritised the clinical, psychological, social, and other problem(s), taking into account the patient's perspective?

2.
Have I performed a sufficiently competent and complete examination to establish the likelihood of competing diagnoses?

3.
Have I considered additional problems and risk factors that may need opportunistic attention?

4.
Have I, where necessary, sought evidence (from systematic reviews, guidelines, clinical trials and other sources) pertaining to the problems?

5.
Have I assessed and taken into account the completeness, quality and strength of the evidence?

6.
Have I applied valid and relevant evidence to this particular set of problems in a way that is both scientifically justified and intuitively sensible?

7.
Have I presented the pros and cons of different options to the patient in a way they can understand, and incorporated the patient's preferences into the final recommendation?

8.
Have I arranged review, recall, referral or other further care as necessary?

Checklist for searching (see Chapter 2)

1.
Decide on the purpose of your search: browsing, seeking an answer to a clinical question, or a comprehensive review (e.g. prior to undertaking a piece of research), and design your search strategy accordingly (Section ‘What are you looking for?’).

2.
Go for the highest level of evidence you can (Section ‘Levels upon levels of evidence’). For example, high-quality synthesised sources (e.g. systematic reviews and evidence-based summaries and syntheses such as Clinical Evidence or NICE guidelines, Section ‘Synthesised sources: systems, summaries and syntheses’) represent a very high level of evidence.

3.
For keeping abreast of new developments, use synopses such as POEMS (‘patient-oriented evidence that matters’),
ACP Journal Club
or
Evidence-Based Medicine
journal (Section ‘Pre-appraised sources: synopses of systematic reviews and primary studies’).

4.
Make yourself familiar with the specialised resources in your own field and use these routinely (Section ‘Specialised resources’).

5.
When searching the Medline database for primary research, you will greatly increase the efficiency of your search if you do two broad searches and then combine them, or if you use tools such as the ‘limit set’ or ‘clinical queries’ function (Section ‘Primary studies—tackling the jungle’).

6.
A very powerful way of identifying recent publications on a topic is to ‘citation chain’ an older paper (i.e. use a special electronic database to find which later papers have cited the older paper, Section ‘Primary studies—tackling the jungle’).

7.
Federated search engines such as TRIP or SUMsearch search multiple resources simultaneously and are free (Section ‘One-stop shopping: federated search engines’).

8.
Human sources (expert librarians, experts in the field) are an important component of a thorough search (Section ‘Asking for help and asking around’).

9.
To improve your skill and confidence in searching, try an online self-study course (Section ‘Online tutorials for effective searching’).

Checklist to determine what a paper is about (see Chapter 3)

1.
Why was the study performed (what clinical question did it address)?

2.
What type of study was performed?

• Primary research (experiment, randomised controlled trial, other controlled clinical trial, cohort study, case–control study, cross-sectional survey, longitudinal survey, case report, or case series)?
  
• Secondary research (simple overview, systematic review, meta-analysis, decision analysis, guideline development, economic analysis)?
  

3.
Was the study design appropriate to the broad field of research addressed (therapy, diagnosis, screening, prognosis, causation)?

4.
Did the study meet expected standards of ethics and governance?

Checklist for the methods section of a paper (see Chapter 4)

1.
Was the study original?

2.
Whom is the study about?

• How were participants recruited?
  
• Who was included in, and who was excluded from, the study?
  
• Were the participants studied in ‘real-life’ circumstances?
  

3.
Was the design of the study sensible?

• What intervention or other manoeuvre was being considered?
  
• What outcome(s) were measured, and how?
  

4.
Was the study adequately controlled?

• If a ‘randomised trial’, was randomisation truly random?
  
• If a cohort, case–control or other non-randomised comparative study, were the controls appropriate?
  
• Were the groups comparable in all important aspects except for the variable being studied?
  
• Was assessment of outcome (or, in a case–control study, allocation of caseness) ‘blind’?
  

5.
Was the study large enough, and continued for long enough, and was follow-up complete enough, to make the results credible?

Checklist for the statistical aspects of a paper (see Chapter 5)

1.
Have the authors set the scene correctly?

• Have they determined whether their groups are comparable, and, if necessary, adjusted for baseline differences?
  
• What sort of data have they got, and have they used appropriate statistical tests?
  
• If the statistical tests in the paper are obscure, why have the authors chosen to use them?
  
• Have the data been analysed according to the original study protocol?
  

2.
Paired data, tails and outliers

• Were paired tests performed on paired data?
  
• Was a two-tailed test performed whenever the effect of an intervention could conceivably be a negative one?
  
• Were outliers analysed with both common sense and appropriate statistical adjustments?
  

3.
Correlation, regression and causation

• Has correlation been distinguished from regression, and has the correlation coefficient (‘
  r
  -value’) been calculated and interpreted correctly?
  
• Have assumptions been made about the nature and direction of causality?
  

4.
Probability and confidence

• Have ‘
  p
  -values’ been calculated and interpreted appropriately?
  
• Have confidence intervals been calculated and do the authors' conclusions reflect them?
  

5.
Have the authors expressed their results in terms of the likely harm or benefit that an individual patient can expect, such as

• relative risk reduction;
  
• absolute risk reduction;
  
• number needed to treat?
  

Checklist for material provided by a pharmaceutical company representative (see Chapter 6)

See particularly
Table 6.1
for questions on randomised trials based on the CONSORT statement

1.
Does this material cover a subject that is clinically important in my practice?

2.
Has this material been published in independent peer-reviewed journals? Has any significant evidence been omitted from this presentation or withheld from publication?

3.
Does the material include high-level evidence such as systematic reviews, meta-analyses, or double-blind randomised controlled trials against the drug's closest competitor given at optimal dosage?

4.
Have the trials or reviews addressed a clearly focused, important and answerable clinical question that reflects a problem of relevance to patients? Do they provide evidence on safety, tolerability, efficacy and price?

5.
Has each trial or meta-analysis defined the condition to be treated, the patients to be included, the interventions to be compared and the outcomes to be examined?

6.
Does the material provide direct evidence that the drug will help my patients live a longer, healthier, more productive and symptom-free life?

7.
If a surrogate outcome measure has been used, what is the evidence that it is reliable, reproducible, sensitive, specific, a true predictor of disease and rapidly reflects the response to therapy?

8.
Do trial results indicate whether (and how) the effectiveness of the treatments differed and whether there was a difference in the type or frequency of adverse reactions? Are the results expressed in terms of numbers needed to treat, and are they clinically as well as statistically significant?

9.
If large amounts of material have been provided by the representative, which three papers provide the strongest evidence for the company's claims?

Checklist for a paper describing a study of a complex intervention (see Chapter 7)

1.
What is the problem for which this complex intervention is seen as a possible solution?

2.
What was done in the developmental phase of the research to inform the design of the complex intervention?