How to Read a Paper: The Basics of Evidence-Based Medicine (32 page)

Box 11.1 Examples of costs and benefits of health interventions

In the appendicitis example, few patients would perceive much freedom of choice in deciding to opt for the operation. But most health interventions do not concern definitive procedures for acutely life-threatening diseases. Most of us can count on developing at least one chronic, disabling and progressive condition such as ischaemic heart disease, high blood pressure, arthritis, chronic bronchitis, cancer, rheumatism, prostatic hypertrophy or diabetes. At some stage, almost all of us will be forced to decide whether having a routine operation, taking a particular drug or making a compromise in our lifestyle (reducing our alcohol intake or sticking to a cholesterol-lowering diet) is ‘worth it’.

It is fine for informed individuals to make choices about their own care by gut reaction (‘I’d rather live with my hernia than be cut open', or ‘I know about the risk of thrombosis but I want to continue to smoke and stay on the [contraceptive] Pill’). But when the choices are about other people's care, personal values and prejudices are the last thing that should enter the equation. Most of us would want the planners and policymakers to use objective, explicit and defensible criteria when making decisions such as, ‘No, Mrs Brown may not have a kidney transplant’.

One important way of addressing the ‘what’s it worth?' question for a given health state (such as having poorly controlled diabetes or asthma) is to ask someone in that state how they feel. A number of questionnaires that attempt to measure overall health status, such as the Nottingham Health Profile, the SF-36 general health questionnaire (widely used in the UK) and the McMaster Health Utilities Index Questionnaire (popular in north America), have been developed. For an overview of all these, see this reference textbook [4].

In some circumstances, disease-specific measures of well-being are more valid than general measures. For example, answering ‘yes’ to the question, ‘do you get very concerned about the food you are eating?’ might indicate anxiety in someone without diabetes but normal self-care attitudes in someone with diabetes [5]. There has also been an upsurge of interest in
patient-specific
measures of quality of life, to allow different patients to place different values on particular aspects of their health and well-being. When quality of life is being analysed form the point of view of the patient, this is a sensible and humane approach. However, the health economist tends to make decisions about groups of patients or populations, in which case patient-specific, and even disease-specific, measures of quality of life have limited relevance. If you would like to get up to speed in the ongoing debate on how to measure health-related quality of life, take time to look up some of the references listed at the end of this chapter [4] [6–8].

The authors of standard instruments (such as the SF-36) for measuring quality of life have often spent years ensuring they are valid (i.e. they measure what we think they are measuring), reliable (they do so every time), and responsive to change (i.e. if an intervention improves or worsens the patient's health, the scale will reflect that). For this reason, you should be highly suspicious of a paper that eschews these standard instruments in favour of the authors' own rough-and-ready scale (‘functional ability was classified as good, moderate or poor according to the clinician’s overall impression', or, ‘we asked patients to score both their pain and their overall energy level from one to ten, and added the results together’). Note also that even instruments that have apparently been well validated often do not stand up to rigorous evaluation of their psychometric validity [8].

Another way of addressing the ‘what’s it worth?' of particular health states is through
health state preference values
—that is, the value which, in a hypothetical situation, a healthy person would place on a particular deterioration in their health, or which a sick person would place on a return to health [9]. There are three main methods of assigning such values.

• Rating scale measurements
  : the respondent is asked to make a mark on a fixed line, labelled, for example, ‘perfect health’ at one end and ‘death’ at the other, to indicate where he or she would place the state in question (e.g. being wheelchair-bound from arthritis of the hip).
  
• Time trade-off measurements
  : the respondent is asked to consider a particular health state (e.g. infertility) and estimate how many of their remaining years in full health they would sacrifice to be ‘cured’ of the condition.
  
• Standard gamble measurements
  : the respondent is asked to consider the choice between living for the rest of their life in a particular health state and taking a ‘gamble’ (e.g. an operation) with a given odds of success, which would return them to full health if it succeeded but kill them if it failed. The odds are then varied to see at what point the respondent decides the gamble is not worth taking.
  

The quality-adjusted life-year or QALY can be calculated by multiplying the preference value for that state with the time the patient is likely to spend in that state. The results of cost–benefit analyses are usually expressed in terms of ‘cost per QALY’, some examples of which are shown in Box 11.2 [10–15]. The absolute cost per QALY is sometimes less important in decision-making than how much the cost per QALY differs between an old, inexpensive therapy and a new, expensive one. The new drug may be only marginally more effective but many times the price! The value used to compare whether the benefit is ‘worth it’ is known as the incremental cost-effectiveness ratio or ICER. A good example of this is the recent introduction of dabigatran (an expensive anticoagulant but one that is less hassle for the patient than warfarin, as it involves fewer blood tests), whose ICER compared to warfarin has been estimated at £13 957 [16].

Until a few years ago, one of my many ‘committee jobs’ was sitting on the Appraisals Committee of NICE—the UK National Institute for Health and Care Excellence, which advises the Department of Health on the cost-effectiveness of medicines. It is very rare for the members of that multi-disciplinary committee to get through a discussion on whether to recommend funding a controversial drug without major differences of opinion surfacing and emotions rising—and, in general, high-quality QALY data tend to generate light rather than heat in such discussions. On the one hand, any measure of health state preference values is a reflection of the preferences and prejudices of the individuals who contributed to its development. Indeed, it is possible to come up with different values for QALYs depending on how the questions from which health state preference values are derived were posed [17].

Box 11.2 Cost per QALY (see references [10–15])

Note that these are 2013 prices, so the absolute values are no longer valid; they nevertheless provide useful
relative
values for example conditions
.

As medical ethicist John Harris has pointed out, QALYs are, like the society which produces them, inherently ageist, sexist, racist and loaded against those with permanent disabilities (because even a complete cure of an unrelated condition would not restore the individual to ‘perfect health’). Furthermore, QALYs distort our ethical instincts by focusing our minds on life-years rather than people's lives. A disabled premature infant in need of an intensive care cot will, argues Harris, be allocated more resources than it deserves in comparison with a 50-year-old woman with cancer because the infant, were it to survive, would have so many more life years to quality-adjust [18].

There is an increasingly confusing array of alternatives to the QALY [4] [6, 19] [20]. Some of the ones that were in vogue when this book went to press include:

• Healthy Years Equivalent or HYE, a QALY-type measure that incorporates the individual's likely improvement or deterioration in health status in the future;
  
• Willingness to Pay (WTP) or Willingness to Accept (WTA), measures of how much people would be prepared to pay to gain certain benefits or avoid certain problems;
  
• Disability-Adjusted Life Year or DALY, used mainly in the developing world to assess the overall burden of chronic disease and deprivation—an increasingly used measure that is not without its critics; and, perhaps most bizarrely
  
• TWiST (time spent without symptoms of disease and toxicity of treatment) and Q-TWiST (quality-adjusted TWiST)!
  

My personal advice on all these measures is to look carefully at what goes into the number that is supposed to be an ‘objective’ indicator of a person's (or population's) health status, and at how the different measures might differ according to different disease states. In my view, they all have potential uses but none of them is an absolute or incontrovertible measure of health or illness! (Note, also, that I do not claim to be an expert on any of these measures or on how to calculate them—which is why I have offered a generous list of additional references at the end of this chapter).

There is, however, another form of analysis that, although it does not abolish the need to place arbitrary numerical values on life and limb, avoids the buck stopping with the unfortunate health economist. This approach, known as
cost–consequences analysis
, presents the results of the economic analysis in a disaggregated form. In other words, it expresses different outcomes in terms of their different natural units (i.e. something real such as months of survival, legs amputated or take-home babies), so that individuals can assign their own values to particular health states before comparing two quite different interventions (e.g. infertility treatment vs cholesterol lowering, as in the example I mentioned in Chapter 1). Cost–consequences analysis allows for the health state preference values of both individuals and society to change with time, and is particularly useful when these are disputed or likely to change. This approach may also allow the analysis to be used by different groups or societies from the ones on which the original trial was performed.

Ten questions to ask about an economic analysis

The elementary checklist that follows is based largely on the sources mentioned in the first paragraph of this chapter. I strongly recommend that for a more definitive list, you check out these sources—especially the official recommendations by the BMJ working group [1].

Question One: Is the analysis based on a study that answers a clearly defined clinical question about an economically important issue?

Before you attempt to digest what a paper says about costs, quality of life scales or utilities, make sure that the trial being analysed is scientifically relevant and capable of giving unbiased and unambiguous answers to the clinical question posed in its introduction (see Chapter 4). Furthermore, if there is clearly little to choose between the interventions in terms of either costs or benefits, a detailed economic analysis is probably pointless.