TORONTO
This is the sixth in a series of 10 articles introducing non-experts to finding medical articles and assessing their value
Pharmaceutical "reps" are now much more informative than they used to be, but they may show ignorance of basic epidemiology and clinical trial design
The value of a drug should be expressed in terms of safety, tolerability, efficacy, and price
The efficacy of a drug should ideally be measured in terms of clinical end points that are relevant to patients; if surrogate end points are used they should be valid
Promotional literature of low scientific validity (such as uncontrolled before and after trials) should not be allowed to influence practice
Pharmaceutical "reps" do not tell nearly as many lies as they used to (drug marketing has become an altogether more sophisticated science), but they have been known to cultivate a shocking ignorance of basic epidemiology and clinical trial design when it suits them. (2) It often helps their case, for example, to present the results of uncontrolled trials and express them in terms of before and after differences in a particular outcome measure. (3) The recent correspondence in the Lancet and BMJ on placebo effects should remind you why uncontrolled before and after studies are the stuff of teenage magazines, not hard science. (4-12)
- identify, for this patient, the ultimate objective of treatment (cure, prevention of recurrence, limitation of functional disability, prevention of later complications, reassurance, palliation, relief of symptoms, etc);
- select the most appropriate treatment, using all available evidence (this includes considering whether the patient needs to take any drug at all); and
- specify the treatment target (to know when to stop treatment, change its intensity, or switch to some other treatment). (13)
For example, in treating high blood pressure, the doctor might decide that:
- the ultimate objective of treatment is to prevent (further) target organ damage to brain, eye, heart, kidney, etc (and thereby prevent death);
- the choice of specific treatment is between the various classes of antihypertensive drug selected on the basis of randomised, placebo controlled and comparative trials-as well as non-drug treatments such as salt restriction; and
- the treatment target might be a phase V diastolic blood pressure (right arm, sitting) of less than 90 mm Hg, or as close to that as tolerable in the face of drug side effects.
If these three steps are not followed (as is often the case-for example in terminal care), therapeutic chaos can result.
- they can considerably reduce the sample size, duration, and, therefore, cost, of clinical trials; and
- they can allow treatments to be assessed in situations where the use of primary outcomes would be excessively invasive or unethical.
In the evaluation of pharmaceutical products, commonly used surrogate end points include:
- pharmacokinetic measurements (for example, concentration-time curves of a drug or its active metabolite in the bloodstream);
- in vitro (laboratory) measures such as the mean inhibitory concentration of an antimicrobial against a bacterial culture on agar;
- macroscopic appearance of tissues (for example, gastric erosion seen at endoscopy);
- change in levels of (alleged) serum markers of disease (for example, prostate specific antigen (14));
- radiological appearance (for example, shadowing on a chest x ray film).
But surrogate end points have some drawbacks. Firstly, a change in the surrogate end point does not itself answer the essential preliminary questions: "what is the objective of treatment in this patient?" and "what, according to valid and reliable research studies, is the best available treatment for this condition?" Secondly, the surrogate end point may not closely reflect the treatment target-in other words, it may not be valid or reliable. Thirdly, overreliance on a single surrogate end point as a measure of therapeutic success usually reflects a narrow clinical perspective. Finally, surrogate end points are often developed in animal models of disease, since changes in a specific variable can be measured under controlled conditions in a well defined population. However, extrapolation of these findings to human disease is likely to be invalid. (15-17)
One important example of the invalid use of a surrogate end point is the CD4 cell count in monitoring progression to AIDS in HIV positive subjects. The CONCORDE trial was a randomised controlled trial comparing early and late start of treatment with zidovudine in patients who were HIV positive but clinically asymptomatic. (18) Previous studies had shown that starting treatment early led to a slower decline in the CD4 cell count (a variable which had been shown to fall with the progression of AIDS), and it was assumed that a higher CD4 cell count would reflect improved chances of survival.
However, the CONCORDE trial showed that, although CD4 cell counts fell more slowly in the treatment group, the three year survival rates were identical in the two groups. This experience confirmed a warning that was issued earlier by authors suspicious of the validity of this end point. (19) Subsequent research in this field has attempted to identify a surrogate end point that correlates with real therapeutic benefit-that is, delayed progression of asymptomatic HIV infection to clinical AIDS, and longer survival time after the onset of AIDS. (20,21) Using multiple regression analysis, investigators in the USA found that a combination of markers (percentage of CD4:C29 cells, degree of fatigue, age, and haemoglobin concentration) was the best predictor of progression. (20)
Other examples of surrogate end points which have seriously misled researchers include ventricular premature beats as a predictor of death from serious cardiac arrhythmias, (22,23) blood concentrations of antibiotics as a predictor of clinical cure of infection, (24) and plaques seen on magnetic resonance imaging in monitoring the progression of multiple sclerosis. (25)
Before surrogate end points can be used in the marketing of pharmaceuticals, those in the industry must justify the utility of these measures by showing a plausible and consistent link between the end point and the development or progression of disease. It would be wrong to suggest that the pharmaceutical industry develops surrogate end points with the deliberate intention to mislead the licensing authorities and health professionals. However, the industry does, theoretically, have a vested interest in overstating its case on the significance of these end points. Given that much of the data relating to the validation of surrogate end points are not currently presented in published clinical papers, and that the development of such markers is often a lengthy and expensive process, one author has suggested setting up a data archive that would pool data across studies. (26)
- See representatives only by appointment. Choose to see only those whose product interests you, and confine the interview to that product
- Take charge of the interview. Do not hear out a rehearsed sales routine but ask directly for the information below
- Request independent published evidence from reputable, peer reviewed journals
- Do not look at promotional brochures, which may contain unpublished material, misleading graphs, and selective quotations
- Ignore anecdotal "evidence," such as the fact that a medical celebrity is prescribing the product
- Using the STEP acronym, ask for evidence in four specific areas:
Safety-the likelihood of long term or serious side effects caused by the drug (remember that rare but serious adverse reactions to new drugs may be poorly documented)
Tolerability-best measured by comparing the pooled withdrawal rates between the drug and its most significant competitor
Efficacy-the most relevant dimension is how the product compares with your current favourite
Price-should take into account indirect as well as direct costs
- Evaluate the evidence stringently, paying particular attention to the power (sample size) and methodological quality of clinical trials, and the use of surrogate end points. Do not accept theoretical arguments in the drug's favour ("longer half life," for example) without direct evidence that this translates into clinical benefit
- Do not accept the newness of a product as an argument for changing to it. Indeed, there are good scientific arguments for doing the opposite (28)
- Decline to try the product via starter packs or by participating in small scale, uncontrolled "research" studies
- Record in writing the content of the interview and return to these notes if the "rep" requests another audience
In conclusion, it is often more difficult than you are being led to believe to weigh the potential benefits of a drug against its risks to the patient and cost to the taxpayer. (29) The difference between the science of critical appraisal and the pharmaceutical industry's well rehearsed tactics of marketing and persuasion should be borne in mind when you are considering "evidence" presented by those with a commercial conflict of interest.
I am grateful to Dr Andrew Herxheimer for advice on this article.
The articles in this series are excerpts from How to Read a Paper: the Basic of Evidence Based Medicine. The book includes chapters on searching the literature and implementing evidence based findings. It can be ordered from the BMJ Publishing Group: tel 0171 383 6185/6245; fax 0171 383 6662. Price [pound sign]13.95 for UK members, [pound sign]14.95 for non-members.
2. Bardelay D. Visits from medical representatives: fine principles, poor practice. Prescrire International 1995;4:120-2. [Back to "Evidence" and marke..]
3. Bero LA, Rennie D. Influences on the quality of published drug studies. Int J Health Technology Assessment 1996;12:209-37. [Back to "Evidence" and marke.., How to get evidence ..]
4. Kleijnen J, de Craen AJ, van Everdingen J, Krol L. Placebo effect in double-blind clinical trials: a review of interactions with medications. Lancet 1994;344:1347-9. [Back to "Evidence" and marke..]
5. Joyce CR. Placebo and complementary medicine. Lancet 1994;344:1279-81. [Back to "Evidence" and marke..]
6. Laporte JR. Figueras A. Placebo effects in psychiatry. Lancet 1994;344:1206-9. [Back to "Evidence" and marke..]
7. Johnson AG. Surgery as a placebo. Lancet 1994;344:1140-2. [Back to "Evidence" and marke..]
8. Thomas KB. The placebo in general practice. Lancet 1994;344:1066-7. [Back to "Evidence" and marke..]
9. Chaput de Saintonge DM. Herxheimer A. Harnessing placebo effects in health care. Lancet 1994;344:995-8. [Back to "Evidence" and marke..]
10. Gotzsche PC. Is there logic in the placebo? Lancet 1994;344:925-6. [Back to "Evidence" and marke..]
11. Rothman KJ. Placebo mania. BMJ 1996;313:3-4. [Back to "Evidence" and marke..]
12. McQuay II, Moore A, Double DB, Georgiou A, Korkia P. Placebo mania. BMJ 1996;313:1008-9. [Back to "Evidence" and marke..]
13. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology-a basic science for clinical medicine. London, Little, Brown, 1991:187-248. [Back to Making decisions abo..]
14. Bostwick DG, Burke HB, Wheeler TM, Chung LW, Bookstein R, Pretlow TG, et al. The most promising surrogate endpoint biomarkers for screening candidate chemopreventive compounds for prostatic adenocarcinoma in short-term Phase II clinical trials. J Cell Biochem 1994;56(suppl 19):283-9. [Back to Surrogate end points]
15. Gotzsche P, Liberati A, Torri V, Rosetti L. Beware of surrogate outcome measures. Int J Health Technology Assessment 1996;12:238-46. [Back to Surrogate end points]
16. Lipkin M. Summary of recommendations for colonic biomarker studies of candidate chemopreventive compounds in Phase II clinical trials. J Cell Biochem 1994;56(suppl 19):94-8. [Back to Surrogate end points]
17. Kimbrough RD. Determining acceptable risks: experimental and epidemiological issues. Clin Chem 1994;40:1448-53. [Back to Surrogate end points]
18. CONCORDE Co-ordinating Committee. CONCORDE MRC/ANRS randomised double-blind controlled trial of immediate and deferred zidovudine in symptom-free HIV infection. Lancet 1994;343:871-81. [Back to Surrogate end points:Features of the idea..]
19. Jacobson MA, Bacchetti P, Kolokathis A et al. Surrogate markers for survival in patients with AIDS and AIDS related complex treated with zidovudine. BMJ 1991;302:73-8. [Back to Surrogate end points:Features of the idea..]
20. Blatt SP, McCarthy WF, Bucko-Krasnicka B, Melcher GP, Boswell RN, Dolan J, et al. Multivariate models for predicting progression to AIDS and survival in HIV-infected patients. J Infect Dis 1995;171:837-44. [Back to Surrogate end points:Features of the idea..]
21. Tsoukas CM, Bernard NF. Markers predicting progression of HIV-related disease. Clin Microbiol Rev 1994;7:14-28. [Back to Surrogate end points:Features of the idea..]
22. Epstein AE, Hallstrom AO, Rogers WJ, Liebson PR, Seals AA, Anderson JL, et al. Mortality following ventricular arrhythrnia suppression by encainide, flecainide and moricizine after myocardial infarction. JAMA 1993, 270, 2451-55. [Back to Surrogate end points:Features of the idea..]
23. Lipicky RJ, Packer M. Role of surrogate endpoints in the evaluation of drugs for heart failure. J Am Coll Cardiol 1993;22(suppl A):179-84. [Back to Surrogate end points:Features of the idea..]
24. Hyatt JM, McKinnon PS, Zimmer GS, Schentag JJ. The importance of pharmacokinetic/pharmacodynamic surrogate markers to outcome. Focus on antibacterial agents. Clin Pharmacokinetics 1995;28:143-60. [Back to Surrogate end points:Features of the idea..]
25. Interferon beta-1b-hope or hype? Drug Ther Bull 1996;34:9-11. [Back to Surrogate end points:Features of the idea..]
26. Aickin M. If there is gold in the labelling index hills, are we digging in the right place? J Cell Biochem 1994;56(suppl 19):91-3. [Back to Surrogate end points:Features of the idea..]
27. Getting good value from drug reps. Drug Ther Bull 1983;21:13-5. [Back to How to get evidence ..]
28. Ferner RE. Newly licensed drugs. BMJ 1996;313:1157-8. [Back to How to get evidence ..]
29. Risk:benefit analysis of drugs in practice. Drug Ther Bull 1995;33:33-5. [Back to How to get evidence ..]
