How Should Clinicians Consider the Results of a Drug Clinical Trial?

Module 4Submodule 4.2

Objective

To enhance prescribers’ ability to:– Critically evaluate clinical trial design

and results – Determine whether or not the results

are sufficiently compelling to change current clinical practice

CONSORT Statement Checklist

• Title & Abstract• Introduction & Background • Methods

– Participants, interventions, measurements, outcomes, sample size, randomization, blinding, statistical methods

• Results– Participant flow, recruitment, baseline data, # participants

analyzed, outcomes & estimation, ancillary analysis, adverse events

• Discussion & Comments– Interpretation, generalizability & overall evidence

Ann Intern Med, 2001;134 (8):657-662. www.annals.org

Selection of Unbiased InformationCheck the title of the papers

•Are they interesting?•Are they useful?•Are they relevant to your practice?

Review the sponsor and authors for any financial relationship with the manufacturer.

• Was the study sponsored by a manufacturer?

• Were the authors independent from the sponsor to: design, organize, analyze, select the results and discuss findings?

• Did the manufacturer review and edit the manuscript?

• Did the manufacturer require written approval of the manuscript before submission for publication?

• Is there a potential for conflict of interest of investigators?

Yes

Go to the next paper

No

ExamplePiperacillin/tazobactam vs imipenem/cilastatin in the treatment of nosocomial pneumonia--a double blind prospective multicentre study. Schmitt DV, et al. Infection. 2006 Jun;34(3):127-34. Effectiveness of discontinuing antibiotic treatment after three days versus eight days in mild to moderate-severe community acquired pneumonia: randomised, double blind study. el Moussaoui R, et al BMJ. 2006 Jun 10;332(7554):1355.

Novel, single-dose microsphere formulation of azithromycin versus 7-day levofloxacin therapy for treatment of mild to moderate community-acquired Pneumonia in adults. D'Ignazio J, et al. Antimicrob Agents Chemother. 2005 Oct;49(10):4035-41. Combination therapy versus monotherapy: a randomised pilot study on the evolution of inflammatory parameters after ventilator associated pneumonia. Damas P, et al. Crit Care. 2006;10(2):R52.

Single-dose azithromycin microspheres vs clarithromycin extended release for the treatment of mild-to-moderate community-acquired pneumonia in adults. Drehobl MA, et al. Chest. 2005 Oct;128(4):2230-7. Community-Acquired Pneumonia Recovery in the Elderly Study Group. Community-Acquired Pneumonia Recovery in the Elderly (CAPRIE): efficacy and safety of moxifloxacin therapy versus that of levofloxacin therapy. Anzueto A, Net al. Clin Infect Dis. 2006 Jan 1;42(1):73-81.

Check the title of the papers

•Are they interesting?•Are they useful?•Are they relevant?

Why is the Author’s Disclosure Important?

“Analysis of randomized clinical trials published in the literature, either medical or surgical specialties, has shown that industry-funded trials are more likely to be associated with statistically significant pro-industry findings” (JAMA 2003;290:921-8)

BMJ 2002;325:249-53, BMC Health Serv Res 2002;2:18-24, JAMA 2003;289:454-465, JAMA 1999;282:1474-5.

Abstract

• Was the research question of interest to you? • Were the clinically relevant outcomes included?• Were the treatment and outcome adequately measured? • Were the study patients similar to your own?• Is the treatment accessible, acceptable and affordable?

Abstract (Example)A Randomized, Double Blind, Placebo Controlled Trial of a Topical Cream

Containing Glucosamine Sulfate, Chondroitin Sulfate, and Camphor for Osteoarthritis of the Knee. Cohen M, Wolfe R, Mai T, Lewis D.

Objective. To assess the ability of a topical preparation of glucosamine sulfate and chondroitin sulfate to reduce pain related to osteoarthritis (OA) of the knee.

Methods. Sixty-three patients were randomized to receive either a topical glucosamine and chondroitin preparation or placebo to be used as required over an 8 week period. Efficacy was assessed using a visual analog scale (VAS) for pain as well as the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC), and the SF-36 questionnaire.

Results. VAS scores indicated a greater mean reduction in pain for the glucosamine/chondroitin preparation group (mean change -3.4 cm, SD 2.6 cm) compared to the placebo group (mean change -1.6 cm, SD 2.7 cm) after 8 weeks. After 4 weeks the difference between active and placebo groups in their mean reduction from baseline was 1.2 (95% CI 0.1 to 2.4, p = 0.03) and after 8 weeks was 1.8 (95% CI for difference between groups, 0.6 to 2.9 cm; p = 0.002).

Conclusion. Topical application of glucosamine and chondroitin sulfate is effective in relieving the pain from OA of the knee and improvement is evident within 4 weeks.

J Rheumatol. 2003 Mar;30(3):523-8.

Introduction/Background– The disease/condition being

evaluated– The type of patients who are normally

affected by the condition– What is the scientific rationale for

conducting the current study.– How the condition is diagnosed (e.g.,

lab values, screening methods)– How the disease is normally treated

(non-pharmacological & pharmacological)

– What is considered a treatment failure

Don’t Skip the Methods and Results Sections

McAlister. J Gen Intern Med 1999;14:236-242, Putnam W, CMAJ 2002;166:1525-1530, Young JM, J Eval Clin Pract 2001;7:201-210, Anne Le Henanft, JAMA 2006;295:1147-1151

Percentage of respondents expressing confidence in basic evidence-based medicine (EBM) skills. *p < 0.01 for comparison between EBM users and nonusers.

Guidelines to Distinguish Useful vs. Useless Results

1. Use of randomization to assign patients to treatment

2. Report of all clinically relevant outcomes 3. Study patients similar to your own?4. Consideration of both clinical and statistical

significance?5. Therapeutic maneuver feasible in your practice6. Inclusion of all patients in the analysis and

justification the authors’ conclusion

Why is it Important to Review the Quality of RCT?

McAlister. J Gen Intern Med 1999;14:236-242, Putnam W, CMAJ 2002;166:1525-1530, Young JM, J Eval Clin Pract 2001;7:201-210, Anne Le Henanft, JAMA 2006;295:1147-1151

Characteristics of the Trial % All Reports

Sample size calculated 78

Use of intention to treat analysis 72

Statistical test considering the non-inferiority or equivalence margin

24

Validity of Clinical Trials

• Comparator Comparability of Effects

Comparability of Treatment Groups at Baseline

Comparability of Information

• Randomization

• BlindingMiettinen OS. Theoretical Epidemiology: Principles of occurrence research in Medicine, 1985:25-35.

Comparator or Control Treatment• Placebo

– Does the treatment work in a group of patients?

• Active – Is the treatment

superior to another active treatment?

NEJM 1994;331:394-398, JAMA 2006;295:1704-1706, Lancet 2005;366:895-906.

Example: ASCOT Study• Multicenter, randomised, controlled trial• Combination of amlodipine + perindopril (intervention) was compared with atenolol + bendroflumethiazide (control)• Effect on non-fatal myocardial infarction and fatal CHD

Randomization

Diabetologia (2004) 47:1175–1187

Women’s Health Initiative Hormone Trial

Subjects and MethodsThe randomization procedure was developed at the WHI Clinical Coordinating Center and implemented locally through a distributed study database, using a “randomized permuted block algorithm, stratified by clinical centre site and age.”

Characteristic

Oestrogen + Progestin, (n=8014) Placebo, (n=7627)

p valuea

n Mean (SD) or % n Mean (SD)

or %

Age at screening, years 8014 63.2 (7.1) 7627 63.3 (7.1) 0.35

  50–59 2706 33.8 2546 33.4

  60–69 3617 45.1 3440 45.1

  70–79 1691 21.1 1641 21.5

Race/ethnicity 0.64

  White 6800 84.9 6468 84.8

  Black 480 6.0 493 6.5

  Hispanic 429 5.4 385 5.0

  American Indian 22 0.3 27 0.4

  Asian/Pacific Islander 171 2.1 155 2.0

  Unknown 112 1.4 99 1.3

Hormone use 0.56

  Never used 5883 73.4 5635 73.9

  Past user 1596 19.9 1515 19.9

  Current user 531 6.6 474 6.2

Table 1 Baseline characteristics of non-diabetic women randomized to oestrogen plus progesterone or placebo in the Women’s Health Initiative

Intervention & Blinding

Blinding

“Participants, clinic staff, investigators and outcomes adjudicators were blinded to treatment assignment. When required for safety or symptom management, an unblinding officer provided the clinic gynecologists with the treatment assignment. Neither the clinic gynecologists nor any of the staff or investigators involved with the clinical care of the participants were involved in assessing the study outcomes.”

Diabetologia (2004) 47:1175–1187

Effect of oestrogen plus progestin on the incidence of diabetes in postmenopausal women: results from the Women’s Health Initiative Hormone Trial

Other Factors To Be Considered When Evaluating the Validity of a Trial

• Contamination• Cross over• Co-intervention• Timing and duration

of intervention• Locale of treatment

And……

• Adherence to treatment

• Follow-up• Drop-out• Patients included

in the analysis• Ethical

considerations

• Null hypothesis (Ho) –

– The statement of “no difference” where the researcher hopes to disprove with the study results.

– The goal of a study is to ‘reject’ the null hypothesis.

Hypothesis Testing

P value: Probability of obtaining a result at least as extreme as the observed result when the null hypothesis is true

The CONSORT diagram showing the flow of participants through each stage of a randomized trial

http://www.consort-statement.org/Statement/figure1.htm

Results. Diagram Flow of Participants

Results• Participant flow• Dates of

recruitment• Baseline data• Numbers

analyzed• Outcomes and

estimation• Ancillary

analysis• Adverse events

CNS Drugs. 2005;19(2):125-36.

Flow Diagram of the Study Population PP: Per Protocol, ITT: Intention to Treat

Brandes, J. L. et al. JAMA 2007;297:1443-1454.

Results

Presented clearly, objectively and in sufficient detail, numbers add up properly, analysis appropriate for the nature of data, interpreted correctly, relevant variables included in the analysis

Estimates of Treatment Effect

Estimate Calculation Example

ARR (RD) CER-EER trial trial

1) 8%-3%=5%2) 0.08%-0.03%=0.05%

RRR CER-EER/ CER 1) (8%-3%)/8%=0.63%2) (0.08%-0.03%)/0.08%=0.63%

NNT 1/ARR or 1/RD 1) 202) 2,000

ARR: Absolute risk reduction, RD: risk difference, RRR: relative risk reduction, CER: control event rate, EER: experimental event rate

Precision of Estimates of Treatment Effect: Confidence Intervals

Effect sizes (95% confidence intervals) in pain relief between topical therapies or nonsteroidal antiinflammatory drugs (NSAID) and placebo or vehicle. Symbols represent the point estimate.J Rheumatol. 2006 Sep;33(9):1841-4.

Superiority Trial

JAMA. 2001;285(15):1987-91. Ann Intern Med, 2001;134:663-94, JAMA. 2006;295(10):1152-60, Control Clin Trials 2002;23:570-583. Lancet 2005;365:573-578. Lancet, 1999;354:716-722. NEJM, 2004;350:2050-2059. Clin Therapeutics. 1996;18(5):797-810.

Type of study Superiority

Null hypothesis There is no difference between valsartan and placebo for reductions in DBP

Alternative hypothesis Valsartan is more efficacious than placebo in reducing high blood pressure

Control Placebo once daily (n=148)

Intervention Valsartan 20 mg once daily (n=140)Valsartan 80 mg once daily (n=150) Valsartan 160 mg once daily (n=148)Valsartan 320 mg once daily (n=150)

Treatment effect Compared to placebo, the reductions of diastolic blood pressure from baseline were 3.37, 5.20, 5.32 and 6.48 mm Hg for valsartan 20, 80, 160 and 320 mg, respectively, at 8 weeks.

The new drug is more efficacious ‘better than’ the control in terms of efficacy or safety by a predefined margin (known as delta, ∆). Example:

Randomized Clinical Trial

Antimicrob Agents Chemother. 2002 Jun;46(6):1746-54.

Objective: To compare the efficacy, safety, and tolerability of moxifloxacin (400 mg IV follow by 400 mg oral moxifloxacin) versus co-amoxiclav (1.2 g IV in a infusion three times a day followed by 625 mg oral co-amoxiclav three times a day), with or without clarithromycin (500 mg) twice daily (i.v. or orally), for 7 to 14 days in adult patients with community-acquired pneumonia requiring initial parenteral therapy.

Results: Although the trial was designed, on the basis of predefined outcomes, to demonstrate the equivalence of the two regimens, the results showed statistically significant higher clinical success rates (for moxifloxacin, 93.4%, and for comparator regimen, 85.4%; difference [Delta], 8.05%; 95% confidence interval [CI], 2.91 to 13.19%; P = 0.004) and bacteriological success rates (for moxifloxacin, 93.7%, and for comparator regimen, 81.7%; Delta, 12.06%; 95% CI, 1.21 to 22.91%) for patients treated with moxifloxacin…. Thus, it is concluded that monotherapy with moxifloxacin is superior to that with a standard combination regimen of a beta-lactam and a beta-lactamase inhibitor, co-amoxiclav, with or without a macrolide, clarithromycin, in the treatment of patients with community-acquired pneumonia admitted to a hospital.

Equivalence and NonInferiority Trials• Equivalence Trials

– The new drug is ‘just as good as’ the control

Antimicrobial agents and Chemotherapy, 2005; 49(10):4035–4041, Pediatrics, 2004; 114(1): e96-e101. JAMA. 2001;285(15):1987-91. Ann Intern Med, 2001;134:663-94, JAMA. 2006;295(10):1152-60, Control Clin Trials 2002;23:570-583. Lancet 2005;365:573-578. Lancet, 1999;354:716-722. NEJM, 2004;350:2050-2059.

For example a randomized, double-blind, noninferiority study was designed to demonstrate that a single 2.0-g oral dose of a novel microsphere formulation of azithromycin was at least as effective as 7 days of levofloxacin, 500 mg/day, in the treatment of adult patients with mild to moderate community-acquired pneumonia.

• Noninferiority Trials– The new drug is ‘not worse than’

the control

These trials are conducted for drugs that might have better safety profile and more tolerability, are easier to administer, and have lower cost.

For example, a trial was designed to determine whether azithromycin was as effective as erythromycin estolate in the treatment of pertussis, using bacterial eradication from the nasopharynx as the primary outcome measure.

Possible Scenarios of Observed Treatment Differences for Adverse Outcomes (Harms) in

Noninferiority Trials

Piaggio, G. et al. JAMA 2006;295:1152-1160.

Systematic Reviews• What type of studies were included?• What sources of information were searched to gather the studies?• What data were analyzed (individual or aggregate)?• Did authors assess validity of individual studies?

User’s Guides to the Medical Literature. Manual for Evidence Based Clinical Practice. Chicago, AMA Press, 2002. Evidence-Based Medicine, Elsevier, 2005.

Discussion/Comments

• Interpretation

• Generalizability

• Overall evidence

• Impact on your practice

Lessons Learned• In this module, you learned:

– That randomization, blinding and choice of control treatment are key features to establish the validity of a randomized clinical trial.

– Other characteristics of clinical trials that are important to determine their validity, such as: contamination, co-intervention, timing of intervention, site of treatment, adherence to treatment, sample size, follow-up, data analysis, ethical considerations and generalizability of results.