Improving Healthcare Outcomes By Leveraging Data for Secondary Use
Outcomes and AEs Steven R. Cummings, MD. Outline: Outcomes Primary and secondary outcomesPrimary and...
-
Upload
reginald-collins -
Category
Documents
-
view
217 -
download
1
Transcript of Outcomes and AEs Steven R. Cummings, MD. Outline: Outcomes Primary and secondary outcomesPrimary and...
Outcomes and AEs Outcomes and AEs
Steven R. Cummings, MD Steven R. Cummings, MD
Outline: OutcomesOutline: Outcomes
• Primary and secondary outcomesPrimary and secondary outcomes
• Surrogate markersSurrogate markers
• Adverse experiencesAdverse experiences
MorboneMorbone
• A company wants help designing a A company wants help designing a trial of Morbone new treatment for trial of Morbone new treatment for osteoporosisosteoporosis
• Animal models: improves bone mass Animal models: improves bone mass and bone strengthand bone strength
• Planning a clinical trialPlanning a clinical trial
• Would like FDA approval to market Would like FDA approval to market MorboneMorbone
Morbone TrialMorbone Trial
Potential outcomesPotential outcomes
• Bone density (BMD) Bone density (BMD)
• Vertebral fractures (by x-ray)Vertebral fractures (by x-ray)
• Hip fracturesHip fractures
Which should be primary?Which should be primary?
How to start?How to start?
• Designate one “primary” and the Designate one “primary” and the others as “secondary” outcomesothers as “secondary” outcomes
Why one “primary” outcome?Why one “primary” outcome?
• To calculate sample sizeTo calculate sample size
• For testing statistical significance For testing statistical significance without penaltywithout penalty
• Greater credibilityGreater credibility
• The FDA requires that an outcome be The FDA requires that an outcome be “primary” in order to approve a drug “primary” in order to approve a drug for that indicationfor that indication–Beneficial effects on secondary Beneficial effects on secondary
outcomes can’t be used for ‘indication’outcomes can’t be used for ‘indication’
Which primary for Morbone?Which primary for Morbone?
• Bone density (BMD) Bone density (BMD)
• Vertebral fractures (by x-ray)Vertebral fractures (by x-ray)
• Hip fracturesHip fractures
ConsiderationsConsiderations
• Clinical importanceClinical importance
• FeasibilityFeasibility–Sample size and costSample size and cost
• Scientific / biological interestScientific / biological interest
• (For new drugs: What does FDA need (For new drugs: What does FDA need in order to approve an indication for in order to approve an indication for prescribing the drug?)prescribing the drug?)
Clinical importanceClinical importance
• Hip fracture: causes almost all of the Hip fracture: causes almost all of the deaths, much of the disability, and deaths, much of the disability, and 75% of the costs of fractures75% of the costs of fractures
• Vertebral fracture by x-ray: about 1/3 Vertebral fracture by x-ray: about 1/3 cause recognized pain and disability. cause recognized pain and disability. Mild changes might not be real Mild changes might not be real fractures.fractures.
• BMD: loss leads to greater risk of BMD: loss leads to greater risk of fracturesfractures
Which Primary Outcome?Which Primary Outcome?Sample sizeSample size
AlternativesAlternatives
• Improvement in BMDImprovement in BMD
• Vertebral fracturesVertebral fractures
• Hip fracturesHip fractures
Sample size/duration Sample size/duration
• 200 / 1 year200 / 1 year
• 3,000 / 3 yrs3,000 / 3 yrs
• 8,000 / 4 yrs8,000 / 4 yrs
Why not make BMD the Why not make BMD the primary outcome?primary outcome?
• Best choice to minimize costBest choice to minimize cost
• Issue: is it a valid surrogate marker Issue: is it a valid surrogate marker of clinical outcomes?of clinical outcomes?
Clean up your language!Clean up your language!
• Not all markers are ‘surrogates’Not all markers are ‘surrogates’
• BiomarkersBiomarkers– Measurement of a process or state.Measurement of a process or state.
• Surrogate markerSurrogate marker– Substitute for clinical outcomeSubstitute for clinical outcome
• Validated surrogateValidated surrogate– You can trust it. Effect of treatment on marker You can trust it. Effect of treatment on marker
has been established to consistently represent has been established to consistently represent the effect on clinical outcome.the effect on clinical outcome.
Surrogate markers for trialsSurrogate markers for trials
• ‘‘A laboratory or physical sign that is A laboratory or physical sign that is used in trials as a substitute for a used in trials as a substitute for a clinically meaningful endpoint.’ clinically meaningful endpoint.’
Criteria for validating a surrogate Criteria for validating a surrogate marker for treatmentsmarker for treatments
• Biologically plausibleBiologically plausible
• Marker strongly predicts the clinical Marker strongly predicts the clinical outcomeoutcome
• Treatment changes the markerTreatment changes the marker
• Treatment changes the rate of Treatment changes the rate of disease in the predicted direction disease in the predicted direction
* Prentice, 1989
Is BMD a valid surrogateIs BMD a valid surrogate
• Biologically plausibleBiologically plausible–Correlation between BMD and bone Correlation between BMD and bone
strength ~0.7 -0.9strength ~0.7 -0.9
Criteria for validating a surrogate Criteria for validating a surrogate marker for treatmentsmarker for treatments
• Biologically plausibleBiologically plausible
• Marker strongly predicts the clinical Marker strongly predicts the clinical outcomeoutcome
* Prentice, 1989
Risk of new vertebral fractures Risk of new vertebral fractures by quartile of BMDby quartile of BMD
Hip BMDHip BMD
00
22
44
66
88
1010
Quartile of Quartile of BMDBMD
11 22 33 44
Spine BMDSpine BMD
00
22
44
66
88
1010
Quartile of Quartile of BMDBMD
11 22 33 44
9.99.9
1.81.8
4.24.25.25.2
9.59.5
2.72.7
3.93.94.64.6
% o
f w
om
en w
ith
fra
ctu
re%
of
wo
men
wit
h f
ract
ure
Bone densityBone density
• Biologically plausible: YESBiologically plausible: YES
• Marker strongly predicts the clinical Marker strongly predicts the clinical outcome: YESoutcome: YES
• Treatment changes the marker*Treatment changes the marker*–YES: treatment improves BMD~5%YES: treatment improves BMD~5%
• Treatment changes the rate of Treatment changes the rate of disease in the predicted direction*disease in the predicted direction*–YES: other trials - 35-50% decrease risk YES: other trials - 35-50% decrease risk
* Prentice, 1989
Are we there yet?Are we there yet?
• BMD does all the right things a BMD does all the right things a surrogate marker should do.surrogate marker should do.
• Anything else?Anything else?
Treatment improves BMDTreatment improves BMD
RxRx BMDBMD
BMD predicts fractureBMD predicts fracture
FxFxBMDBMD
Bone densityBone density
• Do changes in the surrogate (bone Do changes in the surrogate (bone density) account for changes in density) account for changes in reduction in the outcome (fractures)reduction in the outcome (fractures)
* Prentice, 1989
FlourideFlouride
• Increased BMD ~10%Increased BMD ~10%
• IncreasedIncreased the risk of fractures the risk of fractures
* Prentice, 1989
Decreased Vertebral Fracture Risk: Decreased Vertebral Fracture Risk: Predicted from BMD vs. ObservedPredicted from BMD vs. Observed
80806060404020200 0
EstradiolEstradiol
EtidronateEtidronate
EtidronateEtidronate
AlendronateAlendronate
CalcitoninCalcitonin
8 8
55
1313
2020
3 3
PredictedPredicted
Cummings, ASBMR 1997Cummings, ASBMR 1997
Decreased Vertebral Fracture Risk: Decreased Vertebral Fracture Risk: Predicted from BMD vs. ObservedPredicted from BMD vs. Observed
80806060404020200 0
EstradiolEstradiol
EtidronateEtidronate
EtidronateEtidronate
AlendronateAlendronate
CalcitoninCalcitonin
6161
5656
5858
5050
6262
8 8
55
1313
2020
3 3
ObservedObservedPredictedPredicted
Cummings, ASBMR 1997Cummings, ASBMR 1997
The perfect surrogate is the causal The perfect surrogate is the causal pathway by which tx affects the diseasepathway by which tx affects the disease
RxRx FxFxBMDBMD
But treatments may have other effects But treatments may have other effects that could also influence the diseasethat could also influence the disease
RxRx FxFxBMDBMD
??????
TurnTurnoverover
Change in one marker will account forChange in one marker will account foronly part of the effect.only part of the effect.Some changes might also be harmful.Some changes might also be harmful.
Does BMD “Explain” the Reduction in Does BMD “Explain” the Reduction in Fracture Risk? Fracture Risk?
Main methodsMain methods
• Freeman Freeman –For individual data from a trialFor individual data from a trial
• Meta-analysisMeta-analysis–For aggregate data from multiple trialsFor aggregate data from multiple trials
Proving that a change in measurement Proving that a change in measurement predicts effect of treatment on fracture riskpredicts effect of treatment on fracture risk
Two approachesTwo approaches
• Individual level Individual level – How well does How well does changechange in the marker account in the marker account
for the effect in people? for the effect in people?
• Trial levelTrial level– How well does the How well does the changechange in measurement in measurement
predict the clinical results from trials? predict the clinical results from trials?
Validating that a marker is a Validating that a marker is a good surrogategood surrogate
#1#1
• Individual level Individual level –How well does change in the How well does change in the
measurement account for the decrease measurement account for the decrease in fracture risk in people? in fracture risk in people?
–The test: What percent of effect of The test: What percent of effect of decrease in fracture risk ‘explained’ by decrease in fracture risk ‘explained’ by change in the measurementchange in the measurement
Does BMD “Explain” the Reduction in Does BMD “Explain” the Reduction in Fracture Risk? Fracture Risk?
Main methodsMain methods
• Freeman Freeman –For individual data from a trialFor individual data from a trial–Estimates p = proportion of treatment Estimates p = proportion of treatment
effect “explained” by effect “explained” by change change in the in the markermarker
–ß = coefficient for treatmentß = coefficient for treatment–ß* = “adjusted for change in the markerß* = “adjusted for change in the marker– (1 - ß* / ß)(1 - ß* / ß)
Does BMD “Explain” the Reduction in Does BMD “Explain” the Reduction in Fracture Risk? Fracture Risk?
• • Freeman method applied to studies Freeman method applied to studies –FIT trial (alendronate): p = 0.16FIT trial (alendronate): p = 0.16–MORE trial (raloxifene): p = 0.05MORE trial (raloxifene): p = 0.05
• Very little of the treatment effects are Very little of the treatment effects are due to individual improvements in due to individual improvements in spine BMD, as measured by DXA.spine BMD, as measured by DXA.
Does BMD “Explain” the Reduction in Does BMD “Explain” the Reduction in Fracture Risk? Fracture Risk?
• • Freeman method applied to studies Freeman method applied to studies –FIT trial (alendronate): p = 0.16FIT trial (alendronate): p = 0.16–MORE trial (raloxifene): p = 0.05MORE trial (raloxifene): p = 0.05
• Very little of the treatment effects are Very little of the treatment effects are due to individual improvements in due to individual improvements in spine BMD, as measured by DXA.spine BMD, as measured by DXA.
Proving that a change in measurement Proving that a change in measurement predicts effect of treatment on fracture riskpredicts effect of treatment on fracture risk
2nd approach2nd approach
• Individual level Individual level
• Trial levelTrial level–How well does the change in How well does the change in
measurement predict the fracture measurement predict the fracture results from trials? results from trials?
– ‘‘Meta-analysis’ of many trialsMeta-analysis’ of many trials
““Meta-analysis” of trials of Meta-analysis” of trials of antiresorptivesantiresorptives
Each 1% improvement in spine BMD predicts Each 1% improvement in spine BMD predicts .03 (.02 to .05) reduction in risk of vertebral fracture.03 (.02 to .05) reduction in risk of vertebral fracture
““Meta-analysis” of trials of Meta-analysis” of trials of antiresorptivesantiresorptives
Each 1% improvement in spine BMD predicts Each 1% improvement in spine BMD predicts .03 (.02 to .05) reduction in risk of vertebral fracture.03 (.02 to .05) reduction in risk of vertebral fracture
ExpectedExpected
.25.25
ImplicationsImplications
• You cant trust changes in BMD in your You cant trust changes in BMD in your patients as an index of whether treatment patients as an index of whether treatment is working.is working.
• You can’t trust that a drug that improves You can’t trust that a drug that improves BMD will reduce the risk of fracture.BMD will reduce the risk of fracture.
• FDA still requires fractures as the FDA still requires fractures as the endpoint of trials for registering drugs.endpoint of trials for registering drugs.
Surrogate markers that failedSurrogate markers that failed
• Anti-arrhythmic drugs - frequency of Anti-arrhythmic drugs - frequency of ventricular arrythmias - and death ventricular arrythmias - and death
• Inotropic agents for CHF - improved Inotropic agents for CHF - improved cardiac function - mortalitycardiac function - mortality
• Estrogen - LDL/HDL cholesterol - CHDEstrogen - LDL/HDL cholesterol - CHD
• Antiresorptive drugs - BMD - fractureAntiresorptive drugs - BMD - fracture
• And more..And more..
Surrogates and safetySurrogates and safety
• Trials with surrogate endpoints are small Trials with surrogate endpoints are small and shortand short
• Generally too small to detect some Generally too small to detect some important adverse effectsimportant adverse effects
• Usually too short to determine whether Usually too short to determine whether long-term treatment continues to be long-term treatment continues to be effective (or safe)effective (or safe)
Limitations of surrogate markers Limitations of surrogate markers is the main reason for relying on is the main reason for relying on trials with clinical outcomestrials with clinical outcomes
Which Primary Outcome?Which Primary Outcome?
AlternativesAlternatives
• Improvement in BMDImprovement in BMD
• Vertebral fracturesVertebral fractures
• Hip fracturesHip fractures
Sample size/duration Sample size/duration
• 200 / 1 year200 / 1 year
• 3,000 / 3 yrs3,000 / 3 yrs
• 8,000 / 4 yrs8,000 / 4 yrs
Adverse events Adverse events
AEsAEs
• Hugely expensive & time-consumingHugely expensive & time-consuming–May account for 1/4 of the expense of May account for 1/4 of the expense of
drug trialsdrug trials
• Poorly donePoorly done
• Poorly studiedPoorly studied
• An opportunityAn opportunity
Issues re: Adverse EventsIssues re: Adverse Events
• Elicited vs. volunteeredElicited vs. volunteered
• Nuisance AEsNuisance AEs
• Attribution of causeAttribution of cause
AEs for Morbone TrialAEs for Morbone Trial
• The company’s standard approach: The company’s standard approach:
‘‘Record any symptoms or conditions the Record any symptoms or conditions the subject has experienced:subject has experienced:
________________________________________________________________
__________________________________________________________________
An alternative?An alternative?
““Since your last visit, has a doctor told Since your last visit, has a doctor told you you had (check all that apply)”you you had (check all that apply)”
__A blood clot in the leg or lung (venous __A blood clot in the leg or lung (venous thrombosis)thrombosis)
__ An ulcer __ An ulcer
……for all possible diseasesfor all possible diseases
What’s wrong with this approach?What’s wrong with this approach?
Approaches to AEsApproaches to AEsVolunteered vs. elicitedVolunteered vs. elicited
Pro Pro elicited/check listelicited/check list
• More sensitive?More sensitive?
• Easier to codeEasier to code
Con:Con:
• Miss unexpected or Miss unexpected or milder AEs milder AEs
Pro Pro volunteeredvolunteered
• Catch unexpected Catch unexpected AEs AEs
• Fewer AEs?Fewer AEs?
Con:Con:
• Hard to code; costlyHard to code; costly
• Less sensitive?Less sensitive?
Which approach is most likely to find Which approach is most likely to find real AEs? real AEs?
• Evidence is mixedEvidence is mixed
• Sensible approach: standard Sensible approach: standard questions to elicit uncommon AEs questions to elicit uncommon AEs suspected to be related to drug.suspected to be related to drug.
• Additional open ended questions to Additional open ended questions to capture unexpected AEs.capture unexpected AEs.
An approach An approach
• Evidence is mixedEvidence is mixed
• Sensible approach: standard Sensible approach: standard questions to elicit uncommon AEs questions to elicit uncommon AEs suspected to be related to drug.suspected to be related to drug.
• Additional open ended questions to Additional open ended questions to capture unexpected AEs.capture unexpected AEs.
FDA AE classificationsFDA AE classifications
• Serious AEsSerious AEs–DeathsDeaths–HospitalizedHospitalized–Cancer (except skin cancer)Cancer (except skin cancer)–Birth defectsBirth defects
• Reported within 24 hoursReported within 24 hours
• Collect extensive documentationCollect extensive documentation
FDA AE classificationsFDA AE classifications
• Non-serious AEsNon-serious AEs–Anything symptom or clinical eventAnything symptom or clinical event–Regardless of importance or Regardless of importance or
potential relevancepotential relevance
The Bunion ProblemThe Bunion Problem
• FIT Trial of alendronate in 6,400 FIT Trial of alendronate in 6,400 women for 4 yearswomen for 4 years
• Recorded over 20,000 episodes of Recorded over 20,000 episodes of URIs (and thousands of reports of URIs (and thousands of reports of bunions!)bunions!)
• Enormous data management effort Enormous data management effort and costand cost
• How could this be avoided?How could this be avoided?
How to minimize ‘nuisance’AEsHow to minimize ‘nuisance’AEs
• Elicit uncommon, plausible and Elicit uncommon, plausible and important AEsimportant AEs
• Limit collection of non-serious AEs Limit collection of non-serious AEs to samples of subjectsto samples of subjects–1st 300, then stop collecting1st 300, then stop collecting
New approach to classifying AEsNew approach to classifying AEs
• Serious AEsSerious AEs
• Important AEsImportant AEs–Limit activityLimit activity– Initiate a prescription or procedureInitiate a prescription or procedure–E.R. visitE.R. visit
• Minor AEsMinor AEs
AttributionAttribution
• SAEs must be classified asSAEs must be classified as–DefinitelyDefinitely–ProbablyProbably–Possibly, orPossibly, or–Not... Not...
...related to the study drug...related to the study drug
Did the treatment cause the AE?Did the treatment cause the AE?
• 67 year old Morbone volunteer starts 67 year old Morbone volunteer starts taking the study drug. taking the study drug.
• She reports a pruritic blistering rash She reports a pruritic blistering rash on her arms that started 7 days after on her arms that started 7 days after starting and disappeared 2 days after starting and disappeared 2 days after stopping the drug. stopping the drug.
• Your attribution?Your attribution?
AttributionAttribution
• Attributions to the drug are usually Attributions to the drug are usually wrongwrong– (Unless the treatment is known to cause (Unless the treatment is known to cause
the effect)the effect)
ValidationValidation
• Self report is sometimes inaccurateSelf report is sometimes inaccurate
• To get an accurate estimate of the To get an accurate estimate of the effect on plausible and serious effect on plausible and serious outcomes (and benefits), they should outcomes (and benefits), they should be validated.be validated.
• Expensive processExpensive process–Collection of recordsCollection of records–Central adjudication by expertsCentral adjudication by experts
Analysis of AE results of trialsAnalysis of AE results of trials
• Current practice: Current practice: – Intention-to-treat for effectiveness and Intention-to-treat for effectiveness and
safetysafety
• More conservative approachMore conservative approach– ITT for effectivenessITT for effectiveness–Per protocol for AEsPer protocol for AEs
SummarySummaryAssessing AEs in the Morbone TrialAssessing AEs in the Morbone Trial
• Elicit (check list)Elicit (check list)–Hospitalizations, birth defects..Hospitalizations, birth defects..–Disability, new prescriptions and Disability, new prescriptions and
proceduresprocedures
• Open-ended collection of minor AEs Open-ended collection of minor AEs on first 300 for 1 year on first 300 for 1 year
• No attributionNo attribution
• Per protocol analysis of AEsPer protocol analysis of AEs