Endpoint Adjudication: effects of Adjudication in bias, variability, sample size and study power
-
Upload
mimmo-garibbo -
Category
Healthcare
-
view
452 -
download
0
Transcript of Endpoint Adjudication: effects of Adjudication in bias, variability, sample size and study power
© 2014 Syntax for Science SL
Juan V. Torres & Mimmo Garibbo
May 4-5, 2016 | Philadelphia
Effects of central adjudication in bias,
variability, sample size and study power
© 2016 Ethical2 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Introduction
Theoretical Background
– Bias
– Variability
– Sample size & power
Simulations
Conclusions & Further Work
AGENDA
© 2016 Ethical3 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
INTRODUCTION
© 2016 Ethical4 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
DEFINITION
INDEPENDENT REVIEW COMMITTEE
Independent Review Committees
(IRCs) review accumulating data in a
clinical trial and advise the sponsor
(directly or indirectly) on the future
management of the trial.
Endpoint adjudication is an important
task conducted by IRCs.
(EMA (2006) Guideline on data monitoring committees)
© 2016 Ethical5 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
CURRENT USE
ENDPOINT ADJUDICATION
Endpoint adjudication is frequently used in clinical
development.
A survey conducted among 140 organizations showed that:
(Krumholz-Bahner et al. 2015; Ethical GmbH 2015)
69%
Adjudication
USA41%
Europe
41%
© 2016 Ethical6 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
The main advantages of centralized endpoint adjudication are:
Reduce bias: markedly on unblinded studies (Walovitch et al. 2013) but also
present in blinded studies (Tang et al. 2008); also affected by the subjectivity
and complexity of the endpoint (Walovitch et al. 2013).
Reduce variability: central review by a small number of reviewers with
expertise in a specific area may lessen measurement variability. (Dodd et al.
2008).
Reduce sample size: measurement variability makes treatments appear more
similar than they really are, and therefore leads to reduced power to detect true
treatment effects. A reduction in the measurement variability involves a reduction
in sample size.
ADVANTAGES
CENTRALIZED ENDPOINT ADJUDICATION
© 2016 Ethical7 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Tang et al (2008) evaluated
differences between the
assessment of investigators and
IRCs at the time to measure
response rate (RR) and
progression free survival (PFS) in
phase III clinical trials.
Investigators generally over-
estimated RR compared to IRCs.
No significant differences on PFS.
EXAMPLES: REDUCE BIAS
CENTRALIZED ENDPOINT ADJUDICATION
Tang et al (2008)
18 studies
© 2016 Ethical8 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Krajewski et al (2014) found that interobserver variability
varied from 11% to 18% at the time to measure change from
baseline in lesion size (n=173 lesions).
Agreement between the IRC and INV assessments of PFS
status was 76.3% and 75.5% for paclitaxel alone and
paclitaxel + bevacizumab arms, respectively. (Genentech
2007, AVASTIN Briefing book)
Nagler et al (2013) found discrepancies varying up to 9.7%
among 360 measurements of fibrinogen measured by
different technicians.
EXAMPLES: REDUCE VARIABILITY
CENTRALIZED ENDPOINT ADJUDICATION
© 2016 Ethical9 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
It is claimed that centralized endpoint adjudication:
– Increases study complexity
– Increases study cost
– It might be not necessary
DISADVANTAGES
CENTRALIZED ENDPOINT ADJUDICATION
© 2016 Ethical10 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Today’s presentation:
Present key concepts to understand the effect of bias and
variability in terms of sample size and study power.
In working progress:
Assess and provide estimates regarding the effect of bias
and variability in terms of study cost.
Is centralized endpoint adjudication really adding an extra
cost in the study?
OBJECTIVES
© 2016 Ethical11 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
THEORETICAL
BACKGROUND
© 2016 Ethical12 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
BIAS AND PRECISION
© 2016 Ethical13 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Design
Conduct
Analysis
Interpretation
…
Measurement error
Reader(s)
Subject
Site, time, treatment,…
SOURCES OF BIAS
SOURCES OF BIAS & VARIABILITY
SOURCES OF VARIABILITY
© 2016 Ethical14 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Design
Conduct
Analysis
Interpretation
…
Measurement error
Reader(s)
Subject
Site, time, treatment,…
SOURCES OF BIAS
SOURCES OF BIAS & VARIABILITY
SOURCES OF VARIABILITY
© 2016 Ethical15 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
SAMPLE SIZE
Determine the optimum number of subjects required to be
able to arrive at ethically and scientifically valid results.
Given assumptions on:
– Expected treatment effect size
– Expected variability
we calculate how many subjects we need to guarantee a
certain study power preserving the type 1 error.
© 2016 Ethical16 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
VARIABILITY, SAMPLE SIZE & POWER
50% Power
0
Expected
effect
Observed
effects
10Expected tx effect
Lack of efficacy Efficacy
90% Power
0 10Expected tx effect
n = 64, σ = 100 n = 172, σ = 100
n = 64, σ = 60
© 2016 Ethical17 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Two ways to reduce confidence intervals, i.e., increase power:
VARIABILITY, SAMPLE SIZE & POWER
(n = 64, σ = 100)
Increase sample size Reduce variability
(n = 86, σ = 100)
(n = 116, σ = 100)
(n = 172, σ = 100)
(n = 64, σ = 100)
(n = 64, σ = 85)
(n = 64, σ = 73)
(n = 64, σ = 60)
© 2016 Ethical18 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Total sample size for a superiority study to compare two
independent groups can be obtained from the following
formula:
VARIABILITY, SAMPLE SIZE & POWER
Variability
increment
Sample size
increment
1% 2%
5% 10%
10% 21%
20% 44%
It can be seen that an increment
of X% in variability involves an
increment of (X2+2X)% in sample
size.
© 2016 Ethical19 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
SIMULATIONS
© 2016 Ethical20 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Iterations: 2000
Subjects: 130
Sites: 5
Treatment effect: 1
Total variability: 4.7
Variance components:
– Measurement error: 10%
– Site: 10%
– Subjects: 80%, 70%, 60%, 50%
– Readers: 0% , 10%, 20%, 30%
SIMULATION PARAMETERS
ASSUMPTIONS FOR THE SIMULATION ANALYSIS
Power ?
© 2016 Ethical21 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
RESULTS
Variance components Scenario 1 Scenario 2 Scenario 3 Scenario 4
Sites 10% 10% 10% 10%
Measurement 10% 10% 10% 10%
Subjects 80% 70% 60% 50%
Readers 0% 10% 20% 30%
Power
Without Central reading 76% 77% 79% 83%
With Central reading 76% 79% 85% 90%
© 2016 Ethical22 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
CONCLUSIONS
&
FURTHER WORK
© 2016 Ethical23 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
The selection of the most efficient design should be mandatory from the
ethics point of view (avoid unnecessary patient involvement).
Use of centralized endpoint adjudication may reduce error in endpoint
assessment and therefore be used as a tool to reduce sample size or
increase power.
CONCLUSIONS
© 2016 Ethical24 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Assess a larger and more realistic variety of scenarios
– Different endpoints (continuous, ordinal, binary, time to event)
Incorporate costs in the simulations
– Total/partial endpoint adjudication
– Quality control (reassessment)
FURTHER WORK
© 2016 Ethical25 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
Chen, E. X., G. R. Pond, and P. Tang. 2008. “Influence of Independent Review Committees (IRC) on Assessment
of Response Rate and Progression Free Survival in Phase III Clinical Trials.” ASCO Meeting Abstracts
26(15_suppl): 6567.
Dodd, Lori E. et al. 2008. “Blinded Independent Central Review of Progression-Free Survival in Phase III Clinical
Trials: Important Design Element or Unnecessary Expense?” Journal of Clinical Oncology 26(22): 3791–96.
EMA (2006), Committee for Medicinal Products For Human Use (CHMP). Guideline on data monitoring committees
Ethical GmbH. 2015. “Use of Adjudication Methods in Clinical Trials.”
https://www.ethicalclinical.com/eadjudication/adjudication-references (February 17, 2016).
Genentech 2007. AVASTIN Briefing book.
http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/OncologicDrugsAdvisoryC
ommittee/UCM219228.pdf
Krajewski, Katherine M. et al. 2014. “Intraobserver and Interobserver Variability in Computed Tomography Size and
Attenuation Measurements in Patients with Renal Cell Carcinoma Receiving Antiangiogenic Therapy: Implications
for Alternative Response Criteria.” Cancer 120(5): 711–21.
Krumholz-Bahner, S., M. Garibbo, K. A. Getz, and B. E. Widler. 2015. “An Overview and Analysis Regarding the
Use of Adjudication Methods in EU and US Drug Approvals.” Therapeutic Innovation & Regulatory Science 49(6):
831–39.
Nagler, Michael et al. 2013. “Variability between Laboratories Performing Coagulation Tests with Identical
Platforms: A Nationwide Evaluation Study.” Thrombosis journal 11(1): 6.
Tang P. A., Pond G.R., and Chen E. X. Influence of an independent review committee o assessment of response
rate and progression free survival in phase III clinical trials. Annals of Oncology 21: 19–26, 2010
Walovitch, Richard et al. 2013. “Subjective Endpoints in Clinical Trials: The Case for Blinded Independent Central
Review.” Open Access journal of clinical trials 5(september): 111–17.
REFERENCES
© 2016 Ethical26 05/MAY/2016 Effects of central adjudication in bias, variability, sample size and study power
THANKS!
ANY QUESTIONS?
further Info at: www.ethicalclinical.com