Fu#le or Not? An Interim Analysis Case Study · Calculang Operang Characteriscs 1 • Simulate...
Transcript of Fu#le or Not? An Interim Analysis Case Study · Calculang Operang Characteriscs 1 • Simulate...
Fu#leorNot?AnInterimAnalysisCaseStudy
IngridFranklin,VeramedLimitedRosalindWalley,UCBPharma
10thOctober2017
Iן
Iן
Cן
Pן
Sן
Contents2
InterimAnalysisDefined3
RulesofThumb• Mustbepre-specified• Conductwhen≥40%subjectshavecompleted
InterimConcerns• IncreasedtypeIerrorrate(interimforefficacy)• IncreasedtypeIIerrorrate(interimforfuElity)
ICHE9–Astudyshouldonlybestoppedearlyfor:• Ethicalreasons• Unacceptablepower
InterimAnalysis:Analysisofthedataconductedbeforedatacollec3on hasbeencompleted,whichmayresultinastudyadjustment
CaseStudyIntroduc#on4
StudyOverview:Ø PhaseIIstudy.Raredisease.Ø PrimaryEndpoint:TreatmentdifferenceatWeek12(Endof
treatment)indiseaseacEvityscore-conEnuous.Ø Double-Blind,ParallelGroupDesign:
– PlacebovsTreatment(1:1randomisaEon)
TreatmentDifferenceatWeek12
SD False+veRate
Power nperarm(TotalN)
3.8 5 5% 80% 29:29(58)
ClassicalFrequen#stSampleSizeCalcula#on:
Nullhypothesis:Thereisnodifferenceindiseaseac.vityscorebetweentheplaceboandthetreatmentgroupsatweek12.
InterimAnalysisOp#ons5
Ø InterimforFu#lity
Ø InterimforEfficacy
Ø InterimforSampleSizeAdjustment
Ø NoInterimAnalysis
Ini#alDecision:Ø NOTtoincludeaninterim
IncreasedBudget
Norecruitmentconcerns
+
TeamRequestforInterimAnalysis6
Why:Ø SlowRecruitment:
– ConcernaboutwasEngresources– Seniormanagementrequestformalinterimtoassessforfu.lity
Sta#s#cian’sRole:Ø Workwithstudyteamto:
– Createappropriaterule– CalculateitsoperaEngcharacterisEcs
PLANNINGANINTERIMAFTERSTUDYSTART
7
Crea#ngaDecisionRule8
AimofRule:StopstudyearlyifdrugisinefficaciousØ Basicopera#ngcharacteris#cs
Probabilityofstoppingabaddrugatinterim
Probabilityofstoppingagooddrugatinterim
Overallfalsenega#verate
Ø FurtherdetailstoconsiderbeforefinalruleischosenProbabilityabaddrugwouldhavebeensuccessfulatstudy-end
Probabilityagooddrugwouldhavebeensuccessfulatstudy-end
Numberofsubjectssavedifstudyisstopped
9
Ø Rule:If95%CIupperlimit<2,studymaybestoppedforfu#lityØ Toostringent
32
TreatmentDifference
-6 -3 0Placebothreshold
3.8TargetTreatment
Difference
95%CI,N=100%Endofstudy
Opera#ngcharacteris#csimproveasNincrease,butlessresourceswouldbesaved.
95%CI,N=40%Ruleofthumbminimum
95%CI,N=30%EarliestOpportunity
95%CI,N=~70%
Marksthe80%CI
Ø Rule:If80%CIupperlimit<2,studymaybestoppedforfu#lity–toostringent!
PossibleInterimDecisionRule9
10
Ø Rule:If80%CIupperlimit<3.8,studymaybestoppedforfu#lity
32
TreatmentDifference
-6 -3 0Placebothreshold
3.8TargetTreatment
Difference
80%CI,N=100%Endofstudy
Opera#ngcharacteris#csimproveasNincrease,butlessresourceswouldbesaved.
80%CI,N=40%Ruleofthumbminimum
80%CI,N=30%EarliestOpportunity
80%CI,N=~70%
Ø Clinicallyrelevantrule-opera#ngcharacteris#cslookreasonablevisually
PossibleInterimDecisionRule10
Calcula#ngOpera#ngCharacteris#cs
1 • Simulateindividualsubjectprimaryendpointdatafor10,000studies
2 • Calculatemeanandvarianceofsubjectsperarmatstudy-end(N=100%)
3 • Calculatemeanandvarianceofsubjectsperarmatinterim(N<100%)
4 • CalculateCIsformeanforeachsimulatedstudy
5 • Definethresholdusedininterimanalysisrule
6 • Calculateoverallα,β,andopera#ngcharacteris#cs
7 • RepeatforinterimscenariosfordifferenttotalN
Ø Checkrulehasgoodchanceoffindingfu#lityifdrugisinefficaciousØ Aiddecisionofwhentoperforminterimanalysis(N=?)
11
Opera#ngCharacteris#csofPossibleRules12
Ø Overallfalsenega#veprobabilityincreasesslightlyto22%Ø Lowprobabilityofstoppingagooddrug,thereforenegligibleimpactonoverall
falsenega#veprobability
N% N CI Probabilityof stoppingabad
drug
Probabilityof stoppinga gooddrug
30% 18 95% 32% 3% 30% 18 80% 61% 10% 40% 24 95% 45% 3% 40% 24 80% 72% 10% ~70% 40 95% 66% 3% ~70% 40 80% 87% 10%
32% 16% 38% 24% 48%
12% <1% <1% <1% <1% <1%
<1%
Ø Rule:UpperCIlimit<3.8,indicatesfu#lityØ Rule:UpperCIlimit<2.0,indicatesfu#lity
FinalInterimRule13
N% N CI
Probabilityofstoppingatinterim:
Probabilityofsuccessatstudy-endifyoucon#nueat
interim,despiteruleindica#ngyoushouldstop:
Es#mateddatethiscouldbeapplied
No.ofsubjectssavedifwestopforfu#lityBaddrug Gooddrug Baddrug Gooddrug
40% 24 80% 72% 10% 0.3% 4.4% Jul-2017 ~17~50% 30 80% 78% 10% 0.3% 3.5% Sep-2017 ~11~70% 40 80% 87% 10% 0.3% 2.5% Dec-2017 ~2
Ø Rule:Ifupperendofthe80%CI<3.8,thisindicatesfu#lity DifferencetoplaceboatWeek12
Riskofignoringtherule
14
ImpactofInterimonOverallProbabilityofSuccess
0Placebothreshold
3.8TargetTreatment
DifferenceTreatmentDifference
80%CI
CIincludes3.8threshold
53
1.Atinterim 2.Atendofstudy
Ø ReachingStudySuccess:Withaninterim,gooddrugsmustpasstwohurdles
Ø IfagooddrugfailseitherrulethestudyisnotconsideredsuccessfulØ Extentofimpactdependsonstringencyofinterimrule
0Placebothreshold
3.8TargetTreatment
DifferenceTreatmentDifference
95%CI
CIabove0effect
53
IMPLEMENTINGTHEINTERIMANALYSIS
15
InterimRuleChosen…NextSteps16
Ini#alDocumenta#on
Ø Protocolamendment
Ø InterimanalysisSAP
Ø Iden#fica#onofunblindedteam
CROPartnership
Ø Timeframe
Ø Budget
ResultsPresenta#on
Ø Keyresultsslides
Summary17
InanIdealWorld…
Ø Aninterimanalysiswouldbeplannedintheoriginalstudydesign
Ø However,circumstancescanleadtorequestsforaninterimmid-study
InterimDesignProcess–Sta#s#cian’sRoleØ Workwithstudyteamtoiden#fyreasonforinterim
Ø Presentop#onsinasimplis#cmanner
Ø Adviseappropriaterulebasedonopera#ngcharacteris#csØ Iden#fyunblindedteamandpreparenecessarydocumenta#on
Ø Ensuresmoothrunningofinterim
Rן.forthisstudy
:DisclaimerDisclaimer:ןPharmaemployeeandholdsstockandstockoptions.IngridFranklinisan.e
Acknowledgements18