Closed Testing and the Partitioning Principle
description
Transcript of Closed Testing and the Partitioning Principle
![Page 1: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/1.jpg)
Closed Testing and the Partitioning Principle
Jason C. HsuJason C. Hsu
TheThe Ohio State University Ohio State University
MCP 2002MCP 2002August 2002August 2002
Bethesda, MarylandBethesda, Maryland
![Page 2: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/2.jpg)
Principles of Test-Construction Union-Intersection Testing UITUnion-Intersection Testing UIT
S. N. RoyS. N. Roy Intersection-Union Testing IUTIntersection-Union Testing IUT
Roger Berger (1982) Roger Berger (1982) TechnometricsTechnometrics Closed testingClosed testing
Marcus, Peritz, Gabriel (1976) Marcus, Peritz, Gabriel (1976) BiometrikaBiometrika PartitioningPartitioning
Stefansson, Kim, and Hsu (1984) Stefansson, Kim, and Hsu (1984) Statistical Decision Theory and Statistical Decision Theory and Related TopicsRelated Topics, Berger & Gupta eds., Springer-Verlag., Berger & Gupta eds., Springer-Verlag.
Finner and Strassberger (2002)Finner and Strassberger (2002) Annals of StatisticsAnnals of Statistics Equivariant confidence setEquivariant confidence set
Tukey (1953)Tukey (1953)Scheffe (195?)Scheffe (195?)Dunnett (1955)Dunnett (1955)
![Page 3: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/3.jpg)
Partitioning confidence sets
Multiple Comparison with the BestMultiple Comparison with the BestGunnar Stefansson & HsuGunnar Stefansson & Hsu
1-sided stepdown method (sample-determined 1-sided stepdown method (sample-determined steps) = Naik/Marcus-Peritz-Gabriel closed teststeps) = Naik/Marcus-Peritz-Gabriel closed testHsuHsu
Multiple Comparison with the Sample BestMultiple Comparison with the Sample BestWoochul Kim & Hsu & StefanssonWoochul Kim & Hsu & Stefansson
BioequivalenceBioequivalenceRuberg & HsuRuberg & Hsu & G. Hwang & Liu & Casella & Brown & G. Hwang & Liu & Casella & Brown
1-sided stepdown method (pre-determined steps)1-sided stepdown method (pre-determined steps)Roger Berger & HsuRoger Berger & Hsu
![Page 4: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/4.jpg)
Partitioning1.1. Formulate hypotheses HFormulate hypotheses H
00ii: : ii ** for for ii II
iiII ii ** = entire parameter space = entire parameter space
{{ii**: : iiII } partitions the parameter space } partitions the parameter space
2.2. Test each HTest each H00ii
**: : i i **, , iiII, at , at
3.3. Infer Infer ii if H if H00ii
** is rejected is rejected
4.4. Pivot in each Pivot in each ii a confidence set a confidence set CCii for for 5.5. iiII CCii is a 100(1 is a 100(1)% confidence set for )% confidence set for
![Page 5: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/5.jpg)
Partitioning1.1. Formulate hypotheses HFormulate hypotheses H
0i0i: : ii for i for i I I
iiJJ II = entire parameter space = entire parameter space
2.2. For each J For each J I, let I, letJ J
** = = iiJJ ii (( jjJJ jj))cc
3.3. Test each HTest each H0J0J
**: : J J **, J , J I, at I, at
{{JJ**: J : J I} partitions the parameter space I} partitions the parameter space
4.4. Infer Infer JJ if H if H0J0J
** is rejected is rejected
5.5. Pivot in each Pivot in each JJ a confidence set C a confidence set CJJ for for
6.6. J J J J C CJJ is a 100(1 is a 100(1)% confidence set for )% confidence set for
![Page 6: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/6.jpg)
MCB confidence intervals
ii maxmaxjjii jj
[[((YYii maxmaxjjii Y Yjj WW)), , ((YYii maxmaxjjii Y Yjj + + WW))++], ],
ii = 1, 2, … , = 1, 2, … , kk
Upper bounds imply subset selectionUpper bounds imply subset selection Lower bounds imply indifference zone Lower bounds imply indifference zone
selectionselection
![Page 7: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/7.jpg)
Multiple Comparison with the Best
HH0101: Treatment 1 is the best: Treatment 1 is the best
HH0202: Treatment 2 is the best: Treatment 2 is the best
HH0303: Treatment 3 is the best: Treatment 3 is the best
…… Test each at Test each at using 1-sided Dunnett’s using 1-sided Dunnett’s Collate the resultsCollate the results
![Page 8: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/8.jpg)
Union-Intersection Testing UIT
1.1. Form HForm Haa: : H Haiai (an “or” thing)(an “or” thing)
2.2. Test HTest H00: : H H00ii, the complement of H, the complement of Haa
1.1. If reject, infer at least one HIf reject, infer at least one H00ii false false
2.2. Else, infer nothingElse, infer nothing
![Page 9: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/9.jpg)
Closed Testing
1.1. Formulate hypotheses HFormulate hypotheses H00ii: : ii for for ii II
2.2. For each For each JJ II, let , let JJ = = iiJJ ii
3.3. Form closed family of null hypothesesForm closed family of null hypotheses{H{H0J0J: : JJ: : JJ II}}
4.4. Test each Test each HH00JJ at at
5.5. Infer Infer iiJJ ii if all H if all H00J’J’ with with JJ J’J’ rejected rejected
6.6. Infer Infer ii if all H if all H00JJ’’ with with ii J’J’ rejected rejected
![Page 10: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/10.jpg)
Oneway model
YYirir = = ii + + irir, , ii = 0, 1, 2, … , = 0, 1, 2, … , kk, , rr = 1, … , = 1, … , nnii
irir are i.i.d. Normal(0, are i.i.d. Normal(0, 22))
Dose Dose ii “efficacious” if “efficacious” if ii > > 11 + +
ICH E10 (2000)ICH E10 (2000) Superiority if Superiority if 0 0 Non-inferiority if Non-inferiority if < 0 < 0 Equivalence is 2-sidedEquivalence is 2-sided Non-inferiority is 1-sidedNon-inferiority is 1-sided
![Page 11: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/11.jpg)
Closed testing null hypotheses(sample-determined steps)
HH0202: Dose 2 not efficacious: Dose 2 not efficacious
HH0303: Dose 3 not efficacious: Dose 3 not efficacious
HH0101: Doses 2 and 3 not efficacious: Doses 2 and 3 not efficacious
Test each at Test each at Collate the resultsCollate the results
![Page 12: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/12.jpg)
Partitioning null hypotheses(sample-determined steps)
HH0101: Doses 2 and 3 not efficacious: Doses 2 and 3 not efficacious
HH0202: Dose 2 not efficacious: Dose 2 not efficacious
but dose 3 isbut dose 3 is HH0303: Dose 3 not efficacious: Dose 3 not efficacious
but dose 2 isbut dose 2 is Test each at Test each at Collate the resultsCollate the results
![Page 13: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/13.jpg)
Partitioning implies closed testing
PartitioningPartitioning implies closed testing because implies closed testing because A size A size test for H test for H00ii is a size is a size test for test for HH00ii
Reject Reject HH0101: Doses 2 and 3 not efficacious : Doses 2 and 3 not efficacious
implies either dose 2 or dose 3 efficaciousimplies either dose 2 or dose 3 efficacious Reject Reject HH0202: Dose 2 not efficacious: Dose 2 not efficacious
but dose 3 efficaciousbut dose 3 efficacious implies it is not the implies it is not the case case dose 3 is efficacious dose 3 is efficacious but notbut not dose 2dose 2
Reject HReject H0101 and H and H0202 thus implies dose 2 thus implies dose 2
efficaciousefficacious
![Page 14: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/14.jpg)
Intersection-Union Testing IUT
1.1. Form HForm Haa: : H Haiai (an “and” thing) (an “and” thing)
2.2. Test HTest H00: : H H00ii, the complement of H, the complement of Haa
1.1. If reject, infer all HIf reject, infer all H00ii false false
2.2. Else, infer nothingElse, infer nothing
![Page 15: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/15.jpg)
PK concentration in blood plasma curve
![Page 16: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/16.jpg)
Bioequivalence defined
Bioequivalence: clinical equivalence betweenBioequivalence: clinical equivalence between
1.1. Brand name drugBrand name drug
2.2. Generic drugGeneric drug
Bioequivalence parametersBioequivalence parameters AUCAUC = Area Under the Curve = Area Under the Curve CCmaxmax = maximum Concentration = maximum Concentration
TTmaxmax = Time to maximum concentratin = Time to maximum concentratin
![Page 17: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/17.jpg)
Average bioequivalence
NotationNotation
= expected value of brand name drug= expected value of brand name drug
22 = expected value of generic drug = expected value of generic drug
Average bioequivalence meansAverage bioequivalence means
.8 < .8 < //22 < 1.25 for < 1.25 for AUCAUC
andand
.8 < .8 < //22 < 1.25 for < 1.25 for CCmaxmax
![Page 18: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/18.jpg)
Bioequivalence in practice
If If loglog of observations are normal with of observations are normal with means means and and22 and equal variances, and equal variances,
then average bioequivalence becomes then average bioequivalence becomes
loglog(.8) < (.8) < 22 < < loglog(1.25) for (1.25) for AUCAUC
andand
loglog(.8) < (.8) < 22 < < loglog(1.25) for (1.25) for CCmaxmax
![Page 19: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/19.jpg)
Partitioning
Partition the parameter space asPartition the parameter space as
1.1. HH0<0<: : 22 < < loglog(0.8)(0.8)
2.2. HH0>0>: : 22 > > loglog(1.25) (1.25)
3.3. HHaa: : loglog(.8) < (.8) < 22 < < loglog(1.25) (1.25)
Test HTest H0<0< and H and H0>0> each at each at ..
Infer Infer loglog(.8) < (.8) < 22 < < loglog(1.25) if both H(1.25) if both H0<0< and H and H0>0>
rejected.rejected.
Controls Controls PP{incorrect decision} at {incorrect decision} at ..
![Page 20: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/20.jpg)
Dose-Response (Phase II)
![Page 21: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/21.jpg)
Anti-psychotic drug efficacy trial
Dose of Seroquel (mg)Dose of Seroquel (mg)
00 7575 150150 300300 600600 750750
nn 5151 5252 4848 5151 5151 5353
ii 4.784.78 4.224.22 3.743.74 3.563.56 3.583.58 3.933.93
SESE 0.230.23 0.220.22 0.230.23 0.230.23 0.230.23 0.220.22
Arvanitis et al. (1997 Biological Psychiatry)
CGI = Clinical Global Impression
![Page 22: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/22.jpg)
Minimum Effective Dose (MED)Minimu Effective Dose Minimu Effective Dose
== MEDMED
== smallest smallest ii so that so that ii > > 11 + + for all for all jj, , ii jj kk
Want an upper confidence bound MEDWant an upper confidence bound MED++ so that so that
PP{{MEDMED < MED < MED++} } 100(1 100(1)%)%
![Page 23: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/23.jpg)
Closed testing inference
Infer nothing if HInfer nothing if H0101 is accepted is accepted
Infer at least one of doses 2 and 3 effective Infer at least one of doses 2 and 3 effective if Hif H0101 is rejected is rejected
Infer dose 2 effective if, in addition to HInfer dose 2 effective if, in addition to H0101,,
HH0202 is rejected is rejected
Infer dose 3 effective if, in addition to HInfer dose 3 effective if, in addition to H0101,,
HH0303 is rejected is rejected
![Page 24: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/24.jpg)
Closed testing method(sample-determined steps)
Start from HStart from H0101 to H to H0202 and H and H0303
Stepdown from smallest Stepdown from smallest pp-value to largest -value to largest pp-value-value
Stop as soon as one fails to rejectStop as soon as one fails to reject Multiplicity adjustment decreases from Multiplicity adjustment decreases from
kk to to k k 1 to 1 to k k 2 2 to 2 from to 2 from step 1 to 2 to 3 … to step step 1 to 2 to 3 … to step k k 1 1
![Page 25: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/25.jpg)
Tests of equalities(pre-determined steps)
1.1. HH00kk:: 11 = = 22 = = = = kk
HHakak:: 11 = = 22 = = < < kk
2.2. HH0(k0(k1)1):: 11 = = 22 = = = = kk1 1
HHaa((kk1)1):: 11 = = 22 = = < < kk1 1
3.3. HH0(0(kk2)2):: 11 = = 22 = = = = kk2 2
HHaa((kk2)2):: 11 = = 22 = = < < kk22
4.4.
5.5. HH0202:: 11 = = 22 HHaa22:: 11 < < 22
![Page 26: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/26.jpg)
Closed testing of equalities
Null hypotheses are Null hypotheses are nestednested
1.1. Closure of family remains HClosure of family remains H00kk H H0202
2.2. Test each HTest each H00ii at at 3.3. Stepdown from dose Stepdown from dose kk to dose to dose kk1 to 1 to
to dose 2to dose 2
4.4. Stop as soon as one fails to rejectStop as soon as one fails to reject
5.5. Multiplicity adjustment not neededMultiplicity adjustment not needed
![Page 27: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/27.jpg)
Testing equalities is easy
HH00kk: : 11 = = = = kk
HH0202: : 11 = = 22
HH00ii HH00ii
HH00kk: : 11 kk
HH0202:: 11 22
HH00ii HH00ii
![Page 28: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/28.jpg)
Partitioning null hypotheses (for pre-determined steps)
HH0k0k:: Dose Dose kk not efficacious not efficacious
HH0(k-1)0(k-1):: Dose Dose kk efficacious efficacious
butbut dose dose kk1 not efficacious1 not efficacious HH0(k-1)0(k-1):: Doses Doses kk and and kk1 efficacious1 efficacious
butbut dose dose kk2 not efficacious2 not efficacious HH0202:: Doses Doses kk 3 efficacious 3 efficacious
butbut dose 2 not efficacious dose 2 not efficacious Test each at Test each at Collate the resultsCollate the results
![Page 29: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/29.jpg)
Partitioning inference
1.1. Infer nothing if Infer nothing if HH00kk is accepted is accepted
2.2. Infer dose Infer dose kk effective if effective if HH00kk is rejected is rejected
3.3. Infer dose Infer dose kk1 effective if, in addition to 1 effective if, in addition to HH00kk, , HH0(0(kk-1)-1) is rejected is rejected
4.4. Infer dose Infer dose kk2 effective if, in addition to 2 effective if, in addition to HH00kk and and HH0(0(kk-1)-1) , H , H0303 is rejected is rejected
5.5.
![Page 30: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/30.jpg)
Partitioning method (for pre-determined steps) Stepdown from dose Stepdown from dose kk to dose to dose kk1 to 1 to to to
dose 2dose 2 Stop as soon as one fails to rejectStop as soon as one fails to reject Multiplicity adjustment not neededMultiplicity adjustment not needed Any pre-determined sequence of doses can Any pre-determined sequence of doses can
be usedbe used Confidence set given in Hsu and Berger Confidence set given in Hsu and Berger
(1999 (1999 JASAJASA))
![Page 31: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/31.jpg)
Pairwise t tests for partitioning
Size Size tests for tests for HH00kk H H0202 are also size are also size test test for for HH00kk H H0202
HH00kk:: Dose Dose kk not efficacious not efficacious HH0(0(kk-1)-1):: Dose Dose kk1 not efficacious1 not efficacious HH0(0(kk-2)-2):: Dose Dose kk2 not efficacious2 not efficacious HH0202:: Dose 2 not efficaciousDose 2 not efficacious Test each with a size-Test each with a size- 2-sample 1-sided 2-sample 1-sided
tt-test-test
![Page 32: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/32.jpg)
Testing equalities is easy
HH00kk: : 11 = = = = kk
HH0202: : 11 = = 22
HH00ii HH00ii
HH00kk: : 11 kk
HH0202:: 11 22
HH00ii HH00ii
![Page 33: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/33.jpg)
Could reject for the wrong reason
HH00
HHaa
neitherneither
![Page 34: Closed Testing and the Partitioning Principle](https://reader036.fdocuments.us/reader036/viewer/2022062500/56814fde550346895dbda6fc/html5/thumbnails/34.jpg)