Testing Dose-Response with Multivariate Ordinal Data
description
Transcript of Testing Dose-Response with Multivariate Ordinal Data
![Page 1: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/1.jpg)
Testing Dose-Response with Multivariate Ordinal Data
Bernhard KlingenbergAsst. Prof. of StatisticsWilliams College, MA
Paper available at www.williams.edu/~bklingen
In Collaboration with Aldo Solari, Luigi Salmaso and Fortunato Pesarin,
University of Padova
![Page 2: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/2.jpg)
OutlineIntroduction
• Safety and Toxicity Data
• Notation and hypothesis of interest
• Stochastic Ordering
• Theorem (SMH IJD)
Testing SMH• Simple Test statistics
• Permutation Approach
• Step-down methods for indiv. endpoint significance
• Increase power
Example • Parallel, 5 dose group study with rats (8 rats per dose
group) • 25 Adverse Events from exposure to Perchlorethylene
![Page 3: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/3.jpg)
Introduction: Safety and ToxicitySafety and Toxicity Data:
To capture large number of possible manifestations of a dose (exposure) effect on safety or toxicity: Multiple Endpoints
One such collection of endpoints to evaluate neurophysiological effects: Functional Observational Battery (FOB)
Others: Drug Safety, Disease progression
![Page 4: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/4.jpg)
Introduction: FOBGoal: Evaluation of neurophysiological effects to a
toxin (Perchlorethylene)
Data1: 2 groups (No exposure vs. 1.5g/kg exposure) 8 rats in each group Each evaluated at 25 endpoints (various
effects), grouped into 6 domains Response ordinal, on a scale from 1 (no effect)
to 4 (most severe reaction)
1 Moser (1986) Journal of the American College of Toxicology
![Page 5: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/5.jpg)
Introduction: FOB
![Page 6: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/6.jpg)
Introduction: Notationk-dimensional response vectors:
Control Treatment
Random Sample
Control Treatment
Hypothesis of interest: “No dose effect”
),...,( 111 kYY1Y ),...,( 221 kYY2Y
11n11 YY ,,22n21 YY ,,
21 Y Y :H0 12 Y Y :H 1vs.d st
![Page 7: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/7.jpg)
Introduction: No Toxicity“ ”: For all response sequences
Control Treatment
“ “: Stochastically larger 2
Control Treatment
Note: Rejection of H0 does not lead to H1
d ),,( 1 kjj
),,Pr( ),,Pr( 21211111 kkkk jYjYjYjY
st
),,Pr( ),,Pr( 21211111 kkkk jYjYjYjY
),,Pr( ),,Pr( 21211111 kkkk jYjYjYjY
2 Marschall & Olkin (1979) Inequalities: Theory of Majorization and Its Applications
![Page 8: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/8.jpg)
SMHUsually only interested if k margins are equal or not.
I.e., for each adverse event ,
Def.: Simultaneous Marginal Homogeneity (SMH) 3:
hhhhhh jjYjYH all for )Pr( )Pr(: 210
k
hh
d
h
k
h
h YYH1
211
0
Vector of marginal probabilities are equal under the two exposures, for all adverse events
simultaneously
:,,1 kh h
3 Agresti and Klingenberg (2005) JRSS C, Klingenberg and Agresti (2006), Biometrics
![Page 9: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/9.jpg)
SMHSMH with just two adverse events
Control Treatment
1 2 3 4
1
2
…
3
4
Lacrimation
Aro
usal
Lacrimation
Aro
usal
1 2 3 4
1
2
…
3
4
![Page 10: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/10.jpg)
SMHTheorem:
Prior assumption plausible when dealing with adverse events data (increase in exposure shift towards higher outcome categories)
IJD SMH
Cumulative marginal inhomogeneity: )Pr( )Pr( 22 hhhh jYjY
![Page 11: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/11.jpg)
Testing SMHConsequence of Theorem: If prior assumption
plausible, can use permutation approach to test hypothesis of SMH
Test for SMH: Modeling approach via cumulative logits (proportional odds form) 4
Estimation (ML, conditional ML, GEE,…) computationally impossible, Asymptotics invalid
3
4 Han, Catalano, Senchaudhuri, Metha (2004) Exact Analysis of Dose Response for Multiple Correlated Binary Outcomes, Biometrics.
![Page 12: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/12.jpg)
Testing SMHLet
Simple test statistic: Standardized differences in marginal sample proportions
given by (from multinomial assumption):
![Page 13: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/13.jpg)
Testing SMHTo take advantage of ordinal nature: Consider
scoring function
Let be score matrix
Look at difference in mean scores:
Estimate covariance matrix
4
under SMH assuming working independence
![Page 14: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/14.jpg)
Testing SMHTest statistic for sparse data, ignoring correlation
among adverse events:
with
This gives global test of safety/toxicity
Permutation approach: 16!/(8!8!) = 12870 possible permutations, many leading to identical values of
Advantage of permutation approach: Incorporates dependence by resampling entire vectors; exact significance levels
![Page 15: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/15.jpg)
Testing SMHExample: Arousal Endpoint
61.1
Computation with equally spaced scores:
292.82 ,2 ,3,1 33 ,2 ,1,0 t
03.0
01.005.0
01.001.004.0
01.003.002.006.0
Note:
![Page 16: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/16.jpg)
Testing SMHPermutation Distribution:
observed
Perm. Distr.
Asympt. Distr.
![Page 17: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/17.jpg)
Testing SMHIdentifying which individual adverse events are
significant leads to multiple hypotheses testing:
Use test statistic (standardized mean score difference) for individual tests
Multiplicity adjustments via step-down approach of Westfall &Young (1993), using distribution of maximum test statistic
khjjYjYH hhhhhh ,,1, all for )Pr( )Pr(: 210
![Page 18: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/18.jpg)
Testing SMHPermutation Distribution:
Observed maximum
Perm. Distr.
![Page 19: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/19.jpg)
Testing SMH
![Page 20: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/20.jpg)
Testing SMHHow sensitive are results to assigned scores?
Consider the scores that maximize (obtainable via isotonic regression; data-driven)
Appropriate for safety/toxicity data; maximizes the contrast btw. the mean score differences
)( hhz
14.2)(
),1 ,64.0 ,0 ,0(max
max
hh
h
z
Equally spaced scores: 61.1hz
With
maxh
![Page 21: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/21.jpg)
Testing SMH
![Page 22: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/22.jpg)
Testing DomainsDomain effects?
Some endpoints may measure similar effects
Multiplicity adjustment at the endpoint level may be too conservative, leading to some false negatives
Adjusted P-value for domain less than or equal to smallest adjusted P-value within domain
“Proof”: Let endpoint h be in the first domain Dom1:
![Page 23: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/23.jpg)
Testing DomainsImportant Consequence (Robustness Property):
Consonant domain test statistic:
Reject only (at domain level) if at least one endpoint within domain significant
If no significant endpoint, domain also not significant
For domain significance, it is irrelevant how many, potentially
non-significant endpoints are grouped into a domain! *
* Provided the same test statistic is used for all intersection hypotheses
![Page 24: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/24.jpg)
Testing DomainsDissonant domain test statistic:
Accumulate effects over endpoints within domain
Even though no individual endpoint is significant, several marginally significant ones can result in significant domain P-value
![Page 25: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/25.jpg)
SummaryTesting dose-response for multivariate ordinal data
Correlated ordinal responses (typical for toxicity or safety data) are often sparse and imbalanced use permutation approach
Instead of modeling dose-response, we focused on testing SMH vs. stochastic ordering
SMH IJD, but SMH IJD under
Test statistic: zh= (difference in mean scores) / s.e., for
each endpoint, assuming working independence s.e. derived from multinomial model and estimated under
SMH Multiplicity adjusted P-values for each endpoint from
Westfall and Young’s step-down procedure
12 Y Y
st
![Page 26: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/26.jpg)
Conclusions for FOBNeuromuscular domain showed significant effect
Domain P-value 0.003 with dissonant test Domain P-value 0.025 with consonant test
Adverse events in neuromuscular domain that show increased toxicity at the 1.5 g/kg exposure level when compared to control
Sensorimotor domain also shows increased level of toxicity, although no individual adverse event is significant
Gait (p= 0.025)
Hindelimb (p=0.044)
Forelimb (p=0.096)
![Page 27: Testing Dose-Response with Multivariate Ordinal Data](https://reader035.fdocuments.us/reader035/viewer/2022062422/56814029550346895dab897a/html5/thumbnails/27.jpg)
Did not showHow methods extend to several dose levelsEffect of discreteness (used mid P-values
throughout)Combining P-values instead of test statistics, with
functions other than the maximum
Thank you and Go Gators