Sof stat issues_pro

30

Transcript of Sof stat issues_pro

Page 1: Sof stat issues_pro
Page 2: Sof stat issues_pro

Statistical issues in patient reported outcome measures

Jonas Ranstam PhD

RC Syd and Lund University, Lund, Sweden, Email: [email protected]

Page 3: Sof stat issues_pro

Science and uncertainty

“If you thought that science was certain - well, that is just an error on your part.”

Richard P. Feynman

Page 4: Sof stat issues_pro

Uncertainty in clinical research

Generalization from sample to population

Page 5: Sof stat issues_pro

Observedoutcome

Concluded outcome

Inference (naive)

Real outcome

Page 6: Sof stat issues_pro

Observedoutcome

Concluded outcome

Effectmodification

Case-mix

InteractionMis-

classification

Selection

Measurementerrors

Biologicalvariation Co-

morbidity

Real outcome

Inference (scientific)

Uncertaintyacknowledged

Page 7: Sof stat issues_pro

The statistician's task

To eliminate as much uncertainty as possible (by design) and to quantify (in the analysis) what is left.

Page 8: Sof stat issues_pro

Statistical characteristics of PROs

1. Measured on an ordinal scale

2. Discrete distribution

3. Truncated distribution

4. Skewed distribution

5. Floor and/or ceiling effects

Are standard analysis methods appropriate?

Page 9: Sof stat issues_pro
Page 10: Sof stat issues_pro

Lillgraven S, Kristiansen IS, Kvien TK. Comparison of utility measures and their relationship with other health status measures in 1041 patients with rheumatoid arthritis. Ann Rheum Dis 2010;69:1762-1767.

Page 11: Sof stat issues_pro
Page 12: Sof stat issues_pro

The EQ-5D index

Simulation studies at RC Syd show that

Type-1 error rate (risk of false positive findings)

EQ-5D is best analyzed using methods that assume a Gaussian distribution, at least if n is large, between 20-50. Non-parametric alternatives perform poorly with any n value.

Type-2 error rate (risk of false negative findings)

No single method can be recommended. All investigated methods perform poorly for any distributional shape.

Page 13: Sof stat issues_pro

Complications

Longitudinal analyses

(of change, gain, delta-value, etc.)

Page 14: Sof stat issues_pro

Simulated data (n = 1000)

pre - post

correlation (pre, post) = 0

delta = post - pre

correlation (pre, post - pre) = -0.7

Baseline versus change

Page 15: Sof stat issues_pro

Baseline versus change

50% of the “change” can be explained by baseline.

When comparing “change” in different groups, always adjust for imbalance at baseline (e.g. using ANCOVA).

Page 16: Sof stat issues_pro

RTM - Regression to the mean

If the first measurement of a variable is extreme, the second measurement will tend to be closer to the average.

Note, this is a purely statistical phenomenon.

Galton F. Regression towards mediocrity in hereditary stature. J Anth Inst Gr Br Ire.1886;15:246–263.

Page 17: Sof stat issues_pro

Hypothetical example: SF-36 PF

Baseline: mean = 80, SD = 17Follow up: mean = 80, SD = 17p ≈ 1.0

Page 18: Sof stat issues_pro

Hypothetical example: SF-36 PF

Baseline: mean = 48.7, SD = 8.6Follow up: mean = 59.2, SD = 16.7p < 0.001

Page 19: Sof stat issues_pro

RTM - Regression to the mean

The phenomenon explains the placebo effect in clinical trials and apparent treatment effects found in some studies on homeopathic drugs, bible reading, etc.

Page 20: Sof stat issues_pro

Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. Int J Epidemiol 2005;34:215–220

RTM - Easy to quantify (for Normally distributed endpoints)

Page 21: Sof stat issues_pro

Hypothetical example of RTM in SF-36 PF

Mean = 80, SD = 17, cut off = 60

r RTM0.0 28.4 0.1 25.50.2 22.70.3 19.90.4 17.00.5 14.20.6 11.30.7 8.50.8 5.70.9 2.81.0 0

Page 22: Sof stat issues_pro

Regression-to-the-mean

Evaluation of a single group’s development over time should be avoided, or at least include a comparison with the expected RTM effect.

Page 23: Sof stat issues_pro

Practical consequences

When evaluating change, use a control group.

Adjust for baseline imbalance.

The validity of this procedure with multi-modal data (e.g. the EQ-5D index) is unknown.

Page 24: Sof stat issues_pro

Thank you for your attention!

Page 25: Sof stat issues_pro

Can a multi-scale PRO be used as a primary endpoint in a randomized trial?

Page 26: Sof stat issues_pro

Can a multi-scale PRO be used as a primary endpoint in a randomized trial?

Page 27: Sof stat issues_pro

Can a PRO be used as a primary endpoint in a randomized trial?

PRO as primary endpoint

“A PRO measurement can be the clinical trial’s primary endpoint measure, a co-primary endpoint measure ... or a secondary endpoint measure whose analysis is considered according to a hierarchical sequence.“

FDA. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Guidance for Industry.

Page 28: Sof stat issues_pro

Can a multi-scale PRO be used as a primary endpoint in a randomized trial?

PRO and HRQL

“The term PRO is proposed as an umbrella term to cover both single dimension and multi-dimension measures of symptoms, health-related quality of life (HRQL), health status, adherence to treatment, satisfaction with treatment, etc.”

“In the context of drug approval, HRQL is considered to represent a specific type/subset of PROs, distinguished by its multi-dimensionality.”

EMEA. Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products.

Page 29: Sof stat issues_pro

Can a multi-scale PRO be used as a primary endpoint in a randomized trial?

HRQL as primary endpoint

“In general, the methodology for assessing the effect on HRQL is similar to the methodology used in any efficacy trial, except for issues related to the nature of the instruments, which are generally composed of multi-items, and multi-domains.

Briefly, it is recommended that HRQL instrument be previously validated for the condition studied...”

EMEA. Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products.

Page 30: Sof stat issues_pro

FDA. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Guidance for Industry.