Understanding Statistical Power for Non-Statisticians
-
Upload
statistics-data-corporation -
Category
Healthcare
-
view
431 -
download
3
Transcript of Understanding Statistical Power for Non-Statisticians
![Page 1: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/1.jpg)
Understanding Statistical Power for Non-StatisticiansWEBINAR | JUNE 28, 2016 | 12:00PM EASTERN | DIA 2016DALE W. USNER, PH.D. | PRESIDENT | SDC
![Page 2: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/2.jpg)
General Housekeeping
Use mic & speakers or call in Webinar users submit
questions via chat
![Page 3: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/3.jpg)
Dale W. Usner, Ph.D.
20 years in industry 50+ FDA and international regulatory body
interactions Frequent study design support Therapeutic Expertise
Anti-viral/Anti-infective, Cardiovascular, Gastrointestinal, Oncology/Immunology, Ophthalmology, Surgical, Other
![Page 4: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/4.jpg)
Agenda
What is Statistical Power How Assumptions Affect Statistical Power and Sample Size How Power Is Associated With What is Statistically Significant Q&A
![Page 5: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/5.jpg)
What is Statistical
Power?
![Page 6: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/6.jpg)
What is Statistical Power?
Clinical/Medical: If I want to compare our Test Product to the Control Product in systolic blood pressure, what sample size will I need?
Statistician: Assuming a Difference in Means of 10 mm Hg and a common Standard Deviation (SD) of 20 mm Hg, 64 subjects per treatment group are required to have 80% power for a 2-sided = 0.05 test.
![Page 7: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/7.jpg)
What is Statistical Power?
Next?
![Page 8: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/8.jpg)
What is Statistical Power? Questions
What does it mean to have 80% power?
![Page 9: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/9.jpg)
What is Statistical Power? Questions
What does it mean to have 80% power? What does it mean to assume a difference in
means of 10 mm Hg? What if the true diff <10? >10?
![Page 10: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/10.jpg)
What is Statistical Power? Questions
What does it mean to have 80% power? What does it mean to assume a difference in
means of 10 mm Hg? What if the true diff <10? >10?
What role does the SD play? What if the true SD is >20? What if the true SD is <20?
![Page 11: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/11.jpg)
What is Statistical Power? Questions
What does it mean to have 80% power? What does it mean to assume a difference in
means of 10 mm Hg? What if the true diff <10? >10?
What role does the SD play? What if the true SD is >20? What if the true SD is <20?
What happens if the difference observed in the study is <10 mm Hg?
![Page 12: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/12.jpg)
Motivation
Assuming a Difference in Means of 10 mm Hg and a common SD of 20 mm Hg, N = 64 subjects per treatment group are required to have 80% power for a 2-sided = 0.05 test.
The study will demonstrate a statistically significant result if the difference observed in the study is ≥7.0 mm Hg (assuming the observed SD is 20 mm Hg).
![Page 13: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/13.jpg)
Motivation
With 85% Power (N = 73 subjects / Tx Gp) observed differences ≥6.55 mm Hg would yield statistical significance
With 90% Power (N = 86 subjects / Tx Gp) observed differences ≥6.03 mm Hg would be statistical significance
![Page 14: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/14.jpg)
Background
![Page 15: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/15.jpg)
Statistical Inference
Statistical Inference: Drawing conclusions about an entire population based on a sample from that population.
![Page 16: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/16.jpg)
Hypotheses (Efficacy)
Superiority H0: Test Arm is no different from Control Arm H1: Test Arm is different (superior) than Control Arm
Non-Inferiority H0: Test Arm is inferior to Control Arm H1: Test Arm is non-inferior to Control Arm
Desired Outcome: Reject H0 in favor of H1
![Page 17: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/17.jpg)
Statistical Inference: Coin Flip
H0: Proportion of Heads = Prop Tails = 0.50 H1: Proportion of Heads > Prop Tails Flip a coin 4 times with result 3H and 1T. 75%
Heads, should H0 be rejected?
![Page 18: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/18.jpg)
Statistical Inference: Coin Flip
H0: Proportion of Heads = Prop Tails = 0.50 H1: Proportion of Heads > Prop Tails Flip a coin 4 times with result 3H and 1T. 75%
Heads, should H0 be rejected? No, probability of this occurring under H0 is
31.25%
![Page 19: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/19.jpg)
Statistical Inference: Coin Flip
H0: Proportion of Heads = Prop Tails = 0.50 H1: Proportion of Heads > Prop Tails Flip a coin 4 times with result 3H and 1T. 75%
Heads, should H0 be rejected? No, probability of this occurring under H0 is
31.25% Flip a coin 40 times with result 30H and 10T.
75% Heads, should H0 be rejected?
![Page 20: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/20.jpg)
Statistical Inference: Coin Flip
H0: Proportion of Heads = Prop Tails = 0.50 H1: Proportion of Heads > Prop Tails Flip a coin 4 times with result 3H and 1T. 75%
Heads, should H0 be rejected? No, probability of this occurring under H0 is 31.25%
Flip a coin 40 times with result 30H and 10T. 75% Heads, should H0 be rejected? Yes, probability of this occurring under H0 is <0.1%
![Page 21: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/21.jpg)
Define and Power
(Type I Error) is the probability that the study concludes the Test Arm is different from the Control Arm, when the Test Arm truly is no different. This is a regulatory risk.
Power is the probability that the study concludes the Test Arm is different from the Control Arm, when in the Test Arm truly is different. This is sponsor risk.
![Page 22: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/22.jpg)
Continuous & Binary Measures
Continuous Measures Generally testing differences in Means or Medians Standard Deviation also very important
Binary Measures Generally testing differences or ratios of proportions Standard Deviations generally determined by
assumed proportions
![Page 23: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/23.jpg)
Power in Pictures
Consider a therapy designed to lower systolic blood pressure
(SysBP) by an additional 10 mm Hg more than the currently best selling therapy, which has been shown to lower the SysBP to an average of 140 mm Hg.
that each treatment has an SD of 20 mm Hg Assume the data follow normal distributions
![Page 24: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/24.jpg)
Power in Pictures: Sample Size = 1
![Page 25: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/25.jpg)
Power in Pictures: Sample Size = 5
![Page 26: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/26.jpg)
Power in Pictures: Sample Size = 10
![Page 27: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/27.jpg)
Power in Pictures: Sample Size = 30
![Page 28: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/28.jpg)
Power in Pictures: Sample Size = 50
![Page 29: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/29.jpg)
Power in Pictures: Sample Size = 64
![Page 30: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/30.jpg)
Power in Pictures: Sample Size = 73
![Page 31: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/31.jpg)
Power in Pictures: Sample Size = 86
![Page 32: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/32.jpg)
Power in Pictures: Sample Size = 1
![Page 33: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/33.jpg)
Power in Pictures: Sample Size = 5Power: 10%
![Page 34: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/34.jpg)
Power in Pictures: Sample Size = 10Power: 18%
![Page 35: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/35.jpg)
Power in Pictures: Sample Size = 30Power: 47%
![Page 36: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/36.jpg)
Power in Pictures: Sample Size = 50Power: 69%
![Page 37: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/37.jpg)
Power in Pictures: Sample Size = 64Power: 80%
![Page 38: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/38.jpg)
Power in Pictures: Sample Size = 73Power: 85%
![Page 39: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/39.jpg)
Power in Pictures: Sample Size = 86Power: 90%
![Page 40: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/40.jpg)
Standard Powers
80%: If the test product is as efficacious as assumed under H1, 80% of trials should reject H0 in favor of H1 by design. Generally considered to be the lowest targeted power.
85%: 85% of trials should reject H0 in favor of H1 by design.
90%: 90% of trials should reject H0 in favor of H1 by design.
![Page 41: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/41.jpg)
Sponsor Risk
80%: If the test product is as efficacious as assumed under H1, 20% of trials will fail to reject H0 in favor of H1 by design.
85%: 15% of trials will fail to reject H0 in favor of H1 by design.
90%: 10% of trials will fail to reject H0 in favor of H1 by design.
![Page 42: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/42.jpg)
How Assumptions Affect Statistical
Power and Sample Size
![Page 43: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/43.jpg)
Sample Size x Assumed Difference x Power
6 7 8 9 10 11 12 13 140
100
200
300
400
500
SD = 20, 2-sided alpha = 0.05
80% Power85% Power90% Power
Assumed Difference
Tota
l Sam
ple
Size
![Page 44: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/44.jpg)
% Increase in Sample Size (N) for Increasing Power
Assuming a 2-sided = 0.05 test, N increases by: ~14% from 80% to 85% power ~34% from 80% to 90% power
Assuming a 2-sided = 0.10 test, N increases by ~16% from 80% to 85% power ~38% from 80% to 90% power
Assuming a 2-sided = 0.20 test, N increases by ~19% from 80% to 85% power ~46% from 80% to 90% power
![Page 45: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/45.jpg)
Sample Size x Assumed Difference x Standard Deviation
6 7 8 9 10 11 12 13 140
100
200
300
400
500
600
80% Power, 2-sided alpha = 0.05
SD = 16SD = 18SD = 20SD = 22SD = 24
Assumed Difference
Tota
l Sam
ple
Size
![Page 46: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/46.jpg)
% Increase in Sample Size (N) for Increasing Standard Deviation
Regardless of the and power, N increases by a factor of (SDhigh / SDlow)2
![Page 47: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/47.jpg)
Sample Size x Assumed Difference x Alpha (Type I Error)
6 7 8 9 10 11 12 13 140
100
200
300
400
SD = 20, 80% Power
2-sided alpha = 0.202-sided alpha = 0.102-sided alpha = 0.05
Assumed Difference
Tota
l Sam
ple
Size
![Page 48: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/48.jpg)
% Decrease in Sample Size (N) for Increasing Alpha (Type I Error)
Assuming 80% power, N decreases by: ~21% from 2-sided alpha = 0.05 to 0.10 ~42% from 2-sided alpha = 0.05 to 0.20
Assuming 85% power, N decreases by: ~20% from 2-sided alpha = 0.05 to 0.10 ~40% from 2-sided alpha = 0.05 to 0.20
Assuming 90% power, N decreases by: ~18% from 2-sided alpha = 0.05 to 0.10 ~37% from 2-sided alpha = 0.05 to 0.20
![Page 49: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/49.jpg)
Sample Size x Assumed Difference x Power and Ratio of Randomization
6 7 8 9 10 11 12 13 140
100
200
300
400
500
600
SD = 20, 2-sided alpha = 0.05
80% Power80% Power 2:185% Power85% Power 2:190% Power
Assumed Difference
Tota
l Sam
ple
Size
![Page 50: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/50.jpg)
% Increase in Sample Size (N) for Increasing Randomization Ratio
Regardless of the and power, N increases by: ~4.2% from 1:1 to 3:2 randomization ratio ~12.5% from 1:1 to 2:1 randomization ratio ~33.3% from 1:1 to 3:1 randomization ratio
![Page 51: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/51.jpg)
% Decrease in Min Obs Diff Required for Significance with Increasing N
Regardless of the , the Minimum Observed Difference required for Significance decreases by a factor of sqrt(Nlow / Nhigh) With 80% Power, N = 64 / Tx Group required diff is 7.00 mm
Hg With 85% Power, N = 73 / Tx Group required diff is
(7.00*sqrt(64/73)) = 6.55 mm Hg With 90% Power, N = 86 / Tx Group required diff is
(7.00*sqrt(64/86)) = 6.03 mm Hg
![Page 52: Understanding Statistical Power for Non-Statisticians](https://reader036.fdocuments.us/reader036/viewer/2022062503/589d92991a28abfb088b7663/html5/thumbnails/52.jpg)
Effect on Statistical Significance
Increasing Sample Size decreases the minimum difference required to show statistical significance (driven by H0) and therefore increases power.
Required sample size increases with: Increasing: Power, SD, Randomization Ratio Decreasing: Alpha, Assumed Difference