STAT 3120Statistical Methods I

Statistical Power

STAT3120 – Hypothesis Testing

A second calculation that is conducted when conducting Hypothesis Testing is the evaluation of the “Power” of the test.

Recall that Power is the probability of correctly rejecting the null hypothesis when it should be rejected…in other words…the probability of detecting a true effect (1-Type II error).

Question: Why not set both alpha and power at acceptable levels?

STAT3120 – Hypothesis Testing

Answer: Because the Type I and Type II errors are inversely related…as the probability of a Type I error becomes more restricted, the probability of a Type II error increases. Since Power is 1-probability of a Type II, increasing the Type II error, decreases the Power of the test – Type II and Power ARE directly related.

Statistical Power is the probability of rejecting the null hypothesis when it should be rejected. In other words, it is the probability that if a true difference exists, it will be discovered.

Statistical Power is heavily used in medicine, clinical psychology and biology.

Typically, a test must have a Statistical Power of 80% or greater to be considered valid.

Power is a function of three factors:

1. Effect size – i.e., the difference between the two groups or measurements. As the effect size goes up, the power increases.

2. Alpha – As the chance of finding an incorrect significant effect is reduced (Type I error), the probability of correctly finding an effect is also reduced. Typically, alpha is set to be .01 (most conservative and lowers power), .05 or .10 (most risk tolerant and increases power).

3. Sample Size - Increased sample sizes will always produce greater power. But, increasing the sample size can also produce “too much” power – smaller and smaller effects will be found to be significant until at a large enough sample size, any effect is considered to be significant.

STAT3120 – Hypothesis Testing

STAT3120 – Hypothesis Testing

When conducting a one tailed test, the power calculation is executed as:

1-() = 1-P[z<z- (| 0 - a|/ )]

A two tailed test is conducted similarly, with the alpha value associated with the z score divided by two.

STAT3120 – Hypothesis Testing

If you are starting an experiment from scratch, how do you determine the appropriate sample size?

n= 2*(Z +Z )2/Δ2

A two tailed test is conducted similarly, with the alpha value associated with the z score divided by two.

Secret weapon for all of that computational @#%$^&:

proc power;

onesamplemeans test=t

mean =

stddev =

ntotal =

power = .;

run;

STAT3120 – Hypothesis Testing

STAT3120 – Hypothesis Testing

Secret weapon for all of that computational @#%$^&:

proc power; twosamplemeans test=diff meandiff = stddev = npergroup = power = .; run;

proc power;

pairedmeans test=diff

pairedmeans = |

corr =

pairedstddevs = ()

npairs = .

power = ;

run;

Secret weapon for all of that computational @#%$^&:

STAT3120 – Hypothesis Testing