1

Week 8: Type I Error and Type II Error

Statistical Power

2

Type I and Type II error: When we conduct hypothesis testing, we either reject or

fail to reject the null hypothesis. Our decision usually causes four outcomes:

1. Reject the null hypothesis when it is false.

2. Keep the null hypothesis when it is true.

3. Reject the null hypothesis when it is true. (Type I error: alpha )

4. Fail to reject the null hypothesis when it is false. (Type II error: beta )

3

Type I and Type II error:

Decision Made

Null true Null false

Reject Null Type I error (α)

Correct decision (1 – b) POWER!

Fail to reject null

Correct decision

Type II error (b)

State of Nature

4

Another example:

Innocent Guilty

Reject null: Find Guilty!

Incorrect decision Type 1 error (α)

Correct decision (1 – b) or Power

Fail to reject null: Find Innocent

Correct decision

Incorrect decision Type II error (b)

TRUTH

5

Null: Reality: Conclusion: Decision:

There is no difference or relationship.

Null is true: There is no difference or relationship.

Fail to reject the null: Conclude there is no difference or relationship.

Correct decision.

We failed to reject the null hypothesis when it is in fact true.

In reality, there is no significant difference or relationship to find in this case.

Scenario 1:

6

Null: Reality: Conclusion: Decision:

There is no difference or relationship.

Null is false: There is a difference or relationship.

Reject the null: Conclude there is a difference or relationship.

Correct decision.POWER! (1 – b)

We reject the null hypothesis when it is in fact false.

In reality, there is a significant difference or relationship to find in this case.

Scenario 2:

7

Null: Reality: Conclusion: Decision:

There is no difference or relationship.

Null is false: There is a difference or relationship.

Fail to reject the null: Conclude that there is no difference or relationship.

Incorrect Decision: Type II error (b)

We failed to reject the null hypothesis when it is false.

In reality, there was a significant difference or relationship to find in this case and we didn’t find it.

Scenario 3:

8

Null: Reality: Conclusion: Decision:

There is no difference or relationship.

Null is true: There is no difference or relationship.

Reject the null: Conclude there is a difference or relationship.

Incorrect Decision: Type I error (α)

We reject the null hypothesis when it was in fact true.

In reality, there was not a significant difference or relationship to find in this case and we found a difference or relationship.

Scenario 4:

9

Type I and Type II error:

A Type I Error is the false rejection of a true null. It has a probability equal to alpha ( ).

A Type II Error is the false retention of a false null. It has a probability equal to beta ( ).

10

Statistical power: Power is the probability of correctly

rejecting a false null hypothesis. It’s represented by 1- .

Factors affecting statistical power:1. Sample size2. Effect size3. Alpha level4. Directionality

11

Sample Size

Sample size affects power. All else being equal, a larger sample size yields more power than a smaller sample size.

12

Effect Size

Power is influenced by effect size. Effect size is the difference between the value of the null and the value of the alternative, or the desired difference to be detected. All else being equal, a test is more powerful with a larger effect size.

13

Alpha Level

The level of significance, or alpha, influences power. All else being equal, a higher alpha yields higher power. Does this mean we necessarily want to set a high alpha level?

14

Directionality

Power is affected by directionality of the test. All else being equal, a one-tailed test is more powerful than a two-tailed test. Why?

15

In-Class Activity:

Group 1: Influence of alpha level on power

Group 2: Influence of sample size on power Group 3: Influence of effect size on power

Group 4: Influence of directionality on power