What’s in the What’s in the Bag?Bag?Lecture 4Lecture 4

Section 1.4.3Section 1.4.3

Wed, Jan 25, 2006Wed, Jan 25, 2006

How Strong is the How Strong is the Evidence?Evidence?

Rather than give an accept/reject Rather than give an accept/reject answer, we may ask a different answer, we may ask a different question:question: How strong is the evidence against How strong is the evidence against HH00??

The The pp-value-value

pp-value-value – The likelihood of getting – The likelihood of getting by by chancechance a value at least as extreme a value at least as extreme as the one observed as the one observed if if HH00 is true is true..

Two BagsTwo Bags

If the selected token is worth $50, If the selected token is worth $50, what is the what is the pp-value?-value?

Two BagsTwo Bags

-1000 10 20 30 40 60 100050

Bag A

-1000 10 20 30 40 60 100050

Bag B

Two BagsTwo Bags

-1000 10 20 30 40 60 100050

Bag A

-1000 10 20 30 40 60 100050

Bag B

At least as extreme as 50

Two BagsTwo Bags

-1000 10 20 30 40 60 100050

Bag A

-1000 10 20 30 40 60 100050

Bag B

p-value = 2/20 = 0.10

At least as extreme as 50

A Two-Sided TestA Two-Sided Test

Bag F

1 2 3 4 65

Bag E

8 1097

1 2 3 4 65 8 1097

A Two-Sided TestA Two-Sided Test

If the selected token is worth $8, If the selected token is worth $8, what is the what is the pp-value?-value?

First, what is the direction of First, what is the direction of extreme?extreme?

Which values are at least as extreme Which values are at least as extreme as 8?as 8?

1 2 3 4 65

Bag E

8 1097

A Two-Sided TestA Two-Sided Test

1 2 3 4 65

Bag E

Bag F

8 1097

1 2 3 4 65 8 1097

At least as extreme as 8At least as extreme as 8

A Two-Sided TestA Two-Sided Test

1 2 3 4 65

Bag E

Bag F

8 1097

1 2 3 4 65 8 1097

p-value = 12/30 = 0.40

A Two-Sided TestA Two-Sided Test

In a two-sided test, if the null In a two-sided test, if the null distribution is symmetric, then you distribution is symmetric, then you can compute the probability in one can compute the probability in one direction, and then double it to get direction, and then double it to get the the pp-value.-value.

The The pp-value-value

A A smallsmall pp-value is strong evidence -value is strong evidence againstagainst the null hypothesis. the null hypothesis.

Why?Why? A A largelarge pp-value is evidence -value is evidence in favorin favor

of the null hypothesis. of the null hypothesis.

The The pp-value-value

Put differently, a Put differently, a smallsmall pp-value is -value is statisticallystatistically significantsignificant..

A A largelarge pp-value is -value is not statistically not statistically significantsignificant..

This may seem counterintuitive This may seem counterintuitive since, generally speaking, small since, generally speaking, small things are insignificant and large things are insignificant and large things are significant.things are significant.

The The pp-value-value

Prevalence and Cardiovascular DisePrevalence and Cardiovascular Disease Correlates of Low ase Correlates of Low CardiorespiratoryCardiorespiratory Fitness in Adolescents and Adults Fitness in Adolescents and Adults

Two Explanations of Two Explanations of Unusual ObservationsUnusual Observations

The null hypothesis leads us to a certain The null hypothesis leads us to a certain expectation of what the data will show.expectation of what the data will show.

Allowing for some randomness, if the Allowing for some randomness, if the data are close to our expectation, then data are close to our expectation, then we have no reason to doubt the null we have no reason to doubt the null hypothesis.hypothesis.

But if the data deviate far from our But if the data deviate far from our expectation, then we doubt the truth of expectation, then we doubt the truth of the null hypothesis.the null hypothesis.

Two Explanations of Two Explanations of Unusual ObservationsUnusual Observations

Under the null hypothesis, such a Under the null hypothesis, such a deviation would be due to chance.deviation would be due to chance.

The The pp-value measures the likelihood -value measures the likelihood of a deviation as large as the one we of a deviation as large as the one we observed if the null hypothesis is observed if the null hypothesis is true.true. Small deviations are likely.Small deviations are likely. Large deviations are unlikely.Large deviations are unlikely.

Two Explanations of Two Explanations of Unusual ObservationsUnusual Observations

There are two potential explanations There are two potential explanations for any deviation:for any deviation: HH00 is true and the deviation occurred is true and the deviation occurred by by

chancechance.. HH00 is false. is false.

Given the evidence, i.e., the size of Given the evidence, i.e., the size of the deviation, which explanation is the deviation, which explanation is more believable?more believable?

ExampleExample

Consider the Intelligent Design Consider the Intelligent Design hypothesis vs. the Evolution hypothesis vs. the Evolution hypothesis.hypothesis. Which hypothesis uses the “chance” Which hypothesis uses the “chance”

explanation?explanation? Which hypothesis uses a non-random, Which hypothesis uses a non-random,

directed mechanism as the explanation?directed mechanism as the explanation? Which would be the null hypothesis?Which would be the null hypothesis? Which would have the burden of proof?Which would have the burden of proof?