PSY 1950 Null Hypothesis Significance Testing September 29, 2008
Lesson 11 - 1 Significance Tests: The Basics. Vocabulary Hypothesis – a statement or claim...
-
Upload
gwendoline-stanley -
Category
Documents
-
view
221 -
download
2
Transcript of Lesson 11 - 1 Significance Tests: The Basics. Vocabulary Hypothesis – a statement or claim...
Lesson 11 - 1
Significance Tests:
The Basics
Vocabulary• Hypothesis – a statement or claim regarding a characteristic of
one or more populations
• Hypothesis Testing – procedure, base on sample evidence and probability, used to test hypotheses
• Null Hypothesis – H0, is a statement to be tested; assumed to be true until evidence indicates otherwise
• Alternative Hypothesis – H1, is a claim to be tested(what we will test to see if evidence supports the possibility)
• Level of Significance – probability of making a Type I error, α
Steps in Hypothesis Testing
• A claim is made
• Evidence (sample data) is collected to test the claim
• The data are analyzed to assess the plausibility (not proof!!) of the claim
• Note: Hypothesis testing is also called Significance testing
Hypotheses: Null H0 & Alternative Ha
• Think of the null hypothesis as the status quo• Think of the alternative hypothesis as something has
changed or is different than expected
• We can not prove the null hypothesis! We only can find enough evidence to reject the null hypothesis or not.
Hypotheses Cont
• Our hypotheses will only involve population parameters (we know the sample statistics!)
• The alternative hypothesis can be – one-sided: μ > 0 or μ < 0 (which allows a statistician to detect
movement in a specific direction)– two-sided: μ 0 (things have changed)
• Read the problem statement carefully to decide which is appropriate
• The null hypothesis is usually “=“, but if the alternative is one-sided, the null could be too
Three Ways – Ho versus Ha
1. Equal versus less than (left-tailed test)H0: the parameter = some value (or more)H1: the parameter < some value
2. Equal hypothesis versus not equal hypothesis (two-tailed test)H0: the parameter = some valueH1: the parameter ≠ some value
3. Equal versus greater than (right-tailed test)H0: the parameter = some value (or less)H1: the parameter > some value
ba ba
Critical Regions
1 2 3
English Phrases Revisited
Math Symbol English Phrases
≥ At least No less thanGreater than or
equal to> More than Greater than< Fewer than Less than
≤ No more than At mostLess than or
equal to= Exactly Equals Is ≠ Different from
Example 1
A manufacturer claims that there are at least two scoops of cranberries in each box of cereal
Parameter to be tested:
Test Type:
H0:
Ha:
left-tailed testThe “bad case” is when there are too few
Scoops = 2 (or more) (s ≥ 2)
Less than two scoops (s < 2)
number of scoops of cranberries in each box of cereal
If the sample mean is too low, that is a problemIf the sample mean is too high, that is not a problem
Example 2A manufacturer claims that there are exactly 500 mg of a medication in each tablet
Parameter to be tested:
Test Type:
H0:
Ha:
Two-tailed test A “bad case” is when there are too few A “bad case” is also where there are too many
amount of a medication in each tablet
If the sample mean is too low, that is a problem If the sample mean is too high, that is a problem
too
Amount = 500 mg Amount ≠ 500 mg
Example 3A pollster claims that there are at most 56% of all Americans are in favor of an issue
Parameter to be tested:
Test Type:
H0:
Ha:
right-tailed test The “bad case” is when sample proportion is too high
population proportion in favor of the issue
If p-hat is too low, that is not a problem If p-hat is too high, that is a problem
P = 56% (or less)
P > 56%
Conditions for Significance Tests
• SRS– simple random sample from population of interest
• Normality– For means: population normal or large enough
sample size for CLT to apply or use t-procedures– t-procedures: boxplot or normality plot to check for
shape and any outliers – For proportions: np ≥ 10 and n(1-p) ≥ 10
• Independence – Population, N, such that N > 10n
Test StatisticsPrinciples that apply to most tests:
• The test is based on a statistic that compares the value of the parameter as stated in H0 with an estimate of the parameter from the sample data
• Values of the estimate far from the parameter value in the direction specified by Ha give evidence against H0
• To assess how far the estimate is from the parameter, standardize the estimate. In many common situations, the test statistic has the form:
estimate – hypothesized valuetest statistic = ------------------------------------------------------------ standard deviation of the estimate (ie SE)
Example 4
Several cities have begun to monitor paramedic response times. In one such city, the mean response time to all accidents involving life-threatening injuries last year was μ=6.7 minutes with σ=2 minutes. The city manager shares this info with the emergency personnel and encourages them to “do better” next year. At the end of the following year, the city manager selects a SRS of 400 calls involving life-threatening injuries and examines response times. For this sample the mean response time was x-bar = 6.48 minutes. Do these data provide good evidence that the response times have decreased since last year?
List parameter, hypotheses and conditions check
Example 4 cont
Parameter: response times of paramedics to life threatening events
H0:
Ha:
Conditions Check:
1) :
2) :
3) :
μ = 6.7 minutes (unchanged)
μ < 6.7 minutes (they got “better”)
SRS stated in problem statement
Normality n = 400 suggest CLT would apply to x-bar
Independence n = 400 means we must assume over 4000 callseach year that involve life-threatening injuries
Homework
• Pg. 693 & 698
• 11.3, 4, 5, 7, 8, 9
Hypothesis Testing Approaches
• P-Value– Logic: Assuming H0 is true, if the probability of getting a
sample mean as extreme or more extreme than the one obtained is small, then we reject the null hypothesis (accept the alternative).
• Classical (Statistical Significance) – Logic: If the sample mean is too many standard deviations
from the mean stated in the null hypothesis, then we reject the null hypothesis (accept the alternative)
• Confidence Intervals– Logic: If the sample mean lies in the confidence interval about
the status quo, then we fail to reject the null hypothesis
-z*α/2 z*α/2
Reject Regions
x – μ0
Test Statistic: z0 = ------------- z* = invnorm(1-α/2) σ/√n
Reject null hypothesis, if
Left-Tailed Two-Tailed Right-Tailed
Not usually done
z0 < - z*
or
z0 > z*
Not usually done
Confidence Interval Approach
μ0
UBLB
Reject Regions
FTR Region
zα-zα/2 zα/2-zα
Reject Regions
x – μ0
Test Statistic: z0 = ------------- σ/√n
Reject null hypothesis, if
Left-Tailed Two-Tailed Right-Tailed
z0 < - zα
z0 < - zα/2
or
z0 > z α/2
z0 > zα
Classical Approach
P-value
• P-value is the probability of getting a more extreme value if H0 is true (measures the tails)
• Small P-values are evidence against H0
– observed value is unlikely to occur if H0 is true
• Large P-values fail to give evidence against H0
z0-|z0| |z0|z0
P-Value is thearea highlighted
x – μ0
Test Statistic: z0 = ------------- σ/√n
Reject null hypothesis, if
P-Value < α
P-Value Approach
• Probability(getting a result further away from the point estimate) = p-value
• P-value is the area in the tails!!
Example 5: P-Values
For each α and observed significance level (p-value) pair, indicate whether the null hypothesis would be rejected.
a)α = . 05, p = .10
b)α = .10, p = .05
c)α = .01 , p = .001
d)α = .025 , p = .05
e) α = .10, p = .45
α < P fail to reject Ho
α < P fail to reject Ho
α < P fail to reject Ho
P < α reject Ho
P < α reject Ho
Example 4 cont
• What is the P-value associated with the data in example 4?
• What if the sample mean was 6.61?
x – μ0 6.48 – 6.7Z0 = ----------- = -------------- σ/√n 0.10
= -2.2
P(z < Z0) = P(z < -2.2) = 0.0139 (unusual !)
x – μ0 6.61 – 6.7Z0 = ----------- = -------------- = - 0.9 σ/√n 0.10
P(z < Z0) = P(z < -0.9) = 0.1841 (not unusual !)
Two-sided Test P-value
• P-value is the sum of both tail areas in the two sided test case
Statistical Significance
• Statistically significant means simply that it is not likely to happen just by chance
• Significant in the statistical sense does not mean important
• Very large samples can make very small differences statistically significant, but not practically important
Statistical Significance – P-value
When using a P-value, we compare it with a level of significance, α, decided at the start of the test.
•Not significant when α < P•Significant when α ≥ P
Fail to Reject H0 Reject H0
Statistical Significance Interpretation
Remember the three C’s: Conclusion, connection, context
• Conclusion: Either we have evidence to reject H0 in favor of Ha or we fail to reject
• Connection: connect your calculated values to your conclusion
• Context: Always put it in terms of the problem (don’t use generalized statements)
Statistical Significance Warnings
• If you are going to draw a conclusion based on statistical significance, then the significance level α should be stated before the data are produced– Deceptive users of statistics might set an α level
after the data have been analyzed to manipulate the conclusion
– P-values give a better sense of how strong the evidence against H0 is
• This is just as inappropriate as choosing an alternative hypothesis to be one-sided in a particular direction after looking at the data
Summary and Homework• Summary
– Significance test assesses evidence provided by data against H0 in favor of Ha
– Ha can be two-sided (different, ≠) or one-sided (specific direction, < or >)
– Same three conditions as with confidence intervals– Test statistic is usually a standardized value– P-value, the probability of getting a more extreme
value given that H0 is true is small we reject H0
• Homework– 13, 14, 16