Hypothesis Testing
description
Transcript of Hypothesis Testing
Hypothesis Testing
Ginger Holmes Rowell, Ph. D.
April 24, 2002
Hypothesis Testing (HT)
•Uses
•Types
•General Procedures
•Example
•Review of Error Types* Type I & Type II
Hypothesis Testing - Uses• Medical – Clinical Trials
* Show that Drug A lowers cholesterol more than the placebo (2-Sample for difference in means)
* Show that the percent of people who have side effects from from Drug A is NOT different from the group taking the placebo (2-Sample for difference in proportions)
Hypothesis Testing - Uses• Agriculture
* Show that fertilizer A works better than fertilizer B
• Psychology Experiments* Show that Intervention A keeps kids off of
drugs better than what has been done traditionally
• Education* Show that teaching method A is more effective
than teaching method B
Two Broad Categories of HT
Parametric Methods: frequently used, more common
Nonparametric Methods: used if sample size is small and populations are not normal(We will not talk about this type of HT.)
Param etric M ethods (Make assum ptions about the underlying distributions, like norm ality)
Z-test t-test
Mea ns
Zfor la rge sa m ple
Proportion
C hi-Squa re
Va ria nce
1-Sa m ple
ZLa rge Sa m ple
tSm a ll Sa m ple
D ifferences in Mea ns
Proportion ZLa rge Sa m ple
D ifferences in Proportions
F test
D ifferences in Va ria nces
2-Sa m ples
O n e-W ay A N O V A
M ean s
C h i-S q u areG ood n ess o f F it Tes t
P rop ortion s
L even e 's o r B artle tt 's
V arian ces
M ore than 2Independent Sam ples
HT General Procedure - Steps
* Conduct a literature review or pilot study
* Form your hypothesis* Set your threshold of error (determine
desired significance level)
* Design your experiment* Determine which HT to use* Determine Needed Sample Size
HT Procedure: Steps Contin.
* Collect your data
* Enter data into appropriate tool (calculator, statistical software)
* Examine data graphically
* Find descriptive statistics• Use these methods to check for
typos, errors, …
HT Procedure: Steps Contin.
* Find your “test statistic” and use it determine whether your data supports the null or alternative hypothesis.
HT Example
•Research Hypothesis* Using CI Applets will help students
learn more about CI’s
•Set Error Limit: 5% for Type I Error
•Design Experiment* Give a pre-test, students use CI
applet, give a post-test
Statistical Analysis
• Hypothesis* Using the java applets will improve content
knowledge for math teachers• Ho: improvement = 0
• Ha: improvement > 0
• Improvement = Posttest score – Pretest Score
• Statistical Test* Paired difference t-test (small sample)
• Average improvement was approximately normal
HT Example
•Decide which HT to use ???
•Decide the needed sample size
•Collect Data* You helped with that part
•Examine Data* Graphically, descriptive statistics* Check for typos, errors, ...
HT Example - DATA
1 4 3 2 -1 3 -1 0 1 1 2 0
Points Improved = post test score – pre test score
(Each test is scored out of 5 possible points)
HT Example - DATA
1 4 3 2 -1 3 -1 0 1 1 2 0
Variable N Mean StDev SE Mean
Points Improved 12 1.3 1.6 0.463
-1 0 1 2 3 4
C1
(with Ho and 95% t-confidence interval for the mean)
[ ]
X_
Ho
Points Improved
HT Example
•What does our data say?
• Is that enough evidence to reject the null in favor of the alternative?* What do you think?* How much evidence do you need?* Especially with this small sample size.
HT Example - The WORK
• Test statistic
• Interpretation: The test statistic tells you the number of standard deviations that the sample mean (or proportion or variance) falls from the hypothesized value.
0
sample mean hypothesized valuestandard deviation of the mean
t
xt
sn
Using Your TI-83 for HT
•Press STAT>TESTS>t-test* Input: DATA0: 0
* LIST: LI
* Freq: 1: >
* select Calculate
HT Example
• Test Statistic = t= 2.70• What does this tell us? (Remember the Empirical Rule)
• If the null is true, the chance of getting a data set like ours or one that supports the alternative even more is small (1% chance).
• We got our data set & did not make any errors.
• Do you believe the null hypothesis is true?
HT Example
• Conclusion* Reject the null in favor of the alternative
hypothesis
• What does that mean in the context of our problem?* We can expect an average improvement in
content knowledge for math teacher who use the Regression applet
HT Example
•Next Question: How much improvement
•Answer: Find a 95% confidence interval* Review – use your calculator to do
this and interpret the result.
Conclusion: Regression Applet
• A statistically significant improvement in their average content knowledge can be expected for math teachers using the Regression applet (Rice Probability Webs). (t=2.7, n=12, p=0.01)
• We expect (with 95% confidence) that the average improvement in content knowledge of regression will be between 0.2 and 2.3 points on a 5 point scale.
Review
•Null Hypothesis
•Alternative
Type I and II Errors
•Type I Error
•Type II Error
•Power of Test
Sample Size
•What is the effect of sample size on statistical power?
Comments
• I hope you * Understand how to think about
forming a hypothesis* Understand that actually testing the
hypothesis is more than looking at the two sample averages and saying whether you think they are different.
Questions
????????