Tuesday, September 10, 2013 Introduction to hypothesis testing.
-
Upload
terence-cummings -
Category
Documents
-
view
214 -
download
2
Transcript of Tuesday, September 10, 2013 Introduction to hypothesis testing.
Tuesday, September 10, 2013
Introduction to hypothesis testing
Last time:•
Probability & the Distribution of Sample Means
• We can use the Central Limit Theorem to calculate z-scores associated with individual sample means (the z-scores are based on the distribution of all possible sample means).
• Each z-score describes the exact location of its respective sample mean, relative to the distribution of sample means.
• Since the distribution of sample means is normal, we can then use the unit normal table to determine the likelihood of obtaining a sample mean greater/less than a specific sample mean.
Probability & the Distribution of Sample Means
• When using z scores to represent sample means, the correct formula to use is:
Probability & the Distribution of Sample Means
• EXAMPLE: What is the probability of obtaining a sample mean greater than M = 60 for a random sample of n = 16 scores selected from a normal population with a mean of μ = 65 and a standard deviation of σ = 20?
• M = 60; μ = 65; σ = 20; n = 161
5
6560
MM
MZ
Last topic before the exam:• Hypothesis testing (pulls together
everything we’ve learned so far and applies it to testing hypotheses about about sample means).
• Before we move on, questions about CLT, distributions of samples, standard error of the mean and how to calculate it?
Hypothesis testing
• Example: Testing the effectiveness of a new memory treatment for patients with memory problems
– Our pharmaceutical company develops a new drug treatment that is designed to help patients with impaired memories.
– Before we market the drug we want to see if it works. – The drug is designed to work on all memory patients,
but we can’t test them all (the population). – So we decide to use a sample and conduct the following
experiment.– Based on the results from the sample we will make
conclusions about the population.
Hypothesis testing
• Example: Testing the effectiveness of a new memory treatment for patients with memory problems
Memory treatment
No Memorytreatment
Memory patients
MemoryTest
MemoryTest
55 errors
60 errors
5 error diff
• Is the 5 error difference: – A “real” difference due to the effect of the treatment– Or is it just sampling error?
Testing Hypotheses
• Hypothesis testing– Procedure for deciding whether the outcome of a study
(results for a sample) support a particular theory (which is thought to apply to a population)
– Core logic of hypothesis testing• Considers the probability that the result of a study could have
come about by chance if the experimental procedure had no effect
• If this probability is low, scenario of no effect is rejected and the theory behind the experimental procedure is supported
Hypothesis testingCan make predictions about likelihood of outcomes based on this distribution.Distribution of possible outcomes
(of a particular sample size, n)
• In hypothesis testing, we compare our observed samples with the distribution of possible samples (transformed into standardized distributions)
• This distribution of possible samples is often Normally Distributed (This follows from the Central Limit Theorem).
Inferential statistics
• Hypothesis testing– Core logic of hypothesis testing
• Considers the probability that the result of a study could have come about if the experimental procedure had no effect
• If this probability is low, scenario of no effect is rejected and the theory behind the experimental procedure is supported
• Step 1: State your hypotheses• Step 2: Set your decision criteria• Step 3: Collect your data & compute your test statistics • Step 4: Make a decision about your null hypothesis
– A four step program
– Step 1: State your hypotheses: as a research hypothesis and a null hypothesis about the populations• Null hypothesis (H0)
• Research hypothesis (HA)
Hypothesis testing
• There are no differences between conditions (no effect of treatment)
• Generally, not all groups are equal
This is the one that you test
• Hypothesis testing: a four step program
– You aren’t out to prove the alternative hypothesis • If you reject the null hypothesis, then you’re left with
support for the alternative(s) (NOT proof!)
In our memory example experiment:
Testing Hypotheses
μTreatment > μNo Treatment
μTreatment < μNo Treatment
H0:
HA:
– Our theory is that the treatment should improve memory (fewer errors).
– Step 1: State your hypotheses
• Hypothesis testing: a four step program
One -tailed
In our memory example experiment:
Testing Hypotheses
μTreatment > μNo Treatment
μTreatment < μNo Treatment
H0:
HA:
– Our theory is that the treatment should improve memory (fewer errors).
– Step 1: State your hypotheses
• Hypothesis testing: a four step program
μTreatment = μNo Treatment
μTreatment ≠ μNo Treatment
H0:
HA:
– Our theory is that the treatment has an effect on memory.
One -tailed Two -tailedno direction
specifieddirectionspecified
One-Tailed and Two-Tailed Hypothesis Tests
• Directional hypotheses– One-tailed test
• Nondirectional hypotheses– Two-tailed test
Testing Hypotheses
– Step 1: State your hypotheses– Step 2: Set your decision criteria
• Hypothesis testing: a four step program
• Your alpha (α) level will be your guide for when to reject or fail to reject the null hypothesis.
– Based on the probability of making a certain type of error
Testing Hypotheses
– Step 1: State your hypotheses– Step 2: Set your decision criteria– Step 3: Collect your data & Compute sample statistics
• Hypothesis testing: a four step program
Testing Hypotheses
– Step 1: State your hypotheses– Step 2: Set your decision criteria– Step 3: Collect your data & Compute sample statistics
• Hypothesis testing: a four step program
• Descriptive statistics (means, standard deviations, etc.)• Inferential statistics (z-test, t-tests, ANOVAs, etc.)
Testing Hypotheses
– Step 1: State your hypotheses– Step 2: Set your decision criteria– Step 3: Collect your data & compute sample statistics– Step 4: Make a decision about your null hypothesis
• Hypothesis testing: a four step program
• Based on the outcomes of the statistical tests researchers will either:
– Reject the null hypothesis– Fail to reject the null hypothesis
• This could be the correct conclusion or the incorrect conclusion
Error types
• Type I error (α): concluding that there is a difference between groups (“an effect”) when there really isn’t. – Sometimes called “significance level” or “alpha level”– We try to minimize this (keep it low)
• Type II error (β): concluding that there isn’t an effect, when there really is.– Related to the Statistical Power of a test (1-β)
Error typesReal world (‘truth’)
H0 is correct
H0 is wrong
Experimenter’s conclusions
Reject H0
Fail to Reject H0
There really isn’t an effect
There really isan effect
Error typesReal world (‘truth’)
H0 is correct
H0 is wrong
Experimenter’s conclusions
Reject H0
Fail to Reject H0
I conclude that there is an effect
I can’t detect an effect
Error typesReal world (‘truth’)
H0 is correct
H0 is wrong
Experimenter’s conclusions
Reject H0
Fail to Reject H0
Type I error
Type II error
Performing your statistical test
H0: is true (no treatment effect) H0: is false (is a treatment effect)
Two populations
One population
• What are we doing when we test the hypotheses?
Real world (‘truth’)
MA
they aren’t the same as those in the population of memory patients
MA
the memory treatment sample are the same as those in the population of memory patients.
Performing your statistical test• What are we doing when we test the hypotheses?
– Computing a test statistic: Generic test
Could be difference between a sample and a population, or between different samples
Based on standard error or an estimate of the standard error
“Generic” statistical test• The generic test statistic distribution (think of this as the
distribution of sample means)– To reject the H0, you want a computed test statistic that is large– What’s large enough?
• The alpha level gives us the decision criterion
Distribution of the test statistic
α-level determines where these boundaries go
“Generic” statistical test
If test statistic is here Reject H0
If test statistic is here Fail to reject H0
Distribution of the test statistic
• The generic test statistic distribution (think of this as the distribution of sample means)– To reject the H0, you want a computed test statistics that is large– What’s large enough?
• The alpha level gives us the decision criterion
“Generic” statistical test
Reject H0
Fail to reject H0
• The alpha level gives us the decision criterion
One -tailedTwo -tailed Reject H0
Fail to reject H0
Reject H0
Fail to reject H0
α = 0.05
0.025
0.025split up into the two tails
“Generic” statistical test
Reject H0
Fail to reject H0
• The alpha level gives us the decision criterion
One -tailedTwo -tailed Reject H0
Fail to reject H0
Reject H0
Fail to reject H0
α = 0.05
0.05all of it in one tail
“Generic” statistical test
Reject H0
Fail to reject H0
• The alpha level gives us the decision criterion
One -tailedTwo -tailed Reject H0
Fail to reject H0
Reject H0
Fail to reject H0
α = 0.05
0.05
all of it in one tail
“Generic” statistical testAn example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a memory improvement treatment.
• How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
• After the treatment they have an average score of M = 55 memory errors.
• Step 1: State the hypotheses
H0: The treatment sample is the same as (or worse than) the population of memory patients.
HA: The treatment sample does better than the population (fewer errors)
μTreatment ≥ μpop = 60
μTreatment < μpop = 60
“Generic” statistical testAn example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a memory improvement treatment.
• How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
• After the treatment they have an average score of M = 55 memory errors.
• Step 2: Set your decision criteria
μTreatment ≥ μpop = 60
μTreatment < μpop = 60
α = 0.05One -tailed
“Generic” statistical testAn example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a memory improvement treatment.
• How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
• After the treatment they have an average score of M = 55 memory errors.
α = 0.05One -tailed
• Step 3: Collect your data &
μTreatment ≥ μpop = 60
μTreatment < μpop = 60
“Generic” statistical testAn example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a memory improvement treatment.
• How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
• After the treatment they have an average score of M = 55 memory errors.
α = 0.05One -tailed• Step 3: Collect your data &
compute your test statistics
= -2.5
μTreatment ≥ μpop = 60
μTreatment < μpop = 60
“Generic” statistical testAn example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a memory improvement treatment.
• How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
• After the treatment they have an average score of M = 55 memory errors.
α = 0.05One -tailed
• Step 4: Make a decision about your null hypothesis
5%
Reject H0
μTreatment ≥ μpop = 60
μTreatment < μpop = 60
“Generic” statistical testAn example: One sample z-test
Memory example experiment:
• We give a n = 16 memory patients a memory improvement treatment.
• How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, μ = 60, σ = 8?
• After the treatment they have an average score of μ = 55 memory errors.
α = 0.05One -tailed
• Step 4: Make a decision about your null hypothesis- Reject H0
- Support for our HA, the evidence suggests that the treatment decreases the number of memory errors
μTreatment ≥ μpop = 60
μTreatment < μpop = 60