Analyzing Statistical Inferences How to Not Know Null.
-
Upload
aldous-hugh-nelson -
Category
Documents
-
view
216 -
download
0
Transcript of Analyzing Statistical Inferences How to Not Know Null.
![Page 1: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/1.jpg)
Analyzing Statistical Inferences
How to Not Know Null
![Page 2: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/2.jpg)
Agenda
• Inferential stats– Descriptive vs. Inferential– Ramifications of hypothesis testing– Tests of significance
• Action Research– Dissect Sanna paper– Outline paper– Discuss presentations
![Page 3: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/3.jpg)
Teacher Salary Example
• Descriptive Stats– Range of salary distributions– Mean– Percentages of teachers with different levels of
experience and degrees
![Page 4: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/4.jpg)
Teacher Salary Example
• What kinds of questions would we ask if we wanted to compare these 2 groups of teachers?
• Inferential Statistics
![Page 5: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/5.jpg)
Understand the need for using inferential statistics to estimate likely
conclusions
• What Are Inferential Statistics?– Inferential statistics refer to certain
procedures that allow researchers to make inferences about a population based on data obtained from a sample.
– The term “probability,” as used in research, refers to the predicted relative frequency with which a given event will occur (e.g., p-value).
![Page 6: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/6.jpg)
Descriptive vs. Inferential Stats
• Describe the data. • Means, variances,
frequencies.• Important precursor
to inferential stats
• Infer from a sample what is true of a population.
• Rely on descriptive stats.
• Ultimate goal - to draw accurate conclusions about the population
Descriptive Inferential
![Page 7: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/7.jpg)
The Notorious Null
• What is it??– A statement about a relationship– No differences between groups– No relationships between variables
• What assumption should you always make about the null?
Assume the null is accurate
![Page 8: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/8.jpg)
• Null hypothesis differs in most instances from the research hypothesis– which states that one method is expected
to be more effective than another
• Rejecting the null hypothesis provides evidence (but not proof) that the intervention had an effect
The Notorious Null
![Page 9: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/9.jpg)
Hypothesis Testing
• Tests of significance ask this question:
Could these observations really have occurred by chance?
• Example of Jury Selectionp < .0000000000000000014
![Page 10: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/10.jpg)
Probability
• Level of significance or p = probability of being wrong to reject the null (to state there is a true difference, but in reality the difference is from chance)
• In general, research should be p < .05 to be considered significant.
![Page 11: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/11.jpg)
• The decision a researcher must make is:– whether to accept the null hypothesis or to
reject it
• There are four possibilities…
![Page 12: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/12.jpg)
Decisions concerning rejecting the null hypothesis…
The true status of the null hypothesis…
True False
The
res
earc
her’s
dec
isio
n ab
out
the
null
hypo
thes
is…
True
False
CorrectType II
Error (β)
Type I Error (α)
Correct
![Page 13: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/13.jpg)
Consider an example from a kitchen…
• You probably have a smoke alarm where you live.
• You have probably made microwave popcorn or toast that set off your alarm though you had no fire.
TYPE I error
![Page 14: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/14.jpg)
• If you ever took the batteries out of your smoke alarm because you got so annoyed,…
• You ran the risk of having a fire but no alarm.
TYPE II error
![Page 15: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/15.jpg)
• If you had a fire, but your alarm worked, you’re ok.
• If you had no fire, and no alarm, you’re also ok.
• Hence…
![Page 16: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/16.jpg)
Put in cooking terms…
The true status of the kitchen…
No Fire Fire
The
sta
tus
of t
he s
mok
e al
arm
…
NoAlarm
Alarm
CorrectType II
Error (β)
Type I Error (α)
Correct
![Page 17: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/17.jpg)
Back to the null hypothesis…
The true status of the null hypothesis…
True False
The
res
earc
her’s
dec
isio
n ab
out
the
null
hypo
thes
is…
True
False
CorrectType II
Error (β)
Type I Error (α)
Correct
![Page 18: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/18.jpg)
Your Turn for Statistical Fun
• Create a null hypothesis regarding the effectiveness of 2 methods of instruction on student achievement.
• Using the chart on the previous slide, state what is occurring with this particular hypothesis.
• What are ramifications of incorrect decisions?
![Page 19: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/19.jpg)
Probability
• Level of significance or p = probability of being wrong to reject the null (to state there is a true difference, but in reality the difference is from chance)
• In general, research should be p < .05 to be considered significant.
![Page 20: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/20.jpg)
Things that effect p
• Difference between 2 groups
• Sampling and/or measurement error
• Size of the sample
![Page 21: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/21.jpg)
Steps in using inferential statistics
1. Select the test of significance
2. Determine whether significance test will be two-tailed or one tailed
3. Select α (alpha), the probability level (usually <.05)
4. Compute the test of significance5. Consult table to determine the
significance of the results
![Page 22: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/22.jpg)
How to determine p
Tests of Significance
![Page 23: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/23.jpg)
Tests of Significance
• Statistical formulas that enable the researcher to determine if there was a real difference between the sample means
• Examples– t test– ANOVA– Chi-square
![Page 24: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/24.jpg)
t test
• Used to determine whether two means are significantly different at a selected probability level
• Adjusts for the fact that the distribution of scores for small samples becomes increasingly different from the normal distribution as sample sizes become increasingly smaller
• Sample t-table
![Page 25: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/25.jpg)
t test
• If the t value is equal to or greater than the table value, then the null hypothesis is rejected because the difference is greater than would be expected due to chance
![Page 26: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/26.jpg)
Reminder…
• Don’t forget the purpose. You are going through this statistical rigmarole because you want to know
Could these observations really have occurred by chance?
![Page 27: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/27.jpg)
ANOVA
• A comparison of the means for two or more groups
• Example - Do the mean scores differ for the groups using co-operative group, lecture, or web-based instruction?
• The assumption is that randomly formed groups of participants are chosen and are essentially the same at the beginning of a study on a measure of the dependent variable
![Page 28: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/28.jpg)
ANOVA
• F value of ANOVA is similar to t-value in t-test.
• If F value is significant, you know there is a difference somewhere, but have to do post hoc tests to figure out where.
![Page 29: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/29.jpg)
Chi-Square
• Tests differences in frequencies across different categories– Do mothers and fathers differ in their
support of a year-round school calendar?– Do the percentages of undergraduate,
graduate, and doctoral students differ in terms of their support for the new class attendance policy?
![Page 30: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/30.jpg)
Some words about significancesignificance
• “Statistical significance” is a term that refers to some statistical criterion, usually the numerical value of some formula or calculation.
• “Practical significance” means its utility, and that is in the eyes of the beholder. What may be impractical to you or me may be very practical to someone else.
![Page 31: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/31.jpg)
Practical Significance
• An Example – A new reading program shows improved comprehension scores that are statistically significant. However, it takes many hours and dollars to train teachers to use the program. Does it warrant buying the new program?
• Big Question: Is it practical to use the results?
![Page 32: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/32.jpg)
“There is no magical or purely technical way to decide whether or not a statistically significant difference means you should do something different in your school. There are only tools that assist your judgment. There is no escape from using judgment.”
--Gerald W. BraceyReading Educational Research: How
To Avoid Getting Statistically Snookered
![Page 33: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/33.jpg)
Practical Significance
Large sample sizes can produce a statistically significant result even though there is limited or no practical importance associated with the finding.
![Page 34: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/34.jpg)
Effect Size
• Take into account variance, not just the means.
• Refers to the magnitude of a difference.
• Levels you should knowd ≥ .75 = large effect
d ~ .5 = moderate effect
d ~ .3 = small effect
• Good website on Effect Size http://www.cemcentre.org/renderpage.asp?linkID=30325016
![Page 35: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/35.jpg)
Evaluation Criteria
• Basic descriptive statistics are needed to evaluate the inferential results
• Inferential analyses report statistical significance, not practical significance
• Inferential analyses do not indicate internal or external validity
• The results depend on sample sizes
![Page 36: Analyzing Statistical Inferences How to Not Know Null.](https://reader036.fdocuments.us/reader036/viewer/2022070403/56649f2c5503460f94c47baf/html5/thumbnails/36.jpg)
Evaluation Criteria
• The appropriate statistical procedures are used
• The level of significance is interpreted correctly