Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at...
Transcript of Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at...
![Page 1: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/1.jpg)
Simulating with StatKey
Kari Lock Morgan Department of Statistical Science
Duke University [email protected]
Joint Mathematical Meetings, San Diego 1/11/13
![Page 2: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/2.jpg)
StatKey
A set of web-based, interactive, dynamic statistics tools designed for teaching
simulation-based methods at an introductory level.
Freely available at www.lock5stat.com/statkey
No login required Runs in (almost) any browser (incl. smartphones) Google Chrome App available (no internet needed) Standalone or supplement to existing technology
![Page 3: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/3.jpg)
StatKey • Developed by the Lock5 team to accompany our new book, Statistics: Unlocking the Power of Data (although can be used with any book)
• Programmed by Rich Sharp (Stanford), Ed Harcourt and Kevin Angstadt (St. Lawrence)
Robin & Patti St. Lawrence
Eric Duke
Kari Duke
Wiley (2013)
Dennis Iowa State
![Page 4: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/4.jpg)
![Page 5: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/5.jpg)
![Page 6: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/6.jpg)
• What is the average human body temperature?
• Create a confidence interval for average human body temperature based on a sample of size 50 (𝑥 = 98.26)
• Key Question: How much can statistics vary from sample to sample?
• www.lock5stat.com/statkey
Bootstrap Confidence Interval
![Page 7: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/7.jpg)
Bootstrap Confidence Interval
SE = 0.108 Distribution of
Bootstrap Statistics
98.26 2 0.108 (98.044, 98.476)
Middle 95% of bootstrap statistics
![Page 8: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/8.jpg)
Randomization Test
Mednick, Cai, Kanady, and Drummond (2008). “Comparing the benefits of caffeine, naps and placebo on verbal, motor and perceptual memory,” Behavioral Brain Research, 193, 79-86.
• Students were given words to memorize, then randomly assigned to take either a 90 min nap, or a caffeine pill. 2 ½ hours later, they were tested on their recall ability.
• 𝑥 𝑠 − 𝑥 𝑐 = 3 words
• Is sleep better than caffeine for memory?
• Key Question: What kinds of sample differences would we observe, just by random chance, if there were no actual difference?
![Page 9: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/9.jpg)
![Page 10: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/10.jpg)
Randomization Test
p-value Proportion as extreme as observed statistic
observed statistic
Distribution of Statistic Assuming Null is True
![Page 11: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/11.jpg)
• Ability to simulate one to many samples
• Helps students distinguish and keep straight the original data, a single simulated data set, and the distribution of simulated statistics
• Students have to interact with the bootstrap/randomization distribution – they have to know what to do with it
• Consistent interface for bootstrap intervals, randomization tests, theoretical distributions
StatKey Pedagogical Features
![Page 12: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/12.jpg)
• Sleep versus Caffeine:
• t-distribution
• df = 11
Theoretical Distributions
1 2
2 2 2 2
1 2
1 2
15.25 1
3.31 3.55
1
2.252.1
2 12
4ts s
n
X X
n
![Page 13: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/13.jpg)
![Page 14: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/14.jpg)
Theoretical Distributions
p-value
t-statistic
MUCH more intuitive and easier to use than tables!!!
![Page 15: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/15.jpg)
• Chi-square tests • Goodness-of-fit or test for association • Gives 2 statistic, as well as observed and expected counts for each cell • Randomization test or 2 distribution
• ANOVA • Difference in means or regression • Gives entire ANOVA table • Randomization test or F-distribution
Chi-Square and ANOVA
![Page 16: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/16.jpg)
![Page 17: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/17.jpg)
Chi-Square Statistic
Randomization Distribution
Chi-Square Distribution (3 df)
p-value = 0.357
2 statistic = 3.242
2 statistic = 3.242 p-value = 0.356
![Page 18: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/18.jpg)
• Simulate a sampling distribution
• Generate confidence intervals for each simulated statistic, keep track of coverage rate
Sampling Distributions
![Page 19: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/19.jpg)
![Page 20: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/20.jpg)
Sampling Distributions
![Page 21: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/21.jpg)
![Page 22: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/22.jpg)
Descriptive Statistics
![Page 23: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/23.jpg)
Descriptive Statistics
![Page 24: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/24.jpg)
Descriptive Statistics
![Page 25: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/25.jpg)
Descriptive Statistics
![Page 26: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/26.jpg)
Help
• Help page, including instructional videos
![Page 27: Simulating with StatKeykfl5/LockMorgan_JMM_2013.pdf · 2013-01-16 · simulation-based methods at an introductory level. ... Randomization Test Mednick, Cai, Kanady, and Drummond](https://reader034.fdocuments.us/reader034/viewer/2022042317/5f0665d37e708231d417cb09/html5/thumbnails/27.jpg)
Suggestions? Comments? Questions?
• You can email me at [email protected], or the whole Lock5 team at [email protected]