How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics...
Transcript of How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics...
![Page 1: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/1.jpg)
How to Handle Intervals in a Simulation-Based
Curriculum?
Robin LockBurry Professor of Statistics
St. Lawrence University
2015 Joint Statistics Meetings – Seattle, WAAugust 2015
![Page 2: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/2.jpg)
Simulation-Based Inference (SBI) Projects
• Lock5 lock5stat.com
• Tintle, et al math.hope.edu/isi
• Catalst www.tc.umn.edu/~catalst
• Tabor/Franklin www.highschool.bfwpub.com
• Open Intro www.openintro.org
![Page 3: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/3.jpg)
SBI Blogwww.causeweb.org/sbi/
How should we teach about intervals when using simulation-based inference?
![Page 4: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/4.jpg)
Assumptions1. We agree with George Cobb (TISE 2007):
2. Statistical inference has two main components:
“Randomization-based inference makes a direct connection between data production and the
logic of inference that deserves to be at the core of every introductory course.”
• Estimation (confidence interval)
• Hypothesis test (p-value)
![Page 5: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/5.jpg)
Assumptions3. For a randomized experiment to compare two groups:
Confidence interval via simulation?
???
Hypothesis test via simulation?
Randomization (permutation) test
3. For a parameter based on a single sample:
![Page 6: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/6.jpg)
CI: Potential Initial Approaches1. Invert hypothesis tests CI =plausible parameter values that
would not be rejected
2. Bootstrap CI=
Or CI = Percentile Interval
3. Traditional formulas
![Page 7: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/7.jpg)
Example: Proportion of Orange Reese’s Pieces
Sample: n=150 Reese’s Pieces
72 are orange
Key question: How accurate is a proportion estimated from 150 Reese’s pieces?
![Page 8: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/8.jpg)
Invert the TestTest vs. at α=0.05 using for a sample of size n=150.
Say is in the 95% CI for p ⇔ is not rejected.Guess/check: p-value=0.674
p-value=0.0024
p-value=0.104
p-value=0.060
p-value=0.040
p-value=0.050
95% CI for p ( , 0.562)
Repeat for the lower tail or use symmetry
![Page 9: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/9.jpg)
Invert the TestPros: • Reinforces ideas from hypothesis tests• Makes connection with CI as “plausible”
values for the parameter
Cons: • Tedious (especially with randomization
tests)-even with technology• Harder to make a direct connection with
variability (SE) of the sample statistic• Requires tests first• How do we do a CI for a single mean?
![Page 10: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/10.jpg)
BootstrapBasic idea: • Sample (with replacement) from the original sample • Compute the statistic for each bootstrap sample• Repeat 1000’s of times to get bootstrap distribution• Estimate the SE of the statistic
Get a confidence interval with:
orPercentiles from the bootstrap distribution
![Page 11: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/11.jpg)
Simulated Reese’s Population
Sample from this “population”
Original Sample
![Page 12: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/12.jpg)
Original Sample
BootstrapSample
BootstrapSample
BootstrapSample
●●●
Bootstrap Statistic
Sample Statistic
Bootstrap Statistic
Bootstrap Statistic
●●●
Bootstrap Distribution
Many times
We need technology!
![Page 13: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/13.jpg)
StatKeyhttp://lock5stat.com/statkey
![Page 14: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/14.jpg)
StatKeyhttp://lock5stat.com/statkey
![Page 15: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/15.jpg)
SE of
![Page 16: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/16.jpg)
SE of
![Page 17: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/17.jpg)
Version 1 (): Great preparation for moving to traditional methods
Version 2 (): Great at building understanding of confidence intervals
Same process used for different parameters
Bootstrap Confidence Intervals
![Page 18: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/18.jpg)
3.849±2 ⋅0.194=3.85±0.388=(3.46 ,4.24)
![Page 19: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/19.jpg)
Why does the bootstrap
work?
![Page 20: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/20.jpg)
Sampling Distribution
Population
µ
BUT, in practice we don’t see the “tree” or all of the “seeds” – we only have ONE seed
![Page 21: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/21.jpg)
Bootstrap Distribution
Bootstrap“Population”
What can we do with just one seed?
Grow a NEW tree!
𝑥 µ
Estimate the variability (SE) of ’s from the bootstraps
Chris Wild: Use the bootstrap errors that we CAN see to estimate the sampling errors that we CAN’T see.
![Page 22: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/22.jpg)
Transition to Traditional Formulas
𝑆𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 ±2∗𝑆𝐸Use z* or t*
𝑆𝐸=√ �̂� (1− �̂�)𝑛or
or …
![Page 23: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/23.jpg)
BootstrapPros: • Illustrates directly how statistics vary from
sample to sample • Follows naturally from sampling/statistics• generalizes easily to traditional formulas• Same process can be applied to lots of statistics• Can connect to tests, but doesn’t require tests
Cons: • Requires software• Tedious to demonstrate “by hand”• Doesn’t always “work”
![Page 24: How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e735503460f94b7296a/html5/thumbnails/24.jpg)
Want to Know More?
What Teachers Should Know about the Bootstrap: Resampling in the
Undergraduate Statistics Curriculum
Tim Hesterberg
http://arxiv.org/abs/1411.5279
Thanks for [email protected]