Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the...

21
Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University [email protected] [email protected] Panel a 2013 Joint Mathematics Meetings San Diego, CA

Transcript of Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the...

Page 1: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

RandomizationandBootstrapMethodsintheIntroductory

StatisticsCourse

KariLockMorgan RobinLockDukeUniversity St.LawrenceUniversity

[email protected] [email protected]

Panela2013JointMathematicsMeetingsSanDiego,CA

Page 2: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

HowmighttheIntroStatcurriculumchangeto

accommodate/takeadvantageofbootstrap/randomization

methods?

Page 3: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

IntroStat– TraditionalTopics• DescriptiveStatistics– oneandtwosamples• Normaldistributions• Dataproduction(samples/experiments)

• Samplingdistributions(mean/proportion)

• Confidenceintervals(means/proportions)

• Hypothesistests(means/proportions)

• ANOVAforseveralmeans,Inferenceforregression,Chi-squaretests

Page 4: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

IntroStat– RevisetheTopics• DescriptiveStatistics– oneandtwosamples• Normaldistributions• Dataproduction(samples/experiments)

• Samplingdistributions(mean/proportion)

• Confidenceintervals(means/proportions)

• Hypothesistests(means/proportions)

• ANOVAforseveralmeans,Inferenceforregression,Chi-squaretests

• Dataproduction(samples/experiments)• Bootstrapconfidenceintervals• Randomization-basedhypothesistests• Normaldistributions

• Bootstrapconfidenceintervals• Randomization-basedhypothesistests

• DescriptiveStatistics– oneandtwosamples

Page 5: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

WhystartwithBootstrapCI’s?•Minimalprerequisites:

Populationparametervs.samplestatisticRandomsamplingDotplot (orhistogram)Standarddeviationand/orpercentiles

• SamemethodofrandomizationinmostcasesSamplewithreplacementfromoriginalsample

• NaturalprogressionSampleestimate==>Howaccurateistheestimate?

• Intervalsaremoreuseful?Agooddebateforanothersession…

Page 6: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

Example:MustangPrices

Data:Sampleof25MustangslistedonAutotrader.com

Findaconfidence intervalfortheslope ofaregression linetopredictpricesofusedMustangsbasedontheirmileage.

Page 7: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

“Bootstrap”SamplesKeyidea:• Samplewithreplacementfromtheoriginalsampleusingthesamen.

• Computethesamplestatisticforeachbootstrapsample.

• Collectlotsofsuchbootstrapstatistics

Imaginethe“population”ismany,manycopiesoftheoriginalsample.

Page 8: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

Distributionof3000BootstrapSlopes

Page 9: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

UsingtheBootstrapDistributiontoGetaConfidenceInterval– Version#1

Thestandarddeviationofthebootstrapstatisticsestimatesthestandarderrorofthesamplestatistic.

Quickintervalestimate:

𝑂𝑟𝑖𝑔𝑖𝑛𝑎𝑙𝑆𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 ± 2 / 𝑆𝐸ForthemeanMustangslopetime:

)162.0,278.0(058.022.0029.0222.0 −−=−±−=⋅±−

Page 10: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

UsingtheBootstrapDistributiontoGetaConfidenceInterval– Version#2

Keep95%inmiddle

Chop2.5%ineachtail

Chop2.5%ineachtail

95%CIforslope(-0.279,-0.163)

Page 11: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

3.SimulationTechnology?

Fall2010:FathomFall2011:Fathom&Applets

Tactilesimulationsfirst?Bootstrap– No(withreplacementistough)Testforanexperiment– Yes(1or2)

Page 12: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

DesirableTechnologyFeatures?

ThreeDistributions

OnetoManySamples

Page 13: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

DesirableTechnologyFeatures

Page 14: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

4.OneCrankorTwo?

ConfidenceIntervals– Bootstrap– onecrank

SignificanceTests– Two(ormore)cranks

Rulesforselectingrandomizationsamplesforatest.Beconsistentwith:1. thenullhypothesis2. thesampledata3. thewaydatawerecollected

Page 15: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

RandomizationTestforSlope

Page 16: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

5.Testfora2x2Table

Firstexample:ArandomizedexperimentTeststatistic:CountinonecellRandomize:TreatmentgroupsMargins:FixbothLaterexamplesvary,e.g.usedifferenceinproportionsorrandomizeasindependentsampleswithcommonp.

Page 17: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

6.Whatabout“traditional”methods?

AFTERstudentshaveseenlotsofbootstrapandrandomizationdistributions(andhopefullybeguntounderstandthelogicofinference)…

• Introducethenormaldistribution(andlatert)

• Introduce“shortcuts”forestimatingSEforproportions,means,differences,…

Page 18: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

BacktoMustangPricesThe regression equation isPrice = 30.5 - 0.219 Miles

Predictor Coef SE Coef T PConstant 30.495 2.441 12.49 0.000Miles -0.21880 0.03130 -6.99 0.000

S = 6.42211 R-Sq = 68.0% R-Sq(adj) = 66.6%

Page 19: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

7.Assessment?

Newlearninggoals• Understandhowtogeneratebootstrap

samplesanddistribution.• Understandhowtocreaterandomization

samplesanddistribution.• Beabletouseabootstrap/randomization

distributiontofindaninterval/p-value.

Page 20: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

8.Howdiditgo?• Studentsenjoyedandwereengagedwiththenewapproach• Instructorenjoyedandwasengagedwiththenewapproach.• Betterunderstandingofp-valuereflecting“ifH0 istrue”.• Betterinterpretationsofintervals.• Challenge:Few“experienced”studentstoserveasresources.

Page 21: Randomization and Bootstrap Methods in the …Randomization and Bootstrap Methods in the Introductory Statistics Course Kari Lock Morgan Robin Lock Duke University St. Lawrence University

Goingforward

Continuewithrandomizationapproach?

ABSOLUTELY(3sectionsinFall2011)


CTN Research Update - American Psychological … Development Preparation for Implementation Randomization Treatment+Follow-up of Last Participant Data Lock 2013 2014 20162015 B. Tai

CTN Research Update - American Psychological … Development Preparation for Implementation Randomization Treatment+Follow-up of Last Participant Data Lock 2013 2014 20162015 B. Tai

Estimating Sampling Variability Through Bootstrapping · Figure 2: Bootstrap resample of 25 used Mustang cars (Lock Morgan, 2014). Figure 3 plots the sample means from 4000 bootstrap

Estimating Sampling Variability Through Bootstrapping · Figure 2: Bootstrap resample of 25 used Mustang cars (Lock Morgan, 2014). Figure 3 plots the sample means from 4000 bootstrap

1 Randomization Algorithmic design patterns. n Greed. n Divide-and-conquer. n Dynamic programming. n Network flow. n Randomization. Randomization. Allow.

1 Randomization Algorithmic design patterns. n Greed. n Divide-and-conquer. n Dynamic programming. n Network flow. n Randomization. Randomization. Allow.

Introducing Inference with Bootstrap and Randomization Procedures

Introducing Inference with Bootstrap and Randomization Procedures

Using Bootstrap Intervals and Randomization Tests to Enhance Conceptual Understanding in Introductory Statistics Kari Lock Morgan Department of Statistical.

Using Bootstrap Intervals and Randomization Tests to Enhance Conceptual Understanding in Introductory Statistics Kari Lock Morgan Department of Statistical.

Randomization Ppt

Randomization Ppt

A Comparison of Statistical Significance Tests for ...msmucker/... · dent’s t-test, randomization, permutation, bootstrap 1. INTRODUCTION A chief goal of the information retrieval

A Comparison of Statistical Significance Tests for ...msmucker/... · dent’s t-test, randomization, permutation, bootstrap 1. INTRODUCTION A chief goal of the information retrieval

Example 7.1.1: Randomization

Example 7.1.1: Randomization

Randomization workshop

Randomization workshop

MENDELIAN RANDOMIZATION

MENDELIAN RANDOMIZATION

Hypothesis Testing: p-value 2/13/12 Randomization distribution p-value Statistical significance Section 4.2 Professor Kari Lock Morgan Duke University.

Hypothesis Testing: p-value 2/13/12 Randomization distribution p-value Statistical significance Section 4.2 Professor Kari Lock Morgan Duke University.

Wild Bootstrap Randomization Inference for Few …...Wild Bootstrap Randomization Inference for Few Treated Clusters James G. MacKinnon Queen's University jgm@econ.queensu.ca Matthew

Wild Bootstrap Randomization Inference for Few …...Wild Bootstrap Randomization Inference for Few Treated Clusters James G. MacKinnon Queen's University [email protected] Matthew

Basics of Randomization

Basics of Randomization

What Teachers Should Know about the Bootstrap: … tone is informal, with a few stories and jokes. Keywords: Teaching, bootstrap, permutation test, randomization test 1 arXiv:1411.5279v1

What Teachers Should Know about the Bootstrap: … tone is informal, with a few stories and jokes. Keywords: Teaching, bootstrap, permutation test, randomization test 1 arXiv:1411.5279v1

Randomization and Restart Strategiesvanbeek/Publications/wu.pdf · Randomization and Restart Strategies by Huayue Wu Athesis ... distribution that are central to the randomization

Randomization and Restart Strategiesvanbeek/Publications/wu.pdf · Randomization and Restart Strategies by Huayue Wu Athesis ... distribution that are central to the randomization

Randomization & Pseudo-Randomization Part I...Randomization and Pseudo-Randomization RCM Based Methods and Avoiding Confounding The goal of sidestepping confounding by –nding such

Randomization & Pseudo-Randomization Part I...Randomization and Pseudo-Randomization RCM Based Methods and Avoiding Confounding The goal of sidestepping confounding by –nding such

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan Confidence Intervals: Bootstrap Distribution SECTIONS 3.3, 3.4 Bootstrap.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan Confidence Intervals: Bootstrap Distribution SECTIONS 3.3, 3.4 Bootstrap.

Choosing the level of randomization Module 4.2. Unit of Randomization: Individual?

Choosing the level of randomization Module 4.2. Unit of Randomization: Individual?

[BOOK] [Bootstrap] [Awesome] Bootstrap-Programming-Cookbook

[BOOK] [Bootstrap] [Awesome] Bootstrap-Programming-Cookbook

ME Confidence Intervals: Bootstrap Distribution · 2015-02-21 · Bootstrap Sample Statistic Bootstrap Sample Statistic Bootstrap Sample Statistic. . . Bootstrap Sample Statistic

ME Confidence Intervals: Bootstrap Distribution · 2015-02-21 · Bootstrap Sample Statistic Bootstrap Sample Statistic Bootstrap Sample Statistic. . . Bootstrap Sample Statistic