INTRODUCING STATISTICAL INFERENCE USING RANDOMIZATION METHODS

Todd Swanson & Jill VanderStoepHope College Holland, Michigan

WHAT IS THE RANDOMIZATION APPROACH?

In traditional tests, a theoretical distribution is used to determine how unlikely it would be for an observed statistic to occur under a true null hypothesis.

In randomization tests, we don’t use a theoretical distribution, but use simulation to develop a distribution. We simulate the study under a true null hypothesis. After the simulation, we calculate the statistic of interest. We repeat this many times to develop a distribution that

models the null.

HISTORY

Spring of 2009We began writing a curriculum for

introductory statistics in which inference was introduced through randomization methods.

Fall of 2009The new curriculum was first used in all

introductory statistics sections at Hope College.

Spring 2010 and each semester sinceRevision of curriculum

CHANGES IN PEDAGOGY

Topics are introduced through a brief lecture Students work on activities to learn and

reinforce the topics. Tactile learning (shuffling cards and

flipping coins) to estimate p-valuesComputer based simulations (applets)Collecting data and running experiments

All our classes meet in a computer classroom.

CHANGES IN DATA --- REAL RESEARCH

We try to avoid cute, but impractical illustrations of statistics. We include real data and real research that matters.

Homework problems and case studies also involve real statistical data and research.

Each chapter contains a research paper that students read and respond to questions.

EXAMPLES WE CURRENTLY USE

Can dogs sniff out cancer? Is there an advantage to wearing a certain

color of uniform in Olympic combat sports? Does swimming with dolphins ease

depression symptoms? Can babies tell the difference between

naughty and nice? Does the perceived price of a drug influence

its perceived effect? Can students study better when listening to

music with lyrics or without lyrics?

STUDENT PROJECTS

Students complete two in-depth projects Mid-semester they analyze a large data set

that is provided and give a short presentation in class on their findings.

At the end of the semester the students design a study, do a literature review, collect data, and present their results in class. They also write their results up as a research paper.

SEVEN STEP METHOD

Logic ofInference

Scope ofInference

- Significance- Estimation

- Generalize- Cause/Effect

7. Communicate findings

1. Ask a research question

Research Conjecture

2. Design a study

3. Collect data

4. Explore the data

5. Draw inferences beyond the data

6. Formulate conclusions

EXAMPLE: DAY ONE

Can dolphins communicate abstract ideas? In an experiment done in the 1960s, Doris was

instructed which of two buttons to push. She then had to communicate this to Buzz (who could not see Doris). If both he and Doris picked the correct buttons, both dolphins would get a reward.

In one set of trials, Buzz chose the correct button 15 out of 16 times.

Based on these results, do you think Buzz knew which button to push or was he just guessing?

How might we justify an answer? How might we model this situation?

EXAMPLE: DAY ONE

The students, with a little prompting, will quickly come up with the idea of flipping a coin 16 times, count the number of times heads, and repeat this process many times.

The students flip coins and pool their results. We then use a computer applet to quickly

simulate many repetitions of coin flips.

ORIGINAL CONTENT

For the first two years we divided the course into two parts.

The first half of course was based on randomization methods and second half was based on traditional methods.

ORIGINAL CONTENTUNIT 1: RANDOMIZATION METHODS

Chapter 1: Introduction to Statistical Inference: One proportion Coin flips and applets are used to model

null Chapter 2: Comparing Two Proportions

Permutation tests are introduced using playing cards and Fathom

Chapter 3: Comparing Two MeansContinued use of permutation tests with

cards and Fathom. Chapter 4: Correlation and Regression

ORIGINAL CONTENTUNIT 2: TRADITIONAL METHODS

Chapter 5: Correlation and RegressionConfidence Intervals IntroducedPower (strength of evidence) as it relates

to sample size, significance level, and population correlation

Chapter 6: Comparing MeansTwo-sample t-test and ANOVA

Chapter 7: Comparing ProportionsUsing normal distribution and chi-square

Chapter 8: Tests of a Single Mean and ProportionAlso included chi-square goodness of fit

THINGS WE LIKED ABOUT OUR CURRICULUM

Students gained a deeper understanding of inference. Results from CAOS tests were compared to those

done with our traditional course. We were able to put more real research into

our curriculum Case studies, research articles, homework,

examples. A mid-semester project could focus on

inference instead of focusing on descriptive statistics. By mid-semester we were in some ways in a similar

place as we used to be at the end of the semester.

THINGS THAT NEEDED IMPROVEMENT

We had a little trouble getting through all the material.We used a lot more activities and less

lecture. Understanding confidence intervals

needed improvement.Confidence intervals were not introduced

until the second half of the course. In the end, some ideas behind the

randomization methods were forgotten.

For technology we used applets, Fathom, and SPSS.

UPDATED CONTENT

Chapter 1: Introduction to the Statistical Investigative Process One proportion test using randomization

methods focusing on process probabilities. Chapter 2: Random Samples and Variability

Still focusing on a single proportion. Sampling from populations, Normal-based

method, Confidence Intervals Chapter 3: Comparing Two Proportions

Permutation tests and traditional tests and confidence intervals.

Experiments introduced.

UPDATED CONTENT

Chapter 4: Correlation and Regression Scatterplots, Correlation, and Regression Permutation and traditional test for correlation.

Chapter 5: Comparing Means Two-sample tests, paired data, single mean. Permutation tests and traditional methods Confidence intervals

Chapter 6: Comparing Multiple Group Means and Proportions Permutation Test for multiple means and ANOVA Chi-square test for association and goodness-of-

fit

THINGS THAT WENT WELL

Students easily used randomization methods for their final project when conditions weren’t met to perform traditional tests

Better understanding of confidence intervals and how they related to tests of significance

THINGS THAT NEED IMPROVEMENT

Interpretation of confidence intervals. We still have a lot of examples that use data

collected from Hope College students. Led to some non-creative student projects that

used surveys of Hope students about college things: GPA, Greek life, GPA, caffeine consumption, etc.

We want students to think more broadly.

ANOTHER MAJOR UPDATE

We are in the midst of another major update in which we basically follow the same order as we did this past fall.

We are spending more time at the beginning laying out the ground work.

More emphasis on scope of conclusions. More examples from real research. More use of applets for randomization as well

as traditional tests.

A WORK IN PROGRESS

We will continue to revise our materials this year.

Besides the two of us working on this project, we also have:Nathan Tintle, Dordt CollegeBeth Chance, CalPolyAllan Rossman, CalPolySoma Roy, CalPolyGeorge Cobb, Mt Holyoke

THANKS!

More information can be found at www.math.hope.edu/isi

Email Todd: swansont@hope.edu Jill: vanderstoepj@hope.edu