Just-in-Time Teaching:Tilting Classes Across the

AcademyJeff Loats, Arlene Sgoutas, & Randi Smith

(Physics) (Women’s Studies) (Psychology)

Colorado Learning and Teaching with Technology

August, 2016

Presentation Overview• What is it?

Introduction to Just-in-Time Teaching (JiTT)

• Does it work? Data from our courses

• How do I get started? Recommendations for implementing JiTT

In your teaching do you have a method for holding students accountable for preparing for class?

A) I don’t, but I ask/threaten really well.B) I use a paper method (quiz, journal, others?)C) I use a digital method (clickers, others?)D) I have some other method.

3

18%51%15%17%

others)

Consider a typical day in your class. What fraction of students did their preparatory work before coming to class?

A) 0% - 20%B) 20% - 40%C) 40% - 60%D) 60% - 80%E) 80% - 100%

4

29%34%29%13%5%

( others)

Student Preparation Research

Quotes from Sappington, Kinsey, & Munsayac (2002)

"72% of Connor-Greene’s (2000) sample reported that they rarely or never read their assignments by the due date.”"Burchfield and Sappington (2000): On any given day, less than a third of students in this population had adequately prepared for class."Recent USPIRG survey: 70% of students admit that they sometimes don’t evenobtain required textbooks.

JUST-IN-TIME TEACHINGOnline pre-class assignments called WarmUpsFirst half - Students• Conceptual questions, answered in sentences• Graded on thoughtful effortSecond half - Instructor• Responses are read “just in time”• Instructor modifies that day’s plan accordingly.• Aggregate and individual (anonymous) responses

are displayed in class.

Learner Teacher

JUST-IN-TIME TEACHINGA different student role:• Actively prepare for class

(not just reading/watching)• Actively engage in class• Compare your progress & plan accordinglyA different instructor role:• Actively prepare for class with these humans

(not just going over last year’s notes )• Modify class accordingly• Create interactive engagement opportunities

Learner Teacher

TOUTED BENEFITS OF JITT• Student and instructor preparation• Supports development of metacognitive skills• Brings student voices into the classroom• Promotes engagement with “higher-level”

questions (cf. Bloom’s taxonomy)• Consistent with other research-based

instructional strategies (learner-centered, universal design, etc.)

Example Questions & Responses

Suppose you are interested in researching why and how some women fake orgasms. How would you design a study to answer those questions?

Perhaps I’d tell a random group of women to fake an orgasm. Additionally, a control group that is not told to fake an orgasm should be included. After both groups’ ensuing sexual encounters, ask very specific and pointed questions about sexual arousal, climax, or the lack thereof, and the experience overall.

Identify and define the three different conceptualizations of gender and sex that have characterized the study of gender in recent years.

1.) Gender as equivalent to sex: gender is "rooted in biological characteristics," or each animal is assigned a gender according to their sex; females are feminine, males are masculine. 2.)Gender as separate from sex: sex is the physical characteristics (reproductive roles), and gender is "personality roles" constructed from socialization. 3.) Gender as the assignment of meaning to bodies: That gender and sex are both social constructs.

Sounds good in theory, but does it actually work?

Convince me it’s worth my time…

Our Data: Set• We examined data from over 20 courses across

5 disciplines, over 500 students total– Upper- and lower-division courses– Larger and smaller courses– General studies, required major courses,

elective courses• Our implementations of JiTT vary

– Weekly vs. biweekly– 5-20% of final course grade

Our Data: Participation Rates• Response rates (averaged by class) vary from

~50% to ~80% across our two-dozen courses.

• Recall: The average preparation rate reported by Burchfield & Sappington (2000) was ~33%.

Week 1

2 3 4 5 6 7 8 9 10 11 12 13 14 150%

20%

40%

60%

80%

100%

WMS 390Y (N = 19)Avg. = 77%

Week 1

2 3 4 5 6 7 8 9 10 11 12 13 14 150%

20%

40%

60%

80%

100%

PSY 2850 (N = 23)Avg. = 68%

~Daily WarmUps

~Weekly WarmUps

Day 2

4 6 8 10 12 14 16 18 20 22 24 26 280%

20%

40%

60%

80%

100%

Physics 2711 (N = 24)Avg. = 72%

Day 2

4 6 8 10 12 14 16 18 20 22 24 26 280%

20%

40%

60%

80%

100%

Physics 1000 (N = 22)Avg. = 63%

Our Data: WarmUp Correlations• Bulk correlation of Total Grade ↔ WarmUps

– ranges from to – Combined* correlation: (N = 581)

• Hattie (2009) noted that effect sizes for most pedagogical techniques are small to medium (see, e.g.,). From Cohen (1988): is medium effect is large effect

Our Data: Correlation Spread

*Correlations were converted to Fisher Zs, averaged and converted back to Pearson r.

WMS390z

WMS390y

WMS1001

PSY3050

PSY2850

PSY2850

PSY2410

PSY2410

PSY1001

PSY1001

PSY1001

PSY1001

PHY4820

PHY3620

PHY3620

PHY3011

PHY2020

phy1000

phy1000

0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

Total Grade vs. WarmUp Correlations by Class

r = 0.57Combined* correlation

Our Data: Testing Personality Effects• In trying to tease out causality, we are looking

to control for personality traits. – The Big Five: openness, conscientiousness,

extraversion, agreeableness, and neuroticism.– For example: Does the WarmUp ↔ Total

Grade correlation just come from conscientiousness?

• So we cajoled 77 students into an extensive personality assessment.

Our Data: Personality Effect ResultsFrom our 77 students we have

WarmUp ↔ Total Grade: (puzzling)

Variable Statistical Significance

Extraversion 0.460Agreeableness 0.258Conscientiousness 0.227Neuroticism 0.483Openness 0.034WarmUp Grade 0.032

PRELIMINARY RESEARCH SUMMARY Participation rates appear to be quite stable

across disciplines, levels, course size, etc. The importance of WarmUps to learning is not

explained by student conscientiousness.

Is JiTT especially effective in scaffolded topics?

Are students that completed the personality assessment a skewed sample? Why is their WarmUp ↔ Total Grade correlation so low?

Wow—you guys are so convincing. I’m totally sold on this JiTT stuff. Now what?First… pause for questions?

A POSSIBLE PLANChoose one course you will teach next term.A. Write two questions for each class meeting:

1. One lower-level (maybe multi-choice?).One higher-level (sentences).

2. Give yourself 10 minutes to write each oneB. Write a standard (1st) metacognitive question

(What was most interesting or confusing …?)C. Discuss one question at the top of class, and one

in the middle. Use the metacognitive responses as break points or highlights.

OUR SUMMARYJiTT may be among the easiest research-based instructional strategies that you can consistently integrate into your teaching.

JiTT is a research-based instructional strategy that addresses often-neglected areas.

JiTT brings other benefits to students and instructors beyond those measured in most studies.

Strongly Disagree

Strongly Agree

1 2 3 4 5Completing the warm-ups in this class helps to prepare me for class meetings.I do more of my assigned readingfor this class than I would if thereweren't warm-ups assigned.The warm-ups are worth the timeand effort required to completethem because they help me learn.I find it easier to learn the materialin this class because of the warmups.The warm-ups feel like a waste ofmy time.

2.19

3.63

3.89

3.58

3.95

STUDENT SURVEY RESPONSES I

Strongly Disagree

Strongly Agree

1 2 3 4 5The warm-ups are worth enoughpoints/credit toward my final gradein this class, given the time andeffort they require.Overall, I find I have to spend lesstime studying for exams because ofthe learning I've done for the warmups.

I'm better able to determine what Iunderstand well and what I need tostudy more because of the warmups.

My instructor uses the warm-ups ina meaningful way to tailor classmeetings based on our needs.

The warm-ups are a nuisance tocomplete. 2.58

4.09

3.28

2.74

3.59

STUDENT SURVEY RESPONSES II

YOUR SUMMARYIf you want to implement JiTT, what is your next concrete action?

Jeff Loats: jeff.loats@gmail.com, @JeffLoatsArlene Sgoutas: sgoutasg@msudenver.edu Randi Smith: rsmit216@msudenver.eduSlides: www.slideshare.net/JeffLoats

Thanks for your attention!

JITT REFERENCES & RESOURCESSecond Book (interdisciplinary):Simkins, Scott and Maier, Mark (Eds.) (2010) Just in Time Teaching: Across the Disciplines, Across the Academy, Stylus Publishing.Original Book (physics examples):Gregor M. Novak, Andrew Gavrin, Wolfgang Christian, Evelyn Patterson (1999) Just-in-Time Teaching: Blending Active Learning with Web Technology. Prentice Hall. Upper Saddle River NJ.K. A. Marrs, and G. Novak. (2004). Just-in-Time Teaching in Biology: Creating an Active Learner Classroom Using the Internet. Cell Biology Education, v. 3, p. 49-61.Jay R. Howard (2004). Just-in-Time Teaching in Sociology or How I Convinced My Students to Actually Read the Assignment.  Teaching Sociology, Vol. 32 (No. 4 ). pp. 385-390. Published by: American Sociological AssociationStable URL: http://www.jstor.org/stable/3649666S. Linneman, T. Plake (2006). Searching for the Difference: A Controlled Test of Just-in-Time Teaching for Large-Enrollment Introductory Geology Courses. Journal of Geoscience Education, Vol. 54 (No. 1)Stable URL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18Sappington J, Kinsey K and Munsayac K (2002) Two studies of reading compliance among college students. Teaching of Psychology 29(4): 272–274.http://orgs.bloomu.edu/tale/documents/reading_sappington_twostudies.pdfLouis Deslauriers, Ellen Schelew and Carl Wieman (2011). Improved Learning in a Large-Enrollment Physics Class. Science, Vol. 332 no. 6031 pp. 862-864 DOI: 10.1126/science.1201783https://www.sciencemag.org/content/332/6031/862.abstractFreeman S, et al. (2014) Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci USA 111:8410–8415.http://www.pnas.org/content/111/23/8410.abstract

ON-DEMAND SLIDES