Just-in-Time TeachingA Strategy For Success

Arlene Sgoutas, Jeff Loats, Courtney Rocheleau, & Randi Smith

Teaching and Learning with Technology Symposium October 2015

Presentation Overview• What is it?

Introduction to Just-in-Time Teaching (JiTT)

• Does it work? Data from our courses

• How do I get started? Recommendations for implementing JiTT

In your teaching do you have a method for holding students accountable for preparing for class?

A) I don’t, but I ask/threaten really well.B) I use a paper method (quiz, journal,

others?)C) I use a digital method (clickers,

others?)D) I have some other method.

3

17%51%11%17%

(~230 others)

Consider a typical day in your class. What fraction of students did their preparatory work before coming to class?

A) 0% - 20%B) 20% - 40%C) 40% - 60%D)60% - 80%E) 80% - 100%

4

28%34%20%14%5%

(~260 others)

Student Preparation

Quotes from Sappington, Kinsey, & Munsayac (2002)

"72% of Connor-Greene’s (2000) sample reported that they rarely or never read their assignments by the due date.”"Burchfield and Sappington (2000): On any given day, less than a third of students in this population had adequately prepared for class."Recent USPIRG survey: 70% of students admit that they sometimes don’t even obtain required textbooks.

JUST-IN-TIME TEACHINGOnline pre-class assignments called WarmUpsFirst half - Students• Conceptual questions, answered in sentences• Graded on thoughtful effortSecond half - Instructor• Responses are read “just in time”• Instructor modifies that day’s plan accordingly.• Aggregate and individual (anonymous) responses

are displayed in class.

Learner Teacher

JUST-IN-TIME TEACHINGA different student role:• Actively prepare for class

(not just reading/watching)• Actively engage in class• Compare your progress & plan accordinglyA different instructor role:• Actively prepare for class with these humans

(not just going over last year’s notes )• Modify class accordingly• Create interactive engagement opportunities

Learner Teacher

BENEFITS OF JITT• Student and instructor preparation• Supports development of metacognitive skills• Brings student voices into the classroom• Promotes engagement with “higher-level”

questions (cf. Bloom’s taxonomy)• Consistent with other research-based

instructional strategies (learner-centered, universal design, etc.)

Example Questions & Responses

Suppose you are interested in researching why and how some women fake orgasms. How would you design a study to answer those questions?

Perhaps I’d tell a random group of women to fake an orgasm. Additionally, a control group that is not told to fake an orgasm should be included. After both groups’ ensuing sexual encounters, ask very specific and pointed questions about sexual arousal, climax, or the lack thereof, and the experience overall.

Give an example of a time you conformed due to normative influence. Give an example of a time you conformed due to informational influence.

I gave into normative social influence when I bought a pair of Uggs because I felt obligated to follow the norm. I have given into informational social influence when I moved to a new school and had to ask students around here how to act in certain situations because it was not like my other school.

Sounds good in theory, but does it actually work?

Convince me it’s worth my time…

Our Studies• Have examined data from 18 courses

across 5 disciplines– Upper- and lower-division courses– Larger and smaller courses– General studies, required major courses,

elective courses

• Vary in implementation of JiTT– Weekly vs. biweekly– 5-20% of final course grade

Data from Our CoursesOverall Response Rates:

46% - 74% average response rate across the entire 15-week semester.(mean of averages across 18 classes = 59%)

• Recall: ~33% in Burchfield & Sappington (2000)

1 2 3 4 5 6 7 8 9 10 11 12 13 140

20

40

60

80

100

PSY2850: Smith, N = 33

PSY2410: Rocheleau, N = 36 WMS 390Z: Sgoutas, N = 181 2 3 4 5 7 8 9 10 11 14 15 16 17 18 19 22 23 24 25 26 27

0

20

40

60

80

100 PHY2010: Loats, N = 78

1 2 3 4 5 6 7 8 9 10 11 12 130%

20%

40%

60%

80%

100%

PSY1001: Smith, N = 22

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240%

10%20%30%40%50%60%70%80%90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 210%

20%

40%

60%

80%

100% PHY2020: Loats, N = 73

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150%

10%20%30%40%50%60%70%80%90%

100%

Course PerformanceEffect sizes for most pedagogical techniques are small to medium (see, e.g., Hattie, 2009)

Correlations with total grade (w/warm-ups partialed out) across 18 classes range from r = 0.38 – 0.84; Mr = .60 (r2 = .36)• Per Cohen (1988),

r = .30 is medium effect, r = .50 is large effect

Limitations• Could be alternative

explanations/confounds that are uncontrolled in these analyses; e.g.,– Academic motivation– Quality of answer– Conscientiousness– Interest in course

• But pattern repeated across wide variety of courses

Wow—you guys are so convincing. I’m totally sold on this JiTT stuff. Now what?Questions?

A POSSIBLE PLANChoose one course you will teach next term.A. Write two questions for each class meeting:

1. One lower-level (maybe multi-choice?).One higher-level (sentences).

2. Give yourself 10 minutes to write each oneB. Write a standard (1st) metacognitive question

(What was most interesting or confusing …?)C. Discuss one question at the top of class, and one

in the middle. Use the metacognitive responses as break points or highlights.

OUR SUMMARYJiTT may be among the easiest research-based instructional strategies that you can consistently integrate into your teaching.

From an evidence-based perspective, JiTT addresses often-neglected areas.

YOUR SUMMARYIf you want to implement JiTT, what is your next concrete action?Randi Smith: rsmit216@msudenver.eduJeff Loats: jeff.loats@gmail.com, @JeffLoatsCourtney Rocheleau: crochel1@msudenver.eduArlene Sgoutas: sgoutasg@msudenver.edu Slides: www.slideshare.net/JeffLoats

Thanks for your attention!

JITT REFERENCES & RESOURCESSecond Book (interdisciplinary):Simkins, Scott and Maier, Mark (Eds.) (2010) Just in Time Teaching: Across the Disciplines, Across the Academy, Stylus Publishing.Original Book (physics examples):Gregor M. Novak, Andrew Gavrin, Wolfgang Christian, Evelyn Patterson (1999) Just-in-Time Teaching: Blending Active Learning with Web Technology. Prentice Hall. Upper Saddle River NJ.K. A. Marrs, and G. Novak. (2004). Just-in-Time Teaching in Biology: Creating an Active Learner Classroom Using the Internet. Cell Biology Education, v. 3, p. 49-61.Jay R. Howard (2004). Just-in-Time Teaching in Sociology or How I Convinced My Students to Actually Read the Assignment.  Teaching Sociology, Vol. 32 (No. 4 ). pp. 385-390. Published by: American Sociological AssociationStable URL: http://www.jstor.org/stable/3649666S. Linneman, T. Plake (2006). Searching for the Difference: A Controlled Test of Just-in-Time Teaching for Large-Enrollment Introductory Geology Courses. Journal of Geoscience Education, Vol. 54 (No. 1)Stable URL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18Sappington J, Kinsey K and Munsayac K (2002) Two studies of reading compliance among college students. Teaching of Psychology 29(4): 272–274.http://orgs.bloomu.edu/tale/documents/reading_sappington_twostudies.pdfLouis Deslauriers, Ellen Schelew and Carl Wieman (2011). Improved Learning in a Large-Enrollment Physics Class. Science, Vol. 332 no. 6031 pp. 862-864 DOI: 10.1126/science.1201783https://www.sciencemag.org/content/332/6031/862.abstractFreeman S, et al. (2014) Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci USA 111:8410–8415.http://www.pnas.org/content/111/23/8410.abstract

ON-DEMAND SLIDES