Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger...

Experimental Assessment of Learning Outcomes Using a Text Book and Interactive

Package

Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian Schoner, Kari Stachuk and Stacey

Vojtek

University of [email protected]

Sample Course evaluation

“I open the text and my mind goes blank”

What we’ll address:

• An instructional bottleneck/anchor

• A distributed learning approach

• Evaluating alternative approaches

The Instructional Environment

• University of Waterloo: – Professional and technical focus

• Faculty of Environmental Studies– Environment and Business– Environment and Resource Studies– Geography– School of Planning

Course Context

• “Field Ecology”

• 0.5 Credits, 2nd year, compulsory

• Non-scientists

• Applied focus

• 2 h class + 3 h lab/field trip

Technical Material as Instructional bottleneck

• Math-happy and Math-phobic students

• Text book “dry”

• Poor learning outcomes

Life Tables:

• Accounting system for population dynamics

• Understanding comes only by doing

Solution:A Life Tables Interactive Package

• Engage the learner

• Non-threatening “walk through” computation

• Oblige student to “do” life tables

• Emphasise applications

• Link to Field Work Videos

The Life Tables Interactive Package (IP)

Instructional Presentation

• Sampler in class

• Material on server

• Used in 5 course offerings

What we don’t know:

• IP is fun, but do they learn more?

• Is text plus IP better?

• Retention of text vs IP?

Big Picture

• Is CD in back of a text effective?

• Supplement distributed learning with text?

• Abandon text? (Publishing implications!)

What others say:

• Students learn more with computers, faster, liked work more, with better attitudes (Kulic 1994).

• However:– “We need to go beyond generic conclusions

and make statements about the effectiveness of specific types of computer based instruction”

What others say:

• Striking need for evaluation (Baker and O’Neil 1994,Anon 1998)

• Little evaluation of durability of learning effect with technologies (Lesgold 1995)

• Exceptions:– Hiltz (1990)

– Active Learning 8 (1998)

Research Questions

• Different outcomes using IP and text+IP?

• Different retention for IP and text+IP?

• Does text/IP order influence:

– outcomes?– retention?

Experimental design

Treatment Baseline test

1st study

Period

Test IP

vs. Text

2nd Study

Period

Test order of IP and Text

Test retention at 3 mo

Text-Text Text Text-IP

Text IP

IP-Text Text IP-IP

IP IP

Evaluation of Learning

• Before, 30 multiple choice

• At 45 min. 10 of above

• At 90 min. 10 of above

• At 3 months 10 of above

• 48 subjects in 4 treatments (Have 40)

Results

(Tentative)

Statistical Evaluation

• MANOVA on scores

• MANOVA on change in scores

Mean scores

-2

0

2

4

6

8

10

Pre Study After 1period

After 2periods

After 3Months

Book/CD Book/Book CD/Book CD/CD

Change in Mean Score from Pre-Study

-2

0

2

4

6

8

After 1 period After 2 periods After 3 Months

Book/Book Book/CD CD/Book CD/CD

Same except book/IP

At 3 mo all same except Ip/Book

Technical Conclusions

• All approaches work (somewhat!)

• Book/IP is less effective?

• Decay in performance is profound!

• At 3-month retention not radically different

General Conclusion

All methods tried work about the same.

Value of interactive approach must lie in long-term motivation, not in the individual knowledge.

The long term effects on student progress need evaluation

Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger...

Documents

Transcript of Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger...