Streaming Video Tutorials for Complex Laboratory Techniques: Lessons Learned in Sophomore and Junior...

Streaming Video Tutorials for Complex Streaming Video Tutorials for Complex Laboratory Techniques: Laboratory Techniques:

Lessons Learned in Sophomore and Junior Lessons Learned in Sophomore and Junior Biomechanics CoursesBiomechanics Courses

Kristen L. Billiar, Ph.D.Kristen L. Billiar, Ph.D.

Dept of Biomedical Engineering, Worcester Polytechnic InstituteDept of Biomedical Engineering, Worcester Polytechnic Institute

Dept of Surgery, University of Massachusetts Medical SchoolDept of Surgery, University of Massachusetts Medical School

Department of Biomedical Engineering

UMMSUMMS

GoalsGoals

Laboratories in all BME coursesLaboratories in all BME courses Making and interpreting measurementsMaking and interpreting measurements Challenge-based, self-guided, self-pacedChallenge-based, self-guided, self-paced

Particular challenge Particular challenge Rapid growth of BME (40-50 per course)Rapid growth of BME (40-50 per course) Limited resourcesLimited resources

Instructor time (and TA)Instructor time (and TA)Volume of sophisticated equipment Volume of sophisticated equipment Laboratory space (scheduling time)Laboratory space (scheduling time)Funds Funds

Practical vs. theoreticalPractical vs. theoretical Expanding access, not assessing educational Expanding access, not assessing educational

theories or comparing methodstheories or comparing methods

Share approach to problem and assessmentShare approach to problem and assessment Discuss time and effort involved Discuss time and effort involved Describe assessment methods and findingsDescribe assessment methods and findings Initiate dialog – chance to get your feedbackInitiate dialog – chance to get your feedback


UMMSUMMS

Approach and goal of studyApproach and goal of study

Approach - teaching technologyApproach - teaching technology Web-based streaming video tutorialsWeb-based streaming video tutorials Provide the “toolbox” Provide the “toolbox” Economical, sustainable Economical, sustainable

Allow students to perform labs Allow students to perform labs Self-paced, flexible schedule Self-paced, flexible schedule Without constant instructorWithout constant instructor or TA supervisionor TA supervision

Goal - formative study Goal - formative study Assess student learning from the tutorialsAssess student learning from the tutorials Gather student feedback on their usefulness and quality as a Gather student feedback on their usefulness and quality as a

guide for future improvementsguide for future improvements


UMMSUMMS

Tutorials – web-basedTutorials – web-based

TutorialsTutorials How?How? rather than rather than WhatWhat and and WhyWhy Large number (>20), short (< 5 minutes)Large number (>20), short (< 5 minutes) Easy to create and modifyEasy to create and modify11

FormatFormatMS Powerpoint, Camtasia, videoMS Powerpoint, Camtasia, videoSlides, video, audio Slides, video, audio

Design teamDesign team Two professorsTwo professors

Biomedical and Mechanical EngineeringBiomedical and Mechanical Engineering Academic Technology Center StaffAcademic Technology Center Staff

Video, editing, streaming serversVideo, editing, streaming servers Two sophomore-level studentsTwo sophomore-level students

20 hrs/wk x 8wks x 2 = 320 hrs20 hrs/wk x 8wks x 2 = 320 hrs

FundingFunding Two small internal grants, paid for student time and provided accessTwo small internal grants, paid for student time and provided access

1Billiar and Shonat, PEBEL II, 2004


UMMSUMMS

Example tutorial - safetyExample tutorial - safety

Posted on myWPI (Blackboard)Posted on myWPI (Blackboard)

PPT file complete PPT file complete ((linklink)) 23 MB (w/o video clip)23 MB (w/o video clip)

PPS filePPS file 1.2 MB1.2 MB Streaming audioStreaming audio Streaming videoStreaming video Requires fast internet connectionRequires fast internet connection

X


UMMSUMMS

AssessmentAssessmentSophomore-level “Foundations of Biomechanics”Sophomore-level “Foundations of Biomechanics”

large number (n=47), limited use, single laboratorylarge number (n=47), limited use, single laboratory

Junior-level “Experimental Biomechanics”Junior-level “Experimental Biomechanics” small number (n=17), extensive use as they were meant to be, challenge-basedsmall number (n=17), extensive use as they were meant to be, challenge-based

Questionnaires (sophomore-level only)Questionnaires (sophomore-level only) Pre-tutorialPre-tutorial Post-tutorial – with answers and explanationPost-tutorial – with answers and explanation Controlled environment – 2 one-hour sessionsControlled environment – 2 one-hour sessions Automation with BlackboardAutomation with Blackboard

Safety practicumSafety practicum 10-20 minutes/group, Password given10-20 minutes/group, Password given

Grade comparison (sophomore-level only) Grade comparison (sophomore-level only) Quiz assessing knowledge of concept of stress-strain analysis Quiz assessing knowledge of concept of stress-strain analysis

Feedback – usefulness and qualityFeedback – usefulness and quality Likert-typeLikert-type Written responsesWritten responses


UMMSUMMS

Example questionnaireExample questionnaire


UMMSUMMS

Assessment of tutorialsAssessment of tutorials

QuestionnairesQuestionnaires Pre-tutorial scorePre-tutorial score 34% 34% ± 9% ± 9% Post-tutorial scorePost-tutorial score 71% 71% ± 12%± 12%

p < 0.05 paired t-test in all groups (except M5)p < 0.05 paired t-test in all groups (except M5)

0%

20%

40%

60%

80%

100%

M1 M2 M3 M4 M5 M6 S1 S3 S4 S5 S6 S7

Tutorial questionnaire

% correct answers

pre post


UMMSUMMS

Assessment of tutorialsAssessment of tutorials

Compliance with questionnairesCompliance with questionnaires Completed 100%: n = 27 of 47 (57% compliance)Completed 100%: n = 27 of 47 (57% compliance) Completed >90%: n = 36 of 47 (77% compliance)Completed >90%: n = 36 of 47 (77% compliance)

R2 = 0.84

75%

80%

85%

90%

95%

100%

1 2 3 4 5 6 7 8 9 10 11 12

Questionnaire #


UMMSUMMS

Feedback surveyFeedback survey

Please rate the usefulness of each tutorial in helping you to successfu lly complete the lab by checking the one cell for each tutorial that most closely

matches your opinion .

Please rate the general quality of each tutorial by checking the one cell

for each tutorial that most closely matches your opinion .

TUTORIAL

Tutorial M1 Tutorial M2 Tutorial M3 Tutorial M4 Tutorial M5 Tutorial M6

Tutorial S1 Tutorial S3 Tutorial S4 Tutorial S5 Tutorial S6 Tutorial S7


UMMSUMMS

Feedback surveyFeedback survey

What about the Tutorials would you change?What did you especially like about the Tutorials and your experience using them?Please identify aspects of particular Tutorials that were especially problematic.

In each shaded section below, please check the one box that most closely matches your opinion.

In attemptin g to understand how to do the lab, I found these tutorial s, in general and as a whole , were

_ not at all useful. _ something that made me very confused . _ not very useful. _ something that made me a little confused . _ neither not useful nor useful. _ something that neither confused nor clarified. _ somewhat useful. _ something that mad e the material a little clearer. _ very useful. _ something that made the material much clearer. _ much less helpful than written material s. _ much less helpful than one -on-one time with the prof. _ less helpful than written material s. _ less helpful than one -on-one time with the prof. _ as helpful as written material s. _ as help ful as one -on-one time with the prof. _ more helpful than written material s. _ more helpful than one -on-one time with the prof. _ much more helpful than written material s. _ much more helpful than one -on-one time with the prof. _ very difficult to use. _ very inconvenient to use. _ difficult to use . _ inconvenient to use. _ neither difficult nor easy to use . _ neither inconvenient nor convenient to use. _ easy to use . _ convenient to use. _ very easy to use. _ very convenient to use.


UMMSUMMS

Feedback – Likert scaleFeedback – Likert scale

SophomoreSophomore JuniorJunior

completion of surveycompletion of survey 74% (35/47)74% (35/47) 94% (16/17)94% (16/17)

general quality either “good,” “very good,” general quality either “good,” “very good,” or “excellent.”or “excellent.”

70% to 90%70% to 90% 80% to 93%80% to 93%

““easy” or “very easy” to useeasy” or “very easy” to use 60%60% 100%100%

““quite” or “extremely” useful (all but one)quite” or “extremely” useful (all but one) 54% to 72%54% to 72% 73% to 100%73% to 100%

““as helpful,” “more helpful,” or “much more as helpful,” “more helpful,” or “much more helpful” than written materialshelpful” than written materials

80%80% 100%100%

tutorials alone were adequate (or better) in tutorials alone were adequate (or better) in preparing them for their labpreparing them for their lab

60%60% 75%75%

““as helpful” or “more helpful” than one-on-as helpful” or “more helpful” than one-on-one time with the professorone time with the professor

30%30% 75%75%


UMMSUMMS

Quiz grade comparisonQuiz grade comparison

Uniaxi al test data quiz Final course grades 2003

(no lab or demo)

2004 (demo only)

2005 (limited

lab)

2003 (no lab or

demo)

2004 (demo only)

2005 (limited

lab) # students 41 45 46* 41 45 47 Average 74 74 85 85 82 86 SD 18 17 13 13 12 8 CV% 24% 23% 15% 16% 15% 10% For quiz: P<0.001 by ANOVA with 2005 different than 2003 and 2004 (P < 0.05, Dunn’s) For final course grades: P = 0.175 by ANOVA (not significant). *One student did not take this particular quiz

Uniaxial test on leather specimen

0

200

400

600

800

1000

0 5 10 15

Extension (mm)

Load (N)

1st cycle

10th cycle


UMMSUMMS

DiscussionDiscussion

MotivationMotivation Provide labs to students with resource savings Provide labs to students with resource savings

in the long termin the long term Available tutorials (MTS, Microsoft) not well Available tutorials (MTS, Microsoft) not well

suited, much longersuited, much longer General enough for multiple types of users at General enough for multiple types of users at

WPI and beyondWPI and beyond

AssessmentAssessment Effectiveness of tutorials and aspects to Effectiveness of tutorials and aspects to

improveimprove NotNot video vs. written or professor video vs. written or professor NotNot lab vs. no lab lab vs. no lab Controls for summative evaluations not Controls for summative evaluations not

practicalpractical

Resource requirementResource requirement High initial investment (team, time)High initial investment (team, time) Written materials require effort alsoWritten materials require effort also Advantage - technical staff and studentsAdvantage - technical staff and students


UMMSUMMS

DiscussionDiscussion

Overall success – short termOverall success – short term Students passed practicum, completed Students passed practicum, completed

laboratories laboratories Sophomore, junior, and grad as wellSophomore, junior, and grad as well Instructor time in lab minimalInstructor time in lab minimal

QuestionnairesQuestionnaires Controlled viewing, but lower scores than desiredControlled viewing, but lower scores than desired Too many (12 x 2), too much time commitment Too many (12 x 2), too much time commitment

FeedbackFeedback Too broad for one lab in sophomore-level courseToo broad for one lab in sophomore-level course Well received when used as designedWell received when used as designed

Increased use of equipment - Increased use of equipment - time logged time logged Sophomore-level course > 50 hours; ~ 1hr/student Sophomore-level course > 50 hours; ~ 1hr/student Junior-level course >20 hours/student, relatively Junior-level course >20 hours/student, relatively

few scheduling conflictsfew scheduling conflicts

Facilitated challenge-based laboratoriesFacilitated challenge-based laboratories

True time and resource savings - futureTrue time and resource savings - future


UMMSUMMS

AcknowledgementsAcknowledgements

Allen HoffmanAllen Hoffman tutorial contenttutorial content

Megan Murphy and Dan Filipe Megan Murphy and Dan Filipe sophomoressophomores tutorial creationtutorial creation

Paula QuinnPaula Quinn assessment consultantassessment consultant

Adam Jacobi and David Adam Jacobi and David Taranto (WPI ATC)Taranto (WPI ATC)

production supportproduction support

Elias Wilson (TA)Elias Wilson (TA) questionnaire and practicum questionnaire and practicum

administrationadministration

Ross Shonat (NIH)Ross Shonat (NIH) introduction to formal CBIintroduction to formal CBI

FundingFunding WPI Educational Development WPI Educational Development

Council GrantCouncil Grant WPI Teaching Technology WPI Teaching Technology

FellowshipFellowship


UMMSUMMS

Questions?Questions?


UMMSUMMS

Educational frameworkEducational framework

Challenge-based learning (self-guided) Challenge-based learning (self-guided) Aid in the development of Aid in the development of

critical thinking – authentic problemscritical thinking – authentic problems

Building on HPL frameworkBuilding on HPL framework11

Learner-, community-, knowledge-, and Learner-, community-, knowledge-, and assessment-centeredassessment-centered

Self-pacedSelf-paced Allow for differencesAllow for differences Allow greater number of student-hours in Allow greater number of student-hours in

lab lab

The “Challenge” & problem

identification

Generate ideas

Literature & interview

Testing & analysis

Communication of findings

LearningCycle

Research & revise

1Bransford, Brown, and Cocking, 2000

Streaming Video Tutorials for Complex Laboratory Techniques: Lessons Learned in Sophomore and Junior...

Documents

Transcript of Streaming Video Tutorials for Complex Laboratory Techniques: Lessons Learned in Sophomore and Junior...