Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... ·...
Transcript of Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... ·...
![Page 1: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/1.jpg)
Keeping Students Awake – Creating Active Learning Environments with Interactive
Classroom Technology
Innovations in eLearning2 June 2010
![Page 2: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/2.jpg)
Learn. Perform. Succeed.
Outline
• Classroom questioning strategies
• Using technology with classroom questions
• DAU experience with integrating technology with classroom questions
2
![Page 3: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/3.jpg)
Learn. Perform. Succeed.
Classroom questioning strategies
• Examples• Pre-tests to activate prior knowledge and…
– Get the students ready to learn– Can be tied to lesson objectives so instructor can
allocate time to various parts of lesson • Directed “knowledge” questions to check
comprehension of material– At end of lesson topics (every 10-15 minutes)
• “What do you think” questions to generate discussion and give learners time to share experiences
3
![Page 4: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/4.jpg)
Learn. Perform. Succeed.
Using technology with classroom questions
4
• Good questions are still good questions• Helps structure class time by breaking up
material and adding interactivity• Especially helpful for new instructors
• Anonymity of SRS • All students participate – not just the most
out-going ones
![Page 5: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/5.jpg)
Learn. Perform. Succeed.
Example Question – Attitude
5
![Page 6: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/6.jpg)
Learn. Perform. Succeed.
DoD Experience … so far
• Research about other DoD trials
• Results of DAU student and instructor surveys (engagement, satisfaction, assessment)
6
![Page 7: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/7.jpg)
Learn. Perform. Succeed.
DoD Classroom Performance Systems
24 Military Bases17 Federal Agencies23 Military Academies87 Nuclear Submarines3K Army, Navy, AF and
Marine JROTCprograms
. .
.
.
.
.
.
.. ....
. . . . .. ....
. .
.
.
.
. .
.
.
.. ....
. .
.
.
.
. .
.
.
.
. .
.
.
.
. .
.
.
.
. .
.
.
.. ....
. .
.
.
.
. .
.
.
.. ....
. .
.
.
.
. .
.
.
.
. .
.
.
. .
.
.
.
. .
.
.
.. ....
. .
..
.
. .
...
. .
...
. .
...
. .
...
. .
... . .
...
. .
...
. .
...
. .
...
. .
...
. .
...
. .
...
. .
...
. .… .
.. .
. .… .
.. .
. .… .
.. .
. .… .
.. .
. .… .
.. .. .… .
.. .
Over 12,000 instructors and
1 million students use eInstruction
training tools
![Page 8: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/8.jpg)
Learn. Perform. Succeed.
Trident Submarine Training Facility
6 classrooms, n = ~1,1008
![Page 9: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/9.jpg)
Learn. Perform. Succeed.
Trident Submarine Training Facility
6 classrooms, n = ~1,1009
![Page 10: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/10.jpg)
Learn. Perform. Succeed.
Navy Junior ROTC
13 NJROTC units, ~30 students per class
10
![Page 11: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/11.jpg)
Learn. Perform. Succeed.
Provide a global learning environment to support a mission-ready Defense Acquisition Workforce that develops, delivers, and sustains effective and affordable warfighting capabilities.
DAU Mission
Impact acquisition excellence through:Acquisition certification and
leadership trainingMission assistance to
acquisition organizationsOnline knowledge sharing
resourcesContinuous learning assetsStrategic workforce planning
![Page 12: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/12.jpg)
Learn. Perform. Succeed.
DAU Performance Learning Model
Knowledge SharingDAP - Online portal to Big A & HCI knowledgeACC - DoD's online collaborative communitiesVirtual Library - Online connection to DAU research collection
Training CoursesClassroom & online DAWIA, Core Plus, & Executive
Mission AssistanceConsulting - Helping organizations solve complex acquisition problemsTargeted Training - Tailored organizational trainingRapid Deployment Training - On-site & online training on the latest AT&L policies
Continuous LearningCL Modules - Online, self-paced learning modulesConferences - PEO / SYSCOM, Business Managers, DAU Acquisition Community Symposium
Formal & informal learning at the point
of need
![Page 13: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/13.jpg)
Learn. Perform. Succeed.
Defense Acquisition Workforce
As of 30 Sep 09
Career Fields ARMY NAVY/USMC AIR FORCE 4th Estate TOTAL Auditing 0 0 0 3,777 3,777Business, Cost Est., & Fin. Mgt 2,771 2,286 1,845 360 7,262
Contracting 8,391 5,516 7,443 6,305 27,655Information Technology 1,843 1,240 966 309 4,358Life Cycle Logistics 7,952 4,784 1,989 127 14,852Production, Quality & Manufacturing 1,930 2,064 389 4,640 9,023
Program Management 3,452 4,598 4,461 911 13,422SPRDE 10,412 18,328 7,248 1,339 37,327Test and Evaluation 2,235 2,833 3,630 194 7,892Other 1,370 5,323 203 639 7,535Total 40,356 46,972 27,174 18,601 133,103
![Page 14: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/14.jpg)
Learn. Perform. Succeed.
Global Reach to Support Combatant Commands FY09
NORTHCOM ForeignStudents
U.S.Students
CL Grads 1,664 468,868Classroom Grads 20 38,631DL Grads 194 150,853
SOUTHCOM ForeignStudents
U.S.Students
CL Grads 50 533Classroom Grads 0 0DL Grads 15 33
CENTCOM ForeignStudents
U.S.Students
CL Grads 195 1,592Classroom Grads 7 22DL Grads 58 435
PACOM ForeignStudents
U.S.Students
CL Grads 1,212 11,432Classroom Grads 59 441DL Grads 233 1,616
EUCOM ForeignStudents
U.S.Students
CL Grads 884 5,342Classroom Grads 14 92DL Grads 67 460
AFRICOM ForeignStudents
U.S.Students
CL Grads 6 27Classroom Grads 0 0DL Grads 64 20
![Page 15: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/15.jpg)
Learn. Perform. Succeed.
DAU Experience with “Clickers”
• Mission Assistance: 6 events @ 500 students– Army Congressional Fellowship Program– Lean Six-Sigma Green Belt Training– ASMC Professional Development Institute
Faculty gained experience with 23 classes/events and ~ 1,000 learners
• Courses # Classes # StudentsACQ 201 2 72BCF 301 7 170LOG 350 4 100PMT 352 4 120
![Page 16: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/16.jpg)
Learn. Perform. Succeed.
DAU Learner Surveys
• Assessment– No conclusive evidence of improvement on exams in classes where
clickers were used• Level 3 course with higher performing students
• Engagement– Most learners agreed that clickers provided increased involvement– Most learners agreed that clickers helped them stay engaged with the
material and helped them understand
• Satisfaction– Most learners enjoyed using clickers – Most learners agreed that clickers helped them better understand by
catching and correcting mistakes
16
![Page 17: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/17.jpg)
Learn. Perform. Succeed.
DAU Learner Survey Questions1. I have enjoyed using the clickers.2. The clickers helped me to learn material presented in the seminars and exercises.3. The clickers helped me pay closer attention to the ideas and concepts presented in the
seminars and exercises.4. The clickers helped me to better understand the seminars and exercises by catching and
correcting my mistakes.5. Using the clickers after a seminar or exercise wrap-up took time that could have been more
productively devoted to the next exercise.6. I enjoyed the increased involvement the clickers provided.7. The clickers increased my level of anxiety and tension since I had to participate whether I
wanted to or not.8. I do not like active learning and thus do not like the clickers.9. The clicker questions helped me in preparing for the exams.10. Immediate on-screen feedback (compared to my clicker response) helped me determine my
level of understanding.11. All exam reviews should include clicker questions.
17
![Page 18: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/18.jpg)
DAU Learner Survey Responses
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
1 2 3 4 5 6 7 8 9 10 11
4 = Strongly Agree
3 = Agree
2 = Disagree
1 = Strongly Disagree
Survey of 148 learners in 7 courses in 3 disciplines indicated results were consistent with other DoD findings18
![Page 19: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/19.jpg)
Learn. Perform. Succeed.
“Engagement” Comments from Learner Surveys
• The clicker helped to keep me engaged when I may not have been paying as close attention as I should have.
• I loved the clickers! Helped me to focus and be more attentive in class! Also made class more fun!
• The clickers sparked my interest in the subject matter.
• Clickers helped with developing dialogue.
19
![Page 20: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/20.jpg)
Learn. Perform. Succeed.
“Satisfaction” Comments from Learner Surveys
• It is a good learning tool – enjoyed its use.
• Fun and active. More clicker!
• "Clickers" made interaction great!
• The clickers provide … an anonymous way of checking knowledge. – When it’s a hand raising exercise, some people don’t want to guess
because they are afraid they will be wrong. The clickers eliminate that situation since no one knows what any one person answered.
• Time to reflect - gave my brain a chance to absorb content.
20
![Page 21: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/21.jpg)
Learn. Perform. Succeed.
Other Comments from Learner Surveys
• Use clickers more often. Have more questions.
• Include more variety of question types than just (A – D).
• Use the clickers earlier to understand which lessons/points to emphasize based on class understanding/knowledge.
• The clickers did not help me learn or understand.
• Clickers useful – but don’t pay too much $$$ for them!
21
![Page 22: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/22.jpg)
Learn. Perform. Succeed.
• Integrating into legacy courses
• Integrating into new courseware
• Results of instructor surveys
DAU experience with integrating technology with classroom questions
22
![Page 23: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/23.jpg)
Learn. Perform. Succeed.
• TODAY - Substitution– Use of audience response system to replace
existing interactions• Test review• Knowledge check• Class opinion
• TOMORROW – Augmentation– Adding new capabilities to existing curriculum
• Ongoing assessment of students• Use of Smart Boards by instructors and students
Integrating into DAU legacy courses
23
![Page 24: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/24.jpg)
Learn. Perform. Succeed.
Integrating into new classroom courses
• Examples: Requirements Management • Advanced course
• Today – “Jeopardy” style game used to review content
• Tomorrow – use of knowledge check and facilitation
questions
• Unexpected benefit – course instructors more aware of need
for student engagement, moving away from just lecture
24
![Page 25: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/25.jpg)
Learn. Perform. Succeed.
Lean 6-Sigma (LSS) Green Belt Training
• Audience: DoD mid-level supervisors • Duration: 4.5 days, 8 hours per day • Questioning Strategies Employed:
– Course “pretest” about LSS knowledge & confidence to lead team process improvement activities
– Interest & readiness to learn at beginning of each lesson– Subject knowledge at beginning, middle & end of lessons– End of course exam review (Jeopardy style game)
• Comparison of post course (exam) with pre-course (pre-test) demonstrated growth in learner knowledge and confidence
25
![Page 26: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/26.jpg)
Faculty Feedback about “Clickers”
PROs• Forces instructor to think about
questioning• Helps assess audience
understanding • Generates discussion• Enables “shy” students to
engage and get feedback
Cons• CPS won’t fix poorly written
questions• Steep learning curve• Some computers lock up due
to memory limitations• Some believe CPS slows down
the class
Faculty “Pioneers” expressed that clickers are useful for some…but, not necessarily usable with all teaching styles…
![Page 27: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/27.jpg)
Learn. Perform. Succeed.
Summary
Classroom questioning strategies
Using technology with classroom questions
DAU experience with integrating technology with classroom questions
27
![Page 28: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/28.jpg)
ResourcesBuilding Expertise: Cognitive Methods for Training and Performance Improvement, by Ruth Colvin Clark, Third Edition, (San Francisco, Pfeiffer, 2008)
Classroom Performance Improvement Project: Improving Submarine Training by Increasing Lecture Interactivity by Capitan Roy Harkins, USN, Lieutenant Commander Al Brady, USN, and edited by Lieutenant Commander Rod Wheeler, USN (November 2003)
Classroom Response System Bibliography - Center for Teaching, Vanderbilt University by Derek Bruff, Assistant Director, Vanderbilt Center for Teachinghttp://www.vanderbilt.edu/cft/resources/teaching_resources/technology/crs_biblio.htm
Classroom Response System Classroom Guide - Center for Teaching, Vanderbilt Universityby Derek Bruff, Assistant Director, Vanderbilt Center for Teachinghttp://www.vanderbilt.edu/cft/resources/teaching_resources/technology/crs.htm
Questioning Strategies for Audience Response Systems: How to Use Questions to Maximize Learning, Engagement, and Satisfaction by Will Thalheimer, PhD, Work-Learning Research, Inc. Somerville, MA (March 2007) http://www.work-learning.com/catalog
Teaching with Classroom Response Systems: Creating Active Learning Environments, by Derek Bruff, Vanderbilt University, (San Francisco, Jossey-Bass, 2009)
Teaching with Classroom Response Systems Blog by Derek Bruffhttp://derekbruff.com/teachingwithcrs/
![Page 29: Innovations in eLearning 2 June 2010elearningsymposium.gmu.edu/2010/Presentations 2010... · Innovations in eLearning 2 June 2010. ... of how the Def\nse Acquisition University \⠀䐀䄀唀尩](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f84447e708231d4448d82/html5/thumbnails/29.jpg)
Resources, cont.The FIT Model of Instruction: Integrating Student Response Systems into Group Collaborative Learning to Enhance Interaction, Comprehension and Retention by Steve Huff, PhD, B&H Consulting Incorporated (2008)
Turning Talk –“A Social Learning Community” for users of the Turning Talk products but also a good resource on using clickers in general. Specific areas for K-12 and Higher Ed practitioners http://www.turning-talk.com/
The Power of Great Questions, Michael Marquart, T+D Magazine, February 2007
Questions? Comments? Want to Learn More? Contact Us!Steve Brown, Logistics Management, [email protected]
Bill Fast, BCEFM, CNE,[email protected]
Deb Fuller, Instructional Systems [email protected]
Dr. Steve Huff, [email protected]
Alvin Lee, PML, [email protected]