Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial...
-
Upload
joseph-jay-williams -
Category
Education
-
view
118 -
download
1
description
Transcript of Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial...
![Page 1: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/1.jpg)
1
Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning,
Financial Success, and Cognitive & Learning Sciences Research
Joseph Jay WilliamsLytics Lab, Graduate School of Education
Office of the Vice Provost for Online Learning, [email protected]
www.josephjaywilliams.com
[Originally from Trinidad & Tobago – apologies for difficulty in following accent.]
![Page 2: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/2.jpg)
2
Topics
• I. Motivation• II. Learning through
Questions/Assessments• III. General Learning & Problem-
Solving Skills• IV. Performance Support Tools &
Checklists
![Page 3: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/3.jpg)
3
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
![Page 4: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/4.jpg)
4
I. Motivation
• Beliefs about Intelligence• To what extent do you agree?
How smart you are can be significantly changed by your effort and hard work.You have a certain amount of intelligence, and what you do doesn’t really change that much.
• Growth Mindset vs. Fixed Mindset• Dweck, 2008; Yeager & Walton, 2012• Williams, Paunesku, Haley & Sohl-Dickstein, 2013
![Page 5: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/5.jpg)
5
MOOClet 1: Exercise (Khan Academy)
1. Number of Problems Practiced
2. Accuracy
[TEXT SHOWN HERE – Randomize]
![Page 6: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/6.jpg)
6
Embed motivational messages
Practice-as-usual
Remember, the more you practice the smarter you become!
Growth Mindset Message
![Page 7: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/7.jpg)
7
Growth Mindset Message helpful?
• Growth Mindset Message
• "Remember, the more you practice the smarter you become.”,
• "Mistakes help you learn. Think hard to learn from them.”
• Practice-as-usual
Jascha Sohl-Dickstein
![Page 8: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/8.jpg)
8
Results: More motivated?
• Growth Mindset Message > Practice-as-Usual
• reduction in dropout rate• increase in Number of problems practiced • increase in Number Correct
• Accuracy: Problems correct/Problems attempted
• Accuracy improvement
![Page 9: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/9.jpg)
9
Interpreting results
• Obvious?• Skeptical? Anything’s statistically
significant with large N…• Second experiment (run simultaneously)
![Page 10: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/10.jpg)
10
Practice-as-usualGrowth Mindset Message
Positive Message
Different motivational messages
Some of these problems are hard. Do your best!Remember, the more you practice the smarter you become!
![Page 11: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/11.jpg)
11
Does any positive message work?
• Growth Mindset Message
• "Remember, the more you practice the smarter you become.”,
• "Mistakes help you learn. Think hard to learn from them.”
• Positive Message• "Some of these problems are
hard. Just do your best."• "This might be a tough problem,
but we know you can do it.”
• Practice-as-usual
![Page 12: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/12.jpg)
12
Effects of Positive Messages?
• Positive Messages not better than Practice-as-Usual!
• Growth Mindset > Positive Message• How intuitive is this result?• Teachers?• Ongoing: After incorrect answer• www.assistments.org Neil Heffernan• Gates Grant for Modeling Research
Rapid Experimentation: tiny.cc/mechanicalturk
![Page 13: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/13.jpg)
13
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
![Page 14: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/14.jpg)
14
Constraint Satisfaction
Practical ProductResearch
Financial Value
![Page 15: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/15.jpg)
“MOOClets” or Modular Online Resources
• E.g. MOOClet: Math exercise, 5 minute interactive video, 30 minute lesson, Study Strategy coach, Quiz/Assessment
• www.josephjaywilliams.com/mooclets • MOOClets Group Tue 12-1 PST (remote)• tiny.cc/joinmooclets mailing list & updates• Williams, J.J. & Williams, B. A. (2013). Randomized
Experiments as a Methodological and Conceptual Tool for Improving Design of Online Learning. Data Driven Education, NIPS Workshop. tiny.cc/experimentsonlinelearning
15
![Page 16: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/16.jpg)
16
Integrate Diverse Expertise & Innovations
![Page 17: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/17.jpg)
17
Where is relevant research, best practice, technology, funding, market? tiny.cc/mapofspace, tiny.cc/stanfordmap
MOOCs (Coursera, EdX, Udacity)
www.KhanAcademy.org www.Assistments.org
K-12 Ed-Tech space (Bay Area, Boston...)(www.edsurge.com)
(www.edudemic.com )Higher-Ed Technology/LMS
E-Learning & Training forCorporate, State, Federal, Army
Psychological ScientistsComputer Scientists
Education
Corporate Training
Funding Agencies:IES, Dept. of Education, Dept. of Labor,
NIH, NSFPractical Reports:
What Works ClearinghouseIES Practice Guides
University Teaching & Learning CentersDepartment of Education reports
Think Tanks:SRI International
WestEd
Pittsburgh Science of Learning Center
Learning Sciences
Mooc Research Initiative
Open Learning Initiative
Distance LearningAthabasca, Western Governor’s University
Economics of Education
Habit & Behavior Change
Medical Education
Centers/Scholarship of Teaching & Learning
Assessment
PearsonETS
Open Education ResourcesCreative Commons
Foundations:Gates, Spencer, Hewlett, Kresge
Venture CapitalPrivate VCs
New SchoolsImpact Investing
Accelerators:Imagine K-12,
HCI, Design Thinking
Workforce Development
![Page 18: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/18.jpg)
18
Relevant Resources
• www.josephjaywilliams.com/home/useful-software
• www.josephjaywilliams.com/education• www.josephjaywilliams.com/online-education-exp
eriments• tiny.cc/mapofspace• Applying Cognitive Science to Online
Learning. Paper presented at NIPS Workshop on Data Driven Education (tiny.cc/cognitivescienceinmoocs)
• Improving Outcome of Psychosocial Treatments by Enhancing Memory and Learning. Perspectives in Psychological Science.
![Page 19: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/19.jpg)
19
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
![Page 20: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/20.jpg)
Messages in Coursera?
20
Week 6 QuizWhether or not you’re correct on the first try, each attempt at a tough problem can make you better at statistics.
![Page 21: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/21.jpg)
21
Motivational Introductions
Practice-as-usualHave fun in the course!
Anne Friedlander Growth MindsetText version of Mindset MessageMindset Message – Another InstructorPositive EncouragementBelonging
![Page 22: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/22.jpg)
22
Instructor Guide to foster Growth Mindset• tiny.cc/idstrategymindset
Where do you think you will apply this?
Do you have any questions or want clarification?
Click to see any of the Case Studies below
Would you like to?
![Page 23: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/23.jpg)
23
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
![Page 24: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/24.jpg)
24
Topics
• I. Motivation• II. Learning through
Questions/Assessments• III. General Learning & Problem-
Solving Skills• IV. Performance Support Tools &
Checklists
![Page 25: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/25.jpg)
II. Learning through Questions/Assessments
• Discovery of patterns(Williams & Lombrozo, 2010, Cognitive Science)
• Use of prior knowledge(Williams & Lombrozo, 2013, Cognitive Psychology)
• Children’s causal learning (Walker, Williams, Lombrozo, Gopnik, 2012)
• Overgeneralize at expense of specific facts
(Williams et al, 2013, Journal of Experimental Psychology: General)
• Contradictions & belief revision (Williams, et al 2013; 2014; Cog Sci, CHI WiP)
• Comparison of examples (Edwards, Williams, Lombrozo, Gentner, 2013)
GLORP DRENTOnline/Blended
Educational Resources
![Page 26: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/26.jpg)
26
Adding questions/activities
• tiny.cc/augmentedvideo• tiny.cc/augmentedexercise
![Page 27: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/27.jpg)
27
III. General Learning & Problem-Solving Skills
![Page 28: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/28.jpg)
Learning/Self-Questioning Strategies
• Specific Prompt VS.• General Strategy of Self-Questioning?• Teach strategies
(McNamara, 2004, Discourse Processes; Palinscar & Brown, 1984, Cognition and instruction; Williams & Lombrozo, 2010, Cognitive Science)
• Habit & Behavior Change • Theory & Technology: Embed & Remind• Methodology: “longitudinal”, large N
28e.g. Weinstein & Mayer, 1986 Rothman, Baldwin, & Hertel, 2004
![Page 29: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/29.jpg)
29
Embedded “On-Request” Strategy Instruction
Clickable link.
[Click here to learn about the “What? Why? How?” strategy]
![Page 30: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/30.jpg)
30
Self-questioning strategy: What? Why? How?
![Page 31: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/31.jpg)
31
Cues – Reflect & Remind
![Page 32: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/32.jpg)
32
Preliminary Results
N = ~ 800 per condition
• Prompted Problem-solving better than Practice-as-usual.
• Increased Accuracy on intervention and post-intervention problems.
![Page 33: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/33.jpg)
33
Learning Assistant
• tiny.cc/learningassistant
![Page 34: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/34.jpg)
34
Learning Assistant – e.g. 1
![Page 35: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/35.jpg)
35
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
![Page 36: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/36.jpg)
36
Topics
• I. Motivation• II. Learning through
Questions/Assessments• III. General Learning & Problem-
Solving Skills• IV. Performance Support Tools &
Checklists
![Page 37: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/37.jpg)
37
IV. Performance Support Tools & Checklists
![Page 38: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/38.jpg)
38
Coach/Support Learning Behaviorstiny.cc/kalearningcoach
![Page 39: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/39.jpg)
39
A Lesson: Concept + Digital Tool to apply Concept in Future
![Page 40: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/40.jpg)
40
Tool for Difficult Conversations
![Page 41: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/41.jpg)
41
Topics
• I. Motivation• II. Learning through
Questions/Assessments• III. General Learning & Problem-
Solving Skills• IV. Performance Support Tools &
Checklists
![Page 42: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/42.jpg)
42
Guiding Principles
• Multiple Constraints: Practical, Scientific, Financial
• Modularity in Product Design: “MOOClets”• Future-thinking but Actionable• Empirical and Integrative Work• Experiment-Focused Instructional Design• Matchmaking Relevant Researchers• Iterative Improvement• Collaborative Development
![Page 43: Learning Engineering of MOOClets: Simultaneously benefiting Professional Learning, Financial Success, and Cognitive & Learning Sciences Research](https://reader036.fdocuments.us/reader036/viewer/2022070318/557bfc29d8b42aef788b4700/html5/thumbnails/43.jpg)
43
Acknowledgements
• Tania Lombrozo & Tom Griffiths• John Mitchell & Candace Thille• Jascha Sohl-Dickstein• Jace Kohlmeier & Khan Academy• Sam Maldonado• Lytics Lab (lytics.stanford.edu)• VPOL (Vice Provost for Online
Learning, online.stanford.edu)