Assurance of Learning meets Experiential Learning...Experiential Learning and the Assurance of...
Transcript of Assurance of Learning meets Experiential Learning...Experiential Learning and the Assurance of...
Experiential Learning and the Assurance of Learning Challenge
Ft. Lauderdale, Florida January 22, 2019
A workshop for decision makers with a direct impact on experiential program design, management, delivery and evaluation.
©2019Michellana Jester, Kerry Laufer & Shannon McKeenDo not reproduce without permission
Facilitators:
Michellana Jester, MIT Sloan School of [email protected]
Kerry Laufer, Tuck School of Business at [email protected]
Shannon McKeen, UNC Kenan-Flagler Business [email protected]
2©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Morning
• Defining our terms
• Why assurance of learning?
• Introduction to AoL standards
• Panel: Assurance of learning challenges and opportunities
Afternoon
• Operationalizing the standards
• Small group activity
• Discussion: Key insights
• Concluding thoughts
• Evaluation and feedback
Plan for the dayDiscussion topics and activities
3©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Let’s start with some contextWho we are, what we’ve been up to
MBA LEPE
ASSURANCE OF LEARNING
WORKING GROUP
Kerry Laufer
Faculty Advisor, UNC Kenan-Flagler
Shannon McKeenMichellana Jester
Lecturer and Faculty Course Lead, MIT Sloan
Director and Faculty Advisor, OnSite Global Consulting, TuckGO
4©2019 Jester, Laufer & McKeen - Do not reproduce without permission
A lot has happened in 4 yearsA bit about our journey
2016
Survey of 100 schools
Common approaches and evidence of practices that demonstrate an understanding of how students learn from experience, but no standards
Growing interest in AoL
20182017 2019
BizEd Article
Survey assesses status of implementation of 6 AoL standards across LEPE schools
Results presented to LEPE schools at Tepper
Forthcoming:
Assurance of Learning
Standards and Scaling
Strategies to Enable
Expansion of Experiential
Learning Courses in
Management Education
Survey provides update on peer school trends and approaches
Introduction of standards at LEPE Conference at Darden
LEPE schools provide feedback on standards
5©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Yes, 76%
No, 24%
Are you personally involved in EL
13
4
3
Assurance of Learning
Experiential Learning
Other
Describe your role in AoL
We have EL & AoL experience in audience
Source: Pre-workshop survey of attendees, January 2019
6©2019 Jester, Laufer & McKeen - Do not reproduce without permission
71% 71%
94%
Single Session Multi-Session Project-Based
Best description EL types at your institution
Among 17 Respondents
65%
35%
0%
Prevalent Growing Emerging
Prevalence of EL at your institution
Among 17 Respondents
Your institutions have embraced EL
Source: Pre-workshop survey of attendees, January 2019
7©2019 Jester, Laufer & McKeen - Do not reproduce without permission
• Complexity of applying AoL at course level
• Client scope creep in project-based courses
• Ensuring a consistent learning experience across project teams
• Harmonizing across disparate EL activities
Sample of challenges faced
Source: Pre-workshop survey of attendees, January 2019
8©2019 Jester, Laufer & McKeen - Do not reproduce without permission
• Evaluation; data collection; what to assess; how to assess
• Fitting in with institution’s data collection and evaluation methods
• Scaling - every semester is different
• Setting expectations for and of students
Sample of challenges faced
Source: Pre-workshop survey of attendees, January 2019
9©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Our primary focus today
• Applying or modifying assessment tools designed for content-based, face-to-face learning to experiential learning and/or other delivery methods
• Best practices in applying standards
• Learning what to assess and how
• Scaling EL
Opportunities to discuss with colleagues
• Implementing some aspect of EL in all courses
• Benchmarking; trends
• How to integrate EL into an already packed curriculum
• Applying EL in different disciplines (e.g., consulting, investment funds, global)
Your expectations/questions
Source: Pre-workshop survey of attendees, January 2019
10©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Find your connections
1. Introduce yourself to others at your table.
2. Identify as many things as possible that everyone at your table has in common.
3. Stretch your thinking and be creative.
4. Table with the most number and most creative connections wins!
11©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeDefinitions
Definitions Why AoL? Standards Panel
12©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Definitions Why AoL? Standards Panel
Experiential LearningMust be designed to promote student learning and development
A philosophy that informs many methodologies in which educators purposefully engage with learners in direct experience and focused reflection in order to:
• Increase knowledge• Develop skills• Clarify values• Develop people’s capacity to contribute
to their communities
- Association for Experiential Education http://www.aee.org/
Experiential Education
In management education - Any learning that supports students in applying their knowledge and conceptual understanding to real-world problems or situations (Wurdinger & Carlson, 2010).
A cycle of learning that consists of concrete experience, reflective observation, abstract conceptualization, and active experimentation.
As a pedagogy - requires students to actively and iteratively apply and reflect on the concepts, knowledge, and skills acquired in their course of study.
David Kolb, 1984
Experiential Learning
13©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Lecture/readings Case Simulation/Exercise Project-based
Content Instructor created; research driven Often third party author; based on “real” situation
Often third party created; meant to mimic reality to provide opportunity for feedback
Real world, created by process, instructors are guides
Control Very controllable, few variables at play, consistent experience
Controllable, deeper discussions possible
Controlled environment High variability
Outcomes Outcomes more certain, easy to measure with quizes and tests
Lessons from case outlined in teaching notes, measured by discussion or write ups.
Objectives clear, measured by results or performance
Outcomes less certain more difficult to measure
Customization Expected outcomes generally agreed to be same for all
Expected outcomes generally agreed to be same for all
Expected outcomes generally agreed to be same for all
Outcomes can be very different depending on the individual learning goals of the student, their role in the project, their interests and other factors
Assessment Established assessment tools are deemed generally sufficent/acceptable (tests, exams, papers, etc)
Established assessment tools are deemed generally sufficent/acceptable (tests, exams, papers, etc)
Established assessment tools are deemed generally sufficent/acceptable (tests, exams, papers, etc)
Established assessment tools borrowed from the classroom may not be suffcient
Definitions Why AoL? Standards Panel
Project-based EL is “different”
14©2019 Jester, Laufer & McKeen - Do not reproduce without permission
EL Conceptual FrameworkAbbreviated version
Experience Learning Type Exposure – “Lab” Practice – “Workshop” Experience – “Project”
Learning Design Role play, simulation Multi-session, trip Project
Learning Objective Understand content Use knowledge; practice skills Contextual utilization of knowledge and skills
Size/Scope One to many; flexible Few to many; scalable “few” teams of 4 – 6; hard to scale
Specificity of Learning Knowledge and skills Practice and limited application
Integrated application
Duration < 3 class periods > 3 class periods, < 10 days >= ½ semester
Delivery Frequency Options Multiple times a semester 1 -2 times a semester 1-2 times a year
Types of learning Cognitive Cognitive and Behavioral Cognitive, Behavioral and Affective
Pedagogy Instructor led Instructor-aided Student-centered
Definitions Why AoL? Standards Panel
15©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeWhy Assurance of Learning? Why Now?
Definitions Why AoL? Standards Panel
16©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Definitions Why AoL? Standards Panel
“Experience is a funny thing. In any given experience, some people learn valuable lessons.
Other people, in that same experience, learn nothing or even the wronglessons.”
- DeRue et al., 2015
17©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Summary of Key Observations
2
3
AACSB attention to assurance of learning increased; important to understand how our EL efforts fit into our schools’ broader processes and understand what we can do at the course level.
Experiential learning outcomes can be short-lived if not intentionally and individually pursued.
By identifying and applying some standards, LEPE can influence how AoL is applied to experiential learning courses at our schools and assist AACSB’s efforts to improve learning outcomes for all students.
1
Definitions Why AoL? Standards Panel
18©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Requirement:
AACSB Standard 8, 2013 (Curricula Management and Assurance of Learning) - “The school uses well-documented, systemic processes for determining and revising degree program learning goals; designing, delivering and improving degree program curricula to achieve learning goals; and demonstrating that degree program learning goals have been met.”
10%
18%
64%
Percent mentions of AOL in opportunities for improvement
LEPE 2014 LEPE 2015 LEPE + 2016
Source: LEPE+ 2016 survey
Definitions Why AoL? Standards Panel
Interest in AoL has grown as EL has grown
Challenges:• Experiential learning is “best conceived as a process, not in terms
of outcomes (Kolb, 1984)” • Experiential learning outcomes cannot be precisely orchestrated
with so many variables in play. • Established processes and rubrics for measuring learning may not
be sufficient• Gaps between intention and execution
19©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Assurance of Learning steps
1. Define learning goals and outcomes at the degree/program level
2. Align curriculum with goals3. Identify instruments and assessment
measures4. Collect, analyze and disseminate
assessment data5. Use assessment data to improve teaching,
learning and student experience
AACSB Standard 8 (2013)
School uses well-documented, systemic processes to:
1. Determine and revise degree program learning goals
2. Design, deliver and improve curricula to achieve goals;
3. Demonstrate that goals have been met
1. What do we want students to learn?
2. How will they learn it?
3. How will we know they have (or not)?
4. What will we do if they haven’t?
Translation
Definitions Why AoL? Standards Panel
AoL is program-wide outcomes assessmentIt’s about student learning, continuous improvement and accountability
20©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Everyone struggles with AoL
Three Common Problems(General)
1. Individual student performance (not group performance) needs to be assessed.
2. Assessing leadership skills
3. How to use data to improve student learning.
Source: Martell, 2015. Demystifying Assurance of Learning. (AACSB Blog)
This presents opportunties for EL, despite ongoing challenges
Definitions Why AoL? Standards Panel
21©2019 Jester, Laufer & McKeen - Do not reproduce without permission
• Communication skills
• Ethics
• Critical thinking
• Decision-making
• Problem solving
• Ability to integrate across business disciplines
• Global perspective
• Team skills
Overlap: Common ProgramLearning Goals & LEPE School Course
Learning Objectives
Source: AACSB Assurance of Learning Standards: An Interpretation (Revised May 2013) and LEPE+ Survey 2016
Opportunity: EL already aligned with MBA school missionsand common program-level learning goals
Definitions Why AoL? Standards Panel
Opportunities for project-based experiential learning
• Improve student learning
• Better integration with school’s broader learning goals
• Greater alignment with leadership curriculum specifically
• Establish credibility and grow
Source: Demystifying Assurance of Learning. Martell, 2015 (AACSB Blog)
“The primary source of Learning to lead, to the extent that leadership can be learned, is experience”
- McCall, 2004. Leadership development through experience.
22©2019 Jester, Laufer & McKeen - Do not reproduce without permission
But we need to address ongoing challenges
• Experiential learning outcomes hard to measure with so many variables in play.
• Established processes and rubrics for measuring learning may not be sufficient
• Gaps between aspiration/intention and execution
Ongoing challenges for EL
Definitions Why AoL? Standards Panel
Set some standardsEncourage implementation
Monitor our progress
23©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeThe Standards
Definitions Why AoL? Standards PanelDefinitions Why AoL? Standards Panel
24©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Tailor learning outcomes to the individual
Create opportunities for reflection
Provide timely and relevant feedback to teams and individuals
Acknowledge and incorporate the role of emotions
Evaluate in three domains: cognitive, affective and behavioral
Close the loop
1
3
2
4
5
6
AoL Standards for Project-Based EL Coursesin MBA Programs
Definitions Why AoL? Standards Panel
25©2019 Jester, Laufer & McKeen - Do not reproduce without permission
1-Tailor Learning objectives
• Four (max) core learning objectives
• Customize two
• Students share with teammates and program
• Feedback and assessment tailored to match individualized plan.
2-Create reflection opportunities
• At least twice during the project
• Directed to team and to individuals, tied to course-level and individual learning objectives
• Faculty advisors should model good practice; requires training and support
AoL Standards for Project-Based EL Coursesin MBA Programs
• Required reflection assignments and activities for individuals.
• Include 360-degree instruments for all students
• Reflection is structured/focused
• Faculty and staff trained to design and facilitate reflection
3-Provide timely and relevant feedback
Definitions Why AoL? Standards Panel
26©2019 Jester, Laufer & McKeen - Do not reproduce without permission
4-Use emotions
• Negative and positive emotions can be mined to catalyze learning
• Strongest performers understand and manage emotions and use as input for learning
• Faculty and staff should be trained to capitalize on emotions for feedback and coaching
5-Evaluate in 3 domains
• Gap between intention and execution
• Hardest step, because each project is unique
• “Post mortem” at end identify successes and opportunities for improvement
• Focus on what we can learn from data
• Responsibility should be prioritized and assigned; include longitudinal results
AoL Standards for Project-Based EL Coursesin MBA Programs
• Advancement and integration of domains can be promoted through EL
• Every learning objective should be directly associated with one of these three domains
• Faculty and staff need orientation and feedback delivery training in all three domains.
6-Close the loop
Definitions Why AoL? Standards Panel
27©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Early Stage: Student Engagement
Mid Stage:Project In Progress
Final Stage: Assessing the Work
• Tailor Learning Outcomes
Standards In Context
• Create Opportunities for Reflection
• Provide Feedback Early and Often
• Role of Emotion
• Evaluate 3 Domains of Learning • Close the Loop
Definitions Why AoL? Standards Panel
28©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengePanel Discussion
Definitions Why AoL? Standards Panel
29©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Meet our panelists
Jeff Biegenek
Executive Director, MBA Roundtable
Moderator
Executive Director, Eller Graduate School of Business Consulting, University of Arizona
Phil Miller
Asst. Dean MBA & MS programs, Carlson School of Mgmt., University of Minnesota
Sandra Kenny
Executive Director, Eller Graduate School of Business Consulting, University of Arizona,
Definitions Why AoL? Standards Panel
Director of Product Design & Development, Capsim Management Simulations
Jordan NovakBryan Andriano
Executive Director, Global & Experiential Education, George Washington University
30©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Morning
• Defining our terms
• Why assurance of learning?
• Introduction to AoL standards
• Panel: Assurance of learning challenges and opportunities
Afternoon
• Operationalizing the standards
• Small group activity
• Discussion: Key insights
• Concluding thoughts
• Evaluation and feedback
Looking ahead...Discussion topics and activities
31©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeOperationalizing the Standards
32©2019 Jester, Laufer & McKeen - Do not reproduce without permission
“… for the things we have to learn, before we can do them, we learn by doing them.”
AristotleThe Nichomachean Ethics
33©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Tailor learning outcomes to the individual
Create opportunities for reflection
Provide timely and relevant feedback to teams and individuals
Acknowledge and incorporate the role of emotions
Evaluate in three domains: cognitive, affective and behavioral
Close the loop
1
3
2
4
5
6
AoL Standards for Project-Based EL Coursesin MBA Programs
34©2019 Jester, Laufer & McKeen - Do not reproduce without permission
4
10
8
3
1
10
Tailored Learning
Reflection
Feedback
Role of Emotions
Evaluating in 3 Domains
Close the loop
Among 17 respondents
In this group, standards are being applied
Source: Pre-workshop survey of attendees, January 2019
35©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Tailor learning outcomes to the individual
Create opportunities for reflection
Provide timely and relevant feedback to teams and individuals
Acknowledge and incorporate the role of emotions
Evaluate in three domains: cognitive, affective and behavioral
Close the loop
1
3
2
4
5
6
AoL Standards for Project-Based EL Coursesin MBA Programs
36©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Early Stage: Student Engagement
Mid Stage:Project In Progress
Final Stage: Assessing the Work
• Tailor Learning Outcomes
Experiential Learning Standards
• Create Opportunities for Reflection
• Provide Feedback Early and Often
• Role of Emotion
• Evaluate 3 Domains of Learning • Close the Loop
Definitions Why AoL? Standards Panel
37©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Tailored Outcomes
Real, not perfunctory
Online tools in use for incorporating objectives in feedback tools
Both home grown and off the shelf
Decreased reliance on manual processes
Link with elsewhere in degree program
Individualized sessions with project leader or faculty advisor to check in on progress
University of North Carolina-- objectives as part of mid-point feedback session
Incorporate within team charter, roles, learning objectives
Make dynamic and public
Tailor peer feedback
University of Wisconsin & Career Development Plan
University of Arizona Business Consulting & Business Communications
Revisit frequently Delivery at team and individual levels
Leverage technology to scale
38©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Reflection
“Not just stapled on at the end…”
Balancing need for formal assessment and allowing for personal development/growth
Faculty feedback to support of deeper reflection
Haas - moved away from reflection assessments; reintroduced narrative component for context
Start at the beginning
MIT – team and individual reflection framed within three stages of project work
Haas - experiments with creative prompts (YouTube, visual explorer cards)
Alternative methods include papers, journals, video, blog, small & large group debrief, etc.
Feedback prompts to trigger team and individual reflection of performance
Mitigate blind spots
Need for integrated process as opposed to episodic deliverables
Develop ‘comfort’ with reflection among students (and faculty)
UCLA- student reflection at 3 phases of the project
University of Arizona-“premortem” exercise
Reflection and feedback are intertwined
Delivery at team and individual levels
The challenge of grading
39©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Feedback
Fundamentals are in place at most institutions
Online tools in use for collecting and sharing feedback
Both home grown and off the shelf
Notable decrease in reliance on manual processes
Clear infrastructure and process in place
Individualized feedback reports by email
Mandated 1:1 and team sessions
Pulse checks increasingly used (weekly/bi-weekly)
For most, at least two –midpoint and end; some do more
Formal feedback at key point but informally throughout
Vision and role clarity
“This is what we do, how we do it, when, and why”
Person responsible to oversee/execute
Frequency and timeliness
Delivery at team and individual levels
Leverage technology to address scale
40©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Role of EmotionMaking the invisible more visible
Emotion is essential to learning
But, we’re reluctant to move it from the margins
And along the spectrum of users, there are challenges
Increases the strength of memories and help recall
One of the three learning dimension (cognition and environment) which interacts and shapes our learning
Plutchick’s Wheel of Emotions
Decision making with incomplete information and high ambiguity can generate tension
Tools used to assist include high-tech (GlobeSmart; Global Mindset Inventory) and low tech (reflection, team meetings, etc)
Emotion perceived as irrational; a lack of discipline and rigor
Culture/environment may discourage expression of emotion
Faculty advisors may be reluctant to address as a teachable moment
Resource constraints (time, money, people)
Student resistance
Structural/design elements
Lack of faculty awareness, appreciation, interest
It’s embedded & we manage it
41©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Evaluating Using 3 Learning DomainsCognitive, Affective and Behavioral assessment to improve learning outcomes
The 3 domainframework is about learning improvement
The domains are interrelated
There are several key challenges
To assess the thinking, feeling, doing aspects of learning
May help students identify weaknesses and opportunities for improvement
Also knowledge skills and attitude (KSA)
Domains reinforce each other and help with sense making
Cognitive and behavioral are perceived as the easiest to evaluate.
The role of emotion is linked to the learning domains.
Resource constraints (time, money, people)
Identifying tools to measure domain effectiveness
Attention and engagement of students and faculty
Ability to scale
These domains tend to be already embedded in our work
Cognitive (understand) written reports, presentations
Behavioral (perform) –client/team engagement
Affective (motivate)- class participation, reflection
42©2019 Jester, Laufer & McKeen - Do not reproduce without permission
• Identify student learning weaknesses
• Generate ideas for course interventions/changes
• Evaluate ideas and decide on interventions
• Determine roles of faculty coaches/mentors, etc. and where they fit into this process
• Address identified weaknesses
*Adapted from Rexeisen and Garrison, 2013
Closing the Loop Pockets of innovation exist but more infrastructure and standardization needed
It’s complicated and key stakeholder involvement varies
Don’t know how or don’t have time
Resistance and concern
Develop a CTL infrastructure*
Schools struggle to implement a comprehensive AoL process – specifically CTL – across programs
MBA-program level CTL processes and conversations vary
Lack of skill, expertise and/or resources
Unclear who should drive the process
Often takes a back seat to more pressing priorities
Faculty resistance; concerns about another form of evaluation
Administration concerns about another formal hurdle
43©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Mapping Experiential Learning Standards to AoL
• Tailor Learning Outcomes
• Create Opportunities for Reflection
• Provide Feedback Early and Often
• Role of Emotion
• Evaluate 3 Domains of Learning
• Close the Loop
What do we want our students to learn in our EL program?
In what way will we provide opportunities for our students to learn these things?How will they (how will we) know if they have learned these things?
What will we do it they have not learned these things?
Assurance of Learning Steps
1. Define learning goals and outcomes at the degree/program level
2. Align curriculum with goals
3. Identify instruments and assessment measures
4. Collect, analyze and disseminate assessment data
5. Use assessment data to improve teaching, learning and student experience
Experiential Learning Standards
44©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeSmall Group Work
45©2019 Jester, Laufer & McKeen - Do not reproduce without permission
1. What is your reaction to the material
presented?
2. What do we have right?
3. What have we missed?
4. What are your suggestions?
Advancing the Experiential Learning and AoL Challenge Conversation
46©2019 Jester, Laufer & McKeen - Do not reproduce without permission
1. What did you learn from your conversation?
2. What was surprising?
3. In what way did the discussion shed light on the
six standards discussed today? In what way did
the discussion generate more questions for you?
Advancing the Conversation: Reflections and Insights
47©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeConcluding Thoughts
49©2019 Jester, Laufer & McKeen - Do not reproduce without permission
• Evangelize and refine standards
• Develop toolkit/templates
• Longitudinal study (multi-school effort)
• Scaling Experiential learning
• Other MBA Leaders in Experiential Project-Based Education (LEPE) Working groups:
• Faculty Advisor expectations
• Best practices in NDA’s
• Personality/Work style assessments
• R
What comes nextOur wish list is long
50©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Program Learning Goals with AoL Standards
MBA Degree Program Learning Goals
EXL Early Stage:Tailor Learning Outcomes
EXL Mid-Stage:ReflectionFeedback(Role of Emotion)
EXL Final Stage:Evaluating 3 DomainsClose the Loop
Strategic Thinking & Leadership
3 3Demonstrates leadership skills
3
Global Perspective & Cultural Diversity
2 3 3
Ethics & Social Responsibility
1 3Understands ethical consequences
of actions
3Demonstrates ethical behavior
Critical Thinking 2 3Rigorous, sound analysis & decision
making
2Applied business knowledge
Effective Oral & Written Communication
3Effective writing
3Effective oral communication
2Effective business communication
Expected level of engagement to reach learning objective: Strong= 3 Moderate=2 Probable=1
51©2019 Jester, Laufer & McKeen - Do not reproduce without permission
“Anything that we have to learn to do we learn by the
actual doing of it… We become just by doing just acts,
temperate by doing temperate ones, brave by doing
brave ones.”
Aristotle, Niconachaen Ethics, Book II
52©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeThank you!
53©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Surveys:
Experiential Learning and the Assurance of Learning Challenge pre-conference survey. Contact Jeff Bieganek at [email protected] 2018, Contact Shannon McKeen at [email protected] 2017 (Leaders in Experiential Project-Based Education), Contact Shannon McKeen at [email protected]. MBA Roundtable 2016 found at https://www.mbaroundtable.org/files/Survey%20page/Surveys/Managing%20Client%20Based%20Experiential%20Projects%20_Survey%20Summary%20Spring%202016%20.pdf.
References
Professional Associations:
Association for Experiential Educationhttps://www.aee.org/National Society for Experiential Educationhttps://www.nsee.org/8-principles://
54©2019 Jester, Laufer & McKeen - Do not reproduce without permission
2013 Eligibility Procedures and Accreditation Standards for Business Accreditation (last revised July 1, 2018). Retrieved from https://www.aacsb.edu/-/media/aacsb/docs/accreditation/business/standards-and-tables/2018-business-standards.ashx?la=enAACSB assurance of learning standards: An interpretation: AACSB White Paper (2007, November). Retrieved from https://naspaaaccreditation.files.wordpress.com/2014/04/aacsb.pdfClements, M.D. and Cord, B.A. (2013) Assessment guiding learning: Developing graduate qualities in an experiential learning programme. Assessment and Evaluation in Higher Education,38(1)114–24. Demystifying Assurance of Learning: Q&A with Kathryn Martell (2015). Retrieved from https://www.aacsb.edu/blog/2015/october/demystifying-assurance-of-learning-q-and-a-with-kathryn-martell. DeRue, D. S., Ashford, S. J. and Myers, C. G. (2012), Learning Agility: In Search of Conceptual Clarity and Theoretical Grounding. Industrial and Organizational Psychology, 5: 258–279.Dirkx, J. (2008). The meaning and role of emotions in adult learning. In New Directions for Adult Learning and Continuing Education, No. 120, Winter. Eyler, J. (2009). The Power of Experiential Education. Liberal Education, 95(4). Retrieved from https://www.aacu.org/publications-research/periodicals/power-experiential-education.
References (cont.)Articles and online resources:
55©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Gabrielsson, J., Tell, J., & Politis, D. (2010). Business simulation exercises in small business management education: using principles and ideas from action learning. Action Learning: Research and Practice, 7(1), 3-16.Knowles, M. (1984). The Adult Learner: A Neglected Species (3rd Ed.). Houston, TX: Gulf Publishing.Kolb, D. and Fry, R. (1975) Towards an applied theory of experiential learning, in C.Cooper (ed.) Theories of Group Process, London: John Wiley. Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall.Lahey, J. (2016, May 4). To help students learn, engage the emotions. The New York Times. Retrieved fromhttps://well.blogs.nytimes.com/2016/05/04/to-help-students-learn-engage-the-emotions/Laufer, K., McKeen, S., and Jester, M. (2018). Taking Measure of Experiential Learning, BizEd Magazine. Retrieved from https://bized.aacsb.edu/articles/2018/january/taking-measure-of-experiential-learningRevans, R. W. (1982).What is Action Learning? Journal of Management Development, 1(3) 64-75. Retrieved from https://www.emeraldinsight.com/doi/abs/10.1108/eb051529Rexeison, R. and Garrison, M. (2013) Closing-the-loop in assurance of learning programs: Current practices and future challenges. Journal of Education for Business, 88: 280–285.
References (cont.)Articles and online resources (cont.):
56©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeAppendix – Additional Resources
57©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Concrete Experience: Full and unbiased involvement in learning experience
Reflective Observation: Contemplation on one’s experiences from various perspectives
Abstract Conceptualization: Idea formulation and integration
Active Experiment: Incorporation of new ideas into action
Kolb, D.A. (1984) Experiential Learning: Experience as a Source of Learning and Development. Englewood Cliffs, New Jersey: Prentice Hall.
Kolb’s Cycle of Learning
58©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Kolb’s Experiential Learning Model
Note the use of concrete, ‘here-and-now’ experience to test ideas; and use of feedback to change practices and theories (Kolb, 1984). Kolb joins with Dewey to emphasize the developmental nature of learning and with Piaget for an appreciation of cognitive development. He named his model to emphasize the link between Dewey, Lewin, and Piaget; and to stress the role experience plays in learning.
Kolb, D. and Fry, R. (1975) Towards an applied theory of experiential learning, in C.Cooper (ed.) Theories of Group Process, London: John Wiley.
Kolb, D. A. (1984) Experiential learning: Experience as a Source of Learning and Development. Englewood Cliffs, NJ: Prentice Hall
60©2019 Jester, Laufer & McKeen - Do not reproduce without permission
1. Intention2. Preparedness and Planning3. Authenticity4. Reflection5. Orientation and Training6. Monitoring and Continuous Improvement7. Assessment and Evaluation8. Acknowledgment
National Society for Experiential Educationhttps://www.nsee.org/8-principles://
Last Updated on Monday, December 09, 2013 12:16 PM
NSEE’s Eight Principles of Good PracticeFor All Experiential Learning Activities
61©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Why should we care?
The Future of Learning: 4 Key Trends
L ≥ CLearning must be equal to or faster than the rate of change.
61
1. The unbundling of education
2. Personalization through technology
3. Continuous education
4. Creativity orientation
62©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Experiential Learning and the Assurance of Learning ChallengeSurvey data and research in support of standards
Definitions Why AoL? Standards PanelDefinitions Why AoL? Standards Panel
63©2019 Jester, Laufer & McKeen - Do not reproduce without permission
Source LEPE + 2017 survey n=61
01020304050607080
Learning objectives
Primary Ancillary Possibility
Source LEPE+ 2016 survey n = 70
Definitions Why AoL? Standards Panel
Learning objectives are numerous, paradoxical, and often not individualized
General or individualized learning objectives?
79%
29%
General Learning Objectives Individualized LearningObjectives
64©2019 Jester, Laufer & McKeen - Do not reproduce without permission
14
16
26
27
28
33
37
37
39
41
48
40
27
27
23
20
11
15
13
8
INDIVIDUAL LEARNING OBJECTS ARE ESTABLISHED …
MID-COURSE ASSESSMENT(S) TIED TO INDIVIDUAL …
MID-COURSE ASSESSMENT(S) TIED TO TEAM …
FINAL ASSESSMENT TIED TO INDIVIDUAL LEARNING …
INDIVIDUALS RECEIVE FEEDBACK MID -COURSE …
INDIVIDUALS RECEIVE FEEDBACK AT THE END OF …
TEAM RECEIVES FEEDBACK MID -COURSE FROM …
FINAL ASSESSMENT TIED TO TEAM LEARNING …
LEARNING OBJECTIVES ARE ESTABLISHED AND …
TEAM RECEIVES FEEDBACK AT THE END OF THE …
Formal part of program Not formally part of program
Source: LEPE+ Survey 2017; n=62
Feedback from different sources collected and sharedmid-course and at conclusion is common
Definitions Why AoL? Standards Panel
65©2019 Jester, Laufer & McKeen - Do not reproduce without permission
38% 38%
24%
Yes No Maybe
Do you have resources to apply individualized learning
objectives?
Source: LEPE + 2017 Survey; n = 65
Individual student performance (not group performance) needs to be assessed.- Martell (2015)
“Experiences differ immensely because of variables in projects. Therefore, assessments [and learning objectives] must be designed to support……. individual development”- Clements & Cord (2013)
EL Research, AACSB standards, and LEPEfeedback support tailoring
Definitions Why AoL? Standards Panel
66©2019 Jester, Laufer & McKeen - Do not reproduce without permission
The most critical factor for achieving powerful learning outcome from experiential learning program is the inclusion of opportunities for feedback and reflection.
-Eyler, 2009
Reflection on and learning from both individual and collective work experience thus forms the very basis for successful action learning, where participating individuals can develop their self-awareness and capacity to manage both themselves and others more effectively.
- Gabrielsson et al, 2009
Reflection “enable[s] people to more systematically analyze their developmental experiences, identify behavior changes that would improve performance, and regulate emotions that might interfere with experiential learning.”
- AACSB AOL guidelines
Definitions Why AoL? Standards Panel
Reflection is most critical factor for learning from experience
67©2019 Jester, Laufer & McKeen - Do not reproduce without permission
14
16
26
27
28
33
37
37
39
41
48
40
27
27
23
20
11
15
13
8
INDIVIDUAL LEARNING OBJECTS ARE ESTABLISHED …
MID-COURSE ASSESSMENT(S) TIED TO INDIVIDUAL …
MID-COURSE ASSESSMENT(S) TIED TO TEAM …
FINAL ASSESSMENT TIED TO INDIVIDUAL LEARNING …
INDIVIDUALS RECEIVE FEEDBACK MID -COURSE …
INDIVIDUALS RECEIVE FEEDBACK AT THE END OF …
TEAM RECEIVES FEEDBACK MID -COURSE FROM …
FINAL ASSESSMENT TIED TO TEAM LEARNING …
LEARNING OBJECTIVES ARE ESTABLISHED AND …
TEAM RECEIVES FEEDBACK AT THE END OF THE …
Formal part of program Not formally part of program
Source: LEPE+ Survey 2017; n=62
Feedback from different sources collected and sharedmid-course and at conclusion is common
Definitions Why AoL? Standards Panel
68©2019 Jester, Laufer & McKeen - Do not reproduce without permission
• Constructive, actionable, and timely peer and client feedback is valued by students
• Feedback should be continuous, non-intrusive, and integrated into instruction
• Both individual and team performance and outcomes must be assessed
More emphasis needed on feedback qualityand frequency vs. compliance/completion
Definitions Why AoL? Standards Panel
8%
19%
9%
22%25%
< = 4 to 1 5 or 6 to 1 7 or 8 to 1 Btwn 9 &14 to 1
More than15 to 1
Typical Student/Faculty Ratio
69©2019 Jester, Laufer & McKeen - Do not reproduce without permission
• Experiential learning pushes students out of comfort zones and learning can be rooted in emotions
• Frustration, anger, resentment or joy, relief, fulfillment can be mined to catalyze learning
Perhaps the most common expression of strong emotions…occurs around areas of conflict…Differences among students regarding how to proceed with a group project, often lead to feelings of anger or frustration.
-Dirkx, 2008
…the experience gained from action, is, to a large extent, embedded in emotion both positive and negative.
- Gibson, 2006
…far more neural fibers project from the brain’s emotional center into the logical/rational centers than the reverse, so emotion is often a more powerful determinant of our behavior than our brain’s logical/rational process.
-Sylvester, 1994
No two adults perceive the the world in the same way. And adults decide what occurs for them in a learning event.
-Vella, 2002
Emotions can facilitate orimpede learning and should be leveraged
Definitions Why AoL? Standards Panel