Creating a Measurement Strategy to Tell Your Value Story
Transcript of Creating a Measurement Strategy to Tell Your Value Story
World’s largest provider of Journey Analytics solutions
Creating a Measurement Strategy to
Tell Your Value Story
November 2021Jennifer Balcom
Director of Consulting
Lan Tran
Head of Learning & Development Governance, Technology & Operations
2
Head of L&D
Governance, Technology, and Operations
Hi!
I’m Lan!
7
11
Data Storytelling = Visualization + Narrative + Context
Data Visualization = Data Storytelling
“No one ever made a decision because of a number. They need a story.”
- Daniel Kahneman, November 2016
Telling Your Value Story
0
100000
200000
300000
400000
500000
600000
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Learning Hours by Month2019 2020
2019
36Avg Learning Hours / Employee
2020
14Avg Learning Hours / Employee
2019
36Avg Learning Hours / Employee
2020
14Avg Learning Hours / Employee
2019
36Avg Learning Hours / Employee
3 hoursAvg length of course taken by
employees
12%Self-Registrations
2020
14Avg Learning Hours / Employee
1 hourAvg length of course taken by
employees
19%Self-Registrations
The outcome value story …
18
19
Challenges Benefits
Learning is rarely measured at the desired
levels to show value and alignment to
business objectives.
Learning often lacks alignment to the
business and resulting in low application on
the job.
Learning teams aren’t using data to make
decisions.
Defines true Key Performance Indicators
linked to learning effectiveness, efficiency,
value, outcomes and impact.
Identifies where learning is providing the
most value and application in the business.
Outlines roles, plans, and processes for
bringing a strategy to life through reporting,
communication, and change management.
The Need for a Measurement Strategy
Business Objective
Drive bottom-line
profitability
Drive top-line revenue
growth
Cost avoidance:
Minimize costs due to
non-compliance
Cost avoidance:
Minimize talent
acquisition/turnover
cost: “Build vs. Buy”
Leading Indicators
Productivity
Cycle Time
Cost
Scrap
Customer Satisfaction
Sales Performance
Scrap
Safety
Risk Avoidance
Manager Support
Job Application
Manager Support
Learning Effectiveness
Quality
Lagging Indicators
Profit
EBITDA
Revenue
Market Share
Legal Costs
Reputational Damage
Bench Strength
Organizational Flexibility
L&D Value Driver
Maximize Operational Efficiency
Drive Growth
Build / Maintain Foundational
Skills
Mitigate Risk$
How Does L&D Create Value? A Framework for Connecting to Business Outcomes
20
Three Areas You Should be Measuring
Effectiveness
Indicators of impact to performance and business value
▪ Learning effectiveness
▪ Content effectiveness
▪ Instructor performance
▪ Knowledge gain
▪ Application to job
▪ Business Impact
▪ Value / Net Promoter Score
▪ Time to apply on the job
Outcomes
Indicators of impact to outcomes and business results
▪ Employee turnover / retention
▪ Employee engagement
▪ Customer satisfaction
▪ Productivity gains
▪ Business outcomes impacted
▪ Critical skill gains - % of employees
▪ Career movement rate (promotion or moved
into new role with critical skill)
Efficiency & Adoption
Indicators of usage and reach across the organization
▪ Unique enrollments
▪ Average # of hours per employee
▪ Cost per employee
▪ Reach and consumption demographics
▪ Access data
▪ Top course consumption
▪ Voluntary usage
Presentation Title Subhead Can Be Placed Here
Measurement Strategy Framework
1. Establish scalable framework
2. Monitor & manage learning assets
3. Maximize L&D investments
Me
asu
rem
en
t M
atu
rity
Time
4. Demonstrate value
5. Align & partner with business
Tactical, measurement
decisions
Strategy,program
decisions
Impact to business goals
Run L&D like a business
1
2
3
4
5
Adapted from Carnegie-Mellon Capability Maturity Model The aspiration
No consistent process,Ad-hoc methods
Short Term Medium Term Long Term
Key Considerations Leadership, Governance, Culture Roles, Accountabilities, Skills Processes, Results Distribution◆ Standards, Tools, Technology
Presentation Title Subhead Can Be Placed Here
Dimensions for a Sustainable Measurement Strategy
1. Leadership: create
accountability
2. Process: integrate
measurement into work
3. Roles: establish
accountabilities and
performance goals
4. Skills / Behaviors: build
organizational muscle in
measurement and evaluation
5. Standards: establish consistent methods and measures
to streamline data collecting and reporting
7. Results Utilization:
ensure data outputs inform
decisions and actions
6. Technology: enable
the process and minimize
non-value-added work
8. Culture: establish norms
and behaviors that value
and use data
Leadership
Process
Roles
Skills
Standards
Technology
Results
Utilization
Presentation Title Subhead Can Be Placed Here
Dimensions for a Sustainable Measurement Strategy
Dimension Elements Dimension Elements
Leadership • Philosophy and strategy• Governance• Business alignment• Measurement priorities
Standards • Reporting templates• Benchmarks• Performance goals• Decision support tools
Roles • Accountability for using data for decision making has been clearly defined for all key roles Instructors, Designers, Curriculum Managers & Delivery Managers
• Business Stakeholders identified and educated
Technology • Optimized technology for:• Data collection• Analytics• Reporting • Data vizualization• Data integration
Skills / Competencies
• Analytics and interpretation• Consultation skills• Business acumen• Data visualization• Data storytelling
Results Utilization
• Improve business alignment• Enhance or retire courses• Develop instructors• Tell the value story
Processes • Front end analysis• Integrated and consistent
processes• Business rules
Culture • Data informed decisions• Success Recognition• Continuous Improvement• Funded evaluation activities
• Establish Ownership & Accountability
• Ensure High Quality Measurement Practices
• Streamline Processes And Decision Making• When to say “Yes,” “No,” or “Yes, and. . .”
• Enable Consistent Data Collection, Reporting, and Analysis
Without Continually Reinventing The Wheel
The Role of Governance
Ownerversity Measurement StrategyPLAYBOOK
TABLE OF CONTENTS
OVERVIEW….………………………………………………………………………………………………………………………………………………………………..TBD
RITUALS & ROUTINES…………………………………………………………………………………………………….……………………………………………..TBD
CHECKLISTS.…………………………………………………………………………………………………………………..……………………………………………..TBD
TEMPLATES..…………………………………………………………………………………………………………………..………………………………..…………..TBD
STANDARD EVALUATION SURVEYS………………………………………………………………………………………………………………………………..TBD
APPENDIX: DETAILED LEARNING ANALYTICS………………………………………………………………………………………………………………….TBD
Overview
Define the strategy for measuring and reporting content effectiveness and business impact
© 2020 The Kraft Heinz Company
Learning Measurement Opportunities*29
End of 2021
We are here
❖ Image from Explorance
To determine content effectiveness and drive business partnership through actionable analytics
© 2020 The Kraft Heinz Company
Kraft Heinz Measurement Strategy – V130
Demonstrate Business
Value Internally & Externally
Strengthen Business Alignment & Data Driven
Decisions
Act Like an Owner to Continuously Improve
Data Gathering & Reporting
Analytics & ImpactOutcome: Stories &
Actions
© 2020 The Kraft Heinz Company
Actionable Analytics Lead to Proactive Insights 31
Hindsight
Identify trends
Insight
Determine improvements
Foresight
Lead change
Increased engagement
Elevated performance
Improved ROI
ACTION CONTRIBUTORS
Provide data from programs All who take part in the ritual and routine
Analyze the data and create draft full report and executive summary
GTO Learning Analyst and Head of GTO
Review executive summary and full report to provide feedback
Global Learning & Development Academies, DIBs and Leadership, Chief Learning & Diversity Officer
Deliver the executive summary and present the full reportGTO Learning Analyst, Head of GTO and Chief Learning & Diversity Officer
Global Learning & Development Annual Impact Report
Ritual Objective: To communicate team accomplishments for the prior year, measure progress and business impact of learning programs, and lay out future team strategy
Frequency: Annually
When: Targeted for Q2
Ritual Length: Data gathering and analysis starts in January – April, drafts in May, reviews in June
Ritual Owner: GTO Learning Analytics
Tools/Templates: Standard Annual Report Template
Who takes part: GTO Learning AnalystHead of GTOWorkday teamPeople Analytics DIBs, Leadership and Functional AcademiesChief Learning & Diversity Officer
Data from: Google Analytics, Ownerversity, Engagement Survey, Promotion, KPI, Specific Programs (Masters, WE Lead, DIBs, Functional Academies, Ownerversity Day), Global Learning and Development Budget & Spend, Performance data (Owner 360), #LearnLikeAnOwner, industry comparison data
INPUTS
ACTIVITIES
Executive Summary delivered. Approved and published Global Learning & Development Annual Impact Report
OUTPUTS
33• Refine our Learning Measurement
Effectiveness and Analytics Strategy
• Roll out our follow-up evaluation strategy for
key learning programs
• Enhance our reporting related to functional
capabilities and diversity, inclusion, and
belonging learning experiences and their
impact
• Complete the connection from our LMS to the
Global People Analytics Data Lake
• Collaborate with our Global People Analytics
team on showing the impact learning has on
turnover, retention, promotions, sales, KPIs,
etc.
• Enhance our annual Learning Impact Report
to share all the info on how we are helping to
improve the employee experience and
positively impacting our business
What’s next?
Thank You
F O L L O W U S
www.explorance.com