ATD_June2016_How-to-Measure-Application


Leading Indicators

of Success:

Application Rate

Scrap Rate

Use

Activity

Implementation

1. Efficiencies

2. Effectiveness

3. Outcomes

How many of you are measuring…


DRIVER LEVEL ACTIONABLE DATA POINTS

What the CFO wants to know:

5: ROI: What’s the ROI? (Net Benefits ÷ Costs)*100

What the Executive Stakeholders want to know:

4: Impact: Did it improve the bottom line? What are the benefits?

Absenteeism, productivity, quality, sales, time, turnover, efficiency, input/output, penalties/fines, cycle times, downtime, process time, waste, fraud, theft, market share, new accounts, grievances, engagement, churn rate, accidents, complaints, defects, rework, etc.

What the functional leader wants to know:

3: Application: Did they apply it?Scrap learning, Application, Implementation, Use, Increase/Decrease in Activity, etc.

What the training team commonly measures:

2: Learning: Did they learn something?Understanding, Awareness, Knowledge, Skills, Readiness, Confidence, Intent to Use, etc.

1: Reaction: Did they like it? Satisfaction, Relevancy, Perceived importance, etc.

What the LMS captures: 0: Inputs: Did they take it?Utilization, Completions, Efficiencies, Awards, Volume, Costs, Scalability, etc.

Effic

ien

cy

Effe

ctiv

en

es

sO

utc

om

es


90-100% of programs

• System reports

80% of programs

• Evals• Surveys• Tests

30% of programs• Questionnaires• Observation• Projects• Action Plans• Performance Plans


Elapsed Days Since Event

100

90

80

70

60

50

40

30

20

10

Immediate Recall

20 min

1 hour

9 hours

2 days 6 days

31 days

Without context, 50% is

forgotten in one hour

Source: Herman Ebbinghaus, Memory:

Contribution to Experimental Psychology,

1885/1913

Retention (%)

80% is forgotten

after 2 days 90% is forgotten

after 31 days


70

60

50

40

30

20

10

90

100

80

%

1 2 3 54 6

Source: Creative Training Techniques

Without support, learner retention

can drop to 10% in 30 days

Learning activities occurring

6 times, can raise retention

to 90% in 30 days

Single

Event

Learning activity

14

1 4 7 1

0

1

3

1

6

1

9

2

2

2

5

3

0

Day


Manager Involvement

Pre-Work

Event

Post-Work

OJT Support Resources

Manager Support

Application Period

Collect Data


Factors to Consider:

•

•

•

•

•

•

•

•

•

•

•

Data Collection Tools:

•

•

•

•

•

•

•


90-100% of programs

• System reports

80% of programs

• Evals• Surveys• Tests

30% of programs• Questionnaires• Observation• Projects• Action Plans• Performance Plans


•••••••

•

••


60%

50%

40%

30%

20%

10%

0%

20

• A majority (86%) of Skillsoft learners were able to apply what they learned within 6 weeks of the training. Skillsoft’s scrap learning rate was 14% - far below the norm of 55% to 80%.

I was able to apply my training within…

1 Week

50.2%

2-4 Weeks

24.8%

5-6 Weeks

11.3%

I haven’t applied

what I learned

yet, but plan to in

the future.

11.8%

I don’t expect to

use the

knowledge/skills

gained

1.9%

Source: “Moving the Needle: How Skillsoft Impacts Performance of Individuals and Organizations”


Annually, for your organization of 1,000 employees:

= $286,800 of scrap24% scrap3 X $1,195 spend per learner2

X1,000 employees

= $9m value of improvement15% performance improvement3 X

$60,000 average salary X

1,000 employees

= $537,750 of scrap45% scrap1 X $1,195 spend per learner2

X1,000 employees

= 3.6m value of improvement6% performance improvement3 X $60,000

average salary X

1,000 employees

$250,950in Savings

$5.4 millionContribution to Performance

ImprovementThe value of your continuous improvement efforts to your

organization is $5.65 million annually1CEB 2014 Training Effectiveness Dashboard2ASTD 2013 State of the Industry Report3KnowledgeAdvisors Analysis 2014

1. Your L&D team

2. Your functional business leaders / program

stakeholders

3. Your Executive team

How many of you are reporting results to…


1

2

3

4

5

SILOED Learning and talent are disparate processes Culture is hierarchical and not very engaging

HR tends to react to business needs and enforce compliance

TARGETED Pathways between learning and talent begin More executive interest in employee engagement emerges

HR is pursuing more proactive business alignment

TRANSFORMING Learning and talent start to converge with better tools A more people-centric philosophy starts to get established

Employees are more accountable for steering their development & career

CONTINUOUS Engaged employee culture with rapid response to change HR is a workforce architect of advanced strategies and tools

Compelling and integrated learning and talent UX drives high participation

OPTIMIZED Self-developing ecosystem; Highly sought after culture Sophisticated strategies fuel business adaptation

World-class personalized UX heightens engagement

As an organization evolves through the framework, the HR strategy becomes more adaptive and the workforce becomes more agile, resulting in stronger business performance.


ROI Institute

Buy Here

The State of Learning Measurement, Bersin Research Report, July 2015

A Guide to Establishing Value-Based Measures of Success, Skillsoft

OnDemand Webinar, 2013

Center for Talent Reporting – TDRp

ROI Institute

The Relationship Between Learning, Workforce Agility, and Business

Performance, Skillsoft Whitepaper

Challenge the Status Quo with a Smarter Approach to the Global Talent

Crisis, Skillsoft Whitepaper

Skillsoft Learning & Talent Maturity Framework, Skillsoft.com

ROI Institute

Buy Here

http://www.roiinstitute.net/product/10-steps-to-successful-business-alignment/

http://www.bersin.com/library

http://learn.skillsoft.com/index.php/email/emailWebview?mkt_tok=3RkMMJWWfF9wsRonuanNdu/hmjTEU5z16uUvWKW2g4kz2EFye+LIHETpodcMTcZjNbjYDBceEJhqyQJxPr3GK9YN38B4RhnmDA==

http://www.centerfortalentreporting.org/

http://www.roiinstitute.net/

http://learn.skillsoft.com/Website-WP-LearningAgilityandPerformance_RegistrationPage.html

http://learn.skillsoft.com/Website-WP-StatusQuo_RegistrationPage.html

http://learn.skillsoft.com/Website-EVG-WP-TheSkillsoftLearningandTalentMaturityFramework_RegistrationPage.html

http://www.roiinstitute.net/roi-resource-center/?add-to-cart=2855

linkedin.com/company/skillsoft

facebook.com/skillsoft

twitter.com/skillsoft

skillsoft.com

866-757-3177

ATD_June2016_How-to-Measure-Application

Documents

Transcript of ATD_June2016_How-to-Measure-Application