Measuring ROI of Training

“Not everything that can be counted counts, and not everything that

counts can be counted”Albert Einstein

Best Practice in Terms of ROI• Training magazine’s 2003 top 100 companies found that linking training to specific business metrics requires evaluation before, during, and after training initiatives;

• Many of the Top 100 companies ranked by Training magazine tie training initiatives directly to retention efforts;

• Also use other metrics – sales; production etc;

• ROI used as basis for continued improvement of training function;

• Forecast ROI.

The Case for ROI ?AccentureAllstate InsuranceApple ComputerAT&TBristol-Myers SquibbBritish TelecomCaremarkComcastCoors Brewing Co.Deloitte & TouchéDelta AirlinesDHL Worldwide ExpressFederal ExpressGeneral Motors

US Department of DefenseUS Department of LaborUS National Security AgencyUS Department of Veteran AffairsUS General Accountability OfficeUS Office of Personnel ManagementNASA

The Value Hierarchy

Paradigm Shift

ROI -Delivery on the Value Promise

ROI -Delivery on the Value Promise

The promise is in the

way we determine

training needs

The promise is in the

way we determine

training needs

Moving from input (activity) to output (results) based

training

Moving from input (activity) to output (results) based

training

A Quick Diagnostic

How does your company determine

training needs?

Question – we develop and implement training that

Absolutely Un True= 1 ; Absolutely True = 5

Link needs to business solutions 1 2 3 4 5

Based on assessment of current vs. desired performance

1 2 3 4 5

Isolate application and business impact objectives

1 2 3 4 5

Ensures communication of expectations to stakeholders and participants

1 2 3 4 5

Ensures the work environment supports reinforcement of implementation and application

1 2 3 4 5

Builds partnerships with managers 1 2 3 4 5

Measures at more than reaction and learning level

1 2 3 4 5

Communicates the results to all stakeholders

1 2 3 4 5

How Does Your Organization Score

8-15 Activity based

Little Focus on Results

24-31Significant progress

To Results Based

16-23Moderate shift

Or Shift in Certain Areas

32- 40Fully Integrated Results

Based System

Which Individual Areas Do You Need to Address?

Activity Results

No established business need Business case

Little relation to performance Focus on improving performance/change behavior

No/little focus on objectives/competencies

Demonstrating competence over time

No/Little focus on preparing the environment

Prepare environment to ensure transfer and reinforcement

No/Little focus on measuring impact and value

System geared to preparing and proving the case

Mainly input focused Sound balance between input and output

We need to move towards a results

based approach

But what should we put in place?

What Should Be In Place?

Lead project

Evaluation Approach

1. Reaction, Satisfaction

2. Learning3.

Transfer/application

4. Business Impact

5. Return on Investment

6. Intangibles

Element Description

Reaction Happiness with learning interventions

Learning Changes in knowledge, skills and attitudes

Transfer/application Changes in on the job behaviors over time (competence)

Business Impact Changes in impact variables

ROI Intervention benefits in relation to cost

Intangibles Intervention benefits in relation to policy and/or strategic direction and related factors

Evaluation Approach

Evaluation Approach

Customer

Focus

Data

Availability

Value of Case

(relative)

Intangibles

ROI

Business

Impact

Application

and

implementati

on

Learning

Reaction

Element Assessment

Complexity

Limited/Low

Wide/High

Consumer

Client

Readily

Research

Limited/Low

Wide/Complex

Administrator

Business Partner

Intervention Planning Framework

Reaction Evaluation

General need

General need

Learning OutcomesLearning

Outcomes

Learning Material

Achieved?

Assess

More

First Order Analysis and Definition of

Business Need

Identification of Performance

Need

Business Context(Internal and

External)

Competence GapSolution

GenerationIntervention

Strategy

Presenting Problem/

Opportunity

Non Training Solution

Program Design Event PlanningEvent

ManagementEvent Outcomes

Intervention

Planning Framewor

k

TO BE

Data Gathering Approach

Process Framework

Isolation Framework

External Factors

ManagementAttention

Incentives

Systems/ProceduresChanges

LearningProgram

TOTALIMPROVEMENT

AFTERPROGRAM

EFFECT OFLEARNING ON IMPROVEMENT

Calculation

Benefit/Cost Ratio =

BenefitsROI =

Program Benefits

Program Costs

Net Program Benefit

Program CostsX 100

1