Measuring roi of training
-
Upload
preston-union -
Category
Business
-
view
79 -
download
0
description
Transcript of Measuring roi of training
Measuring Measuring ROI of ROI of
TrainingTraining
AgendaAgenda
1. Measuring the Effectiveness of Training
Program
2. Measuring ‘Return on Investment’ of Training
3. Enhancing the Effectiveness and ROI of
Training
A.Measuring the Effectiveness A.Measuring the Effectiveness of Training Programof Training Program
The Four Levels of EvaluationThe Four Levels of Evaluation
Level 1 - Reaction
Level 2 - Learning
Level 3 – Behavior Application
Level 4 – Performance Impact
Four Levels Four Levels of Training of Training
EffectivenessEffectiveness
Level 1 - ReactionLevel 1 - Reaction
Evaluate trainees’
reactions to the program:
Did they like the program?
Did they like the
facilitators?
Did they like the training
accommodation and
facilities?
Level 1 - Level 1 - ReactionReaction
Guidelines for Evaluating ReactionGuidelines for Evaluating Reaction
1. Determine what you want to find out
2. Design a form that will quantify reactions
3. Encourage written comments and suggestions
4. Get 100 percent immediate response
5. Get honest response
6. Develop acceptable standards
7. Measure reactions against standards, and take
appropriate action
8. Communicate reaction as appropriate
Level 2 - LearningLevel 2 - Learning
Measuring learning means
determining one or more of
the following :
• What knowledge was
learned?
• What skills were
developed or improved?
• What attitudes were
changed?
Level 2 - Level 2 - LearningLearning
Guidelines for Evaluating LearningGuidelines for Evaluating Learning
1. Use a control group if practical
2. Evaluate knowledge, skills and/or attitudes both
before and after the program
3. Use a paper-and-pencil test to measure knowledge
4. Use a performance test to measure skills
5. Get 100 percent immediate response
6. Use the results of evaluation to take appropriate
action
Guidelines for Evaluating LearningGuidelines for Evaluating Learning
Experimental Group Control Group
Pre Test Score 45 46
Post Test Score 55 48
Gain 10 2
Pretest and Posttest Scores on Change Management Training
Example :
Level 3 – Behavior ApplicationLevel 3 – Behavior Application
• The frequency of
application of new
skills/knowledge/
attitudes (on the job)
• The effectiveness of the
skills/knowledge/
attitudes (as applied on
the job)
Level 3 – Level 3 – Behavior Behavior ApplicationApplication
Guidelines for Evaluating LearningGuidelines for Evaluating Learning
1. Use a control group if practical
2. Allow time for behavior change and application to take
place
3. Evaluate both before and after the program if practical
4. Survey and/or interview one or more of the following :
trainees, their immediate supervisor, their subordinates,
and others who often observe their behavior
5. Get 100 percent response or a sampling
6. Repeat the evaluation at appropriate times
7. Consider cost versus benefits
Level 4 – Performance ResultsLevel 4 – Performance Results
• Indicate the extent to which you
think this program has
influenced each of these
measures in your work unit,
department, or business unit:• Productivity• Quality• Customer Response Time• Cost Control• Employee Satisfaction• Customer Satisfaction• Other
Level 4 – Level 4 – PerformancePerformance ResultsResults
Guidelines for Evaluating LearningGuidelines for Evaluating Learning
1. Use a control group if practical
2. Allow time for results to be achieved
3. Measure both before and after the program if practical
4. Repeat the evaluation at appropriate times
5. Consider cost versus benefits
6. Be satisfied with evidence if proof is not possible
Performance IndicatorsPerformance Indicators
HARD DATA INDICATORS
• Duration involved
• Number of defect products
• Sales volume
• Customer satisfaction index
• Response time to orders
• Others
Performance IndicatorsPerformance Indicators
SOFT DATA INDICATORS (intangible impacts)
• Job satisfaction
• Conducive working relationship
• Effective communication
• Stress rate
• Quality in decision-making
Example : Measuring Training ResultsExample : Measuring Training Results
0.00
50.00
100.00
150.00
200.00Program : TQM TQM
TrainingTraining
Results after 3 3
months of months of
training,training, number
of defects
dropped to 80
units/day
120 units
80 units
Before trainingBefore training After trainingAfter training
0.00
10.00
20.00
30.00
40.00
50.00
Example : Measuring Training ResultsExample : Measuring Training Results
Program :
Sales TrainingSales Training
Results after 3 3
months trainingmonths training,
number of sales
per salesman
increase to 30
units/month.
20 units
30 units
Before trainingBefore training After trainingAfter training
B.Measuring B.Measuring Return on Investment of Return on Investment of
TrainingTraining
Benefits of ROI of TrainingBenefits of ROI of Training
• Measure contribution
• Set priorities
• Focus on results
• Alter management perceptions of training
ROI of Training Model ROI of Training Model
Collect Collect DataData
Isolate the Isolate the Effects of Effects of TrainingTraining
Convert Data Convert Data to Monetary to Monetary
ValuesValues
Calculate ROI Calculate ROI of Trainingof Training
Tabulate Tabulate Program Program
CostsCosts
Identify Identify Intangible Intangible BenefitsBenefits
Net Program BenefitsNet Program Benefits
Program CostsProgram CostsX 100X 100
Return on Investment FormulaReturn on Investment Formula
ROI =ROI =
ExampleExample : :
• Costs per program (25 participants)Costs per program (25 participants) RsRs
88,50088,500
• Benefits per program (1st year)Benefits per program (1st year) Rs Rs 230,625230,625
Rs 230,625 – 88,500Rs 230,625 – 88,500
Rs 88,500Rs 88,500ROI =ROI = X 100X 100
ROI = 161 %ROI = 161 %
Collecting DataCollecting Data
• Identify appropriate
performance indicators
• Develop a collection plan
Collect Collect DataData
Example of Performance IndicatorsExample of Performance Indicators
• Units produced• Items sold• Work backlog• New accounts opened• Productivity• Inventory turnover• Etc.
OutputOutput
• Equipment downtime• Overtime• Time to project
completion• Processing time• Repair time• Lost time days• Etc
TimeTime
Example of Hard IndicatorsExample of Hard Indicators
• Unit costs• Variable costs• Overhead costs• Operating costs• Number of cost
reduction• Etc.
CostCost
• Scrap• Waste• Rejects• Error rates• Rework• Product defects• Product failure
QualityQuality
Example of Performance ResultsExample of Performance Results
• Some performance results after training Some performance results after training
program:program:
• Rejections were reduced from 11 % to 7.4 %
• Absenteeism was reduced from 7 % to 3.25 %
• The annual turnover rate was reduced from 30
% to 16 %
• Customer complaints were reduced by 65 %
Isolating the Effects of Training Isolating the Effects of Training
Methods to Methods to Isolate the Isolate the Effects of Effects of TrainingTraining
Using Using Control GroupControl Group
Trend Trend LinesLines
Participants Participants EstimateEstimate
Supervisors of Supervisors of Participants Participants
EstimateEstimate
Using Using Control Control GroupGroup
• A control group arrangement can be
used to isolate training impact.
• With this strategy, one group
receives training, while another,
similar group does not receive
training.
• The difference in the performance of
the two groups is attributed to the
training program.
Isolating the Effects of Training Isolating the Effects of Training
Trend Trend LinesLines
• Trend lines are used to project the
values of specific output variables if
training had not been undertaken.
• The projection is compared to the
actual data after training, and the
difference represents the estimate
impact of training.
Isolating the Effects of Training Isolating the Effects of Training
Trend Lines AnalysisTrend Lines Analysis
Jan Feb Mar Apr Jul Jun May Aug
At the beginning of May, a Sales training Program session was
held
Vo
lum
e o
f S
ales
The
difference
represents
the estimate
impact of
training.
Trend Projection
Actual sales performance
Participants and Participants and Supervisors of Supervisors of
Participants Participants Estimate of Estimate of
Training’s ImpactTraining’s Impact
• This method rests on the assumption that
participants (and their supervisors) are
capable of estimating how much a
performance improvement is related to the
training program.
• Because their actions have produced the
improvement, participants (and their
supervisors) may have very accurate input
on the issue.
• They should know how much of the
change was caused by applying what they
have learned in the program.
Isolating the Effects of Training Isolating the Effects of Training
Participants and Participants and Supervisors of Supervisors of
Participants Participants Estimate of Estimate of
Training’s ImpactTraining’s Impact
• Typical Questions to Estimate :Typical Questions to Estimate :
• What percent this improvement
can be attributed to the application
of skills/techniques/knowledge
gained in the training program?
• What confidence do you have in
this estimate, expresses as a
percent?
• What other factors contributed to
this improvement in performance?
Isolating the Effects of Training Isolating the Effects of Training
1 Training Program 50% 70%
2 Change in Procedures 10% 80%
3 Adjustment in Standards 10% 50%
4 Revision to Incentive Plan 20% 90%
5 Increased Management Attention 10% 50%
6 Other - -
Percent Improvement
Caused by
Confidence Expressed as
a PercentFactors Which Influenced
ImprovementNo.
Isolating the Effects of Training Isolating the Effects of Training
Example of a Participant’s EstimationExample of a Participant’s Estimation
The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training a usable training factor value of 35 %factor value of 35 %
1 Training Program 50% 70%
2 Change in Procedures 10% 80%
3 Adjustment in Standards 10% 50%
4 Revision to Incentive Plan 20% 90%
5 Increased Management Attention 10% 50%
6 Other - -
Percent Improvement
Caused by
Confidence Expressed as
a PercentFactors Which Influenced
ImprovementNo.
Isolating the Effects of Training Isolating the Effects of Training
Example of a Participant’s EstimationExample of a Participant’s Estimation
• The confidence percentage is multiplied by the estimate (50 % x 70 %) to
produce a usable training factor value of 35 %• This adjusted percentage is then multiplied by the actual amount of
improvement (post-program minus pre-program value) to isolate the portion
attributed to training• The adjusted improvement is now ready for conversion to monetary values,
and used in the return on investment
Converting Data to Monetary ValuesConverting Data to Monetary Values
Converting Converting Data to Data to
Monetary Monetary ValuesValues
• Steps to Convert Data to Monetary Steps to Convert Data to Monetary
ValuesValues
1. Focus on a unit of improvement
2. Determine a value of each unit
3. Calculate the change in performance
data
4. Determine an annual amount of change
5. Calculate the annual value of
improvement
Steps to Convert Data to Monetary ValuesSteps to Convert Data to Monetary Values
An example to illustrate the steps to convert data to monetary values
1. Focus on unit improvement
• One grievance reaching step two in the
four-step grievance resolution process
StepsSteps IllustrationIllustration
2. Determine a value of each
unit
• Using internal experts, the cost of an
average grievance was estimated to be
Rs,500 when considering time and direct
costs
Steps to Convert Data to Monetary ValuesSteps to Convert Data to Monetary Values
3. Calculate the change in
performance data
• Six months after the program was completed,
total grievances per month reaching step two
declined by ten. • Seven of the then grievance reductions were
related to the program as determined by
supervisors (isolating the effects of training)
StepsSteps IllustrationIllustration
4. Determine an annual amount for the change
• Using the six month value, seven per month
(grievance reductions), yields an annual
improvement of 84 (7 x 12 months)
Steps to Convert Data to Monetary ValuesSteps to Convert Data to Monetary Values
5. Calculate the annual value of improvement
• Annual value = 84 x Rs500 = Rs42,000
StepsSteps IllustrationIllustration
Tabulating Tabulating cost of the cost of the programprogram
• Cost components that should be
included are :
• The cost to design and develop
the program
• The cost of all program materials
provided to each participant
• The cost for facilitator
• The cost of the facilities of the
training program
Tabulating Cost of the ProgramTabulating Cost of the Program
Tabulating Tabulating cost of the cost of the programprogram
• Cost components that should be
included are :
• Travel, lodging, and meal costs
for the participants
• Salaries, plus employee benefits
of the participants who attend
the training
• Administrative and overhead
costs of the training function,
allocated in some convenient
way
Tabulating Cost of the ProgramTabulating Cost of the Program
C.Enhancing Training C.Enhancing Training EffectivenessEffectiveness
How Effective is Your Training Program?How Effective is Your Training Program?
Broad and Newstrom (1992) report studies have
shown less than 30% of what is30% of what is actually taught actually taught
transfers to the job in a way that enhances transfers to the job in a way that enhances
performanceperformance.
Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packedstrategies to ensure high payoff from training investments. Reading, MA: Addison-Wesley.
Source of Barriers to TrainingSource of Barriers to Training Transfer Transfer
• Lack of reinforcement on the job
• Interference from immediate (work) environment
• Nonsupportive organizational culture
• Trainees’ perception of impractical training programs
• Separation from the inspiration or support of the trainer
The Transfer PartnershipThe Transfer Partnership
Trainee Trainee recognizes need for new skills
Trainee Trainer Trainer designs and/or delivers learning experiences
Trainee Trainer Manager supports learning and application on the job
Manager
ManagerManager
After TrainingAfter Training
• Plan trainees’ reentry
• Provide opportunities to practice new skills
• Have trainees participate in transfer-related
decisions
• Reduce job pressures initially
• Give positive reinforcement
• Schedule trainee briefings for co-workers
• Set mutual expectations for improvement
• Arrange proactive (refresher) sessions
• Provide and support the use of job aids
TrainerTrainer
After TrainingAfter Training
• Provide follow-up support
• Conduct evaluation surveys and provide
feedback
• Develop and administer recognition system
• Provide refresher/problem-solving sessions
Trainee Trainee
After TrainingAfter Training
• Practice self-management
• Review training content and learned skills
• Develop a mentoring relationship
• Maintain contact with training ‘buddies’
THANK YOUTHANK YOU