Post on 06-Apr-2018
8/2/2019 Measuring Training
1/49
1VRAJEEV
MeasuringROI of
Training
8/2/2019 Measuring Training
2/49
2VRAJEEV
Agenda
1. Measuring the Effectiveness of TrainingProgram
2. Measuring Return on Investment of Training
3. Enhancing the Effectiveness and ROI of
Training
8/2/2019 Measuring Training
3/49
3VRAJEEV
A.Measuring the Effectivenessof Training Program
8/2/2019 Measuring Training
4/49
4VRAJEEV
The Four Levels of Evaluation
Level 1 - Reaction
Level 2 - Learning
Level 3 Behavior
ApplicationLevel 4 Performance Impact
Four Levelsof Training
Effectiveness
8/2/2019 Measuring Training
5/49
5VRAJEEV
Level 1 - Reaction
Evaluate trainees
reactions to the program:
Did they like the program?Did they like the
facilitators?
Did they like the training
accommodation and
facilities?
Level 1 -Reaction
8/2/2019 Measuring Training
6/49
6VRAJEEV
Guidelines for Evaluating Reaction
1. Determine what you want to find out
2. Design a form that will quantify reactions
3. Encourage written comments and suggestions
4. Get 100 percent immediate response
5. Get honest response
6. Develop acceptable standards
7. Measure reactions against standards, and takeappropriate action
8. Communicate reaction as appropriate
8/2/2019 Measuring Training
7/497VRAJEEV
Please give us your frank reactions and comments. They will help us to evaluate
this program and improve future programs.
Program :
Facilitator :
1. How do you rate the subject (interest, benefit, etc)
a. Excellent
b. Very Good
c. Good
d. Fair
e. Poor
2. How do you rate the facilitator? (knowledge, ability to deliver and communicate?)
a. Excellent
b. Very Goodc. Good
d. Fair
e. Poor
3. How do you rate the facilities? (comfort, convinience, etc)
a. Excellent
b. Very Good
c. Goodd. Fair
e. Poor
4. What would have improved the program?
Sample ofReaction
Form
8/2/2019 Measuring Training
8/498VRAJEEV
Level 2 - Learning
Measuring learning means
determining one or more of
the following :
What knowledge was
learned?
What skills were
developed or improved? What attitudes were
changed?
Level 2 -Learning
8/2/2019 Measuring Training
9/499VRAJEEV
Guidelines for Evaluating Learning
1. Use a control group if practical
2. Evaluate knowledge, skills and/or attitudes bothbefore and after the program
3. Use a paper-and-pencil test to measure knowledge
4. Use a performance test to measure skills
5. Get 100 percent immediate response
6. Use the results of evaluation to take appropriateaction
8/2/2019 Measuring Training
10/4910VRAJEEV
Guidelines for Evaluating Learning
Experimental Group Control Group
Pre Test Score 45 46Post Test Score 55 48
Gain 10 2
Pretest and Posttest Scores on Change Management TrainingExample :
8/2/2019 Measuring Training
11/4911VRAJEEV
Level 3 Behavior Application
The frequency ofapplication of newskills/knowledge/ attitudes (on the job)
The effectiveness of theskills/knowledge/ attitudes (as applied onthe job)
Level 3 BehaviorApplication
8/2/2019 Measuring Training
12/4912VRAJEEV
Guidelines for Evaluating Learning
1. Use a control group if practical
2. Allow time for behavior change and application to takeplace
3. Evaluate both before and after the program if practical
4. Survey and/or interview one or more of the following :trainees, their immediate supervisor, their subordinates,and others who often observe their behavior
5. Get 100 percent response or a sampling
6. Repeat the evaluation at appropriate times
7. Consider cost versus benefits
8/2/2019 Measuring Training
13/4913VRAJEEV
Example of Survey to Measure Behavior Application
Understanding and Motivating
1. Trying to understand my subordinates 5 4 3 2 1
2. Listening to my subordinates 5 4 3 2 1
3. Praising good work 5 4 3 2 1
4. Talking with subordinates about
their family and personal interests
5. Asking my subordinates for their ideas 5 4 3 2 1
6. Applying "Management by Walking Around" 5 4 3 2 1
Time and energy spent after the prgram compared to time and energy spent before
the program
5 4 3 2 1
Instruction: The objective of this questionnaire is to determine the extent to which those whoattended the recent program on Leadership have applied the principles and techniquesthat they learned there to the job.
Circle the answer that you consider appropriate for each question.5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less
8/2/2019 Measuring Training
14/4914VRAJEEV
Level 4 Performance Results
Indicate the extent to which youthink this program hasinfluenced each of these
measures in your work unit,department, or business unit: Productivity Quality Customer Response Time Cost Control Employee Satisfaction Customer Satisfaction Other
Level 4 PerformanceResults
8/2/2019 Measuring Training
15/4915VRAJEEV
Guidelines for Evaluating Learning
1. Use a control group if practical
2. Allow time for results to be achieved
3. Measure both before and after the program if practical4. Repeat the evaluation at appropriate times
5. Consider cost versus benefits
6. Be satisfied with evidence if proof is not possible
8/2/2019 Measuring Training
16/4916VRAJEEV
Performance Indicators
HARD DATA INDICATORS
Duration involved
Number of defect products Sales volume
Customer satisfaction index Response time to orders
Others
8/2/2019 Measuring Training
17/49
17VRAJEEV
Performance Indicators
SOFT DATA INDICATORS (intangible impacts)
Job satisfaction
Conducive working relationship Effective communication Stress rate Quality in decision-making
8/2/2019 Measuring Training
18/49
18VRAJEEV
Example : Measuring Training Results
0.00
50.00
100.00
150.00
200.00Program : TQM
Training
Results after
3 months oftraining, number
of defects
dropped to 80
units/day
120 units
80 units
Before training After training
8/2/2019 Measuring Training
19/49
19VRAJEEV
0.00
10.00
20.00
30.00
40.00
50.00
Example : Measuring Training Results
Program :
Sales Training
Results after 3
months training,
number of sales
per salesman
increase to 30
units/month.
20 units
30 units
Before training After training
8/2/2019 Measuring Training
20/49
20VRAJEEV
B.MeasuringReturn on Investment of
Training
8/2/2019 Measuring Training
21/49
21VRAJEEV
Benefits of ROI of Training
Measure contribution
Set priorities
Focus on results
Alter management perceptions of training
8/2/2019 Measuring Training
22/49
22VRAJEEV
ROI of Training Model
CollectData
Isolate theEffects ofTraining
Convert Datato Monetary
Values
Calculate ROIof Training
TabulateProgram
Costs
IdentifyIntangibleBenefits
8/2/2019 Measuring Training
23/49
23VRAJEEV
Net Program Benefits
Program CostsX 100
Return on Investment Formula
ROI =
Example :
Costs per program (25 participants) Rs 88,500
Benefits per program (1st year) Rs 230,625
Rs 230,625 88,500
Rs 88,500ROI = X 100
ROI = 161 %
8/2/2019 Measuring Training
24/49
24VRAJEEV
Collecting Data
Identify appropriateperformance indicators
Develop a collection plan
Collect
Data
8/2/2019 Measuring Training
25/49
25VRAJEEV
Example of Performance Indicators
Units produced Items sold Work backlog New accounts opened
Productivity Inventory turnover Etc.
Output
Equipment downtime Overtime Time to project
completion
Processing time Repair time Lost time days Etc
Time
8/2/2019 Measuring Training
26/49
26VRAJEEV
Example of Hard Indicators
Unit costs Variable costs Overhead costs Operating costs
Number of costreduction
Etc.
Cost
Scrap Waste Rejects Error rates
Rework Product defects Product failure
Quality
8/2/2019 Measuring Training
27/49
27VRAJEEV
Example of Performance Results
Some performance results after trainingprogram:
Rejections were reduced from 11 % to 7.4 %
Absenteeism was reduced from 7 % to 3.25 %
The annual turnover rate was reduced from 30% to 16 %
Customer complaints were reduced by 65 %
8/2/2019 Measuring Training
28/49
28VRAJEEV
Isolating the Effects of Training
Methods toIsolate theEffects ofTraining
UsingControl Group
TrendLines
Participants
EstimateSupervisors of
ParticipantsEstimate
8/2/2019 Measuring Training
29/49
29VRAJEEV
UsingControlGroup
A control group arrangement can beused to isolate training impact.
With this strategy, one group
receives training, while another,similar group does not receivetraining.
The difference in the performance ofthe two groups is attributed to thetraining program.
Isolating the Effects of Training
8/2/2019 Measuring Training
30/49
30VRAJEEV
TrendLines
Trend lines are used to project thevalues of specific output variables iftraining had not been undertaken.
The projection is compared to theactual data after training, and thedifference represents the estimateimpact of training.
Isolating the Effects of Training
8/2/2019 Measuring Training
31/49
31VRAJEEV
Trend Lines Analysis
Jan Feb Mar Apr JulJunMay Aug
At the beginning ofMay, a Sales training
Program session washeld
V o
l u m e o f
S a
l e s
The difference
represents the estimate
impact of training.
Trend Projection
Actual sales performance
8/2/2019 Measuring Training
32/49
32VRAJEEV
Participants andSupervisors of
ParticipantsEstimate of
Trainings Impact
This method rests on the assumption thatparticipants (and their supervisors) arecapable of estimating how much aperformance improvement is related to the
training program.
Because their actions have produced theimprovement, participants (and theirsupervisors) may have very accurate input
on the issue. They should know how much of the
change was caused by applying what theyhave learned in the program.
Isolating the Effects of Training
8/2/2019 Measuring Training
33/49
33VRAJEEV
Participants andSupervisors of
ParticipantsEstimate of
Trainings Impact
Typical Questions to Estimate :
What percent this improvementcan be attributed to the application
of skills/techniques/knowledgegained in the training program?
What confidence do you have inthis estimate, expresses as a
percent?
What other factors contributed tothis improvement in performance?
Isolating the Effects of Training
8/2/2019 Measuring Training
34/49
34VRAJEEV
1 Training Program 50% 70%
2 Change in Procedures 10% 80%
3 Adjustment in Standards 10% 50%
4 Revision to Incentive Plan 20% 90%
5 Increased Management Attention 10% 50%
6 Other - -
PercentImprovement
Caused by
ConfidenceExpressed as
a PercentFactors Which Influenced
ImprovementNo.
Isolating the Effects of Training
Example of a Participants Estimation
The confidence percentage is multiplied by the estimate (50 % x 70 %) to produce a usable training factor value of 35 %
8/2/2019 Measuring Training
35/49
35VRAJEEV
1 Training Program 50% 70%
2 Change in Procedures 10% 80%
3 Adjustment in Standards 10% 50%4 Revision to Incentive Plan 20% 90%
5 Increased Management Attention 10% 50%
6 Other - -
PercentImprovement
Caused by
ConfidenceExpressed as
a PercentFactors Which Influenced
ImprovementNo.
Isolating the Effects of Training
Example of a Participants Estimation
The confidence percentage is multiplied by the estimate (50 % x 70 %) toproduce a usable training factor value of 35 %
This adjusted percentage is then multiplied by the actual amount ofimprovement (post-program minus pre-program value) to isolate the portionattributed to training
The adjusted improvement is now ready for conversion to monetary values,
and used in the return on investment
8/2/2019 Measuring Training
36/49
36VRAJEEV
Converting Data to Monetary Values
ConvertingData to
Monetary
Values
Steps to Convert Data to MonetaryValues
1. Focus on a unit of improvement
2. Determine a value of each unit
3. Calculate the change in performancedata
4. Determine an annual amount of change
5. Calculate the annual value ofimprovement
8/2/2019 Measuring Training
37/49
37VRAJEEV
Steps to Convert Data to Monetary Values
An example to illustrate the steps to convert data to monetary values
1. Focus on unitimprovement
One grievance reaching step two in thefour-step grievance resolution process
Steps Illustration
2. Determine avalue of each
unit
Using internal experts, the cost of anaverage grievance was estimated to beRs,500 when considering time and directcosts
8/2/2019 Measuring Training
38/49
38VRAJEEV
Steps to Convert Data to Monetary Values
3. Calculate thechange in
performancedata
Six months after the program was completed,total grievances per month reaching step two
declined by ten. Seven of the then grievance reductions were
related to the program as determined bysupervisors (isolating the effects of training)
Steps Illustration
4. Determine anannual amountfor the change
Using the six month value, seven permonth (grievance reductions), yields anannual improvement of 84 (7 x 12 months)
8/2/2019 Measuring Training
39/49
39VRAJEEV
Steps to Convert Data to Monetary Values
5. Calculate theannual value of
improvement
Annual value = 84 x Rs500 = Rs42,000
Steps Illustration
8/2/2019 Measuring Training
40/49
40VRAJEEV
Tabulatingcost of the
program
Cost components that should beincluded are :
The cost to design and developthe program
The cost of all program materialsprovided to each participant
The cost for facilitator
The cost of the facilities of thetraining program
Tabulating Cost of the Program
8/2/2019 Measuring Training
41/49
41VRAJEEV
Tabulatingcost of the
program
Cost components that should beincluded are :
Travel, lodging, and meal costs
for the participants Salaries, plus employee benefits
of the participants who attendthe training
Administrative and overheadcosts of the training function,allocated in some convenientway
Tabulating Cost of the Program
8/2/2019 Measuring Training
42/49
42VRAJEEV
C.Enhancing TrainingEffectiveness
8/2/2019 Measuring Training
43/49
43VRAJEEV
How Effective is Your Training Program?
Broad and Newstrom (1992) report studies have
shown less than 30% of what is actually taughttransfers to the job in a way that enhances
performance .Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed strategies to ensure high payoff from training investments. Reading, MA: Addison- Wesley.
8/2/2019 Measuring Training
44/49
44VRAJEEV
Source of Barriers to Training Transfer
Lack of reinforcement on the job
Interference from immediate (work) environment
Nonsupportive organizational culture Trainees perception of impractical training programs
Separation from the inspiration or support of the trainer
8/2/2019 Measuring Training
45/49
45VRAJEEV
The Transfer Partnership
Trainee Trainee recognizes need fornew skills
Trainee Trainer Trainer designs and/ordelivers learning experiences
Trainee Trainer Manager supports learningand application on the job
Manager
8/2/2019 Measuring Training
46/49
46VRAJEEV
Manager
After Training
Plan trainees reentry Provide opportunities to practice new skills
Have trainees participate in transfer-relateddecisions
Reduce job pressures initially
Give positive reinforcement
Schedule trainee briefings for co-workers
Set mutual expectations for improvement
Arrange proactive (refresher) sessions
Provide and support the use of job aids
8/2/2019 Measuring Training
47/49
47VRAJEEV
Trainer
After Training
Provide follow-up support Conduct evaluation surveys and provide
feedback
Develop and administer recognition system
Provide refresher/problem-solving sessions
8/2/2019 Measuring Training
48/49
48VRAJEEV
Trainee
After Training
Practice self-management Review training content and learned skills
Develop a mentoring relationship
Maintain contact with training buddies
8/2/2019 Measuring Training
49/49
THANK YOU