Measuring ROI in Training with Case Studies
-
Upload
shyam-sunder -
Category
Business
-
view
8.863 -
download
0
description
Transcript of Measuring ROI in Training with Case Studies
ROI INDIA
2 day Workshop
Reaction Objectives
1) Perceive it to be relevant to your work
2) Find it important to your success on the job
3) Find the case studies relevant and useful
Learning Objectives
1) Identify the different evaluation levels
2) Align programs to business needs
3) Indentify latest 3 ways to collect data
4) Describe at least 2 ways to isolate the effects of the program
5) Calculate ROI given benefits & cost
Application Objectives
1) Develop an evaluation plan for a specific program
Impact Objectives
1) Improve the effectiveness of the programs
2) Redesign ineffective programs
3) Show ROI on your programs
Six months after completing the workshop, you should:
Case Study[ Reliance Insurance Organization ]
Page 1.4
Discussion Questions
1) Is this situation typical?
2) What are the basic issues in the case?
3) How could this situation be avoided in the future?
4) What should Marge do? Be specific?
5) What should Frank be expecting in the future?
Activity Based Result Based
No business need for the program Program linked to specific business needs
No effort to prepare the work environment
to support program
Environment prepared to support program
Reporting on programs is input focused Reporting on programs is output focused
Objectives are defined at lower levels Objectives are defined at higher levels
is Training Evaluation
ARTScience
1954
1959
1970
Kirkpatrick Model Phillips ROI Model
ReactionLevel 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
Inputs / IndicatorsLevel 0 100%
Level 5
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned ActionLevel 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
90–100%
Inputs / IndicatorsLevel 0 100%
Level 5
Learning Learning 70–80%
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
90–100%
70–80%
20–30%
Inputs / IndicatorsLevel 0 100%
Level 5
Learning
Implementation
If the participants know the L3 behaviors are being
tracked they will be more likely to apply
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
90–100%
70–80%
20–30%
10–20%
Inputs / IndicatorsLevel 0 100%
Level 5
Customer satisfaction
Sales
Units manufactured
Students graduated
Costs
Quality complaints
Time to project completion
Rework
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
90–100%
70–80%
20–30%
10–20%
Inputs / IndicatorsLevel 0 100%
Level 5
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
90–100%
70–80%
20–30%
10–20%
01–05%
Inputs / IndicatorsLevel 0 100%
ROE / ROI Level 5
(ROE)
(ROI)
Benefits/ Cost Ratio =
Program Benefits
Program Cost
ROI =
Net Program Benefits
Program Costs
X 100
Benefits/ Cost Ratio =
2, 30, 000
88, 000
ROI =
2,30,000 – 88,000
88, 000
Program Benefits from Participants (1st year) = ` 2,30,000
Costs per Program (25 participants) = ` 88,000
= 2.61
= 161% X 100
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
10 step by step of collecting, analyzing, reporting, using systematic & methodic
90–100%
70–80%
20–30%
10–20%
01–05%
Inputs / IndicatorsLevel 0 100%
Return on InvestmentLevel 5
Develop
Objectives of
Solution (s)
Develop
Evaluation
Plans and
Baseline Data
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation
Level 2: Learning Level 4: Business Impact
Level 1: Reaction,
Satisfaction, and
Planned Actions
Level 3:
Application/ Implementation
Evaluation Planning Data Collection
THE PROCESS MODEL
Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:ROI
Intangible Measures
Data Analysis Reporting
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
10 step by step of collecting, analyzing, reporting, using systematic & methodic
12 guiding principles to be conservative in approach
90–100%
70–80%
20–30%
10–20%
01–05%
Inputs / IndicatorsLevel 0 100%
Return on InvestmentLevel 5
Job Aid
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
10 step by step of collecting, analyzing, reporting, using systematic & methodic
12 guiding principles to be conservative in approach
5 books 40 books,3000 case studies
Slowly phasing out 3000 organizations in 50 countries
90–100%
70–80%
20–30%
10–20%
01–05%
Inputs / IndicatorsLevel 0 100%
Return on InvestmentLevel 5
Myths of ROI Evaluation
1) Soft skills can not be measured
2) We are not yet there for ROI
3) We don‘t believe in ROI
4) My senior management doesn‘t want to know ROI
5) ROI Measurement is too expensive
6) ROI can‘t be measured and reported by HR & Training professionals
An Evaluation
FrameworkCase Applications
and Practice
A Process
Model
Operating
Standards and
Philosophy
Implementation
Strategy
Pieces of Puzzle
All program are ‗not evaluated’ at all
levels…
Level Measurement category Current status Evaluation
Targets
0 Inputs and Indicators: 100%
1 Reaction and planned action: 100%
2To measure what the participants have
learned in the program-90%
3 Application and Implementation: 50%
4 Impact and consequences: 30%
5 Return on investment: 10%
Level Measurement category Current status Evaluation
Targets
0 Inputs and Indicators:Done for less then 25% of the
programs100%
1 Reaction and planned action:Done for 100% of the workshops
conducted.100%
2To measure what the participants have
learned in the program-
Done for more than 60-70% of the
training programs by way of Tests
and role plays.
90%
3 Application and Implementation:Done for less than 10% of the
training programs50%
4 Impact and consequences:Not done for any training program
previously30%
5 Return on investment:Not done for any training program
previously15%
Tata Sky
What is your organization’s
current status?
Develop
Objectives of
Solution (s)
Develop
Evaluation
Plans and
Baseline Data
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation
Evaluation Planning Data Collection
OUTPUT, TIME, COSTS, QUALITY
CUSTOMER SERVICE
• Customer Complaints
• Customer Satisfaction
• Customer Dissatisfaction
• Customer Loyalty
• Customer Retention
• Customer Value
• Lost Customers
EMPLOYEE DEVELOPMENT
/ADVANCEMENT
• Promotions
• Capability
• Intellectual Capital
• Programs Completed
• Requests for Transfer
• Performance Appraisal Ratings
• Readiness
• Networking
INITIATIVE/INNOVATION
• Creativity
• Innovation
• New Ideas
• Suggestions
• New Products and Services
• Trademarks
• Copyrights and Patents
• Process Improvements
• Partnerships
• AlliancesWORK
CLIMATE/SATISFACTION
• Grievances
• Discrimination Charges
• Employee Complaints
• Job Satisfaction
• Organization Commitment
• Employee Engagement
• Employee Loyalty
• Intent to Leave
• Stress
IMAGE
• Brand Awareness
• Reputation
• Leadership
• Social Responsibility
• Environmental Friendliness
• Social Consciousness
• Diversity
• External Awards
WORK HABITS
• Tardiness
• Violations of Safety Rules
• Communication Break-downs
• Excessive Breaks
Hard Data Soft Data
Easy to quantify Difficult to quantify
Easy to assign a rupee value Difficult to assign a rupee value
Objectively based Subjectively based
Common measures Less credible as a performance measure
Can be measured Can be measured
Analysis
Design
Development
Implementation
Evaluation
}
The best way to Predict a future is to …
create it.―
‖
ProjectBusiness Alignment and Forecasting The ROI Process Model
Business Alignment Model
Learning Needs
Preference Needs
Measurement and
Evaluation
Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application ObjectivesPerformance Needs
Impact ObjectivesBusiness Needs
Payoff Needs ROI Objectives
End HereStart Here
5
4
3
2
5
4
3
2
1 1
ROI Needs
Analysis
Project
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application ObjectivesPerformance Needs
Impact ObjectivesBusiness Needs
Payoff Needs ROI Objectives
End HereStart Here
5
4
3
2
5
4
3
2
1 1
( Supervisors need )
Counseling & Discussion Skills
Cause & Effect
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
Absenteeism is causing
15,00,000 monthly
Unplanned absenteeism is 16% and
increasing. Greater than benchmarked 5%
Discussion between team member
and supervisor are not occurring
Counseling and discussion skills
Counseling skills must be practical
and should happen immediately
ROI Objectives
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs5
4
3
2
1
Absenteeism is causing
15,00,000 monthly
Unplanned absenteeism is 16% and
increasing. Greater than benchmarked 5%
Discussion between team member
and supervisor are not occurring
Counseling and discussion skills
Counseling skills must be practical
and should happen immediately
4/5 on the practicality of the
program and relevance to job
Participants will learn key
counseling skills
Counseling discussion conducted in
95% situations when an unexpected
absence occurs
Reduce absenteeism to 5%
ROI of 20%
Reaction Objectives
Learning Objectives
Application Objectives
Impact Objectives
V Model
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application ObjectivesPerformance Needs
Impact ObjectivesBusiness Needs
Payoff Needs ROI Objectives
End HereStart Here
5
4
3
2
5
4
3
2
1 1
Absenteeism is causing
`15,00,000 monthly
Unplanned absenteeism is 16%
and increasing. Greater than
benchmarked 5%
Discussion between team
member and supervisor are
not occurring
Deficiency in counseling and
discussion skills
Counseling skills must be
practical and should happen
immediately
4/5 on the practicality of
the program and relevance
to job
Participants will learn key
counseling skills
Counseling discussion conducted in 95% situations
when an unexpected absence occurs
Reduce absenteeism to 5%
ROI of 20%
Reaction Questionnaire at
the end of the program
Practice session during program and
facilitator assessment
Follow up questionnaire to
participants to check the
frequency of discussions
Monitor absenteeism
records for 6 months
Calculate ROI, BCR
V Model –Tata Sky
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application ObjectivesPerformance Needs
Impact ObjectivesBusiness Needs
Payoff Needs ROI Objectives
End HereStart Here
5
4
3
2
5
4
3
2
1 1
Increase Sales
Increase EPRS
Profiling of the customer,
Standardization in customer demo,
& transaction
Greeting
Asking right questions to the prospects to
understand the wants and needs,
Match the needs with features & benefits
ILT and role plays 4/5 on the practicality of the
program and relevance to job
Participants will learn key interaction
& influencing skills
Use the skills consistently, which is 100% of the
times
+ve ROI
Track business dashboards
every month
Calculate ROI, BCR
Increase walk-in conversion ratio
Increase the subscription recharge by 10%
Increase Sales by 10%
Action plan activity sheet
Facilitator assessment through role
plays
Reaction and planned action through a
questionnaire at the end of the program
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs5
4
3
2
1
Increase Sales
Increase EPRS
Increase walk-in conversion
Profiling of the customer,
Standardization in customer
demo, & transaction
Greeting, Asking right questions to the
prospects to understand the wants and
needs, Match the needs with features &
benefits
ILT and role plays
Sales Revenue
EPRS Revenue
4/5 on the practicality of the
program and relevance to job
Participants will learn key
interaction & influencing skills
Use the skills consistently, which is
100% of the times
+ve ROI
Increase Sales by
Increase the subscription recharge by
Increase walk-in conversion ratio
V Model – Publishing Firm
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application ObjectivesPerformance Needs
Impact ObjectivesBusiness Needs
Payoff Needs ROI Objectives
End HereStart Here
5
4
3
2
5
4
3
2
1 1
Potential project loss of
`1,25,00,000 a year
-Improve quality of emails
-Decrease supervisors monitoring
time
Excellent email communication
skills for a global audience
Structure, tone, style and grammar
and global culture
Blended – ILT, LMS and coaching Overall score of 90%
Participants will learn email skills
Use appropriate email writing skills 100% of the
times while writing emails
Error rates to zero
Decrease managers monitoring time to nil
ROI of 50%
Online reaction Questionnaire
Online pre and post assessment
Email scorecard
Performance records &
Interview
Calculate ROI, CBR
Each level includes:• Data Sources• Data Collection• Key Questions• Key Issues
Collect Data After Solution
Implementation
Isolatethe Effects
of the Solution
Convert Data To Monetary
Value
CalculateThe Return On
Investment
TabulateCosts of Solution
IdentifyIntangibles
Significant Influences • Policy Statement• Procedures and Guidelines• Staff Skills• Management Support• Technical Support• Organizational Culture
Phillips ROI Methodology TM
Develop Report and
CommunicateResults
Level 4 Level 3 Level 3 and 2 Level 1
Level 3Level 4
Level 5
Level 5
Results-Based Approach
Intangible Benefits
Specify Skill/Knowledge
Deficiencies of Affected Population
TrainingRequired
?
Level 2
Problem/Opportunity Present or Anticipated
Identify Job Performance
Needs, Gaps, and Why
Identify Solutions
Identify Preferences
Develop Objectives/ Evaluation Strategy
Identify TransferStrategy Options and
L-2 & L-3 Support For All Stakeholders
Identify Business, Needs, Gaps, and Stakeholders
Design Solution and StakeholderComponents
ConsiderResources/ Logistics Delivery
Develop Content/ Materials
Conduct/ Implement Solution
Level 1Level 2
Implement Pre-activity
Implement Transfer Strategy
- Solution- Pre-work Activity/Transfer Strategy
Page 3.9
( More Examples )
1 Reaction
2 Learning
3 Application
4 Impact
5 Return On Investment
Develop
Objectives of
Solution (s)
Develop
Evaluation
Plans and
Baseline Data
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation
Evaluation Planning Data Collection
Initial collection of data which serves as a basis for
COMPARISON with the subsequently acquired data.
125 Kgs 86 Kgs
Existing Expected
1. Data collection Plan
2. ROI analysis plan
( sample )
Level Objectives Measures/Data
Data
Collection
method
Data Source Timings Responsibility
1 Positive reactionAn overall average
rating of 4/5 Questionnaire
Participants of the
program
At the end of
each workshopFacilitator
2Will be able to demonstrate all
the 3 skills during the role playFacilitators rating
Observation
of skill practice
by facilitator
FacilitatorDuring the
workshopFacilitator
3
(1)Will be able to assess all the
EZs for adherence to the
taught processes
(2)Will be able to bring all the
employees to a consistent level
of display of learned skills.
Completion of Action
plan
Action plan
sheet
Retail Operation
Incharges(ROIs)
Retail Operation
Managers(ROMs)
30 – 45 days
after the
workshop
Regional Managers
Retail Operations
4
(1)The walk in conversion
ratio will increase
(2)The sales will increase
(3)The customer retention will
increase
Monthly data from the
Retail Operation
Vertical of the
Experience Zones
Business
performance
monitoring
Dashboard,
Retail Operations
45 – 90 days
after the
workshop
Regional Managers
Retail Operations
5 ROI No Objective was set for this level in the beginning. The expectation was to have a positive ROI.
Level Objectives Measures/Data
Data
Collection
method
Data Source Timings Responsibility
1Reaction to the program,
content and facilitator.
An overall average
rating of 4.5/5
Online
questionnaire
Participants of the
program
At the end of
each workshopFacilitator
2 Assess the email writing skillsScore above 90% in
the post assessmentOnline test Participant
At the end of
each workshopFacilitator
3
Demonstrate appropriate
structure, tone, style and
grammar in emails
100% score on the
score cardScore card Participant
Weekly
monitoring of
emails for 3
months
Facilitator
4
Improve quality of emails
Decrease managers monitoring
time
Decrease email errors
to zero
Decrease manager‘s
monitoring time to nil
Performance
records,
Interview
Managers3 months after
trainingFacilitator
5 ROI 50% ROI was set.
Data Items
(Usually
Level 4
Method for
Isolation
Methods of
Converting
Data to
Monetary
Values
Cost Categories Intangible
Benefits
Communication
Targets for Final
Report
Other
Influences/
Issues
During
Application
Monthly
conversion
ratio
Participant
estimate
(ROMs &
ROI)
Standard
Value.
(1)Facilitation Fees
(2)Program Materials
(3)Market survey Fees
(4)Facility costs
(5)Participants salaries
ROMs
ROIs
Exec. V P
Retail
operations
Any change
in market
factor
Tata Sky – Sales Mystery Shopping
• Ambience ( 4 parameters )
• Grooming ( 4 parameters )
• Opening the call (15 parameters )
• Developing/ Identifying the need ( 8 parameters )
• Proposing a solution ( 4 parameters )
• Objection Handling ( 5 parameters )
• Closing (4 parameters )
• Follow up ( 2 parameters )
• Process adherence ( 4 parameters )
• Telephone Related (1 parameters )
Publishing Firm – Score from emails, reports
Parameters Pre-Assessment
Grammar 63.8 %
Structure 85.4%
Tone & Style 58.3%
Average Score 70.2%
3.26 / Appendix A.4
Develop
Objectives of
Solution (s)
Develop
Evaluation
Plans and
Baseline Data
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation
Level 2:
Learning
Level 4:
Business Impact
Level 1: Reaction,
Satisfaction, and
Planned Actions
Level 3:
Application/ Implementation
Evaluation Planning Data Collection
1. Relevance to work
2. Important to success
3. Would recommend to others
4. Amount of new information
5. Method of delivery
6. Learning Environment
7. Planned Action
1. Skills
2. Knowledge
3. Capacity
4. Capability
5. Awareness
6. Confidence
A. Survey
B. Test
C. Questionnaire
D. Interview
E. Focus Group
F. Observation
G. Performance Records
S.No InstrumentLevel
1
Level
2
Level
3
Level
4
1 Survey
2 Test
3 Questionnaire
4 Interview
5 Focus Group
6 Observation
7 Performance Contracts
Impact Questionnaire for Leadership Development Program
Page 4.11
Page 4.21
Page 4.22
Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:ROI
Intangible Measures
Data Analysis Reporting
Case StudyFirst Bank (page 5.3)
Discussion Questions
1) Is this situation typical
2) Should CEO drop the issue?
3) What are some approaches to resolve the dilemma?
4) What would you do?
External Factors
Management
Attention
Incentives
Systems/Procedures
Changes
Programs
TOTAL
IMPROVEMENT
AFTER
PROGRAM
EFFECT OF
PROGRAM ON
IMPROVEMENT
1. Use of a control group arrangement
2. Trend line analysis of performance data
3. Participant‘s estimate of the program‘s impact (percent)
4. Supervisor‘s estimate of the program‘s impact (percent)
5. Management‘s estimate of the program‘s impact (percent)
6. Use of experts/previous studies
No Program
Program
Measurement
Measurement
Control Group
Experimental
20%
10%
J F M A M J J A S O N D J
REJECT
RATES
18.5% Pre Program Six-Month Average
Projected Average —
Using Pre Data as a Base
14.5%
7% Post Program
Six-Month
Average
CPI Program Conducted
MONTHS
1. If the training program would not been implemented, would this trend
continue on the same path established before the training?
2. No other variables and influences entered the process after the training was
conducted
Analysis
Design
Development
Implementation
Evaluation
}
• Participants estimate
• Supervisor estimate
• Customer Input
• Experts Inputs
Francis Galton
Sales %
Attributable to
training
Confidence Level
ERPS %
Attributable to
training
Confidence
Level
Total Average for
the Group42% 86% 42% 88%
Corrected Group average for sales 36% of the increase in Sales of Boxes
Corrected Group average for
Subscription Recharge(EPRS)37% of Increase in Monthly Recharge( EPRS)
Data
Source
Time Savings
in Hours
Attribution
Level to
Training
Confidence
Level
Adjusted
Value
Decrease in
Supervisor
monitoring
time
Supervisors 1.2 hours / day 60% 100%21.5 days /
year
Project &
Strategic
Head
2 hours / week 60% 100% 7.8 days / year
Decrease in
escalations of
poor quality
Project Head - 60% 100% 4.2 escalations
Case StudyNational Bank
Page 5.10
Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:ROI
Intangible Measures
Data Analysis Reporting
• Converting output to contribution - standard value (profit/savings)
• Converting the cost of quality - standard value
• Converting employee‘s time (using compensation)
• Using historical costs/savings
• Using internal and external experts
• Using data from external databases/studies
Service/Production Workers - Hourly (e.g. Courier) 40 - 70%
Skilled Hourly (e.g. Machinist) 75 - 100%
Clerical/Administrative (e.g. Scheduler) 50 - 80%
Professional (e.g. Sales Representative, Nurse, Accountant) 75 - 125%
Technical (e.g. Computer Technician) 100 - 150%
Engineers (e.g. Chemical Engineer) 200 - 300%
Specialists (e.g. Computer Software Designer) 200 - 400%
Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%
Middle Managers (e.g. Department Manager) 125 - 200%
Exit cost of previous employee Lost productivity
Recruiting cost Quality problems
Employee cost Customer dissatisfaction
Orientation cost Loss of expertise/knowledge
Training cost Supervisor‘s time for turnover
Wages and salaries while training Temporary replacement costs
Step 1 Define the unit of measure
(Increase in sales, decrease in absenteeism, time saving)
Step 2 Determine the value of each unit
One unit of sales, one cost of an absentee, one hour of time saving)
Step 3 Calculate the change in performance
Change during the reporting period
Step 4 Determine the annual amount of change
Step 5 Calculate the total value of improvement
The value of one unit of improvement multiplied by the amount of annual change during
the reporting period
Step 1 Define the unit of measure
Unit of measure = one less grievance
Step 2 Determine the value of each unit
Using internal experts on the labour relations staff, the cost of an average grievance was
estimated to be 18,000, considering time and direct cost
Step 3 Calculate the change in performance
Six months after the program, total grievances per month had declined by 10.
Supervisors isolated the effects of the program, determining that 7 of the 10 grievance
reductions were because of the program
Step 4 Determine the annual amount of change
Using the adjusted value of 7 per month yields an annual improvement of 84
Step 5 Calculate the total value of improvement
Annual value = 84 x 18,000
= 15, 12,000
Step 1 Define the unit of measure
One new customer,
Subscription recharge for a month
Step 2 Determine the value of each unit
Average revenue earned is Rs. 300 for a set top box(Finance dept. worked out).
Increased subscription recharge per month= Rs. 21,27,878 (after deduction 70% cost)
Step 3 Calculate the change in performance
Average increment in sales boxes is 1016. Out of which 36% was attributed to training.
(1016 x 36)/ 100 = 365
The percentage increase in subscription recharge attributable to training is 37%.
(2127878 x 37)/ 100 = 7,87,315
Step 4 Determine the annual amount of change
Annual value of increased sales in the set top box : 300 x 365 x 12 = 13,14,000
Annual value of increased in subscription recharge : 7,87,315 x 12 = 94,47,780
Step 5 Calculate the total value of improvement
Total benefit : 13,14,000 + 94,47,780 = 1,07,61,780
TURNOVER COSTS SUMMARY
Entry Level - Hourly, Non Skilled (e.g. Fast Food Worker) 30 - 50%
Service/Production Workers - Hourly (e.g. Courier) 40 - 70%
Skilled Hourly (e.g. Machinist) 75 - 100%
Clerical/Administrative (e.g. Scheduler) 50 - 80%
Professional (e.g. Sales Representative, Nurse, Accountant) 75 - 125%
Technical (e.g. Computer Technician) 100 - 150%
Engineers (e.g. Chemical Engineer) 200 - 300%
Specialists (e.g. Computer Software Designer) 200 - 400%
Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%
Middle Managers (e.g. Department Manager) 125 - 200%
No
Is there a
standard
value?Yes
Is there a
Method to
get there?
Move to
Intangible
Benefits
Can we get
there with
minimum
resources?
No
Yes
No No
Can we
convince
executive of
credibility in
2 minutes?
Convert
data and
add to
numerator
Yes Yes
Add to
numerator
Move to
Intangible
Benefits
Move to
Intangible
Benefits
Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:ROI
Intangible Measures
Data Analysis Reporting
• Needs Analysis cost
• Development cost
• Delivery cost
• Evaluation cost
• Others
S. No The Cost category
Value( In
Indian
Rupees)
1. Facilitation fees for the Master trainer 2,00,000
2. The cost for the content 12,00,000
3. The cost for the Market survey 25,000
4.
Participants Salaries(As per managerial grades):
M1-2 Participants( @ Rs.10000 per day) 40,000
M2 – 2 Participants( @ 7000 per day) 28,000
M3 – 2 Participants( @ 4000 per day) 16,000
M4 – 10 Participants( @ 2800 per day) 56,000
M5 – 22 Participants ( @ 1400 per day) 61,600
5.Cost of internal venue for the six batches done for the Company staff (ROIs
and the ROMs).This included rent of venue and food.1,50,000
6.Cost of the 25 training batches for the Experience Zone Employees (@
Rs.25000 per batch).This included rent of venue and food.6,25,000
7.Cost of 6 refresher batches – required to tackle the employee churn (@
Rs.25000 per batch).1,50,000
Total Costs for the training program 25,51,600
Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:ROI
Intangible Measures
Data Analysis Reporting
Benefits/ Cost Ratio =
Benefits/ Cost Ratio
Program Cost
ROI = Program Benefits – Program Costs
Program Costs
X 100
Program Benefits = 1,07,58,691
Program Costs = 25,51,600
Benefits/ Cost Ratio =
1,07,58,691
2551600
= 4.21
ROI =
1,07,58,691 – 25,51,600
25,51,600
= 321% X 100
Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:ROI
Data Analysis Reporting
Brand Awareness
Job satisfaction
Organizational Commitment
Work Climate
Employee Engagement
Employee Stress
Employee Innovation
Employee Networking
Employee Collaboration
Employee partnering
Customer Satisfaction
/Dissatisfaction
Reputation
Image
Customer Complaints
Customer Response Time
Customer Loyalty
Teamwork
Cooperation
Conflict
Decisiveness
Communication
Accountability
Alliances
Awards
Capability
Capacity
Creativity
Clarity
Caring
Corporate Social
Responsibility
Human Life
Intellectual Capital
Leadership
Poverty
Sustainability
Work/Life Balance
• The confidence of the Experience zone employee increased due to the training. This
reflected in all the activities of the store. The morale of the experience zone employee as
well that of the owner increased. The effect was noticeable but since we could not measure
the difference in confidence pre and post training, this was reported as an intangible benefit.
• The better customer management at the stores reduced the incoming call load at the
contact center. As we could not use any method to isolate the effect we decided to keep it
as an intangible method.
• Better handling of customers during the time of sale also ensured that they recharged for the
first month more regularly. This reduced the casual churn of the customers due to not
recharging their accounts in time. Here again since we could not find a way to isolate this
particular effect of training we decided to list this as an intangible benefit.
• The ROIs got skilled on the job coaching. This will help in better channel management.
Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:ROI
Intangible Measures
Data Analysis Reporting
• General information
• Methodology for impact study
• Data analysis
• Costs
• Results
• Barriers and enablers
• Summary of findings
• Conclusions and recommendations
• Exhibits
Chain of
Impact
Difficulty of
Assessment
Power to
Show Results
Frequency of
Use
Value of
Information
Lowest Lowest Frequent Low
Reaction
Learning
Application
Impact
ROI
Highest Highest Infrequent High
Shyam [email protected]
Principal & Founding Partner
GreenBooks & ROI India
Facebook: www.facebook.com/shyamsunder.a
Twitter: www.twitter.com/shyamsunder