ROI Of Training 04

8/8/2019 ROI Of Training 04

1/36

ROI OF TRAININGROI OF TRAINING

Session 6 Isolating theSession 6 Isolating the

Results/Effects Of the TrainingResults/Effects Of the Training


2/36

Pre-Amble

Isolating the results/effect of the training effort isthe most difficultand also most importantelement of building the case for value (ROI);

Because of this, value at the higher levels isseldom determined;

However, the credibility not only of your study toshow value, but also the training function isbased on this element.


3/36

Pre-Amble

The complexity of isolatingthe effect, is a function of

the variables that are inplay at any moment intime in an organization


4/36

Pre-Amble

TheChallenge


5/36

Pre-Amble

An acceptabledegree ofaccuracy


6/36

External FactorsManagement

Attention

Incentives

Systems/ProceduresChanges

Learning

Program

TOTAL

IMPROVEMENTAFTERPROGRAM

EFFECT OFLEARNING ON

IMPROVEMENT


7/36

How then doyou isolate the

effects?


8/36


9/36

Control groups Trend line analysis of performance data Participants estimation of impact

Supervisors estimation of impact Managements estimation of impact Use of previous studies Use of customer input

Subordinates report of other factors Estimating the impact of other factors

Isolating the Effects of theIntervention


10/36


External Factors

Management

Attention

Incentives

Systems/Procedures

Changes

Learning

Program

ELIMINATE


11/36

Control groups Trend line analysis of performance data Participants estimation of impact

Supervisors estimation of impact Managements estimation of impact Use of previous studies Use of customer input

Subordinates report of other factors Estimating the impact of other factors



12/36


External Factors

Management

Attention

Incentives

Systems/Procedures

Changes

Learning

Program

CALCULATE

AND ATTRIBUTE THEREST


13/36

Control Groups

Most accurate isolation approach; Involves the use of an experimental group

that is impacted by the intervention and a

control group that is not; Composition of both groups should be as

identical as possible and, if feasible, theselection of participants for each group

should be on a random basis.


14/36

Control Group Designs

ControlGroup

ExperimentalGroup

M1

M1 Program

M2

M2


15/36

Question isWhat Do You

Control?


16/36


17/36


18/36

Federal Express ROI analysis using control groups as a goodexample of the state of the art in measuring ROI. The studyfocused on 20 employees who went through an intense,redesigned two-week training program soon after being hired todrive company vans. Their performance was compared with acontrol group of 20 other new hires whose managers were told todo no more or less on-the-job training than they normally would.

Performance was tracked for the two groups for 90 days incategories such as accidents, injuries, time-card errors, anddomestic air-bill errors. The ten performance categories wereassigned monetary values by experts from engineering, finance,and other groups. The intervention demonstrated that theperformance of the highly trained employees was superior tothat of the group that did not receive the upgraded training and

also resulted in a 24% return on investment.

Also look at CLC Best PracticeDocument
http://../Corporate%20Leadership%20Council/CLC11R8ARD.pdfhttp://../Corporate%20Leadership%20Council/CLC11R8ARD.pdfhttp://../Corporate%20Leadership%20Council/CLC11R8ARD.pdfhttp://../Corporate%20Leadership%20Council/CLC11R8ARD.pdf


19/36

Federal Express demonstrates: PracticalityNon disruptive;Relatively accurate;Line involvement (and commitment);ROI is possible if you know what you are

aiming for (measures)

Also look at CLC Best PracticeDocument


20/36

PracticalExercise


21/36

Accra Wholesale Brickworks, find that thereproduction targets are not being met. TheCompany is midsize but established in the City.

It employs a production team consisting ofthree foreman, that supervise approximately 8workers each and these teams takeresponsibility for procurement of the rawmaterials, mixing, filling the moulds, drying

and baking. The production function ismanaged by a section manager.

Practical Exercise


22/36

Practical Exercise

Using the control group method:

How would you selects your experimental

group vis your control group What will you control

How will you apply the Trend lineAnalysis Technique?


23/36

Trend-line Analysis

Basic format less accurate than control group Draw a trend-line using previous performance data as

a base When intervention is conducted, actual performance is

plotted and compared to the trend line Improvement in actual performance over what thetrend line predicted can then be reasonably attributedto the intervention

The primary advantage of this approach is that it issimple, inexpensive, and takes very little effort;

However not accurate as it does not isolate into otherfactors;

However a good first order analysis.


24/36

AveragePost Program 35

AveragePre Program 55

Time

Comp

laints

Sexual

HarassmentPrevention Program

Formal Internal Complaints of Sexual Harassment

O N D J F M A M J J A S O N D J F M AM J J A S O

Trend-line Analysis


25/36

Trend line Use Case Study(Micro Electronics Incorporate

Practical Exercise


26/36

Estimate of Impact by Key Role Players

Easy to implement; Based on the assumption that participants

are capable of determining or estimating howmuch of a performance improvement/impact

is related to the intervention; Because their actions have produced the

improvement, participants may have veryaccurate input on the issue;

Although an estimate, this value will usually

have considerable credibility withmanagement because participants are at thecenter of the change or improvement;

High face validity.


27/36

i f b l


28/36

The individuals who do not respondto the questionnaire or provide non-usable data on the questionnaire areassumed to report noimprovement/impact;

Extreme data and unrealistic claimsare omitted from the analysis;

Only annualized values are used, itis assumed that there are nobenefits from the program after thefirst year of implementation;

The confidence level, expressed as apercent, is multiplied by theimprovement value to reduce theamount ofthe improvement by thepotential error;

If value is expressed it is alsofactored by the confidence value.

Adjustmentsto

improvevalidity

Estimate of Impact by Key RolePlayers


29/36


% ProgramContributed

toImprovement

Confidenceof

Statementas %

Factor TotalImprovemen

t(annualized) Value

Contributionto

Improvement

60 % (.6) 40% (.4) .24 1000 240

60 % (.6) 40% (.4) .24 1000 240

50% (.5) 50% (.5) .25 1000 250


30/36


The processcan beexpanded

toinclude:

Supervisors ManagementOtherstakeholders


31/36

Expert Input

Select carefully, based on their knowledge of theprocess and situation;

If capacity does not exist in the organization; Will probably use techniques that most training

practitioners can implement; Organization culture will determine whether the

outcome will be accepted; If linked to skills transfer it could be worthwhile; Often top management place more confidence in

outside experts than in their own staff.


32/36


33/36

Obtaining Input From a TargetGroup

In some situations it could be feasible to calculate the impactof factors (other than the intervention) that influenced theimprovement and then conclude that the intervention iscredited with the remaining portion;

In terms of this approach, the intervention claims credit forimprovement that cannot be attributed to other factors;

The method is appropriate when the other factors are easilyidentified and the appropriate mechanisms are in place tocalculate their impact on the improvement;

The process can be very credible if the method used to isolatethe impact of other factors is credible.


34/36

Which Strategy?

Control group feasibility; Availability of historical data; Expertise to conduct study;

Capability and willingness toestimate impact; Can non-training effects be

isolated? What are the constraints in

terms of using the data

collection technique andisolation strategy.


35/36

Credibility of Data is InfluencedBy:

Reputation of the source ofthe data;

Reputation of the source ofthe study;

Motives of the researchers; Personal bias of audience; Methodology of the study; Assumptions made in the

analysis; Realism of the outcome data; Type of data; Scope of analysis.


36/36

ROI Of Training 04

Documents

Transcript of ROI Of Training 04