8/2/2019 ID - Session 6

1/18

Kirkpatricks

Levels of Evaluation

Presentation by: Hemant Swarnkar1

8/2/2019 ID - Session 6

2/18

2

Donald Kirkpatrick

Kirkpatrick developed a model of training

evaluation in 1959

Arguably the most widely used approach

Simple, Flexible and Complete

4-level model

8/2/2019 ID - Session 6

3/18

The Four Levels

3

8/2/2019 ID - Session 6

4/18

Level evaluationtype (whatismeasured)

evaluation descriptionand characteristics

examples of evaluation toolsand methods

relevance and practicability

1 Reaction Reactionevaluation is how thedelegates felt about the

training or learningexperience.

'Happy sheets', feedbackforms.Verbal reaction, post-training

surveys or questionnaires.

Quick and very easy to obtain.Not expensive to gather or to analyse.

2 Learning Learning evaluation isthe measurement ofthe increase inknowledge - before and

after.

Typically assessments or testsbefore and after the training.Interview or observation canalso be used.

Relatively simple to set up; clear-cutfor quantifiable skills.Less easy for complex learning.

3 Behaviour Behaviour evaluation isthe extent of appliedlearning back on the job- implementation.

Observation and interviewover time are required toassess change, relevance ofchange, and sustainability of

change.

Measurement of behaviour changetypically requires cooperation and skillof line-managers.

4 Results Results evaluation isthe effect on thebusiness orenvironment by the

trainee.

Measures are already in placevia normal managementsystems and reporting - thechallenge is to relate to the

trainee.

Individually not difficult; unlike wholeorganisation.Process must attribute clearaccountabilities.

8/2/2019 ID - Session 6

5/18

5

Relationship Between Levels

Each subsequent levelis predicated upondoing evaluation atlower level

A Level 3 will be ofmarginal use, if a Level

2 evaluation is notconducted

Level 1 - Reaction

Was the environment suitable for learning?

Level 2 - KnowledgeDid they learn anything

Level 3 - BehaviorKSA being used on the job?

Level 4 - ResultsWas it worth it?

8/2/2019 ID - Session 6

6/18

6

Types of Assessments Used at Each Level

Level 1 - Reaction

Was the environment suitable for learning?

Level 2 - KnowledgeDid they learn anything

Level 3 - BehaviorKSA being used on the job?

Level 4 - ResultsWas it worth it?

Type Form

SummativeCorrelation of businessresults with otherassessment results

Summative Observation ofPerformance

360 Survey

Diagnostic

Summative

Self-assessment

Test

Reaction

Formative

Survey

Real-time Polling

Quizzing

8/2/2019 ID - Session 6

7/18

7

Reaction - What Is It?

How favorablyparticipants react to thetraining (Customersatisfaction) Collects reactions to

instructor, course, andlearning environment

Communicates to

trainees that theirfeedback is valued

Can provide quantitativeinformation

8/2/2019 ID - Session 6

8/18

8

Reaction - What It Looks Like

Questionnaire - Most commoncollection tool

Content: I enjoyed the content.

Methods: The seminar approachhelped me learn.

Media: The AVs were helpful to me.

Trainer style: I liked the instructor.

Facilities: The room was useful formy learning.

Course materials: The materialsprovided make my learning better.

SD: 1 2 3 4 5 6 :SA

8/2/2019 ID - Session 6

9/18

9

Reaction - How to Perform

Determine what you wantto find out

Design a form tocollect/quantify reactions

Do Immediately

Develop acceptable

scoring standards Follow-up as appropriate

8/2/2019 ID - Session 6

10/18

10

Learning - What Is It?

Knowledge

Skills Attitudes

8/2/2019 ID - Session 6

11/18

11

Learning - What ItLooks Like

Media used to measurelearning: Text: I felt the textbook help me

learn.

Voice: The audio materialsincreased my knowledge. Demonstration: I learned well from

the demonstration.

Methods used to measure

learning: Interviews Surveys Tests (pre-/post-) Observations Combinations

8/2/2019 ID - Session 6

12/18

12

Learning - How toPerform

Use a control group, iffeasible

Evaluate knowledge,skills, and/or attitudesbefore and after

Get 100% participation or

use statistical sample Follow-up as appropriate

8/2/2019 ID - Session 6

13/18

13

Behavior - What Is It?

Transfer of knowledge, skills,and/or attitude to the realworld

Measure achievement ofperformance objectives

8/2/2019 ID - Session 6

14/18

14

Behavior - What It Looks Like

Observe performer, first-hand

Survey key people who

observe performer Use checklists,questionnaires, interviews,or combinations

I believe coming in early ishelpful.

I work better on the newproduction system.

8/2/2019 ID - Session 6

15/18

15

Behavior - Howto Perform

Evaluate before andafter training

Allow ample time

before observing Survey key people Consider cost vs.

benefits 100% participation or a

sampling Repeated evaluations

at appropriate intervals Use of a control group

8/2/2019 ID - Session 6

16/18

16

Results - What Is It?

Assesses bottom

line, final results

Definition of resultsdependent upon thegoal of the trainingprogram

8/2/2019 ID - Session 6

17/18

17

Results - What

It Looks Like Depends upon objectives of

training program Quantify: Bottom line,

productivity, improvement

Proof vs. Evidence Proof is concrete Evidence is soft

Sample Questions: I believe the new system has

improved productivity.

My training has allowed me tobe more productive.

My boss is a better leaderafter my training.

8/2/2019 ID - Session 6

18/18

18

Results - Howto Perform

Use a control group

Allow time for resultsto be realized

Measure before andafter the program

Consider cost versus

benefits Be satisfied with

evidence when proof isnot possible