Hosted by Lambda Solutions and Limestone Learning

Measuring the Impact of eLearningTurning Kirkpatrick’s Four Levels of Evaluation Upside Down

E M P O W E R TA L E N T

Your Hosts

Sean HouganMarketing Coordinator

Paula YunkerManaging Partner &

Instructional Designer

Agenda

1. Lambda Solutions and Limestone Learning

2. Why learning evaluation is important.

3. Kirkpatrick’s four levels of evaluation explained.

4. Aligning learning to organizational goals.

5. Typical challenges implementing evaluation in an organization.

6. Practical strategies for implementing learning evaluation.

7. Favorite learning evaluation resources.

8. Q & A

• Our approach

• Clients:

Totara & Moodle are 80% more cost

effective

Managed cloud hosting reduces

your IT operating costs by more than

40%

Over 12 years of experience and 600 customer

implementations

We Reduce The Cost Of Learning

Industry Leading Clients

Poll #1

What do you hope to gain from this webinar?

Why Learning Evaluation Is Important?

A Little History First…

Kirkpatrick’s Four Levels Of Evaluation

Level 1 - Reaction

• Measures to what

degree learners react

favorably to the

learning event.

• The “smile sheet”.

Poll #2

Yes or no? Do you use level 1 learning evaluation?

Level 2 - Learning

• Measures to what degree learners acquire the

intended knowledge, skills and attitudes based

on their participation in a learning event.

Poll #3

What level 2 learning evaluation do you use in your training courses/programs?

• Pre-evaluation (formative)

• During the learning event (formative)

• At the end of the learning event (summative)

• Level 2 isn’t used.

Level 3 - Behaviour

• Measures to what

degree learners apply

what they learned

during training when

they are back on the

job.

Poll #4

Yes or no? Do you or your organization use level 3

learning evaluation?

Level 4 – Results

• Measures to what

degree targeted

business outcomes

occur as a result of the

learning event(s).

Poll #5

Yes or no? Do you or your organization use level 4

learning evaluation?

Level 5 - ROI

Turning The 4 Levels Upside Down

Aligning Learning To Organizational Goals

Strategies For Aligning Learning

• Step 1: Identify the business need.

• Step 2: Create the learning plan

• Step 3: Measure the business results.

Step 1: Identify The Business Need

Ask your key stakeholders and business partners:

• What desired results do you want to realize?

• What are the goals & business outcomes ?

• What needs to change to meet those goals & business outcomes?

Step 1: Identify The Business Need (cont.)

• What KPIs should change?

• What needs to be learned to make those changes happen?

• What will success look like to you?

Step 2: Create The Learning Plan

Develop a learning and evaluation plan to meet the learning requirements based on step 1 answers:

• What needs to be learned to achieve business outcomes?

• What required KPAs are needed?

• What learning experiences and practice are needed to become proficient?

Step 2: Create The Learning Plan (cont.)

• When and how will these learning experiences and practice be best accomplished?

• How and when will the learning be measured?

• How will learning be applied to the job and how will that application be reinforced on the job?

Step 3: Measure The Business Results

To identify how learning aligns to business outcomes and meets stakeholder/business partners needs:

• What changes in business results & key performance indicators have been realized from the training program?

Step 3: Measure The Business Results (cont.)

• For every dollar invested in training how many dollars does the organization get in return? Is the ROI significant enough to justify training?

• What key intangible benefits (soft data) have been gained?

Poll #6

What are some examples of soft data?

Challenges Implementing Evaluation

Typical Challenges

• Lack of buy-in from the top.

• Ownership.

• Unable/unwilling to measure and track evaluation on the job.

• Lack of reinforcement on the job.

Strategies For Implementing Evaluation

Level 4 Implementation Strategies

• Identify what training has the great impact.

• Determine how important the need is to demonstrate value.

• Involve stakeholders/ business partners

• Determine who is responsible?

• Start gathering data early.

Level 3 Implementation Strategies

• Design relevant follow-up questions.

• Administer 3 months post-training event individually, in groups, by survey.

• Ask similar questions to others to get a fuller “picture”.

• Create a Learning Action Plan.

• Learning contract between the learner and the supervisor/manager.

• Create pre-evaluation that aligns to learning content/objectives.

• Design a variety of informal formative evaluation tools; at least one per lesson/objective.

• Design an end-of-course summative evaluation; use a checklist or rating scale to evaluate learning.

Level 2 Implementation Strategies

• Go beyond a “smile sheet”.

• Focus questions on the learning experience.

• Make evaluation statements learner-centered.

• Create questions that align with higher evaluation levels.

• Consider using a 7, 9 or 11 point rating scale; only label each end of the scale.

Level 1 Implementation Strategies

Favorite Learning Evaluation Resources

• Evaluation websites.

• Books/reference material.

• Kirkpatrick Evaluation LinkedIn group.

• Training and certification.

• Industry and best practice research sites.

• Limestone Learning blog.

Questions/Comments

info@LambdaSolutions.net1-877-700-1118

pyunker@limestonelearning.com1-888-463-8489

Paula YunkerManaging Partner

Sean HouganMarketing Coordinator

TOLL FREE +1.877.700.1118

EMAIL info@lambdasolutions.net

CONTACT US lambdasolutions.net/contact-us