Comp 20 - Training & Instructional Design Unit 6 - Assessment This material was developed by...

Comp 20 - Training & Instructional Design

Unit 6 - Assessment

This material was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003.

Learning Objectives

Component 20/Unit 6Health IT Workforce Curriculum

Version 2.0/Spring 20112

Evaluation & Learner Assessment in Instructional Design

Component 20/Unit 6 3Health IT Workforce Curriculum


Purpose of Program Evaluation



The Evaluation Plan



The Evaluation Plan Cont’d



Develop a Logic Model



Simple Logic Model



Logic Model: EHR Training Workshop



Purpose of the Logic Model



Assessment & Evaluation Definitions



Writing Assessment Plans



Logic Model: EHR Training Workshop



Process & Formative Evaluation Questions



Process Evaluation Plan

Questions Collection Methods

Instruments

Amount of money & time

Receipts Proposed budgetTimesheets

Number of sessions delivered

Training curriculum

Program requirements

Number of persons trained

Attendance records

Attendance records

Population trained Attendance records

Job descriptions



Process Evaluation Plan Cont’d

Questions Collection Methods

Instruments

How well were the sessions delivered?

End of workshop evaluation

Self-report pencil and paper of online survey

What was the level of participation?

Number & quality questions askedComputer activity

Observation rubric

Were participants satisfied?

End of workshop evaluation

Self-report pencil and paper or online survey



Outcome Evaluation Questions



Outcome/Learner Assessment



Outcome/Learner Assessment Cont’d



Types of Learner Assessments

Tests or Quizzes

Specified increase in the number of correct responses as compared to a pretest or a specified number of students who are able to attain a targeted level of correct responses.

Observations Specific student behaviors, e.g. to accomplish a specific task

Student Writing Specific content knowledge or structural characteristics such as organization, development, or other specified features.

Oral Level of participation, specific content knowledge, specific thinking skills, or other indications of student progress as indicated by student comments.

Self Assessment

Student comments, written statements, visual indicators, or other indications from students regarding their progress towards the learning goals.



Using Rubrics



Sample Rubric



Outcome Evaluation Questions



Outcome Evaluation/Learner Assessment Plan

Questions Collection Methods Instruments

To what extent did knowledge of the system improved?

Written quizzesParticipation in discussions

Test rubricsDiscussion protocol

To what extent did skills & behaviors improve?

Performance task assessed in workshop, and later on the job

Performance task rubric

Which participants demonstrated a change in skills and behaviors?

Compile performance tasks results and group by job description and department

Personnel recordsPerformance task rubric



How to Use Evaluation Results



Communicating Evaluation Results



Summary



Comp 20 - Training & Instructional Design Unit 6 - Assessment This material was developed by...

Documents

Transcript of Comp 20 - Training & Instructional Design Unit 6 - Assessment This material was developed by...