1 System for Administration, Training, and Educational Resources for NASA Training Evaluations.
-
Upload
roland-merritt -
Category
Documents
-
view
213 -
download
0
Transcript of 1 System for Administration, Training, and Educational Resources for NASA Training Evaluations.
1
System for Administration, Training, and Educational Resources for NASA
Training Evaluations
2
Agenda
Lesson 1: Overview for Training Evaluation Lesson 2: Research Item Evaluation
Surveys Lesson 3: Gather and Analyze
Survey Results Lesson 4: Evaluate Learning Using Pre-
and Post-Test
3
Lesson 1Overview for Training Evaluations
4
Lesson Objectives
Identify all levels of evaluation and explain which evaluation tools you will use in SATERN.
Understand evaluation terms and definitions Explain how evaluations are implemented at
NASA Identify SATERN configurations needed to
support evaluations
5
Evaluating Training Results Using SATERN
Levels of Evaluation
6
How Evaluations are Implemented at NASA
A questionnaire survey will automatically be assigned to leaners after successful completion of some of the instructor-led course or an online module.
SATERN administrators can use their own discretion and decide whether to use Level 2 evaluations pre- and post-tests.
NASA will continue to use Metrics that Matter surveys. Administrators will not create or track follow-up evaluations in SATERN.
7
Configuration to Support Evaluations Completion Status Triggers Survey Assignment
Rating Scale Used for Surveys
StronglyDisagree
Disagree Neither Agreenor Disagree
Agree Strongly Agree
1 2 3 4 5
8
Lesson Wrap-Up Identified all levels of evaluation and explained
which evaluation tools you will use in SATERN. Understand evaluation terms and definitions Explained how evaluations are implemented at
NASA Identified SATERN configurations needed to
support evaluations
9
Lesson Check
10
Lesson 2Research Item Evaluation Surveys
11
Lesson Objectives Identify the purpose of an item evaluation Identify possible question types for a item
evaluation survey Research and preview item evaluation surveys
used in SATERN Determine configuration options for item
evaluation surveys Associate an item evaluation survey with an item
12
Item Evaluations Collect Data on Learner Satisfaction Assesses a learner’s reaction to a training event Most effective if delivered immediately after the
training event ends Sometimes referred to as a ‘smile sheet’ Measures how well learners liked the training
13
Question Types for Item Evaluations
Rating Scale
One Choice
Multiple Choice
Open Ended
14
Lab 1
Research and Preview a Published
SATERN Survey
15
Configuration Options for Level 1 Surveys
16
How to Associate a Survey with an Item
There are two ways to make an association between an item and a survey:• Item Usage tab of
questionnaire survey
• Evaluations tab of item record
17
Lab 2
Method 1: Associate an Item
with a Survey
18
Lab 3
Method 2: Associate a
Survey within an Item
19
Lesson Wrap-Up
Identified the purpose of an item evaluation Identified possible question types for a item
evaluation survey Researched and previewed item evaluation
surveys used in SATERN Determined configuration options for item
evaluation surveys Associated an item evaluation survey with an
item
20
Lesson Check
21
Lesson 3Gather and Analyze Survey Results
22
Lesson Objectives Launch and complete assigned surveys from the
learner side of SATERN Check learners’ progress from a survey record Run reports from the survey record Run item evaluation reports from the reports tab Remove item evaluations from learner records
23
How Learners Complete Surveys
An alert displays on their SATERN home page
A learner is also sent a notification If an answer is missed, a warning message displays to inform the learner to complete the question Learners can save the survey and return later to continue editing
24
Lab 4
Launch and Complete an
Assigned Survey
25
Check Progress from the Survey Record
26
Reports Related to Evaluations
Item Evaluation Report Item Evaluation by Individual Response Report Item Evaluation by Instructor Report
27
Lab 5
Run an Item Evaluation Report
28
Removing Outdated Surveys
The best way to capture accurate results is to have learners complete the survey soon after training. If a considerable amount of time has passed without survey completion, you may want to remove the survey from the learner record.
29
Lab 6
Method 1: Remove Surveys via
Learner Needs Management
30
Lab 7
Method 2: Remove Surveys Directly from the Learner
Record
31
Lesson Wrap-Up
Launched and completed assigned surveys from the learner side of SATERN
Checked learners’ progress from a survey record Ran reports from the survey record Ran item evaluation reports from the Reports tab Removed item evaluations from a learner record
32
Lesson Check
33
Lesson 4Evaluate Learning Using Pre- and Post-Tests
34
Lesson Objectives Explain the purpose of Level 2 evaluations Associate a pre-test and post-test as Level 2
evaluations
35
Level 2 Learning Evaluations
Purpose of a pre- and post-exam is to assess the learner population’s knowledge of the content prior to the training, and then just after the training.
If the training program is effective, there will be an increase in knowledge when the two exam results are compared
36
Lab 8
Associate Pre- and Post-Exams with
an Item
37
Lesson Wrap-Up
Explained the purpose of Level 2 evaluations Associated a pre-test and post-test as Level 2
evaluations
38
Lesson Check
39
Course Summary Identified all levels of evaluation and explained
which evaluation tools are used in SATERN Understand evaluation terms and definitions Explained how evaluations are implemented at
NASA Identified SATERN configurations needed to
support evaluations