1 Research on the Design of Technology-Based Training Richard E. Clark Rossier School of Education...

1

Research on the Design of Technology-Based

Training

Richard E. ClarkRossier School of Education

Center for Cognitive TechnologyUniversity of Southern California

[email protected]

ICT Immersive Training WorkshopMay 19, 2005

Marina del Rey California

mailto:[email protected]

2

Topics for Today’s WorkshopYou will be able to answer five questions:1. How does our cognitive architecture

influence learning from training?2. Do media influence learning? What are the

six principles for effective media-based training?

3. What new and promising training design models are out there right now?

4. What motivates us to learn and how can we influence it during training?

5. How can we decide whether training should be delivered in the classroom or by multi media?

3

Have you ever?

• Spaced out when you were studying a chapter in a book?

• Had difficulty memorizing lists • Noticed that you stopped listening to

someone who was giving you information and missed some of it?

• Had trouble remembering when you were under pressure?

4

I. How does our cognitive architecture influence learning from training?

1. We have a very limited thinking space, easily overloaded to protect us from scrambling our brain ( 3 +/- 1 information chunks/thoughts at one time)

– When overloaded, our brain short circuits and interferes with the mental resources needed to learn

– Novices most vulnerable – experts insulated– Overload is not noticed – may be enjoyable?

To overcome this limit we have two different but interacting knowledge systems

– Declarative (what, why, where, when) – Procedural (how)

5

I. Cognitive Architecture

2. Declarative system is information rich and helps us handle novelty and develop new procedures to achieve goals

– All new knowledge is declarative – about 10%– Knowledge stays flexible while it is being used to

achieve our goals– Once it is successful it gradually becomes fixed,

procedural, automated and unconscious – 90%• Types: concepts, processes and principles• Formats: Sensory Images, symbols

6


3. Procedural system operates outside of consciousness and so avoids 3 chunk limit on thinking

– Goal of nearly all training is to teach procedures – Consist of conditions (when, where) actions (what to

do) and decisions (alternatives and criteria for deciding)

– Expertise 90% automated & unconscious– Experts 50% inaccurate so “telling them how it is

for an expert” is not enough

7


4. Once we accept a new goal, our “learning system” is structured to select and/or develop a procedure for achieving it.

– General learning procedures include:• Plans to achieve the goal• Connections to relevant prior knowledge, • Ways to select the important and ignore the rest• Self monitoring for accuracy and results feedback• Procedures for using feedback to tune the plan

8


5. Our “learning system” requires external help to learn when information is relatively new

– Effective help procedures include:• Plans to achieve the goal (and clear goals)• Connections to relevant prior knowledge (examples)• Ways to select the important and ignore the rest• Self monitoring for accuracy and results feedback• Procedures for using feedback to tune the plan

9


5. Successful training provides clear goals and supports the learning strategies that trainees can’t or won’t provide for themselves.

– Provide a clear “3C” goal (concrete, current, challenging), motivate them to accept it and persist at achieving it

– Show them how to reach it with a “worked example”– Immerse them in a variety of realistic examples – Support their learning strategies as they develop

their own version of a procedure

10

II. Do media influence learning?1. Media influence the efficiency (cost, speed)

of learning but not whether someone learns.– Learning is due to instructional strategies and

the relevance of the information we provide.– More than one medium will result in comparable

learning of any task

2. Instructional strategies influence learning. They are external representations of internal cognitive strategies needed to learn

– Examples: Goals/objectives, learning plans, connections to prior experience (examples, analogies), practice exercises, progress evaluation, feedback

11

II. Do media influence learning?

3. Multi-media presentations often cause cognitive overload due to media features that split attention and require integration

– Caused by background music, irrelevant sounds, animated “help” characters, separated video and print explanations, narration over print etc.

4. Goal is to lower cognitive load and focus attention only on essential, goal-relevant information in multi media programs

How? Six Research Based Principles Next

12

Six Cognitive Principles about Media

1 Contiguity principle.– Integrate all visual and aural information -

use both together at the same time and in the same instructional space

2 Multiple modalities principle.– Visual and sound modalities better than

either alone – use both where possible

13

Spatial Contiguity PrinciplePeople learn better when corresponding words and graphics are placed near rather than far from each other on the screen.As a possible modification to the annotated illustrations, you might be tempted to separate the text and graphics. Your rationale might be that the learner can concentrate on the text and then concentrate on the graphics, and thus gain two exposures to the material. This strategy, however, violates the spatial contiguity principle.

When the driver steps on the car’s brake pedal, a piston moves forward inside the master cylinder. The piston forces brake fluid out of the master cylinder and through the tubes to the wheel cylinders. In the wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move. These smaller pistons activate the brake shoes. When the brake shoes press against the drum, both the drum and the wheel stop or slow down.

This is a screenful of printed text describing how brakes work.

See graphics

Return

When you click on “See graphics” you see these graphics depicting how brakes work.

Problem: When graphics are separated from the text, learners have difficulty in making connections between corresponding words and graphics.

14

Modality PrinciplePeople learn better from animation and narration than from animation and on-screen text.

A solution to the split attention problem is to present spoken words rather than printed words, thus reducing the load on the visual system.

“When the driver steps on the car’s brake pedal, a piston moves forward inside the master cylinder.”

“In the wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.”

“The piston forces brake fluid out of the master cylinder and through the tubes to the wheel cylinders.”

Each sentence is spoken at a slow pace as the animation shows what happens when the driver steps on the brake petal.

The words are spoken rather than printed on the screen.

15


3 Coherence Principle– Eliminate all extraneous words, sounds

(e.g. music) and pictures to reduce load– If information is not essential to achieve

the goal, drop it or risk distracting people

4 Narration principle– Narration is better than text in media

displays – people listen more effectively than they read. Natural voice narration is better than machine voice.

16

Coherence Principle People learn better from multimedia messages when extra words, pictures, and sounds are excluded rather than included.It may be tempting to embellish a presentation by adding background sounds or music. However, the added materials can distract the learner and disrupt the learner’s building of a causal chain.

“When you step on your car’s brake pedal, a piston moves forward inside your master cylinder.”

“In your wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.”

“The piston forces brake fluid out of your master cylinder and through the tubes to your wheel cylinders.”

To add interest, you are tempted to add background music such as aninstrumentalloop.

To add interest, you are tempted to add background sounds such as as swooshing sound of the piston

To add interest, you add background sounds such as as sound of flowing liquid.

To add interest, you add background sounds such as as mechanical sound of the piston moving.

The music continues.

The music continues.

17

“When you step on your car’s brake pedal, a piston moves forward inside your master cylinder.”

“In your wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.”

“The piston forces brake fluid out of your master cylinder and through the tubes to your wheel cylinders.”

To create a sense of social interaction, the narration is spoken in a non-accented human voice that conveys friendliness, intelligence, and energy.

The voice principle can be applied to conversational style as in this example, or to formal style.

Narration Principle People learn better from narration than text and better yet when the narration has a human voice

with a standard accent than a machine voice or an accented voice.Another possible way to create a sense of learning partnership is to use a standard accented human voice that the learners interpret as friendly, intelligent, and dynamic. The reason is that learners may try harder to understand when they accept the speaker as a social partner.

18


5 Reverse expertise principle– Most load principles needed for novice

and/or low spatial ability learners, not for experts (but lower load does not hurt experts)

6 Worked-example principle– Discovery problem-based learning

imposes high cognitive load with little benefit – use worked examples based on cognitive task analysis

19

Worked example: How to fix instructional materials that are classified as split attention risks.

1.Read the text material, and separate the text into segments where each segment describes or explains one area or element in the visual.

2. Identify the location in the visual that each segment of the text is discussing.

3. Decide wither the visual can be reformatted so that you can insert each segment of text immediately next to the part that it is discussing or whether you must draw arrows from the part being discussed to a text fragment that is placed as close as possible to the part?

4. Revise the visual so that text and visual are fully integrated.

20

Worked example: How to fix instructional materials that are split

attention risks.Test : To test Insulation Resistance from conductors to earth. How conducted : i )

Disconnect appliances and busways during these tests. Make sure main switch is "on" and all fuses are "in". Remove main earth from neutral bar and set meter to read insulation. Connect one lead to earth wire at MEN bar and take first measure by connecting the other lead to the active. Take next measure by connecting the lead to the neutral.ii) If resistance is not high enough in either of the two tests in i) then measure each circuit separately.Results required :i) At least One Mega ohm ii) Same result as i) above

21

Corrected example:

22

IV. What new and promising training design models are available?

Reminder: • Subject matter experts design most training• Expert’s knowledge is automated and

unconscious • When asked “how do you do that” expert

explanations are approximately 50% wrong• Less than 20% of what is learned in training

transfers back to performance• ISD models are suited to less complex

knowledge

23

New complex knowledge training design models: • Use Cognitive Task Analysis for accuracy• Mix declarative and procedural knowledge• Provide cognitive learning support• Provide “Part” and “Whole” task practice• Advanced assessment possibilities• Examples: vanMerrieboer’s 4C/ID and Clark’s

Guided Experiential Learning (GEL)

IV. What new and promising training design models are available?

24

How Is CTA Conducted?

Three or more experts interviewed separately:• Sequence of activities (before, during, after)• Conditions (when, cues, indications, contra

indications)• Actions (How to complete each step)• Decisions (When, Alternatives, Criteria)• Concepts, processes, principles operating

in this task – case study examples• Standards of performance• Equipment, materials

25

New 10 Stage Design System:(vanMerrienboer' s 4C/ID system)

1 Decompose the skill2 Sequence task

classes3 Design learning tasks4 Analyze mental

models - process5 Analyze cognitive

strategies6 Design supportive

information

7 List rules or procedures

8 Prerequisite knowledge

9 Design JIT information

10 Design part-task and whole task practice

26

A

B

C

analyze non -recurrent aspects

analyze recurrent aspects

analyze rules

analyze prerequisiteknowledge

analyzecognitivestrategies

analyzementalmodels

decomposethe complex skill

sequence skill clustersand/or task classes

A

B

C

analyze non -recurrent aspects

analyze recurrent aspects

analyze rules

analyze prerequisiteknowledge

analyzecognitivestrategies

analyzementalmodels

decomposethe complex skill

sequence skill clustersand/or task classes

27

Overview of GEL Design

Select Course& SME’s

Identify Many Job and Mission Problems

CognitiveTask Analysis

Identify• How to act and decide• New concepts, processes• Equipment and materials• Performance standards

Design• Lessons • Immersive demonstrations• Checklists for practice• Job aids for transfer• Guided practice exercises• Feedback for practice

Select delivery media

Design• Four level evaluation• Test of prior knowledge• Transfer letters

28

Using Guided Experience to Design Courses

Problem Selection and Examples

• Identify prior experience of trainees• Identify five large, authentic field problems

• Easy to very complex• Create “worked examples” of the problems

• Use cognitive task analysis• What are conditions, input and output•Solve for prior experience of trainees

Select Course& SME’s

Identify Many Job and Mission Problems

Cognitive Task Analysis

29

Conceptual Knowledge About Field Problems

• Collect information about• New concepts (definition and example)• Process (how it works – big picture)• Procedure (How to do it, conditions and

consequences)

Identify• How to act and decide• New concepts, processes• Equipment and materials• Performance standards

30

Design Lessons

• Sequence groups of problems into lessons• First performed in field are first taught• If no fixed sequence, easy before difficult

• Develop goals for each lesson• Remember definitions of concepts• Remember description of processes• Practice and perform procedure• Remember conditions and consequences


31

6 Instructional Methods for Each Lesson

Lesson Sequence:1. Goals –

• You will learn how to (REMEMBER, DO…)2. Reasons

• Value of learning - consequences of not learning

• What you already know that you should use3. What You Need to Know to Perform

• Teach new concepts and processes needed to learn procedure


32

Lesson Sequence:

4. Demonstrate procedure• Worked example – authentic setting• Instructor or Model should be credible, similar

5. Practice with incomplete problems• Trainees use partly worked example or demo• Practice more complex/difficult problems that

integrate previous learning• Gradually fade support – training wheels come

off!6. Feedback focused on strategy improvement


33

Select delivery media Design• Four level evaluation• Test of prior knowledge• Transfer letters

Media Selection,Evaluation and Transfer

• Select media based on context, practice and cost• Evaluate on four levels

1. Reactions (motivation - confidence, value)2. Learning (use practice exercises to evaluate)3. Transfer (check with supervisor/commander)4. Impact (did it make a difference to bottom

line?)• Send letter to commander asking for transfer

help

34

Assessment Possibilities

Stages in the development of expertise– Strategy assembly, gradual automation– Links between sub goals and explanation– Increasing speed with more practice– Reliable, valid measurement of expertise– Clear targets for corrective feedback– Connections between expertise and

measured performance outcomes

35

Evidence for Effectiveness?

Trainees learn faster, make fewer mistakes:• No differences on routine, directly

observable actions• Large differences on implicit, complex

mental strategies – Novices learn 25% faster (range 20-80%)– Made 70% fewer mistakes on cognitive

process– Mistakes of control groups are more serious

36

Evidence for Effectiveness?• Strategy sequence learned more quickly• Fewer and less important decision errors• More accurate explanations of decisions• Speed of automation increased• Better assessment of developing expertise • Coaching and feedback targets obvious• Learning and transfer increased but not

motivation

37

IV. What motivates us to learn and how can we influence it during training?

• Learning is 50% motivation• Three motivation outcomes:

– active choice, – persistence and – mental effort

• Active choice may be increased by multimedia

• Persistence may increase initially but it often decreases quickly

• Evidence that new media may decrease mental effort invested in learning

38

3. Design to support learner motivation

• Web persistence rates 50% less than classroom

• Problem with value for course content, social connections, poor planning and/or misperceived demands

• Apply motivation research (provide value, clear learning and performance goals, support confidence and planning)

39

V. Deciding whether training should be delivered live or by multi media?

1) Can application context be simulated on web?

2) Can timing, complexity of feedback be accommodated on web?

3) If “no” to 1) or 2), offer classroom instruction.

4) If 1) and 2) are “yes” Use ingredients method of cost/utility analysis – if web is cost beneficial, use it.

40

In Conclusion: Avoiding Misconceptions

• Start by analyzing problems to be solved with web-based courses

• Separate design AND development for learning AND motivation

• Use cognitive task analysis to capture expertise www.ctaresources.com

• Use worked examples and scaffolding to support constant, incremental practice

• Reduce cognitive load to the minimum when training complex knowledge

41

Research on the Design of Technology-Based

Training

Richard E. ClarkRossier School of Education

Center for Cognitive TechnologyUniversity of Southern California

ICT Immersive Training WorkshopMay 19 2005

Marina del Rey California

1 Research on the Design of Technology-Based Training Richard E. Clark Rossier School of Education...

Documents

Transcript of 1 Research on the Design of Technology-Based Training Richard E. Clark Rossier School of Education...