The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.
-
date post
15-Jan-2016 -
Category
Documents
-
view
215 -
download
0
Transcript of The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.
![Page 1: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/1.jpg)
The final resting place The final resting place for all this research…for all this research…
Ron Laughery, Ph.D.
University of Colorado
![Page 2: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/2.jpg)
Items to be covered…• What is the problem this
research is trying to solve from an operational perspective?
• What is the basic human performance modeling and simulation approach that this research will feed?
• What is the specific tool and architecture that we are working to advance?
• What are the issues in moving this research into practice?
![Page 3: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/3.jpg)
What is the problem this research is trying to solve from an operational perspective?
• The £5,000,000,000 question…– In about 1995, Robin Miller, an operational
analyst with the MoD asked us this question and made this statement at a meeting:
• “A question I get all the time can be summed up as this – should we invest £5B in new kit, or should we instead invest that £5B in training? If your models can’t help me answer that question, you’re not doing your job.”
• We are trying to ensure that we are doing our job in Mr. Miller’s eyes
![Page 4: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/4.jpg)
What is the basic human performance modeling and simulation approach that
this research will feed?• In military and civilian systems, decisions are
increasingly being made on the basis of model based analyses– System effectiveness depends upon…
SoftwareSoftware
HumansHumansHumans
HardwareHardware
![Page 5: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/5.jpg)
Two basic approaches to modeling human/system performance
• Reductionist– Breaking human activity and interaction
with the system into discrete activities
![Page 6: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/6.jpg)
Advantages/disadvantages of reductionist modeling approach
• Advantages– Intuitive– Level of detail determined by need– Basic data are usually available or easily obtained– Consistent with many military systems and
operational analysis models
• Disadvantages– Often requires extensive subject-matter expert
input
![Page 7: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/7.jpg)
Second approach to modeling human/system performance
• First principled/cognitive models– Based on theories of the underlying
mechanisms that facilitate human behavior
perception
Workingmemory
Long-termmemory
Iconicstorage
Centralprocessing
Responsemechanisms
![Page 8: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/8.jpg)
Example: ACT-R Representation and Equations
RetrievalGoal
ManualVisual
Productions
Intentions Memory
MotorVision
World
Ai =Bi + Wj ⋅Sji +σAj∑
Bi =ln tj−d
j∑
Ui =Pi ⋅G−Ci +σU
Pi =Succi
Succi +Faili
Ti =F ⋅e−Ai
Activation
Learning
Latency
Utility
Learning
IF the goal is to categorize new stimulus and visual holds stimulus info S, F, TTHEN start retrieval of chunk S, F, T and start manual mouse movement
S 20 1Size Fuel Turb Dec
L 20 3 Y
Stimulus
ChunkBi
SSL S13
![Page 9: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/9.jpg)
Advantages/disadvantages of the first principle approach
• Advantages– Requires less data input from either
experiments or subject matter experts– More first-principle based and, if component
models are valid, easier to defend
• Disadvantages– Model construction can be quite cumbersome
for simple tasks– We don’t have enough real first-principle
models of human performance
![Page 10: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/10.jpg)
A strategy that has worked- a hybrid approach
• The flexibility of reductionist models combined with the power of first principles of human behavior is the formula for success
perception
Workingmemory
Long-termmemory
Iconicstorage
Centralprocessing
Responsemechanisms
![Page 11: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/11.jpg)
Reductionist modeling with Task Network Modeling
• Largely involves the extension of a task analysis into a network defining sequencing
![Page 12: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/12.jpg)
Going from a task network to a running computer model
• Add timing information and task/system interdependencies
![Page 13: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/13.jpg)
Add human decision making strategies
• Any defined branch point represents a need for a decision
• Logic and rule sets, goal seeking, naturalistic
?
![Page 14: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/14.jpg)
Then, develop a scenario, equipment model and/or links to other simulations
![Page 15: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/15.jpg)
Run the model to collect human/system performance data
![Page 16: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/16.jpg)
Combining First Principles of human behavior with Task Network Models
• For the past 16 years, we have been embedding and linking first principle models of human performance into our tools including– Cognitive workload and human response– Micro models of human time and accuracy– Human error and system response to error– Performance shaping factor effects– Linkage to anthropometric, biomechanical models– Goal driven task scheduling– Naturalistic Decision Making– Situation awareness modeling– Integration of cognitive engineering models such as ACT/R
• Predicting training effects is still the Predicting training effects is still the weakest link!weakest link!
![Page 17: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/17.jpg)
Improved Performance Research Improved Performance Research Integration Tool (IMPRINT): Integration Tool (IMPRINT): Capability and ApplicationCapability and Application
![Page 18: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/18.jpg)
What Does IMPRINT Do?What Does IMPRINT Do? It helps you... Set realistic system requirements Identify future manpower & personnel constraints Evaluate operator & crew workload Test alternate system-crew function allocations Assess required maintenance manhours Assess performance during extreme conditions Examine performance as a function of personnel
characteristics, training frequency & recency Identify areas to focus test and evaluation resources
![Page 19: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/19.jpg)
IMPRINT Architecture - IMPRINT Architecture - Operations ModelingOperations Modeling
StochasticDiscrete Event
Simulation
StochasticDiscrete Event
Simulation
Task Time & Error Data -Estimates & Requirements
Unit &ForceUnit &Force
System MeasuresSystem
Measures
TaskLibraries
TaskLibraries
WorkloadFuture Manpower
Performance Shaping Functions- personnel
- training- stressors
WorkloadFuture Manpower
Performance Shaping Functions- personnel
- training- stressors
WorkloadFuture Manpower
Performance Shaping Functions- personnel
- training- stressors
![Page 20: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/20.jpg)
Improved Performance Research Improved Performance Research Integration Tool (IMPRINT)Integration Tool (IMPRINT)
![Page 21: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/21.jpg)
IMPRINT Architecture - IMPRINT Architecture - Maintenance ModelingMaintenance Modeling
Send systems on missions as defined by scenarioSimulate need for maintenance
Systems Readyfor Next Mission
Repair systems
Manpower Pool
corrective& continue
mission
combatdamage
corrective& stop
mission
preventive
Repair Parts
![Page 22: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/22.jpg)
Who Has IMPRINT?Who Has IMPRINT?
Army Navy Air Force Other Government Contractors University
108
23
9
12
108
19
279 and growing
![Page 23: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/23.jpg)
Mental WorkloadMental Workload
Degree of Resource Use?Which Brain Resources Involved?
Mission Tasks
1. monitoralarms
2. decideresponseaction
3. pull trigger
.
.
.n. task n
Visual
Cognitive
Auditory
Psychomotor
Cognitive
0.0 - No Cognitive Activity1.0 - Automatic (simple association)1.2 - Alternative Selection3.7 - Sign/Signal Recognition4.6 - Evaluation/Judgment (consider
single aspect)5.3 - Encoding/Decoding, Recall6.8 - Evaluation/Judgment (consider
several aspects)7.0 - Estimation, Calculation,
Conversion
Psychomotor
Auditory
Visual
Cognitive
0.0 - No Cognitive Activity1.0 - Automatic (simple association)1.2 - Alternative Selection3.7 - Sign/Signal Recognition4.6 - Evaluation/Judgment (consider
single aspect)5.3 - Encoding/Decoding, Recall6.8 - Evaluation/Judgment (consider
several aspects)7.0 - Estimation, Calculation,
Conversion
Psychomotor
Auditory
Visual
![Page 24: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/24.jpg)
Task type (Taxon*) MOPP Heat Cold Noise Sleepless Hours
Visual T A TNumerical A TACognitive A TAFine Motor Discrete T A TFine Motor ContinuousGross Motor Light T TGross Motor HeavyCommo. (Read & Write) ACommo. (Oral) T A A
T = affects task time, A = affects task accuracy, TA= affects both
Current IMPRINT Implementation: Current IMPRINT Implementation: Stressors by Task TypeStressors by Task Type
* O’Brien, L. H., Simon R. and Swaminathan, H. (1992). Development of the Personnel-Based System Evaluation Aid (PER-SEVAL) Performance Shaping Functions. ARI Research Note 92-50
![Page 25: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/25.jpg)
Approach to modeling human response to stressors
Performance multipliersas a function of time since sleep
6 12 18 24 30 36 42 48 54 60 66 700
0.2
0.4
0.6
0.8
1
1.2
time since sleep
performance multiplier
attention
perception
cognition
psychomotor
physical
The general effects...
Detect ring - 50% attention, 50% perception
Select menu item using a mouse - 40% attention, 60% psychomotor
Interpret customer’s request for information - 100% cognitive
On a specific task….
• attention performance multiplier = .82
• perception performance multiplier = .808
• cognition performance multiplier = .856
• psychomotor performance multiplier = .784
- physical performance multiplier = .727
under specificconditions...
leads to this specificeffect at this time…..
task time = 112.3% of normalTime to Prepare to Engage
at 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
![Page 26: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/26.jpg)
Use task network models to study aggregate effects of PSFs
Net Performance Op 6
0
2
4
6
8
10
12
14
16
18
20
100500900130017002100250029003300370041004500490053005700610065006900730077008100
Time (sec)
Wo
rklo
ad0
20
40
60
80
100
120
Uti
lizat
ion
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
![Page 27: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/27.jpg)
What view of training is in What view of training is in IMPRINT now?...IMPRINT now?...
![Page 28: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/28.jpg)
What we really need for a reasonably What we really need for a reasonably accurate representation of training…accurate representation of training…
• We need these functional relationships…– For different task types (the taxonomy)– For different “types” of training
Performance
Amount of training received
Classroomtraining
Simulatortraining
Fieldtraining
No training Retraining
![Page 29: The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649d4a5503460f94a273f1/html5/thumbnails/29.jpg)
Big questions…Big questions…
• Purpose of models– Design of optimal training systems– Design of systems considering training
• Taxonomies– Training environment– Task type
• Scope/complexity of tasks studied– Do small tasks scale to large tasks?
• How do we treat Retention