DDMLab – September 27, 2006 - 1 ACT-R models of training Cleotilde Gonzalez and Brad Best Dynamic...
-
date post
21-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of DDMLab – September 27, 2006 - 1 ACT-R models of training Cleotilde Gonzalez and Brad Best Dynamic...
DDMLab – September 27, 2006 - 1
ACT-R models of training
Cleotilde Gonzalez and Brad Best
Dynamic Decision Making Laboratory (DDMLab)
www.cmu.edu/ddmlab
Carnegie Mellon University
DDMLab – September 27, 2006 - 2
Our Goals in the MURI Project
• Create computational models that will be used as predictive tools for the different effects resulting from the application of empirically-based training principles
• The predictive training models will help:o manipulate a set of training, task and ACT-R
parameterso determine speed and accuracy as a result of the
parameter settings and the training principleso Easily generate predictions of training effects for
new manipulations
DDMLab – September 27, 2006 - 3
Agenda
• Prolonged work and the speed-accuracy tradeoff
o The data entry task
o ACT-R models of fatigue effects
• Repetition priming effect
o Initial ACT-R models of repetition priming
o Predictions to be verified in current data collection
• Training difficulty principle
o The radar task
o ACT-R models of consistent and varied mapping effects
DDMLab – September 27, 2006 - 4
Data entry is ubiquitous in human life
DDMLab – September 27, 2006 - 5
Opposing processes affect performance• From Healy et al., 2004: Prolonged work results in distinctive opposite effects
(facilitative and inhibitory)
• Proportion of errors increases while response time decreases.
DDMLab – September 27, 2006 - 6
The 2x2 levels of ACT-R http://act.psy.cmu.edu
(Anderson & Lebiere, 1998)
Declarative Memory Procedural Memory
Symbolic
Chunks: declarative facts
Productions: If (cond) Then (action)
SubSymbolic
Activation of chunks (likelihood of
retrieval)
Conflict Resolution (likelihood of use)
DDMLab – September 27, 2006 - 7
RetrievalGoal
ManualVisual
Productions
Intentions Memory
MotorVision
World
Ai Bi Wj S ji Aj
Bi ln t j d
j
Ui Pi G Ci U
Pi Succ i
Succi Faili
Ti Fe Ai
Activation
Learning
Latency
Utility
Learning
IF the goal is to categorize new stimulus and visual holds stimulus info S, F, TTHEN start retrieval of chunk S, F, T and start manual mouse movement
S 20 1Size Fuel Turb Dec
L 20 3 Y
Stimulus
ChunkBi
SSL S13
ACT-R equations http://act.psy.cmu.edu
(Anderson & Lebiere, 1998)
DDMLab – September 27, 2006 - 8
Chunk Activation
baseactivatio
n
activation
= +
Activation makes chunks available to the degree that past experiences indicate that they will be useful at the particular moment.
Base-level: general past usefulnessAssociative Activation: relevance to the general contextMatching Penalty: relevance to the specific match requiredNoise: stochastic is useful to avoid getting stuck in local
minima
Higher activation = fewer errors and faster retrievals
associativestrength
sourceactivation( )*
A i Bi Wj Sjij MPk Simkl N(0,s)
k
similarityvalue
mismatchpenalty( )*+ +noise
DDMLab – September 27, 2006 - 9
Production compilation
• Basic idea:
o Productions are combined to form a macro production faster execution
o Rule learning:
• Retrievals may be eliminated in the process
• Practically: declarative procedural transition
o Production learning produces power-law speedup
• The power-law function does not appear in the compilation mechanism, rather the power-law emerges from the mechanism
DDMLab – September 27, 2006 - 10
ACT-R Models of Fatigue Effects in Data Entry
• ACT-R Model 1: Prolonged work and the speed-accuracy tradeoff (Experiment 1 from Healy, Kole, Buck-Gengler and Bourne, 2004)
• ACT-R Model 2: Speed-accuracy trade-off changes for motoric and cognitive components (Experiment 2 from Healy, Kole, Buck-Gengler and Bourne, 2004)
• ACT-R Model 3: How do cognitive and motoric stressors affect different response time components: articulatory suppression and weight (Experiment 1 from Kole, Healy ad Bourne, 2006)
DDMLab – September 27, 2006 - 11
Encode next number
Retrieve key location
Type next number
Hit Enter
All numbers encoded
All numbers typed
Initiationtime
Executiontime
Conclusiontime
ACT-R model of the data entry task
DDMLab – September 27, 2006 - 12
ACT-R Model 1
Error Proportion Total RT
R2=.89 R2=.89
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
1 2 3 4 5 6 7 8 9 10
Block
Err
or
pro
po
rtio
n
Observed
Predicted
2.5
2.55
2.6
2.65
2.7
2.75
1 2 3 4 5 6 7 8 9 10
Block
To
tal
Res
po
nse
Tim
e (m
sec)
Oserved
Predicted
DDMLab – September 27, 2006 - 13
ACT-R model 1
Encode next number
Retrieve key location
Type next number
Hit Enter
All numbers encoded
All numbers typed
Initiationtime
Executiontime
Conclusiontime
• Speedup: Production compilation:
o From Visual Retrieval (key loc) Motor
o To Visual Motoro faster access to key
loc
• Accuracy ↓: Degradation of source activation (W):
A i Bi Wj Sjij MPk Simkl N(0,s)
k
DDMLab – September 27, 2006 - 14
• ACT-R Model 1: Prolonged work and the speed-accuracy tradeoff (Experiment 1 from Healy, Kole, Buck-Gengler and Bourne, 2004)
• ACT-R Model 2: Speed-accuracy trade-off changes for motoric and cognitive components (Experiment 2 from Healy, Kole, Buck-Gengler and Bourne, 2004)
• ACT-R Model 3: How do cognitive and motoric stressors affect different response time components: articulatory suppression and weight (Experiment 1 from Kole, Healy ad Bourne, 2006)
ACT-R Models of Fatigue Effects in Data Entry
DDMLab – September 27, 2006 - 15
ACT-R Model Accuracy
0.82
0.84
0.86
0.88
0.9
0.92
0.94
1 2 3 4 5 6 7 8 9 10
Block
Pro
po
rtio
n C
orr
ect
Observed
Predicted
R2=.68
DDMLab – September 27, 2006 - 16
ACT-R Model 2
1
1.02
1.04
1.06
1.08
1.1
1.12
1.14
1.16
1.18
1.2
1 2 3 4 5 6 7 8 9 10
Block
Init
iati
on
Tim
e (s
ec)
Observed
Predicted
Initiation Time Conclusion Time
R2=.85 R2=.81
0.24
0.25
0.26
0.27
0.28
0.29
0.3
1 2 3 4 5 6 7 8 9 10
BlockC
on
clu
sio
n T
ime
(sec
) Observed
Predicted
DDMLab – September 27, 2006 - 17
ACT-R model 2
Encode next number
Retrieve key location
Type next number
Hit Enter
All numbers encoded
All numbers typed
Initiationtime
Executiontime
Conclusiontime
• Speedup: Production compilation
o From Visual Retrieval (key loc) Motor
o To Visual Motoro faster access to key loc
o Gradual decrease in goal value (G)
• Accuracy ↓:
o A gradual decrease in source activation (W)
A i Bi Wj Sjij MPk Simkl N(0,s)
k
Uiii CGPU
DDMLab – September 27, 2006 - 18
• ACT-R Model 1: Prolonged work and the speed-accuracy tradeoff (Experiment 1 from Healy, Kole, Buck-Gengler and Bourne, 2004)
• ACT-R Model 2: Speed-accuracy trade-off changes for motoric and cognitive components (Experiment 2 from Healy, Kole, Buck-Gengler and Bourne, 2004)
• ACT-R Model 3: How do cognitive and motoric stressors affect different response time components: articulatory suppression and weight (Experiment 1 from Kole, Healy ad Bourne, 2006)
ACT-R Models of Fatigue Effects in Data Entry
DDMLab – September 27, 2006 - 19
ACT-R Model 3Total response time
2
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
1 2 3 4 5 6 7 8 9 10
Block
To
tal
Res
po
nse
Tim
e (m
sec)
Suppression
Silent
Human data ACT-R Prediction
DDMLab – September 27, 2006 - 20
ACT-R Model 3Proportion correct
0.78
0.8
0.82
0.84
0.86
0.88
0.9
0.92
1 2 3 4 5 6 7 8 9 10
BlockP
rop
ort
ion
Co
rrec
t
Suppression
Silent
Human data ACT-R Prediction
DDMLab – September 27, 2006 - 21
ACT-R model 3
Encode next number
Retrieve key location
Type next number
Hit Enter
All numbers encoded
All numbers typed
Initiationtime
Executiontime
Conclusiontime
• Same as Model 2:
• With articulatory suppression
• Not a ‘fit’ but a prediction
DDMLab – September 27, 2006 - 22
Summary of Act-R models of Fatigue
• The model provides detailed predictions of the speed and accuracy tradeoff effect with prolonged work in the data entry task:
o Decreased RT by production compilation
o Increase Errors by gradual decrease in source activation
• Fatigue may affect cognitive and motor components differently:
o strong effects of fatigue by gradual decrease in goal value
o no effects on motor components
• The ACT-R model suggest that cognitive fatigue may arise from a cognitive control and motivational processes (Jongman, 1998)
DDMLab – September 27, 2006 - 23
Agenda
• Prolonged work and the speed-accuracy tradeoff
o The data entry task
o ACT-R models of fatigue effects
• Repetition priming effect
o Initial ACT-R models of repetition priming
o Predictions to be verified in current data collection
• Training difficulty principle
o The radar task
o ACT-R models of consistent and varied mapping effects
DDMLab – September 27, 2006 - 24
Empirical test of model’s predictions(Gonzalez, Fu, Healy, Kole, and Bourne, 2006)
• Effects of number of repetitions and delay on repetition primingo How performance deteriorates with different delays after
training?o How re-training may help retention of skills?
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0 2 4 6 8 10 12 14 16 18
Number of days between end of training and testing
RT
dif
fere
nc
e (
Tra
inin
g-T
es
t)
no re-train
Re-train 1
Re-train 2
DDMLab – September 27, 2006 - 25
Agenda
• Prolonged work and the speed-accuracy tradeoff
o The data entry task
o ACT-R models of fatigue effects
• Repetition priming effect
o Initial ACT-R models of repetition priming
o Predictions to be verified in current data collection
• Training difficulty principle
o The radar task
o ACT-R models of consistent and varied mapping effects
DDMLab – September 27, 2006 - 26
Threat Sensors
Weapon System Sensors
Radar Grid
Quiet Airspace Report
Target Set (Memory Set)
Total Block Score
Guns Ignore
Missiles
F D
G J
The Radar Task(From Gonzalez and Thomas, under review;
Gonzalez, Thomas, and Madhavan, under review)
DDMLab – September 27, 2006 - 27
The Radar Task(From Gonzalez and Thomas, under review;
Gonzalez, Thomas, and Madhavan, under review)
DDMLab – September 27, 2006 - 28
Current data fits to human data:effects of mapping and load
RADAR: Model Latency at Training
0
200
400
600
800
1000
1200
1400
CM 1 - 1 CM 4 - 4 VM 1 - 1 VM 4 - 4
Condition
Res
pons
e T
ime
(ms)
Human
Model
DDMLab – September 27, 2006 - 29
ACT-R Model of automaticity
Focus on Next Frame
Attend to Next Target
CM: Target Same Type
Target Found
VM: Target Same Type
Try Retrieving
Not Target(Retrieval Failure)
Rehearse Memory Set
Respond: Target Found
Target(Retrieval Success)
Respond: Target Not Found
No More TargetsTarget Different Type
as MS
DDMLab – September 27, 2006 - 30
Consistent Mapping Conditions
Focus on Next Frame
Attend to Next Target
CM: Target Same Type
Target Found
Rehearse Memory Set
Respond: Target Found
Respond: Target Not Found
No More TargetsTarget Different Type
as MS
DDMLab – September 27, 2006 - 31
Varied Mapping Conditions
Focus on Next Frame
Attend to Next Target
VM: Target Same Type
Try Retrieving
Not Target(Retrieval Failure)
Rehearse Memory Set
Respond: Target Found
Target(Retrieval Success)
Respond: Target Not Found
No More TargetsTarget Different Type
as MS
DDMLab – September 27, 2006 - 32
Current model issues
• The model depends on knowing when to retrieveo If the target is the same type as the memory set, in CM
that means it’s the target, but in VM it may be a distractero When are participants aware of the condition they’re in?
• False alarm rates should indicate when participants think they’re in CM, but actually are in VM
o Adaptation should happen over time in VM false alarm rates
o Latency may increase in VM due to fewer skipped retrievals (participants may initially think they’re in VM and actually get the target without doing a retrieval)
DDMLab – September 27, 2006 - 33
Summary of this year’s accomplishments
• Generation of new ACT-R models to demonstrate the Training Difficulty hypothesis in the radar task
o effects of consistent and varied mapping with extended task practice
o Initial tuning of the model with experimental data collected
• Enhancement of ACT-R models and creation of new models that reproduce the speed-accuracy tradeoff effect for the data entry task
• Enhancement of current ACT-R models of repetition priming and depth of processing for the data entry task
• Initial development of a general model of ACT-R fatigue that can be applied to any existing model
DDMLab – September 27, 2006 - 34
Plans for next year• On the data entry task:
o Report on the cognitive functions and mechanisms corresponding the speed-accuracy trade off in prolonged work, based on ACT-R/empirical results
o Enhance and create the models corresponding to the repetition priming and depth of processing effects
• Generate a predictive training tool for the data entry task in which training, task, and ACT-R parameters can be manipulated to produce speed and accuracy results
• On the Radar task:
o Reproduce the effects of the difficulty of training hypothesis
o Produce new predictions on the effects of stimulus-response mappings
• Add learning to condition determination