Post on 29-Aug-2014
description
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Human Error
Anna L Cox | University College London | anna.cox@ucl.ac.uk
overview
• a basic taxonomy of human error• theory: memory for goals• factors that increase error• reducing error
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Why study human error?
• practically: to avoid catastrophic accidents.
• theoretically: to predict when one errs.•Cog Sci: it’s the same system that
produces correct behaviour most of the time! Tells us a lot about human cognition.
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Beatson Oncology Case • LN, age 15• chemotherapy & 19 of 20 or 21 planned
radiotherapy treatments•Because the tumor type, location, and
extent and the patient size, age, and medical condition vary, the treatment for each patient is unique
• Each dose 58% HIGHER than it should have been….and no one noticed
• Scaling factor wasn’t entered
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Blame• "the error ... was procedural and was not associated in any way with faults or deficiencies in the Varis 7 computer system" (Johnston, 2006; ii).
• “... at no point in the investigation was it deemed necessary to discuss the incident with the suppliers if this equipment since there was no suggestion that these products contributed to the error.” (Johnston, 2006; 2)
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Accidents due to Human Errors
0%
20%
40%
60%
80%
100%
Petrochemicalplants
Medicine Worldwide Jetcargo
US nuclearpower plants
Automobiles Air trafficcontrol
Human Error Others
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Annual Death Rates in US
329 395914986
43649
98000
0
50000
100000
commericalaviationdeaths
drowningdeaths
deaths fromfalls
motor vehicledeaths
deaths frommedicalerrors
One jumbo jet crash every day
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Denise Melanson
• infusion pump containing a four day dose of two chemotherapy drugs, 5-fluorouracil and cisplatin, to administer to herself at home.
• pump dispensed the dose in four hours and not four days
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Blame
• The nurse incorrectly calculated the dose•A second nurse checked the calculation
but didn’t notice the error• The drug bottle first displayed the per
24hr dose, (and then, in brackets the per 1hr dose)
Anna L Cox | University College London | anna.cox@ucl.ac.uk
OptiClik
Anna L Cox | University College London | anna.cox@ucl.ac.uk
JB’s video
http://www.youtube.com/watch?v=Hm7k0TRaPHI
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Human Errors
Slips
A system fails to achieve theintended outcome in a planned sequence
of mental or physical activities
Incorrect execution of a correct action sequence
Correct execution of an incorrect action sequence
Mistakes
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Human Errors
Slips Mistakes
ExecutionSlips
EvaluationSlips
KnowledgeBased
RuleBased
• ---------• ---------• ---------
• ---------• ---------• ---------
• ---------• ---------• ---------
• ---------• ---------• ---------
Anna L Cox | University College London | anna.cox@ucl.ac.uk
System Activities
User Activities
Physical System
Goal
Delete file
IntentionUse delete key
ActionSpecification
Select file + hit delete”
ExecutionUse mouse and keyboard
to perform action
sequence
Evaluation Form sub-goal
Interpretation Nothing happened
Perception Screen doesn’t change
Norman’s Action Theory
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Mistakes
User Activities
System Activities
Goal
Physical System
Execution
SpecificationAction
Intention Evaluation
Interpretation
Perception
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Mistakes
Knowledge-Based Rule-Based
• Faulty conceptual knowledge• Incomplete knowledge
• Biases and faulty heuristics• Incorrect selection of knowledge
• Information overload
• Misapplication of good rules
• Encoding deficiencies in rules
• Action deficiencies in rules
• Dissociation between knowledge and rules
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Slips
User Activities
System Activities
Goal
Interruptions
Execution Slips Evaluation Slips
Intention
SpecificationAction
Execution
Physical System
Evaluation
Interpretation
Perception
Anna L Cox | University College London | anna.cox@ucl.ac.uk
“I intended to pick up my prescription on my way home. I
drove home directly.”
Capture Slip: automatic activation of a well-learned routine that overrides the current intended activity .
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Double Capture Slip: The unintended activation of a related strong action schema.
“I meant to take off only my shoes, but took my socks off
as well.”
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Omission Slip: due to interruptions.
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Loss of Activation Slip
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Description Slip: incomplete or ambiguous specification of intention that is similar to a familiar intention.
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Associative Activation Slip: activation of similar but incorrect schemas.
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Perceptual Confusion Slip
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Repetition of Action Slip: repetition of an correctly performed action.
A nurse repeated radiation therapy
to a patient three times in a row, due to poor feedback. The patient died
three months later.
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Cross-talk Slip (concurrent): action components are exchanged between two or more concurrent actions.
Physical Systems
Goal
Intention
ActionSpecification
Execution
Evaluation
Interpretation
Perception
Physical Systems
Goal
Intention
ActionSpecification
Execution
Evaluation
Interpretation
Perception
English writing Dutch reading
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Cross-talk Slip (sequential): action components are exchanged between two or more sequential actions.
Physical Systems
Goal
Intention
ActionSpecification
Execution
Evaluation
Interpretation
Perception
Physical Systems
Goal
Intention
ActionSpecification
Execution
Evaluation
Interpretation
Perception
“I had just finished talking on the phone when my secretary ushered in some visitors. I got up from
behind the desk and walked to greet them with my hand outstretched saying ‘Smith speaking’.”
Talking on the phone (previous) Greeting (current)
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Applying Norman’s Action Cycle to Error Classification
• Errors are sometimes made when programming infusion pumps
• These errors might occur when entering numbers
•What causes these errors and when to they occur?
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Number Entry Error Taxonomy
Anna L Cox | University College London | anna.cox@ucl.ac.uk
#errordiary
•We’re tweeting when we make an error•Have a go at classifying these errors
using Norman’s action cycle• Tweet when you make an error
Anna L Cox | University College London | anna.cox@ucl.ac.uk
issues•Norman relied on self-report data of errors- gives a sense of common everyday errors- no reliable estimate of frequency of error- cannot determine cause of error
•move to the lab- difficulty is that errors are normally infrequent- this brings about issue for how to operationalize design and conduct statistical analysis - self-monitoring- slip errors occur with routine, procedural tasks - critical to train participants to criterion
Anna L Cox | University College London | anna.cox@ucl.ac.uk
post-completion error• routine, procedural task- goal-directed task- many subtasks
• post-completion step- the final step in a procedure- completed after the goal is achieved
• post-completion error (PCE)- missing the post-completion step- US ATM, photocopying, filling petrol, etc
Anna L Cox | University College London | anna.cox@ucl.ac.uk
example: photocopying
Anna L Cox | University College London | anna.cox@ucl.ac.uk
memory for goals
• goals are declarative memory representations defined by activation• most active goal directs behavior• activation declines over time, rehearsal can strengthen goal activation• associative links between goals propagate activation through network• completing ‘make copies’ lessens activation of ‘remove original’
Time
Threshold
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Working memory capacity & workload
Anna L Cox | University College London | anna.cox@ucl.ac.uk
experiment•memory for goals predicts that working memory load should mediate likelihood of error
• seminal study: Byrne & Bovair (1997) - working memory capacity: with larger capacity the goal can be actively rehearsed, making error less likely- memory load: with increased load the main goal will decay more rapidly so high chance of error because post-completion step less likely to reach activation threshold
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Byrne & Bovair’s results
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Interruptions
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Interruptions in the real world• Nurses are frequently
interrupted during their duties
• 20% of the time they don’t go back to the task they were working on when they were interrupted
• Sometimes it’s because that task is redundant
• Sometimes it’s because they’ve forgotten
• What are the implications for patient care?
Anna L Cox | University College London | anna.cox@ucl.ac.uk
•Could argue that she wasn’t paying attention
• She was blamed for not looking at the device
•But what about when we do direct our attention to the device again
•Does an interruption have any effect on what happens next?
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Anna L Cox | University College London | anna.cox@ucl.ac.uk
effect of interruption• Li et al. (2008): multi-step procedural task making doughnuts with purposeful post-completion step
• interrupted to switch tasks to pack doughnuts
Anna L Cox | University College London | anna.cox@ucl.ac.uk
effect of interruptioner
ror r
ate
Interruption position
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Reducing errors (I)
• cues
Anna L Cox | University College London | anna.cox@ucl.ac.uk
reducing errors
• system redesign- post-completion errors can often be designed out - e.g., ATM: card then cash vs. cash then card- but not always feasible for large complex systems
• provide explicit cues- must be visually salient, just-in-time, and meaningful- cues that are not specific are ineffective- habituation is a concern
Anna L Cox | University College London | anna.cox@ucl.ac.uk
visual cueing: Chung& Byrne (2008)
Anna L Cox | University College London | anna.cox@ucl.ac.uk
visual cueing, revisited
Anna L Cox | University College London | anna.cox@ucl.ac.uk
cue effectiveness
Anna L Cox | University College London | anna.cox@ucl.ac.uk
But…
•Not necessarily the perfect design solution
• The ‘just-in-time’ aspect can make it hard to design in
•And people become habituated to cues that are in the world all the time and so they stop working
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Reducing errors (II)
• Slow down!
Anna L Cox | University College London | anna.cox@ucl.ac.uk
effect of interruptionEr
ror
Rate
(%
)
Interruption Positiondata from Li, S.Y.W., Blandford, A., Cairns, P., & Young, R.M. (2008). Journal of Experimental Psychology: Applied, 14, 314-328.
Anna L Cox | University College London | anna.cox@ucl.ac.uk
distribution of errors
•working on routine task
•interruption occurs in between subtasks
•on resumption most common error is to repeat or skip a step
•this is because of competition between goals at time of retrieval
data from Trafton, J. G., Altmann, E. M., & Ratwani, R. M., (2009). A memory for goals model of sequential action. International Conference of Cognitive Modeling, 2009.
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Speed-accuracy tradeoff• some errors are caused by failure of memory
•memory is sensitive to changes in speed-accuracy tradeoff criterion
• error rate should be reduced if people invest time in thinking about where they were before resuming
•we use an enforced lockout procedure
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Anna L Cox | University College London | anna.cox@ucl.ac.uk
lockout reduces error rateEr
ror
rate
(%
)
Condition
Anna L Cox | University College London | anna.cox@ucl.ac.uk
interruption-only trialsFr
eque
ncy
Resumption time (seconds)
Anna L Cox | University College London | anna.cox@ucl.ac.uk
find out more
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Lockouts in design?
•What would YOU do if you were locked out of a system after an interruption?
Anna L Cox | University College London | anna.cox@ucl.ac.uk
implications for design• errors will occur, even with skilled users• errors reduced by good system design- well structured tasks- few interruptions- low memory demand- easy to select buttons- salient display (particularly mode indicators)- certain visual cues can be useful
• aid recovery from errors- make actions reversible- make the results of each action apparent
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Summary
• Slip errors occur infrequently – but are persistent• Increasing speed => increasing error rate• Interruptions => increasing error rate
• …err what’s it like on a hosptial ward??
• Cues: have to be “just-in-time” and very aggressive
• Take your time – slow down!
Anna L Cox | University College London | anna.cox@ucl.ac.uk
reading• core
- Byrne, M. D., & Bovair, S. (1997). A working memory model of a common procedural error. Cognitive Science, 21, 31-61.- Norman, D.A. (1981). Categorization of action slips. Psychological Review, 88, 1-15.
• supplementary- Chung, P.H., & Byrne, M.D. (2008). Cue effectiveness in mitigating
postcompletion errors in a routine procedural task. International Journal of Human-Computer Studies, 66, 217-232. - Li, S.Y.W., Blandford, A., Cairns, P., & Young, R.M. (2008). The Effect of interruptions on postcompletion and other procedural errors: An account based on the activation-based goal memory model. Journal of Experimental Psychology: Applied, 14, 314-328.- Ratwani, R.M., McCurry, J.M., & Trafton, J.G. (2008). Predicting postcompletion errors using eye movements. In Proceedings of CHI’08 (pp. 539-542). New York, NY: ACM Press.- Reason, J. (1990). Human Error. Cambridge University Press.
Anna L Cox | University College London | anna.cox@ucl.ac.uk
Microwave Racing video
http://www.youtube.com/watch?v=Bzy5hVvbei8