Post on 24-Dec-2015
The Psychology of Avoiding Disaster Readiness Disasters
Robin Dillon-Merrill
Catherine H. Tinsley
The McDonough School of Business
Georgetown University
Precursors to Catastrophes
People are confronted by the same threats year after year: Hurricanes along the Southeastern coast Floods and tornados in the Midwest Wildfires in the West Mud slides & earthquakes in California
When catastrophes occur that were preceded by near miss events, the question becomes: Were near miss events ignored?
Anecdotal Evidence
Governor Haley Barbour of Mississippi: “hurricane fatigue”He feared that his constituents were not evacuating in response to the Katrina
threat because they had successfully weathered earlier storms.
A former FEMA official described an agency that was responding to “business as usual”(i.e., treating Katrina like past hurricanes)]
Individual statements: “I survived Camille; my house is sturdy; I am staying put”
Organizational decision making: “This is how we have responded to hurricane warnings in the past.” [1] Quotes from The Washington Post, September 11, 2005 pp. A6-A7
Gap in Current Disaster Research
Research has shown that the level of preparedness is significantly linked to personal experience with disasters (Lindell and Perry, 2000, Wenger, 1980, and Dooley, et al., 1992).
But these experiences can either lead to greater awareness and preparedness or to greater complacency and fatalism offering no conclusions as to why the variation exists(Tierney, Lindell, and Perry, 2001, Jackson, 1981, and Mileti and O’Brien,
1992),.
It is precisely their interpretations of the outcomes of prior disasters and why these outcomes unfolded that will influence their subsequent perceptions of, and preparations for, future disaster events (Lindell and Perry, 1992).
Opportunities for New Orleans to have Learned Prior to Katrina
Hurricane Ivan- 2004: cat 4-5 (140-155 mph winds)- Predicted 25% chance stay on direct track to New Orleans
(actual landfall in Mobile Bay, Alabama 2am Sept. 16)- By noon Sept. 15 (when storm turned) estimated 600,000
out of 1.2 million evacuated New Orleans- 2/3 of non-evacuees (with means to evacuate) didn’t
evacuate because they felt safe in their homes. Others were discouraged by negative experiences with past evacuations
- 120,000 NO residents did not have cars- Superdome was used to shelter non-evacuees
Opportunities for New Orleans to have Learned Prior to Katrina
Hurricane Pam Simulation conducted July 2004- 8 day table-top exercise with over 250 officials participating- Assumed 120 mph (cat 3) slow moving storm- Assumed more than 1 million evacuated- Recognized that the levees would be overtopped- Recognized the need to rely on state resources for shelters
for 3-5 days- Focused recommendations for managing the aftermath of
the catastrophe (i.e., search & rescue, debris removal, etc.) rather than for minimizing the magnitude of the catastrophe (i.e., improving evacuation and sheltering strategies remained open issues)
- A second exercise in summer of 2005 didn’t take place because of lack of funding
Precursor’s Influence
Decision Makers attend to near-misses Near-miss information is incorporated into decision calculus Near-misses will systematically bias decision making Towards more risk
Near-misses can be evidence of a system’s vulnerability or of a system’s resilience
Resilience > Vulnerability Good Fortune is Discounted
What is a Near-Miss?
Near-miss An event that has some probability of a negative (even
fatal) outcome and some probability of a positive (safe) outcome, and the actual outcome is non-hazardous
A success that could have been a failure except for good luck
0
0.1
0.2
0.3
0.4
0.5
0.6
0 2 4 6 8 10
Severity Index
Pro
ba
bil
ity
Load (fL(x)) Capacity (fC(x))
SUCCESS SUCCESS
NEAR MISSOr HIT
L1
L2
L3
LMAX
C1
C2
C3CMIN
Definitions:
Lx < CMIN = Success
Cx > LMAX = Success
CMIN < Lx < LMAX ANDLx > Cx = Hit
CMIN < Lx < LMAX ANDLx < Cx = Near-miss
Examples:
L1 < CMIN : SUCCESS
C3 > LMAX : SUCCESS
L3 > C1 : HIT
L2 < C2 : NEAR-MISS
L2 < C1 : NEAR-MISS
L3 < C2 : NEAR-MISS
What is a Near-Miss?
First Studies
Simulation of a Mars Rover mission Limited battery life (8 days) 5 travel days to destination Rewarded $5 extra dollars for each battery day extra
Weather forecast for each day Mild weather or 95% chance of severe storm
Severe dust storms can cause catastrophic failure 40% catastrophic failure if drive through severe storm 100% safe if stop & deploy wheel guards
Operational decisions (stop/ go) for day 6-~13 Decide to drive or stop & deploy wheel guards
Manipulation check, risk propensity, and engagement
Manipulation
Near-Miss Of 5 days before you started operating the rover, had 3
days of severe storms and rover had driven successfully through these
Of 5 days before you started operating the rover, all mild weather
Results-- NASA
Drive on Day 6 Stop on Day 6 Total
Near-miss condition Count 6 5 11
Expected Count 3.1 7.9 11
Residual 2.9 -2.9
Row % 54.5% 45.5% 100%
No near-miss Count 0 10 10
Expected Count 2.9 7.1 10
Residual -2.9 2.9
Column % 0% 100% 100%
Total Count 6 15 21
Row% 28.6% 71.4% 21
Results-- Students
Drive on Day 6 Stop on Day 6
Total
Near-miss condition Count 22 4 26
Expected Count 16.8 9.2 26
Residual 5.2 -5.2
Row % 84.6% 15.4%
No near-miss Count 11 14 25
Expected Count 16.2 8.8 25
Residual -5.2 5.2
Row % 44% 56%
Total Count 33 18 51
Row% 64.7% 35.3%
Results Experiment 2 Those who USED probability information
Drive on Day 6 Stop on Day 6 Total
Near-miss condition Count 7 19 26
Expected Count 9 17 26
Residual -2 2
Row% 20.8% 79.2%
No near-miss Count 12 17 29
Expected Count 10 19 29
Residual 2 -2
Row % 41.4% 58.6%
Total Count 19 36 55
Expected Count 19 36 55
Row% 31.2% 67.9% 100%
Results: Experiment 2 Those who did NOT use probability information
Drive on Day 6 Stop on Day 6 Total
Near-miss condition Count 18 10 28
Expected Count 13.8 14.2 28
Residual 4.2 -4.2
Row % 68.8% 31.2%
No near-miss Count 11 20 31
Expected Count 15.2 15.8 31
Residual -4.2 4.2
Row % 35.5% 64.5%
Total Count 29 30 59
Expected Count 29 30 59
Row % 52.4% 47.6% 100%
Second Studies
Given biases exist, how does that influence how managers are evaluated within an organization?
Failures and successes are attributed to poor decision making
Is there another variable? Loma Prieta 1989 (Friday afternoon rush hour) Northridge 1994 (4:30 am on a holiday)
If all outcomes are a function of decision quality and luck, how do we evaluate others’ decision processes?
Outcome Bias (Baron & Hershey, 1988, Allison, et al., 1996)
The outcome systematically influences people’s evaluations of the quality of the decision making
Hindsight bias (Fischoff, 1982) Anchor on outcomes Exaggerate what could have been anticipated at time of
decision Misremember one’s own predictions to be consistent with now-
known outcomes
Suggest we will anchor on outcomes
Biases in Decision making
Hypothesis 1
H1a: Managers whose decisions result in a miss (organizational success) will have their decision making evaluated in a significantly more favorable light than managers whose decisions result in a hit (organizational failure)
H1b: Managers whose decisions result in a miss (organizational success) will be judged to be more competent, to be more intelligent, to have more leadership ability, and to be more promotable than managers whose decisions result in a hit (organizational failure)
What happens with near-misses?
• Recall that a near miss is both• Evidence of a system’s resilience• Evidence of a system’s vulnerability
• And what if we know the outcome was derived, in part, from good luck?
• Prospect theory: reference points
• Norm theory: • Immutable features give you class of events to categorize something• Mutable features (easily imagined as different) give you contrast events• What is easily imagined mutable feature?• Failure• Thus near miss = miss and near miss contrasted with failure
• Suggests near-misses more likely to be coded as successes than as failures• Suggests we will discount other’s good luck
Hypothesis 2
H2a: Managers whose decisions result in a near-miss will have their decision making evaluated more favorably than managers whose decisions result in a hit and judged less favorably than managers whose decisions result in a miss.
H2b: Managers whose decisions result in a near-
miss will be judged more competent, more intelligent, to have more leadership ability, and to be more promotable than managers whose decisions result in a hit and judged less competent, less intelligent, to have less leadership ability, and to be less promotable than managers whose decisions result in a miss.
Hypothesis 3
H3: Managers whose decisions result in a near-miss will be judged closer to those whose decisions ended in a miss than to those whose decisions ended in a hit.
Method
Case study loosely based on development details from past unmanned NASA missions
Development problems Challenges interacting across NASA development centers A skipped peer review Mission not delayed over a last-minute potentially fatal problem
(considered highly unlikely)
Three different outcomes Success: Launch and deployment successful (no problem
shortly after launch) Failure: Problem shortly after launch, because of spacecraft’s
orientation to sun, problem is catastrophic Near-miss: Problem shortly after launch, because of
spacecraft’s orientation to sun, not a problem, data collection is successful
Participants
89 undergraduate students 98 MBA students 24 NASA managers
Sample Differences
In general, NASA managers tended to be a bit easier on Chris Rated decision to launch higher (p<.05)
NASA mean = 3.7, MBA mean = 3.4, UG mean = 3.0 Were marginally more likely to promote Chris (p=.1)
NASA mean = 3.8, MBA mean = 3.3, UG mean = 3.3 Were significantly less likely to fire Chris (p<.001)
NASA mean = 3.0, MBA mean = 4.2, UG mean = 4.3
No significant interaction effects between sample and condition
Decision making ability
2(not at
all)
4 (somewhat)
Competence
2 (not at
all)
4 (somewhat)
2(not at
all)
4 (somewhat)
Intelligence
2 (not at
all)
4 (somewhat)
Leadership ability
2 (very bad)
4 (neutral)
Decision to promote
2(very bad)
4 (neutral)
Decision to fire
2(very bad)
4 (neutral)
Decision to proceed without peer review
2(very bad)
4 (neutral)
Decision to launch without redesign
ALL PARTICIPANTS
6(greatly)
6 (greatly)
6(greatly)
6(greatly)
6 (very good)
6 (very good)
6 (very good)
6 (very good)
- Failure- Near-miss- Success
p<0.001
p<0.01
p<0.001
p<0.001
p<0.001
p<0.001
3 5
p<0.05
p=.11
3
3
3
3
3
3
3
5
5
5
5
5
5
5
p<0.001
Summary
Rated managers whose decisions resulted in organizational success significantly more favorably than mangers whose decisions resulted in failures
Rated mangers whose decisions, BUT FOR LUCK, would have resulted in failures more favorably than those whose decisions resulted in failure
Did not hold managers accountable for faulty decision making if results in good organizational outcome, EVEN WHEN SUCCESS IS BECAUSE OF LUCK
Implications for organizations
Near-misses categorized as misses rather than hits, meaning organizations fail to take advantage of learning opportunities Generally lack the formal failure investigation board
Near-miss bias may make organizations more risky May explain the normalization of deviance (Vaughan, 1996)
Without obvious failures, events that once caused concern become accepted as normal occurrences.
If those experiencing near-misses are promoted through organizational ranks, given they make more risky subsequent decisions, organizations will come to embrace more and more risk.
What to do about all this?
Knowledge and recognition that biases exist Hindsight, Outcome, and Near-Miss Bias Decisions do have a luck component
Developing an Effective Lessons Learned System Effectiveness of Lessons Learned systems are dependent on
completeness of data A complete data set requires noticing both failures and
successes and being able to distinguish near-misses
How can you increase chances of acknowledging both successes and failures Improve group decision making: groupthink, escalation, abilene
Avoiding Groupthink
Monitor team size (<10) Provide face-saving mechanism for dissent and
changing one’s mind Don’t be a bystander because fearful of appearing
foolish (evaluation apprehension) Discuss risks before benefits Discuss how things might have failed Encourage & track alternative viewpoints Get external observers
Avoiding Escalation
All advice for avoiding groupthink, plus: Set resource limits up front Recognize sunk costs
Avoiding Abilene Paradox
All advice for avoiding groupthink, plus: Generate solution alternatives without evaluation
(brainstorming) Conduct a private vote (Delphi) Create norms for expression of controversial views
(rotating devil’s advocate)
Future Work
Determine what factors may help mitigate a near-miss bias
Determine what effect the accumulation of the near-miss bias may have as an inhibitor to organizational learning