Managing Risk - washto.orgManaging Risk Travis McGrath, PhD, PE Chief Operations Officer March 2020....
Transcript of Managing Risk - washto.orgManaging Risk Travis McGrath, PhD, PE Chief Operations Officer March 2020....
1
Managing RiskTravis McGrath, PhD, PEChief Operations Officer
March 2020
Safety Share… and Risk Share
2
What does “risk” mean to you?
3
What does risk look like for a DOT?
4
Design Risk
5
Design errors and omissions
Environmental Process Risk
6
Procurement Risk
7
1.00 1.02 0.99
1.03 1.04
1.19
1.23
0.95
1
1.05
1.1
1.15
1.2
1.25
2012 2013 2014 2015 2016 2017 2018
INFL
ATIO
N IN
DEX
YEAR
ITD Construction Cost Index
Nominal 3%/yr Inflation
ITD Index - Actual
8
Construction Risk
9
Operations/Maintenance/Resilience Risks
10
O&M/Resilience Risks… from our customers
11
Employee Safety Risks
No employees were harmed in the making of this photo.
But he did win a Daytime Emmy.
Risks from Our Own Risk Management
12
ITD Enterprise Risk Dashboard – Fall 2019
Enterprise Risks – the ‘umbrella’
What other risks do you face in your work?
14
So…
How do we define risk?
How can we manage risk?
15
16
Risk: Potential event or condition that could harm (threat) or improve (opportunity) our ability to achieve goals
Generally described in terms of a likelihood and an impact, either qualitatively or quantitatively
Risk Response: Action or response to improve a risk. Strategies typically include:
• Terminate (Avoid)• Treat (Reduce threats)• Take Advantage of (opportunities)• Transfer• Tolerate (Accept residual risk)
Risk and Risk Response
Approaches for Characterizing Risk
Risk List / Risk Paragraph• Brainstorm a list of risks relative to an assumed ‘base’ case• Can sort the list based on which risks you feel are most significant• In a paragraph, describe what actions you will take to respond to key risks
• Not the recommended approach, but if you absolutely don’t have time for anything else, or it’s a simple project, this is better than nothing
17
Risk Category / Type of Impact
EA required instead of CE Environmental / Time, Cost
Delays getting RR agreement Third Party / Time
Asphalt unit price way higher Procurement / Cost
Have to pave ramps too Construction / Cost
Approaches for Characterizing RiskQualitative• Descriptors like Low, Medium, High or Green, Yellow, Red• Useful for quickly identifying critical risks to guide initial responses• Caution!
– Very approximate/imprecise – don’t base big decisions on qualitative ratings– Probability and impact ratings must be defined for common understanding– Don’t use non-linear or undefined scales (like often done with “1 to 5” scales)
18
VH = Very HighH = HighM = MediumL = Low
Rare Unlikely Possible Likely Very Likely
Very Significant M M H VH VH
Major L M M H VH
Moderate L M M M H
Minor L L L M M
Insignificant L L L L M
Likelihood Rating
Aggr
egat
e Im
pact
(acr
oss a
ll im
pact
type
s)
Approaches for Characterizing RiskQuantitative
Quantify • Likelihood of occurrence (probability or frequency)• Impact of occurrence (e.g., additional $, additional time)
Useful for • Combining different types of impacts into a single measure (“total risk”)• Making more defensible and transparent decisions
19
3.12.7
2.52.3
1.71.11.1
0.80.3
0 1 2 3 4
C5. Uncertain construction sequencingP6. Delays getting railroad agreements
P4. Delays in agency approvalsD12. Permitting issues
R1. Difficult ROW AcquisitionIdentified Minor Risks (aggregate)
Unidentified Risks (aggregate)U1. Difficult utility relocations
D1. EIS Re-evaluation required
Average Delay to Project Schedule (months)
Risk Event
20
Risk Assessment/Management Process
Identify Risks
Establish “Base”
Assess / Analyze Risks and Establish Risk Tolerance
Plan and Implement Risk
Responses
Document, Monitor, and
Update
Purpose: Proactively identify, assess, prioritize, and mitigate risks to the DOT,to better understand and control outcomes
What events could occur and change your planned outcomes?
How can you manage those potential events and their outcomes if they occur?
https://www.fhwa.dot.gov/GoSHRP2/Solutions/Renewal/R09
21
Risk Assessment/Management Process
Identify Risks
Establish “Base”
Assess / Analyze Risks and Establish Risk Tolerance
Plan and Implement Risk
Responses
Document, Monitor, and
Update
RA/RM Process applies at any level• Enterprise Risk Management (ERM)• Program risk management
• Asset Management (TAMP)• Asset Resilience (all-hazards)• Transportation Improvement
Program (TIP/STIP)• Individual project risk management
But specific tools or approach may vary at each level (e.g., qualitative vs. quantitative)
Illustration
Qualitative RA/RM for a simple project
(i.e., don’t need a probability expert to do this)
22
Establish Project Base
• The planned project, based on assumptions used to develop:
– scope of work
– cost estimate
– schedule estimate
– third-party engagement (subs, client, public, etc.)
– logistical plans
• Generally optimistic - even if it wasn’t intended to be - since PMs are generally optimistic about their project’s outcome
• A basis against which risks can be identified and measured
If you don’t first define a Base, you can’t characterize risk
Risk assessment can’t “cover” for poor Base cost and schedule estimates
Project Base - Example
• DOT scope of work: Design and construction inspection for a proposed permanent soil-nail wall, to be built through an existing basement wall next to the roadway. DOT/consultant are proposing an innovative drilling method and nail bar design as a cost-saving alternative to a soldier-pile & tieback system. Consultant is performing the design. The DOT is contracting the driller.
• Key assumptions and conditions– Soil nailing is appropriate for the purpose and site conditions– The innovative drilling method / bar design will perform adequately– The innovative approach can be accepted/permitted quickly– The drilling contractor (from another state) will
perform adequately– The existing geotechnical information is is
sufficient for completing design (i.e., no surprises or misinterpretations)
Identify Risks
• Goal: comprehensive, non-overlapping list of risks (both threats and opportunities) to the DOT
• Suggested risk identification process1. For the current project:
• Identify issues of concern (what keeps you awake at night?)• Consider how Base assumptions might change or be invalid
2. Brainstorm with others what other problems or improvements could occur, considering collective experience on other projects
3. Consult risk checklists to “prompt” anything you might have missed4. Document in risk register – sort and categorize logically
• Guidance– Document all credible possibilities. Think broadly; don’t dismiss prematurely.
Avoid overconfidence. It can happen to your project.– Consider the whole project; most schedule risk is prior to construction!– Several tens of risks is typical without overlapping or artificial separation
Identify Risks• Sources of project risk to DOT
– Uncertainty in conditions (e.g., ground, groundwater, unusual weather, market conditions, force majeure)
– The prime contractor and subcontractors (contract language, unclear expectations, relationship, performance)
– Design and CE&I consultants (unclear scope, not meeting Standard of Care)
– Third parties (e.g., permitting agencies, railroads, utilities, public, legislative)
– The DOT’s own performance – particularly technical and managerial, including over-optimism, poor assumptions, and mistakes. For example:
• Failing to hold consultants and contractors to acceptable quality or agreed-to schedules
• Failing to manage scope or document / approve changes
• Inadequate technical work or review
– Any other potential changes relative to Base assumptions, whether real or perceived by someone else
Identify Risks –Example
Project Risk Register Project Number:Project Manager:
Date:Before (Without) Risk Response
Risk Category Unique Risk Number Risk Title Summary of the Risk to DOT Probability of
OccurrenceConsequence if Risk Occurs
Risk Severity
Category 1 1 0Category 1 2 0Category 1 3 0Category 1 4 0Category 1 5 0Category 2 6 0Category 2 7 0Category 2 8 0Category 3 9 0Category 3 10 0
00
Excerpt from Simple Risk Register Template, page 1 of 2
Identify Risks –ExampleRisk Category Unique Risk
Number Risk Title Summary of the Risk to DOT
Design D1 Additional information needed for wall design
Existing information from others might not be adequate for us to complete our soil-nail design. Could lead to either additional borings or more conservative wall design, at owner's expense.
Construction C1 Groundwater conditions worse than anticipated
Could create excessive pressures behind the wall and therefore unanticipated movement. Could require additional dewatering and/or remedial wall support.
Design D2Soil nailing isn't appropriate for this site, but not discovered until construction
Separate from the issue of innovative technology (see below). The previous designer believed soil nailing won't work. Could require re-design of more robust system or switch back to soldier piles. This re-design would be DOT's risk.
Construction C2 Innovative technology doesn't perform as intended
The innovative drilling method and bar design might not perform as needed (e.g., can't make capacity), requiring additional nails, grout, or a switch to traditional soil-nail technology. Since DOT is proposing this technology in lieu of existing design, client might not be willing to pay for significant changes if technology doesn't work. Hence, risk to DOT.
Construction C3Drilling contractor doesn't bring the required skill
Contractor has a good reputation in another state, but is untried in our area. Could cause quality issues and/or delays.
Construction C4Drilling contractor has significant problems with drilling equipment
Brings the wrong equipment, poorly-maintained equipment, or suffers signficant break-down. Could cause delays and extended overhead costs. DOT would look to Prime to pay these costs, but time risk is DOT's.
Third-Party / Political P1 Difficult City approval process for proposed
approach
The City might be nervous about soil nailing and/or the proposed innovative technology. Could cause delays and additional cost to work through issues. Prime would not be happy.
Third-Party / Political P2 Prime contractor resists the change to soil
nailing
The Prime contractor might have reasons to stick with original design (e.g., already started procurement of soldier piles and tiebacks). Could cause delays and/or switch back to soldier piles.
Construction C5Extended poor weather impacts construction
Especially if construction is delayed into the wet season. Could result in delays and extra costs.
Construction C6Experienced DOT construction staff not available when needed
Multiple construction projects underway. We can't afford to delay this project due to staffing limitations, and we're unlikely to staff this with inexperienced people, so the impact would likely be to other DOT projects.
Construction C7Underestimate the costs to design and staff the CE&I Could lead to project write-offs if we can't get change-orders approved.
Design D3 Errors in Consultant design Could require re-design plus could result in delay claims from the drilling and/or Prime contractors
Assess Risks /Perform Risk Analysis
• Goal: Evaluate risk ‘severity’ to prioritize for mitigationFor many ‘typical’ DOT projects, qualitative RA (using ratings) is appropriate.For complex or risky projects, specialists can help PMs with a quantitative approach.
• Simplified qualitative risk analysis process and guidance– Define the ratings for probability and impact – use linear scales if possible– Assess each risk in terms of the ratings
• Generally define impacts first, then a consistent probability (no unique combinations, so try for a reasonable, matched pair)
• Consider all impact types (e.g., $, time) – use combined total (e.g., equivalent $) if possible
• Use relevant data if available. But most likely you’ll have to draw from your team’s expertise and judgment → subjective assessments
– The goal is unbiased, defensible assessmentsBe honest. Avoid overconfidence and “wishful thinking”
– After all the risks are assessed, prioritize based on risk ratings (e.g., P x C) to “set the stage” for risk-response planning
Assess Risks / Perform Risk Analysis -Example
Qualitative Rating Scale for Probability of Risk Occurrence
Rating Range Average
1 <20% 15%2 20% - 35% 30%3 35% - 55% 45%4 55% - 70% 60%5 > 70% 75%
Qualitative Rating Scale for Consequences if Risk Occurs
Corresponding Cost Impact to
Project
RatingPercent of
Estimated Project Cost Column1
1 <= 2.5%2 5.0%3 7.5%4 10.0%5 >= 12.5%
Corresponding Probability
Note: These are linear scales, which means for example:• A “4” is twice as
much as a “2”• A “risk” P x C of
4 x 2 is the same as 2 x 4
Before (Without) Risk Response
Risk Category Unique Risk Number Risk Title Summary of the Risk to DOT Probability of
OccurrenceConsequence if Risk Occurs
Risk Severity
Design D1Additional information needed for wall design
Existing information from others might not be adequate for us to complete our soil-nail design. Could lead to either additional borings or more conservative wall design, at owner's expense.
4 1 4
Construction C1 Groundwater conditions worse than anticipated
Could create excessive pressures behind the wall and therefore unanticipated movement. Could require additional dewatering and/or remedial wall support.
3 2 6
Design D2Soil nailing isn't appropriate for this site, but not discovered until construction
Separate from the issue of innovative technology (see below). The previous designer believed soil nailing won't work. Could require re-design of more robust system or switch back to soldier piles. This re-design would be DOT's risk.
2 5 10
Construction C2 Innovative technology doesn't perform as intended
The innovative drilling method and bar design might not perform as needed (e.g., can't make capacity), requiring additional nails, grout, or a switch to traditional soil-nail technology. Since DOT is proposing this technology in lieu of existing design, client might not be willing to pay for significant changes if technology doesn't work. Hence, risk to DOT.
3 5 15
Construction C3 Drilling contractor doesn't bring the required skill
Contractor has a good reputation in another state, but is untried in our area. Could cause quality issues and/or delays. 3 3 9
Construction C4Drilling contractor has significant problems with drilling equipment
Brings the wrong equipment, poorly-maintained equipment, or suffers signficant break-down. Could cause delays and extended overhead costs. DOT would look to Prime to pay these costs, but time risk is DOT's.
2 4 8
Third-Party / Polit P1Difficult City approval process for proposed approach
The City might be nervous about soil nailing and/or the proposed innovative technology. Could cause delays and additional cost to work through issues. Prime would not be happy.
4 2 8
Third-Party / Polit P2Prime contractor resists the change to soil nailing
The Prime contractor might have reasons to stick with original design (e.g., already started procurement of soldier piles and tiebacks). Could cause delays and/or switch back to soldier piles.
3 1 3
Construction C5Extended poor weather impacts construction
Especially if construction is delayed into the wet season. Could result in delays and extra costs. 1 2 2
Construction C6Experienced DOT construction staff not available when needed
Multiple construction projects underway. We can't afford to delay this project due to staffing limitations, and we're unlikely to staff this with inexperienced people, so the impact would likely be to other DOT projects.
2 2 4
Construction C7Underestimate the costs to design and staff the CE&I
Could lead to project write-offs if we can't get change-orders approved. 2 3 6
Design D3 Errors in Consultant design Could require re-design plus could result in delay claims from the drilling and/or Prime contractors 2 4 8
Assess Risks / Perform Risk Analysis -Example
Plan Risk Responses (Mitigation)
• Goal: Plan cost-effective responses for at least the most significant risks
• Suggested risk-response planning process and guidance
1. Be sure to address the highest-priority risks (i.e., highest risk ratings, from the sorted risk register)
2. Brainstorm practical, cost-effective responses to manage each risk; e.g.,• Can you modify the design to avoid or reduce/mitigate the risk?• Can you contractually transfer the risk to a party who can better manage it?
3. Evaluate the combined effectiveness of the planned responses for each risk, in terms of the residual (remaining) consequence and/or probability of occurrence. Avoid over-optimism.
4. Select cost-effective responses for implementation (mitigation benefit > cost)• Can DOT tolerate the residual risks? (risk tolerance defined by DOT leadership)• Include a contingency ($, time) to “cover” some of the overall residual risk
5. Record the results in the risk register and re-sort to identify the most significant residual (post-mitigation) risks
Plan Risk Responses (Mitigation) –Example
Excerpt from Simple Risk Register Template, page 2 of 2
Project Risk Register Project Number:Project Manager:
Date:After (With) Risk Response
Risk Category Unique Risk Number Risk Title Summary of the Risk
to DOT Planned Risk ResponseResidual
Probability of Occurrence
Residual Consequence if Risk Occurs
Residual Risk
Severity
Timing for Implementation
Risk Owner and Response
OwnerCategory 1 1 0Category 1 2 0Category 1 3 0Category 1 4 0Category 1 5 0Category 2 6 0Category 2 7 0Category 2 8 0Category 3 9 0Category 3 10 0
00
Plan Risk Responses (Mitigation) –Example
Unmitigated After (With) Risk Response
Unique Risk Number Risk Title Risk
Severity Planned Risk ResponseResidual
Probability of Occurrence
Residual Consequence if Risk Occurs
Residual Risk
Severity
Timing for Implementation
Risk Owner and Response Owner
D1 Additional information needed for wall design 4
Minimal risk to DOT. However, would request additional borings (client cost) in proposal. If not granted, design more conservative wall to account for the increased uncertainty/risk.
4 During proposal preparation PM
C1Groundwater conditions worse than anticipated 6 Design redundant drainage for "worst case" 2 2 4 During design Wall Designer
D2Soil nailing isn't appropriate for this site, but not discovered until construction
10No risk responses identified, other than not proposing soil nailing as an alternative (i.e., which would mean not doing this project).
10
C2 Innovative technology doesn't perform as intended 15
Perform Verification tests prior to production to determine what capacity can be achieved, and modify design if needed prior to construction. Should reduce the likelihood, but not the cost impact to DOT.
2 5 10Document in proposal and write into specifications
PM
C3Drilling contractor doesn't bring the required skill 9 Write specifications to require submittal and approval of
crew experience and permission for crew substitutions. 2 3 6Document in proposal and write into specifications
PM
C4Drilling contractor has significant problems with drilling equipment
8Ensure our contract with the driller places responsibility for equipment downtime (and related consequences) on the contractor.
2 2 4During contract negotiations with driller
PM
P1 Difficult City approval process for proposed approach 8 Meet with City to understand their requirements for
approval, and to establish issue-resolution process. 2 2 4 During design PM
P2Prime contractor resists the change to soil nailing 3 No risk responses identified, although will meet with
team prior to moving forward with proposal. 3 Now PM
C5 Extended poor weather impacts construction 2 No risk responses identified. 2
C6Experienced DOT construction staff not available when needed
4No risk responses identified, although will meet with other DOT PMs to alert them to the issue and encourage an office-wide staffing plan.
4 Now PM
C7 Underestimate the costs to design and staff the CE&I 6
Write the proposal to address uncertainty in construction duration; propose time-and-materials compensation to allow for extended schedule.
1 3 3 During proposal and estimate preparation PM
D3 Errors in Consultant design 8 Ensure adequate internal check and review of design calcs and specifications 1 4 4 During design PM, Wall Designer
Implement, Monitor, and Update• Goal: Control risks by selecting, assigning, and implementing
cost-effective risk responses
• Risk management implementation guidance
– For each risk to be mitigated:
• Assign owner(s) for the risk and the selected risk responses
• Provide the resources and support to implement the planned responses
• Monitor the effectiveness of the risk response
• Update and revise responses as appropriate (e.g., if the risk changes or the response isn’t working as planned)
• If the risk occurs, allocate contingency to cover its residual impact
• If the risk is averted or the project advances beyond the phase in which the risk could occur, “retire” the risk
– Be vigilant for new risks as the project evolves– Remember to keep management up to date, as appropriate
Questions about the Risk Management Process?
36
38
Hot Topic in AASHTO:“Risk and Resilience”
What does“transportation system resilience”
mean to you?
39
40
Example definitions from the pros:
• “… to anticipate… and adapt to changing conditions, and withstand,… and recover rapidly from disruptions” (FHWA)
• “… absorb, recover from, or more successfully adapt to adverse events” (National Research Council)
• From reliability theory: the ability to resist downtime, and to recover quickly if it ‘goes down’
System Resilience
41
Those definitions share two common elements:
Resistance of system to disruption
Recovery of the system if disrupted
System Resilience
42
One perspective:
Natural and human-induced risks affect system resilience, either by threatening or enhancing resilience
How we respond to those risks also affects system resilience
How are Risk and Resilience Related?
What does risk to resilience look like?
43
44
Risk to Resilience during PlanningWill this thing last until we get money to replace it?
Risk to Resilience during Design
45
Uncertainty in design requirements / conditions
46
Risk to Resilience during Construction
47
Risk to Resilience during Construction
48
Risk to Resilience during Operations
Risk to Resilience during Operations
50
Risk to Resilience during Operations
51
Risk to Resilience during Operations
52
Risk to Resilience during Operations
53
Sources of Risk to Resilience
Limited funding
Constraints on how funding is used
Changing political priorities
Uncertainty in design requirements / under-designed
Changing system use
Design errors and omissions
Poor-quality construction
Unsuitable materials
Construction impacts/accidents
Accelerating climate change• Loss of permafrost• Sea-level rise / erosion• Increasing variability in extreme
weather events
User-caused damage
Inadequate maintenance practices
Slow emergency response
Cyber attacks
Uncertainty in ground conditions • General geologic / geotechnical /
hydrogeological• Seismic• Thermal• Geohazards
54
AASHTO Subcommittee on Risk Management
Purpose: Help you benefit from Risk Management
Guidance and implementation assistance Share best practices / Community of Practice Support research; e.g., NCHRP 20-24(129) - scoping study for all-hazards risk
and resilience analysis NCHRP 08-118 – risk assessment techniques for asset
management
Questions?
55
56
Enjoy your stay in Coeur d’Alene!