Advancements in Managing Uncertainty in Remedial Options ...
Transcript of Advancements in Managing Uncertainty in Remedial Options ...
RPIC Edmonton 2015
David Wilson, Stantec Consulting Ltd.
June 4, 2015
Advancements in Managing Uncertainty in Remedial Options Analysis and Remedial Action Plan Development for Northern Sites
Agenda
1 Uncertainty in ROA and RAP Development
2 Project Impacts due to Uncertainty
3 Identifying and Characterizing Uncertainty
4 Uncertainty and Risk: Mitigation and Management
5 Case Examples
Uncertainty in ROA and RAP Development
“We demand rigidly defined areas of doubt and uncertainty!” - Douglas Adams, The Hitchhiker's Guide to the Galaxy
1
Definitions
Risk • Treasury Board (2010b): “the effect of uncertainty
on objectives” or “the expression of likelihood and impact of an event”
Uncertainty • Treasury Board (2010a): “the state, even partial, of
deficiency of information related to understanding or knowledge of an event, its consequence, or likelihood”
• Environment Canada (2012): the “state of having limited knowledge where it is impossible to exactly describe an existing state or future outcome”
Types of Uncertainty
Aleatory or Exogenous Uncertainty • statistical variability and
heterogeneity of the system (e.g., standard deviation of sample results)
Epistemic Uncertainty • model and parameter
uncertainty (e.g., infiltration rate)
Deep Uncertainty (CDMU, 2013) • uncertainty about
fundamental processes or assumptions
Often forgotten: also includes scenario and decision-rule uncertainty
ROA and RAP Development • Federal 10-step process – Step 7: Develop Remedial/
Risk Management Strategy • Consider project approach (remediate or risk
manage; generic or site-specific) • Determine remedial/risk management objectives • Then consider remedial/risk management options
Sources of Uncertainty SOURCE e.g.s: • Background conditions
poorly defined • CSM not fully developed • Some COCs not identified • Impacts not delineated • Decision criteria not
defined
Uncertainty is generally identified within a phase but is often not carried to a subsequent phase (e.g., from ESA to ROA/RAP; from RA to ROA/RAP; from ROA to RAP; from ROA/RAP to Cost)
NORTHERN SITE ‘MAGNIFIERS’ • Limited time/samples • Limited site-specific physical
system data • Limited historical data • Risk assessment exposure
scenarios not ‘typical’ • Identifying/engaging
stakeholders challenging
Project Impacts Due to Uncertainty
"No matter how good the team or how efficient the methodology, if we’re not solving the right problem, the project fails“ - Woody Williams
2
Impacts of Uncertainty
Uncertainty in CSM • Uncertainty in remedy performance/project risk
of remedy failure • Errors in identifying appropriate technologies for
actual site conditions and fate and transport mechanisms
Uncertainty in Project Objectives • Inappropriate clean-up goals/drivers Northern/Remote Site Challenges • Use/evaluation of emerging technologies difficult • Impact of short field season and seasonal demob
After ITRC, 2011
Impacts of Uncertainty
Quality • DQOs not met
• Range of impact (COCs, volumes, risk y/n)
Schedule • Range of task durations
• Ill-defined go/no go criteria
Cost • Range of task costs
Quality
Schedule Cost
Quality
Schedule Cost
Quantifying Project Impacts
Quality, e.g. • Significant risks not
identified • Remediation over-scoped Schedule, e.g. • Assessment extended/
revisited • Remediation extended Cost, e.g. • Exceed estimates • Not optimized Every uncertainty will impact all three project dimensions – they are inextricably interlinked
Projects reside here
Identifying and Characterizing Uncertainty
“Madness is the result not of uncertainty but of certainty” - Friedrich Nietzsche
3
Identifying Uncertainty
Phase I/II/III ESAs • Site history known and documented?
• How mature is the CSM?
• Contaminant concentrations compared to background?
RAs • Model and parameter uncertainty (e.g., contaminant
fate + transport; exposure factors) defined?
• COCs and pathways supported by CSM?
Remediation/Risk Management Objectives • Future use decided? R/RM scenario and decision-rule
uncertainty defined?
Characterizing Uncertainty
Aleatory or Exogenous Uncertainty • Measures of statistical variability:
o standard deviation (parametric and non-parametric)/ standard error; variogram
• Bayesian probabilities
Epistemic Uncertainty • Model sensitivity analysis
• Monte Carlo simulation
Deep Uncertainty • Expert judgment
• Pairwise comparison of significance
Quantifying Potential Impacts Source Soil impacts not fully delineated Background not characterized COCs not identified Model uncertainty Decision-rule uncertainty
Characterization # samples/m2 or m3
# background samples by media Predict likelihood – compare to analogy sites Range of results from family of models Objectives for site defined?
Quantification Impacted soil volume range: x m3 ± y m3
Parameter inclusion as COC: Y/N/Unk. Parameter inclusion as COC: Y/N/Unk. Level of confidence in model result Identify reliance of decisions on site objectives
The Project Risk Register
What is it? Tool for estimating the impact of project risk on project outcomes How should it be developed? List identified and characterized hazards (including uncertainties), determine likelihood of occurrence and consequence (i.e., risk score) How can it be used? Identify, implement and monitor risk mitigation and management measures
Methods: Mitigating Uncertainty
Exogenous Uncertainty • Variability is due to spatial, temporal, or individual randomness and
cannot be decreased by further data collection: it’s impact can only be (and should be) managed
Epistemic and Deep Uncertainties • Rank first by risk significance (e.g., hazardous vs. non-hazardous,
COC HQs), then by magnitude • Work down ranking, and answer the questions:
• Does the uncertainty span a decision threshold? [e.g., remediate or risk manage; on-site or off-site disposal]
• Worth investing in reduction of uncertainty (mitigate), or better to manage? [Cost-benefit analysis]
Decision Threshold Examples
Ex1 - Risk Assessment Scenario: HHRA HQ for primary
COC, based on UCLM, yields a value of 0.3, exceeding threshold of 0.2. ‘Expected’ HQ is <0.2
Decision: to remediate or risk manage
Options: 1) accept conservative case and remediate; 2) invest in additional analysis to reduce epistemic uncertainty
Ex2 - CSM Scenario: discontinuous
permafrost site, groundwater shows metals impacts, one season of sampling, nearest surface water body 25 m
Decision: to include a gw-sw pathway
Options: 1) accept conservative case of a pathway; 2) invest in additional field work to reduce uncertainty
Cost-Benefit Example
Situation • Phase III ESAs have been completed on several abandoned
mine sites grouped into a project. Significant areas of PHC and metals-contaminated soils have been delineated to varying degrees of certainty – there is high confidence that all COCs have been identified, but low confidence in quantity estimates. Is investment in additional delineation warranted?
Analysis Scenarios • Scenario 1: additional season of assessment, followed by
remediation
• Scenario 2: remediation
Analysis Method • Comparison of expected costs including uncertainty impact
Cost-Benefit Example cont’d
Scenario 1 • Cost of additional season
of assessment
• Cost of remediation, using expected lower uncertainty impacted soil quantity
Scenario 2
• Cost of remediation, using higher uncertainty impacted soil quantity
Investment not Warranted
Cost ($M): Assessment $0.45 Remediation $25.0
Cost ($M): Remediation $23.5
References and Tools References • Environment Canada. 2012. Federal Contaminated Sites Action Plan (FCSAP)
Environmental Risk Assessment Guidance. ISBN no. 978-1-100-22282-0. Cat. no. En14-19/1-2013E-PDF.
• Committee on Decision Making Under Uncertainty (CDMU). 2013. Environmental Decisions in the Face of Uncertainty. Board on Population Health and Public Health Practice. National Academy of Sciences. ISBN 978-0-309-13034-9.
• Interstate Technology & Regulatory Council (ITRC). 2011. Project Risk Management for Site Remediation. RRM-1. Washington, D.C.: Interstate Technology & Regulatory Council, Remediation Risk Management Team. www.itrcweb.org.
• TBS. 2010a. Framework for the Management of Risk. • TBS. 2010b. Guide to Integrated Risk Management. Distribution Fitting • ProUCL. www.epa.gov/OSP/hstl/tsc/software.htm. • Visual Sample Plan (VSP). vsp.pnnl.gov. Monte Carlo Simulation Tools • Oracle Crystal Ball.
www.oracle.com/technetwork/middleware/crystalball/overview/index.html. • Palisades @Risk. www.palisade.com.
Questions?
David Wilson, M.A.Sc., P.Eng. Senior Associate Stantec Ottawa (613) 738-6091 [email protected]