Seeing the real picture – strategies for dealing with ......Slide 1 Seeing the real picture –...

Seeing the real picture – strategies for dealing with uncertainties in risk assessment models Simon Firth, Firth Consultants Ltd Brownfield Risk & Remediation, 13th September 2017

Uncertainty in risk assessment Source

Pathways

Receptors

Cs CPOE

Normally conduct quantitative risk assessment to determine: • If risk is acceptable or not • Determine remediation criteria

Assess risk by: a. Compare concentration along pathway with guideline value; or b. Compare exposure with a toxicity reference value

How does uncertainty affect results?

The importance of uncertainty • How confident are we that we have made the right decision?

– If we decide that no further action is required, how certain are we that the receptors are protected?

– If we decide to remediate or mitigate the risks, how certain are we that we are not undertaking unnecessary work? Could we be doing more harm than good, spending resources and creating other risks for little benefit?

• Assessing and managing uncertainty is a key part of risk assessment and fundamental to good risk based management

US EPA Risk Assessment Guidance for Superfund Sites:

Types of uncertainty

How wrong

could we be?

Conceptual

Model

Parameter

Areas of uncertainty • Source

– Where? – source dimensions – What? – chemicals of concern – How much? - concentrations

• Pathway – What is(are) the pathway(s)? – how are contaminants likely to

get from A to B – Pathway properties – that affect contaminant fate and transport

• Receptor – Exposure characteristics – Sensitivity

Assessing uncertainty • Qualitative

– How conservative is each parameter/assumption? – How likely are we to have under- or over-estimated risks?

• Sensitivity analysis – Can help identify key uncertain parameters – Can give a feel for extent to which risks could be over- or under-

estimated

• Probabilistic modelling – Can help quantify likelihood of unacceptable risk occurring – But can also introduce new uncertainties

Qualitative analysis of uncertainties • C4SL project used a semi-quantitative approach to help

assess the overall level of conservatism • Each parameter was scored according to its likely level of

conservatism

Parameter Evaluation of uncertainty

Estimation of soil-gas concentration + / +++

Estimation of indoor air concentration using J&E ● / +++

Soil to plant concentration factors - / ++

Produce consumption rates ● / +

Homegrown fraction - / +

Overall evaluation of C4SL exposure modelling estimates for benzene for residential land-use

Highly conservative

• Scores combined to assess over-all level of conservatism

Sensitivity analysis

Min Max Justification

8970 11800 Range in reported concentrations

100 1000 Range reported in EA, 2002

25 100 0.5x and 2x value used for assessment

3 9 Reasonable range based on site data

6 14 Reasonable range based on site data

1.8 2.1 Range for Mercia Mudstone in ConSim

0.2 0.299 Range calculated using RTM porosity calculator

0.0018 0.012 Range in estimates from site

0.1 0.51 Range in estimates from falling head tests

37.5 150 0.5x and 2x value used for assessment

0.001 0.004 Range in measured values from site

Single Dual Both options tested

• Example using RTM model:

Probabilistic modelling

• Example using ConSim:

• But there will still be uncertainty in the results!

5 (black) 10 (red) 25 (yellow ) 50 (cyan)75 (grey) 90 (green) 95 (blue) WQS: 0.5 [mg/l]

Time [years]100.090.080.070.060.050.040.030.020.010.0D

ilute

d C

once

ntra

tion

Uns

atur

ated

Pat

hway

4.0

3.5

3.0

2.5

2.0

1.5

1.0

0.5

0.0


Time [years]100.090.080.070.060.050.040.030.020.010.0D

ilute

d C

once

ntra

tion

Uns

atur

ated

Pat

hway

2.5

2.0

1.5

1.0

0.5

0.0

Using single (most likely) value for Kd


Time [years]100.090.080.070.060.050.040.030.020.010.0D

ilute

d C

once

ntra

tion

Uns

atur

ated

Pat

hway

1.5

1.0

0.5

0.0

Using single (most likely) value for Kd + half-life


Time [years]100.090.080.070.060.050.040.030.020.010.0D

ilute

d C

once

ntra

tion

Uns

atur

ated

Pat

hway

1.0

0.5

0.0

Using single (most likely) value for Kd + half-life + infiltration

• Monte Carlo modelling can help to quantify uncertainty

How certain do we need to be?

Pred

icte

d co

ncen

trat

ion

Scenario 1 Scenario 3 Scenario 5 Scenario 4 Scenario 2 Scenario 6

Range in estimates Most likely

Guideline value

Reducing uncertainty • Gather more data, e.g.

– More samples – Field testing of hydraulic conductivity – Bioaccessibility testing

• Measure rather than model, e.g. – Soil vapour concentrations – Concentrations in river

• Use a lines of evidence approach to constrain uncertainties, e.g. – Soil, groundwater and soil vapour concentrations to constrain source

dimensions – Calibrating model to observed concentrations

Source concentrations - soil

Mean = 496 mg/kg Confidence limits (using Chebychev) LCL95 = -382 mg/kg UCL95 = 1374 mg/kg

Source concentrations - soil

Composite samples

• Composite samples can be useful for reducing analytical costs

• But do lose information on spatial variability

Groundwater samples

Vapour samples

Dotted lines represent long-term geomean

Predicted concentration from groundwater concentrations

Communicating uncertainty • Very important to communicate level of

confidence/uncertainty in risk assessment results to decision makers/stakeholders

• Modelling reports should have a section on uncertainty – What are the key uncertainties – How could these affect model results

Key points • Uncertainty is integral to risk assessment process

– What are the key uncertainties? – How do these affect overall conclusions of the risk assessment?

• Measures to reduce uncertainty – Gather more data – Measure rather than model – Use all available lines of evidence

• Uncertainties and their impact on risk assessment conclusions should be communicated to decision makers/stakeholders

Thanks for listening!

Seeing the real picture – strategies for dealing with ......Slide 1 Seeing the real picture –...

Documents

Transcript of Seeing the real picture – strategies for dealing with ......Slide 1 Seeing the real picture –...