Information Needs for Developing Robust Adaptive Strategies Robert Lempert Director, RAND Pardee...
-
Upload
alexia-casey -
Category
Documents
-
view
213 -
download
0
Transcript of Information Needs for Developing Robust Adaptive Strategies Robert Lempert Director, RAND Pardee...
Information Needs for Developing Robust Adaptive Strategies
Robert Lempert
Director,
RAND Pardee Center for Longer Range Global Policy and the Future Human Condition
Workshop on Climate Science Needed to Support Robust Adaptation Decisions
Georgia Tech
February 6, 2014
2
Climate-Related Decisions Poses Both Analytic and Organizational Challenges
Climate-related decisions involve:• Incomplete information from new, fast-moving, and
sometimes irreducibly uncertain science
• Many different interests and values
• Long-time scales
• Near certainty of surprise
How to make plans more robust and adaptable while preserving public accountability?
Public planning should be:• Objective
• Subject to clear rules and procedures
• Accountable to public
3
Concept of Decision Support Focuses Attention on How (in Addition to What) Information is Used
Decision support:
• Bridges supply and demand with a focus on decision processes
• Represents organized efforts to produce, disseminate, and facilitate the use of data and information to improve decisions
• Includes as key elements:
– Recognition that decision processes are at least as important as decision products
– Co-production of knowledge between users and producers
– Institutional stability
– Design for learning
Supply and demand of scientific information may be mismatched
Sarewitz and Pielke (2007) NRC (2009)
4
Should Climate Science Inform Exploratory, Rather Than Consolidative, Models?
• Consolidative models:
– Bring together all relevant knowledge into a single package which, once validated, can be used as a surrogate for the real world
– Are often used for prediction
• Exploratory models:
– Map assumptions onto consequences, without privileging any one set of assumptions
– Cannot be validated
– Can, when used with appropriate decision processes and experimental designs, provide policy-relevant information
Bankes (1993)Weaver et. al. (2013)
5
Consolidative Models Useful for ‘Predict Then Act’ Analysis
• This traditional approach to risk management works well when the future– Isn’t changing fast– Isn’t hard to predict– Doesn’t generate much disagreement
What will future conditions be?
Under those conditions, what is
best near-term decision?
How sensitive is the decision to
those conditions?
Analytics aims to provide ranking of strategies, which implies consolidative models
6
Predict Then Act Methods Can Backfire in Deeply Uncertain Conditions
• Uncertainties are underestimated
• Competing analyses can contribute to gridlock
• Misplaced concreteness can blind decisionmakers to surprise
What will future conditions be?
Under those conditions, what is
best near-term decision?
How sensitive is the decision to
those conditions?
7
Believing Forecasts of the Unpredictable Can Contribute to Bad Decisions
• In the early 1970s forecasters made projections of U.S energy use based on a century of data
Gross national product (trillions of 1958 dollars)
2.22.0
1.81.6
1.4
1.2
1.0.8
.6
.4
.2
0180
Energy use (1015 Btu per year)
0
Historical trend
continued1970
1920 19291940
19501960
1910
1973
19731900
1890
20 40 60 80 100 120 140 160
1975 Scenarios
8
Believing Forecasts of the Unpredictable Can Contribute to Bad Decisions
Gross national product (trillions of 1958 dollars)
2.22.0
1.81.6
1.4
1.2
1.0.8
.6
.4
.2
0180
Energy use (1015 Btu per year)
0
Historical trend
continued1970
1920 19291940
19501960
1910
1973
19731900
1890
20 40 60 80 100 120 140 160
2000 Actual
1990
19801977
1975 Scenarios2000 Actual
1990
19801977
In the early 1970s forecasters made projections of U.S energy use based on a century of data
… they all were wrong
Deep uncertainty occurs when the parties to a decision do not know or do not agree on the likelihood of alternative
futures or how actions are related to consequences
9
RDM Manages Deeply Uncertain Conditions by Running the Analysis Backwards
1. Start with a proposed strategy
2. Use multiple model runs to identify conditions that best distinguish futures where strategy does and does not meet its goals
3. Identify steps that can be taken so strategy may succeed over wider range of futures
Proposedstrategy
Identify vulnerabilities of
this strategy
Develop strategy adaptations to
reduce vulnerabilities
“RDM (Robust Decision Making) Process”
Analytics aims to provide concise summary of key tradeoffs, a useful function of exploratory models
10
Over 15 years, HCMC has planned multi-billion dollar flood investments using best available projections
Flood Risk Management Study for Ho Chi Minh City Provides an Example
11
Conditions have diverged from projections and the city is at significant risk
12
Today, HCMC seeks an innovative, integrated flood risk management strategy
How Can HCMC Develop This Plan When Today’s Predictions Are No More Likely To Be Accurate?
World Bank WPS-6465 (2013)
13
Simulation Model Projects Flood Risk From Estimates of Hazard, Exposure, and Vulnerability
Hazard
• Future rainfall intensity
• Height of the Saigon River
Exposure
• Population in the study area
• Urban form
Vulnerability
• Vulnerability of population to flood depth
Risk = Expected Number of People Affected By Floods Each year
Risk Model
14
Model Projections Depend on Values of Six Deeply Uncertain Parameters
RainfallIncrease +0% + 35%
Increase River Height 20 cm 100 cm
Population7.4 M 19.1 M
Urban FormGrowth in Outskirts Growth in Center
Poverty Rate2.4 % 25 %
Vulnerability Not VulnerableVery Vulnerable
15
Traditional Planning Asks “What Will The Future Bring?”
Infrastructure performance under best estimate future
Risk to the Non-Poor
Ris
k t
o t
he
Po
or
Risk Model
Best Estimate Future
16
Ris
k t
o t
he
Po
or
Infrastructure UnderAlternative Future
Risk Model
We Run HCMC’s Infrastructure Plan Through 1000 Different Combinations of Conditions
Alternative Future Future
Risk to the Non-Poor
What factors best distinguish this
region, where risk is reduced for both
poor and non-poor?
17
6% increase
45cm riseInfrastructure
1. Under What Future Conditions Is HCMC’s Infrastructure Vulnerable?
Infrastructure fails to reduce risk when:> 6% increase in rainfall intensity,> 45 cm increase in river height, OR> 5% poverty rate
6%
45cm
5%
In this region, risk is reduced for both poor and non-poor
18
6% increase
45cm rise
2. Are Those Conditions Sufficiently Likely To Warrant Improving HCMC’s Plan?
Infrastructure
IPCC SREX Mid Value (20%)
IPCC SREX High Value (35%)
MONRE SLREstimate 30 cm
NOAA SLR + SCFC Subsidence75 cm
Even stakeholders with diverse views can agree that its worth considering improvements to current plan
19
We Consider A Range of Options for Integrated Flood Risk Management
1. Rely on current infrastructure
5. Capture Rain Water
“Soft Options”
3. Relocate Areas
4. Manage Groundwater
2. Raise Homes
20
Infrastructure
NOAA SLR + SCFC Subsidence (75 cm)
IPCC SREX Mid Value (20%)
IPCC SREX High Value (35%)
Elevate(23%, 55cm)
+
How Will Adding “Soft” Options Improve Our Strategy?
MONRE SLREstimate 30 cm
21
NOAA SLR + SCFC Subsidence (75 cm)
What Are Tradeoffs Between Robustness And Cost?
Stakeholders debate about how much robustness they can afford
(which is much more useful than debating what the future will be)
Relocate(17%, 100cm)
High Cost
Low Cost
Rainwater(10%, 100cm)
Elevate(23%, 55cm)
Groundwater(7%, 55cm)
MONRE SLREstimate 30 cm
IPCC SREX Mid Value (20%)
IPCC SREX High Value (35%)
xInfrastructure
22
RDM Approach Also Used to Help Develop New Plans for Managing Colorado River
2012 Bureau of Reclamation study, in collaboration with seven states and other users:
•Assessed future water supply and demand imbalances over the next 50 years
•Developed and evaluated opportunities for resolving imbalances
23
RDM Embeds Analytics in a “Deliberation with Analysis” Decision Support Process
Scenarios that Illuminate
Vulnerabilities
Robust Strategy
Deliberation
AnalysisDeliberation with Analysis
Participatory Scoping
Scenario Exploration
and Discovery
Key products include:
1.Scenarios that illuminate vulnerabilities of plans
2.New or modified plans that address these vulnerabilities
3.Tradeoff curves that help decision makers choose robust strategies
New Options
Case Generation
Tradeoff Analysis
24
Decision Structuring: Work with Decision Stakeholders to Define Objectives/Parameters
1. DecisionStructuring
Deliberation with Stakeholders
• Metrics that reflect decision makers’ goals
• Management strategies (levers) considered to pursue goals
• Uncertain factors that may affect ability to reach goals
• Relationships among metrics, levers, and uncertainties
Informationneeded to organize
simulation modeling
Also called “XLRM”
25
Case Generation: Evaluate Strategy in Each of Many Plausible Futures
2. CaseGeneration
Simulating Futures
Large database of simulations
model results
(each element shows
performance of a strategy in one future)
• Strategy• Plausible
assumptions• Potential
outcomes
100s/1000s of cases
26
Scenario Generation: Mine the Database of Cases to Identify Policy-Relevant Scenarios
Scenarios that illuminate
vulnerabilities of proposed
strategy
3. ScenarioDiscovery . . .. . ......
Un
cert
ain
inp
ut
vari
able
2
1. Indicate policy-relevant cases in database of simulation results
2. Statistical analysis finds low-dimensional clusters with high density of these cases
3. Clusters represent scenarios and driving forces of interest to decisionmakers
Uncertain input variable 1
Parameter 1
Par
amet
er 2 Strategy
successful
Strategy less successful
27
Tradeoff Analysis: Help Decisionmakers to Compare Tradeoff Among Strategies
Robust strategy or
information to enable
decision-makers to make more
robust strategy
4. TradeoffAnalysis
Visualization helps decisionmakers
compare strategies
28
Analysis with Colorado System Simulations Reveal Key Vulnerabilities
++
24,000 climate projections•Recent historic•Paleo records•Model projections•Paleo-adjusted model projections
Several demand projections
24,000 climate projections•Recent historic•Paleo records•Model projections•Paleo-adjusted model projections
Several demand projections
Baseline strategyBaseline strategy
Reclamation’s Colorado River Simulation SystemReclamation’s Colorado River Simulation System
++
RAND, RR-242-BOR
29
Used Climate Information to Assess Vulnerabilities of Current Management Plans
Current plans fall short if:•Temperature greater than 2°F•Any decrease in precipitation
30
Initial Actions(dependent on beliefs)
Initial Actions
Contingent Actions
Common Options Strategy
Analysis Can Support Deliberation Regarding Near- and Longer-Term Actions
31
What about probabilities?
32
RDM Considers Sets of Alterative Probability DistributionsExpected value of strategy s for distribution (x) is given by
HARD1.Choosing what strategies to consider2.Choosing what futures to consider3.Calculating the performance of strategy s in some future x4.Knowing – and convincing other people that you know – the true probability distribution
EASY•Calculating the integral for any once you have 1-3 above
Thus RDM considers many probability distributions over the set of futures x -- NOT a uniform distribution
33
Some Strategies Are Robust Over a Wide Range of Probability Estimates
This chart:•Shows expected cost to taxpayers from re-authorizing U.S. Terrorism Risk Insurance Act•Quoted on floor of US Senate by a proponent•Called “insidious” by opponents•Usefully informed Congressional debate
CBO, TreasuryAssumption
RAND, MG-679-CTRMP
34
Observations
• Predict-the-act approaches promote restricted view of demand for climate information– Generally seek to aggregate information into consolidative models
aimed at prediction
• “Backwards” analyses like RDM effectively employ a much wider range of products from climate science– Integrated models of entire system (exploratory and consolidative)
– Models of individual components of full system
– Scientific understanding not well-embodied in models
– Any probabilistic information, however imprecise
• RDM creates demand for decision support methods and tools for managing and summarizing large and diverse sets of information in a decision-relevant context.
35
More InformationMore Information
http://www.rand.org/international/pardee/
http://www.rand.org/methods/rdmlab.html
Thank you!
36
37
Flexible Options Can Prove Effective
38
Research Portfolio Includes Developing and Evaluating Improved Tools
Tools to draw meaning from
informationUsers
Tools to represent
information
Analytic tools to facilitate RDMExploratory modeling methods run computer simulation models many times to create a database that links a wide range of assumptions to their consequences
Scenario Discovery methods uses cluster analysis on these databases of model results to simply characterize the future conditions where a the proposed strategy does not meet its goals
Tool development efforts include:1.Facilitating design of strategies with “pareto-satisficing surfaces”2.Higher dimensional scenario discovery3.Adaptive sampling
39
Portfolio Analysis Reveals Key TradeoffsP
erce
nt o
f Seq
uenc
esV
ulne
rabl
e in
Lo
wes
t Str
eam
flow
Con
ditio
ns
Lee Ferry Deficit Vulnerability Lake Mead Vulnerability
40
“XLRM” Framework for theColorado River Basin Study
40
Uncertain Factors (X) Options and Strategies (L)
Demand Conditions (6)
Supply Conditions (4)• Observed Resampled (103 traces)• Paleo Resampled (1,244 traces)• Paleo Conditioned (500 traces)• Downscaled GCM Projected (112
traces)
System Operations Conditions (2)
Options for demand reduction and supply augmentation (40)
Portfolios of many options designed to adjust over time in response to new information (4)•Near-term actions•Signposts•Contingent actions
Relationships or Models (R) Performance Metrics (M)
Colorado River Simulation System (CRSS)
• Water delivery (5)• Electric power (3),
Recreation (11), Ecological (5), Water quality (1), and Flood control (1)
41
Colorado Basin Study Followed This Process
Deliberate:•Participants to decision define objections, options, and other parameters
Analysis:•Participants work with experts to generate and interpret decision-relevant information
Dozens of workshops with many stakeholders over
course of study
Interactivevisualizations
Revised instructions
1. Start with proposed policy and its goals
2. Identify futures where policy fails to meet its goals
3. Identify policies that address these vulnerabilities
4. Evaluate whether new policies are worth adopting