Uncertainty Subgroup ARB Expert Workgroup June 17, 2010 1.
-
Upload
prudence-mccarthy -
Category
Documents
-
view
214 -
download
0
Transcript of Uncertainty Subgroup ARB Expert Workgroup June 17, 2010 1.
Uncertainty SubgroupARB Expert Workgroup
June 17, 2010
1
• Michael O’Hare, chair• Wes Ingram, ARB staff
• Paul Hodson• Stephen Kaffka• Keith Kline• Michelle Manion• Richard Nelson• Mark Stowers
• With assistance from Richard Plevin, UCB
2
Uncertainty and ILUC
• How has uncertainty about the ‘real value’ of ILUC resulting from substitution of biofuel B for (fossil or other) fuel F been described?
• Given a characterization (PDF function, range, table of estimate values, etc.) ‘how big’ is the uncertainty in a given estimate? Across all estimates?
• How can it be reduced?• How should LCFS implementation accommodate it?
• Cost of error• Objective function
3
Implementation
Estimate physical GWI for MJ/MJ substitution of fuels
GHG emissions
Analyze optimal operationalGWI and choose (implement)
Fuel system responds withblends and driving behavior; nature“responds” with real GWI
Social cost(T etc.) 4
Three distinct variables for a single fuel
• g*ij = estimate of “physical” GWI of fuel i from model j
• i = real “physical” GWI: if fuel i is substituted for fuel k on a MJ/MJ basis, additional GHG release is (i - k). i is a random variable.
• gi = operational GWI of fuel i used in LCFS implementation
5
Sources of uncertainty
• Errors in assumed conceptual frameworks: relationships between biofuel policy and land use change
• Errors and variety in model structure• Errors in model parameters• Intrinsic variation in real-world variablesAre new model results converging on a narrow
range of values?
6
Uncertainty in existing models
• Modelers’ uncertainty reporting
• Variation among model results
• Random error vs. bias
• Can a probability distribution be constructed?
7
Estimates of ILUC emissions from corn ethanol
Estimates of ILUC emissions from corn ethanolStudy Target Year
Shock size(109 L)
ILUC Estimate(g CO2 MJ-1)
ILUC Range(g CO2 MJ-1)
Searchinger et al. 2008
2016 56 104 20 – 200a
Hertel et al. 2010 2001b 50 27 15 – 90c
Dumortier et al. 2009
2018/2019 30 n/a 21 – 118d
USEPA 2010 2012 7.5 81 62 – 104e
USEPA 2010 2017 14 58 43 – 76e
USEPA 2010 2022 10 34 25 – 45e
Al-Riffai et al. 2010 2020f 0.47 36 36 – 53g
CARB 2009 2001b 50 30 18 – 44
Tyner et al. 2010 2015 variable - 14 – 23
Plevin et al. (95% interval)
- - - 20 – 140
Plevin et al. (full range)
- - - 10 – 340
a. Calculated from reported sensitivity results.
b. Analysis was performed using the GTAP-6 database, based on 2001 data, but the results were adjusted post facto to account for the 10% greater average corn yield in 2010.
c. Range is based on a combination of high and low values for various uncertain economic model parameters.
d. Range is based on evaluating alternative model assumptions.
e. Range is 95% CI around mean considering only the uncertainty in satellite data analysis and carbon accounting.
f. Analysis was performed using the GTAP-7 database, based on 2004 data, using the model to project out to 2020.
g. Effect of additional 106 GJ after meeting 5.6% mandate. Higher value is for greater trade liberalization.
Notes about prior table
10
Economic model uncertainty dominates
Source: Plevin, O’Hare et al. 201011
EPA Uncertainty AnalysisSoybean biodiesel
N.B. Does not include uncertainties related to economic modeling or production period.
12
Large, but nebulous uncertainty
“The lesson for policymakers is that results from economic models depend heavily on assumptions, and because we are trying to predict long-run human behavior, there can be legitimate differences in these assumptions.”– Dumortier et al. 2009
“[T]his modeling project has demonstrated how the current limits to data availability create significant uncertainty regarding outcomes predicted by these policy simulations.”– Al-Riffai et al. (IFPRI) 2010
[O]ur experience with modeling, data, and parameter estimation and assumptions leads us to conclude that one cannot escape the conclusion that modeling land use change is quite uncertain. Of course, all economic modeling is uncertain, but it is important to point out that we are dealing with a relatively wide range of estimation differences.– Tyner et al. 2010 13
Uncertainty characterization is inconsistent and incomplete
• Searchinger et al.: Halving and doubling average emissions per hectare, other local SA
• Dumortier et al.: different FAPRI versions; crop yields; GREET vs BESS; w/wo forest conversion
• Hertel et al.: Systematic Sensitivity Analysis on few parameters, with notable limitations.
• USEPA: Uncertainty analysis of remote sensing and emission factors.
• CARB: OAT sensitivity analysis
14
Reducing uncertainty
• Better models• More models• Fewer models• Better data for models• Selection of “best” modelsMost of this work is being done in other
subgroupsUncertainty will not be reduced to zero
15
Policy and implementation challenge
• The LCFS requires publishing a GWI with infinite precision for each fuel, but our knowledge of the ‘real’ GWI is imperfect
• What is the best GWI to use within that framework?
16
Recall: three distinct variables for a single fuel
• g*ij = estimate of “physical” GWI of fuel i from model j
• i = real “physical” GWI: if fuel i is substituted for fuel k on a MJ/MJ basis, additional GHG release is (i - k). i is a random variable.
• gi = operational GWI of fuel i used in LCFS implementation
17
Optimal value for GWIElements of choice
Action: Implement a set of values gi for fuel GWI’sChoose objectiveFactors:
Distribution of i (real values): data is {g*ij}System response R({i}, {gi})Cost of ‘errors’ i – gi , R ≠ Ρ (real response)
The most likely value of i is not necessarily the optimal value of gi (safety factor principle).
18
Optimal gi’s
}{,,, iiihf ggVE R
Classical rule: Max{g} Expectation of value
19
where f and h are distributions of and R
20
What is the goal of the LCFS?• Climate
– Minimize difference i – gi ?
– Minimize T at a date (end of LCFS?)– Minimize additional GHG in atmosphere at a date (end of LCFS?) – Minimize total forcing up to a date?– Minimize probability that a condition will fail (LCFS increases one
of the above)?
• Other– Minimize social cost (including cost of T, food effects,
socioeconomic/welfare effects…?)– As of when (discounted how?)
21
Why might the cost of error be asymmetric?
• Could it be better/worse for i – gi to be +10 than -10? (Using too much/too little biofuel)
• ILUC is irreversible even if biofeedstock production stops.
• Encourage/obstruct advanced biofuels industry.
• Food effects from overuse of biofuel
22
Why might the cost of error be non-linear?
• Catastrophic events (Greenland ice cap, Gulf stream, peat feedback, etc.)
• Socio-economic effects• Other costs may be non-linear even if
symmetric:– Encourage/obstruct advanced biofuels industry.– Food effects from overuse of biofuel
23
Recognition of uncertainty entails attention to cost of error
• Asymmetric distribution of i means E(i) is not necessarily the optimal gi .
• This means the cost of error (functional form and asymmetry) also affects choice of gi
24
25
Uncertainty-explicit policy
• Few examples to learn from in fuel policy
• More theory, few examples in climate policy
• Many examples in health and safety regulation– Safety factor concept– Risk aversity for large costs
26
Policy MetricMetric
UncertaintyProxy
UncertaintyTreatment of Uncert.
Acid rain program
Tons SO2 and NOX
Low Low
Conservative
assumptions
PM2.5 standards
Measured PM2.5 conc.
Low Medium98th
percentile value used
Kyoto protocol
Direct CO2e Low MediumError
propagation recommended
RFS2Life cycle
CO2eHigh High
Partial uncertainty
analysis
LCFSLife cycle
CO2eHigh High None
How much investment in reducing uncertainty?
• --LUC is a complex social and biophysical phenomenon with significant variation across space and time.
• --Validating the role of any one factor/driver of LUC is difficult, at best, and very information-intensive.
• --So, even with substantial additional investments in GTAP, some uncertainties surrounding current iLUC estimates are irreducible.
28
“...in many situations, limitations of data, scientific understanding, and the predictive capacity of models will make (uncertainty) estimates unavailable, with the result that they must be supplemented with other sources of information."
• --Morgan et al. (2010), "Best Practices for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decisionmaking."
• "Expert elicitation," i.e., a formal assessment by individual experts, based on the full range of scientific evidence, of their best judgement of a subjective probability distribution for the value in question.
• --Expert elicitations have been used to address many questions of climate science (e.g., role of aerosols).
• --While not a substitute for further research, expert elicitation can allow for a formal expression of diversity of opinion not fully reflected in the literature
29
Extending policy scope
• The greatest uncertainty and largest GHG effects are associated with a few landscape types/areas
• A separate policy restricting use of such landscapes for economic purposes may reduce uncertainty and the risk of policy failure more than improved analysis and modeling.
30
Expert elicitation
• --Experts could review full suite of information defining the influence of biofuels on iLUC, not only from GTAP, but including other models (FASOM/FAPRI), databases, and studies from other disciplines
• --In lieu of quantitative values for iLUC, experts could recommend adaptive decision rules/guidance for categorizing feedstocks and fuel pathways according to relative iLUC risk, based on an assessment of all available information
31
Recommendations
• Model estimates should present complete, systematic uncertainty characterization
• Expert elicitation may help infer distribution of i from {g*ij}
• CARB should establish objectives• Cost of error should be examined and estimated• CARB should implement gi with explicit attention to cost
of error and distribution of i
• CARB should explore mechanisms to incorporate estimates from multiple models in determining gi.
32