Robust unit commitment using the parametric cost function ... presentation FERC June 2017.pdfSlide 1...
Transcript of Robust unit commitment using the parametric cost function ... presentation FERC June 2017.pdfSlide 1...
Slide 1
Robust unit commitment using the parametric cost function approximation
FERC, Washington, D.C.
June 25-26, 2017
Raymond PerkinsWarren B. Powell
CASTLE LabsPrinceton University
http://www.castlelab.princeton.edu
Slide 1© Warren B. Powell, 2014
Mission statement Main points:
» Classical view is that ISOs use a “deterministic model” for unit commitment.
» Considerable research has been done on the “stochastic unit commitment problem” that uses a stochastic lookahead model using scenario trees.
» Our thesis: • “stochastic programming” is a form of policy (a stochastic lookahead
model) for solving a stochastic problem (the real world).• ISOs use a modified deterministic lookahead model, where the
modifications enforce reserve requirements to ensure a robust solution.• We call this a parametric cost function approximation, and argue that this
is also a form of policy for solving stochastic unit commitment problems.» We will describe weaknesses in the use of scenario trees, and argue
why the parametric CFA (which is current industry practice) is likely to be much more effective for handling uncertainty in this context.
Slide 2
Variability and uncertainty
Slide 3
Win
d
So
lar
Variability and uncertainty
Illustration of forecasted wind power and actual» The forecast (black line) is deterministic (at time t, when the forecast
was made). The actuals are stochastic.
This is our forecast of the wind power at time t’, made at time t.
'ttf
This is the actual energy from wind, showingthe deviations from forecast.
Slide 4Current timet ' Some point in the futuret
Variability and uncertainty Forecasts evolve over time as new information arrives:
Slide 5
Actual
Rolling forecasts, updated each hour.
Forecast made at midnight:
Variability and uncertainty
ISOs handle uncertainty using a sequence of decisions» Day-ahead, intermediate term and real-time planning
each address different types of decisions
Slide 6
© 2010 Warren B. Powell Slide 7
Lecture outline
General modeling framework Stochastic lookahead policies The parametric cost function approximation Conclusions
Slide 7
© 2010 Warren B. Powell Slide 8
Lecture outline
General modeling framework Stochastic lookahead policies The parametric cost function approximation Conclusions
Slide 8
General modeling framework The objective function
Given a system model (transition function)
We refer to this as the base model to avoid confusions with lookahead models we will introduce later.
00
min , ( ) |
T
t t tt
C S X S S
Decision function (policy)State variableCost function
Finding the best policy
Expectation over allrandom outcomes
1 1, , ( )Mt t t tS S S x W
Slide 9
General modeling framework
There are two fundamental strategies for solving sequential decision problems:» Policy search – Search over a parameterized class of
functions for making decisions to optimize some metric.
» Lookahead approximations – Approximate the impact of a decision now on the future.
00
min , ( | ) |
T
t t tt
E C S X S S
Policy search Policy search – Two types of policies:
» Analytical functions that directly map states to actions (“policy function approximations”)
• Lookup tables – “when in this state, take this action”
• Parametric functions– Order-up-to policies: if inventory is less than s, order up to S.
• Locally/semi/non parametric– Release rate from a reservoir as a function of reservoir level
» Minimizing analytical approximations of costs and/or constraints (“cost function approximations”)
• Optimizing a deterministic model modified to handle uncertainty (buffer stocks, schedule slack)
( )( | ) arg min ( , | )
t t
CFAt t tx
X S C S x
X
Lookahead policies Lookahead approximations – Approximate the impact of a
decision now on the future:» Approximate lookahead models – Optimize over an approximate
model of the future:• Replace uncertain future with a deterministic approximation• Model future with a small sample of uncertain outcomes
» Approximating the value of being in a downstream state using machine learning (“value function approximations”)
( ) arg min ( , ) ( , )t
VFA x xt t x t t t t t tX S C S x V S S x
*' ' ' 1
' 1
( ) arg min ( , ) min ( , ( )) | | ,
t
T
t t x t t t t t t t tt t
X S C S x C S X S S S x
*' ' ' 1
' 1( ) arg max ( , ) max ( , ( )) | | ,
t
T
t t x t t t t t t t tt t
X S C S x C S X S S S x
Lookahead policies
The ultimate lookahead policy is optimal
Slide 13
Expectations that we cannot compute
Maximization that we cannot compute
Designing policies
The ultimate lookahead policy is optimal
Instead, we have to solve an approximation called the lookahead model:
» A lookahead policy works by approximating the lookahead model.
Slide 14
*' ' ' 1
' 1( ) arg max ( , ) max ( , ( )) | | ,
t
T
t t x t t t t t t t tt t
X S C S x C S X S S S x
*' ' ' , 1
' 1( ) arg max ( , ) max ( , ( )) | | ,
t
t H
t t x t t tt t tt t t tt tt t
X S C S x C S X S S S x
Lookahead policies
We use a series of approximations:» Horizon truncation – Replacing a longer horizon
problem with a shorter horizon» Stage aggregation – Replacing multistage problems
with two-stage approximation.» Outcome aggregation/sampling – Simplifying the
exogenous information process» Discretization – Of time, states and decisions» Dimensionality reduction – We may ignore some
variables (such as forecasts) in the lookahead model that we capture in the base model (these become latentvariables in the lookahead model).
1) Policy function approximations (PFAs)» Lookup tables, rules, parametric/nonparametric functions
2) Cost function approximation (CFAs)»
3) Policies based on value function approximations (VFAs)»
4) Direct lookahead policies (DLAs)» Deterministic lookahead/rolling horizon proc./model predictive control
» Chance constrained programming
» Stochastic lookahead /stochastic prog/Monte Carlo tree search
» “Robust optimization”
Four (meta)classes of policies
( )( | ) arg min ( , | )
t t
CFAt t tx
X S C S x
X
,' ',..., ' 1
( ) arg min ( , ) ( , )tt t t H
TLA Dt t tt tt tt ttx x t t
X S C S x C S x
( ) arg min ( , ) ( , )t
VFA x xt t x t t t t t tX S C S x V S S x
' '' 1
( ) arg min ( , ) ( ) ( ( ), ( ))t
TLA St t tt tt tt tt
t tX S C S x p C S x
xtt , xt ,t1,..., xt ,tT
,' ',..., ( ) ' 1
( ) arg min max ( , ) ( ( ), ( ))tt t t H t
TLA ROt t tt tt tt ttx x w W t t
X S C S x C S w x w
[ ( )] 1t tP A x f W
Polic
y se
arch
Look
ahea
dap
prox
imat
ions
Lookahead policies
Lookahead policies peek into the future» Optimize over deterministic lookahead model
The base model
. . . .
The
look
ahea
dm
odel
t 1t 2t 3t
Slide 17
Lookahead policies
Lookahead policies peek into the future» Optimize over deterministic lookahead model
. . . .
The
look
ahea
dm
odel
t 1t 2t 3t The base model
Slide 18
Lookahead policies
Lookahead policies peek into the future» Optimize over deterministic lookahead model
. . . .
The
look
ahea
dm
odel
t 1t 2t 3t The base model
Slide 19
Lookahead policies
Lookahead policies peek into the future» Optimize over deterministic lookahead model
. . . .
The
look
ahea
dm
odel
t 1t 2t 3t The base model
Slide 20
© 2010 Warren B. Powell Slide 21
Lecture outline
General modeling framework Stochastic lookahead policies The parametric cost function approximation Conclusions
Slide 21
Stochastic lookahead policies
Stochastic lookahead» Here, we approximate the information model by using a
Monte Carlo sample to create a scenario tree: 1am 2am 3am 4am 5am …..
Change in wind speed
Change in wind speed
Change in wind speedSlide 22
Stochastic lookahead policies
We can then simulate this lookahead policy over time:
. . . .
t 1t 2t 3t
The
look
ahea
dm
odel
The base modelSlide 23
. . . .
t 1t 2t 3t
Stochastic lookahead policies
We can then simulate this lookahead policy over time:
The
look
ahea
dm
odel
The base modelSlide 24
. . . .
t 1t 2t 3t
Stochastic lookahead policies
We can then simulate this lookahead policy over time:
The
look
ahea
dm
odel
The base modelSlide 25
. . . .
t 1t 2t 3t
Stochastic lookahead policies
We can then simulate this lookahead policy over time:
The
look
ahea
dm
odel
The base modelSlide 26
Stochastic lookahead policies
Two stage lookahead approximation
Slide 27
0x
1) Schedule steam
1
2
N
1W
2) See wind:
1 2, ,..., Tx x x3) Schedule turbines
Stochastic lookahead policies
Creating wind scenarios (Scenario #1)
Slide 28
Stochastic lookahead policies
Creating wind scenarios (Scenario #2)
Slide 29
Stochastic lookahead policies
Creating wind scenarios (Scenario #3)
Slide 30
Stochastic lookahead policies
Creating wind scenarios (Scenario #4)
Slide 31
Stochastic lookahead policies
Creating wind scenarios (Scenario #5)
Slide 32
Stochastic lookahead policies
The two-stage approximation
0
1) Schedulesteam
x
2) See wind:
3) Schedule turbinesSlide 33
Stochastic lookahead policies
Slide 34
Load shedding event:
Stochastic lookahead policies
Slide 35
Load shedding event:
Stochastic lookahead policies
Slide 36
Load shedding event:
Actual wind
Hour ahead
forecast
Stochastic lookahead policies
Slide 37
Note the dip in steam just when the outage occurs – This is because we forecasted that wind would cover the load.
Stochastic lookahead policies
Slide 38
When we allow the model to see into the future (the drop in wind), it schedules additional steam the day before. But only when we need it!
Stochastic lookahead policies
The two-stage approximation
0
1) Schedulesteam
x
2) See wind:
3) Schedule turbinesSlide 39
Downward wind shift
© 2010 Warren B. Powell Slide 40
Lecture outline
General modeling framework Stochastic lookahead policies The parametric cost function approximation Conclusions
Slide 40
1) Policy function approximations (PFAs)» Lookup tables, rules, parametric/nonparametric functions
2) Cost function approximation (CFAs)»
3) Policies based on value function approximations (VFAs)»
4) Lookahead policies» Deterministic lookahead/rolling horizon proc./model predictive control
» Chance constrained programming
» Stochastic lookahead /stochastic prog/Monte Carlo tree search
» “Robust optimization”
Four (meta)classes of policies
( )( | ) arg min ( , | )
t t
CFAt t tx
X S C S x
X
,' ',..., ' 1
( ) arg min ( , ) ( , )tt t t H
TLA Dt t tt tt tt ttx x t t
X S C S x C S x
( ) arg min ( , ) ( , )t
VFA x xt t x t t t t t tX S C S x V S S x
' '' 1
( ) arg min ( , ) ( ) ( ( ), ( ))t
TLA St t tt tt tt tt
t tX S C S x p C S x
xtt , xt ,t1,..., xt ,tT
,' ',..., ( ) ' 1
( ) arg min max ( , ) ( ( ), ( ))tt t t H t
TLA ROt t tt tt tt ttx x w W t t
X S C S x C S w x w
[ ( )] 1t tP A x f W
A deterministic lookahead model» Optimize over all decisions at the same time
» In a deterministic model, we mix generators with different notification times:
• Steam generation is made day-ahead• Gas turbines can be planned an hour ahead or less
Parametric cost function approximation
Gas turbines
' ' 1,...,24
' ' 1,...,24
' ''
( )
min ( , )
tt t
tt t
t H
tt ttx t ty
C x y
Slide 42
Steam generation
A deterministic lookahead policy» This is the policy produced by solving a deterministic
lookahead model
» No ISO uses a deterministic lookahead model. It would never work, and for this reason they have never used it. They always modify the model to produce a robust solution.
Parametric cost function approximation
Steam generation
Slide 43
' ' 1,...,24
' ' 1,...,24
' ''
( )
( )= min ( , )
tt t
tt t
t H
t t tt ttx t ty
X S C x y
Gas turbines
A robust CFA policy» The ISOs introduce reserves:
» This modification is a form of parametric function (a parametric cost function approximation). It has to be tuned to produce a robust policy.
Parametric cost function approximation
Slide 44
max, ' , ' '
max, ' , ' '
Up-ramping reserve
Down-ramping reserve
upt t t t tt
downt t t t tt
x x L
x x L
' ' 1,...,24
' ' 1,...,24
' ''
( )
( | )= min ( , )tt t
tt t
t H
t t tt ttx t ty
X S C x y
Parametric cost function approximation
An energy storage problem:
Slide 45
Parametric cost function approximation
Benchmark policy – Deterministic lookahead
Slide 46
Parametric cost function approximation
Parametric cost function approximations» Replace the constraint
with:» Lookup table modified forecasts (one adjustment term for
each time in the future):
» Exponential function for adjustments (just two parameters)
» Constant adjustment (one parameter)
Slide 47
't t
' ' ' 'wr wd Ett tt t t ttx x F
'wrttx '
wdttx
2 ( ' )
' ' 1 '
t twr wd Ett tt ttx x e F
' ' 'wr wd Ett tt ttx x F
Parametric cost function approximation
Optimizing the CFA:» Let be a simulation of our policy given by
» We then compute the gradient with respect to
» The parameter is found using a classical stochastic gradient algorithm:
We tested several stepsize formulas and found that ADAGRAD worked best:
Slide 48
( ) ( , )F F
( , )F
0
( , ) ( ), ( ( ) | )T
t t tt
F C S X S
1 1( , )n n n nnF
Parametric cost function approximation
Optimizing the CFA:» We compute the gradient by applying the chain rule
» Where the interaction from one time period to the next is captured using
» Assuming there are no integer variables, these equations are quite easy to compute.
» For real stochastic unit commitment problems, we are going to need to use a derivative-free algorithm.
Slide 49
Parametric cost function approximation
Optimal adjustment parameters for each model
Slide 50
Lookup tableConstant parameter
Constant parameter
Lookup table
Exponential function
Exponential function
More accurate forecasts Less accurate forecasts
1.0
0.0
1.0
0.0
Parametric cost function approximation
Improvement over deterministic benchmark:
» Significant improvement over deterministic benchmark (an untuned lookahead policy).
Slide 51
Decreasing forecast accuracy:
Parametric cost function approximation
0
20000
40000
60000
80000
100000
120000
140000
1 36
71
106
141
176
211
246
281
316
351
386
421
456
491
526
561
596
631
666
701
736
771
806
841
876
911
946
981
1016
10
51
1086
11
21
1156
11
91
1226
12
61
1296
13
31
1366
14
01
1436
14
71
1506
15
41
1576
16
11
1646
16
81
1716
17
51
1786
18
21
1856
18
91
1926
19
61
1996
MW
5‐min Time Intervals
SMART‐ISO ‐ Unconstrained Grid ‐ 22‐28 Jul 2010 Wind Buildout 4 ‐ No ramping reserves
Actual Demand (Exc) Simulated (Used) Wind Simulated Storage Power Simulated Fast Power Simulated Slow Power
Slide 52
Parametric cost function approximation
0
20000
40000
60000
80000
100000
120000
140000
1 36
71
106
141
176
211
246
281
316
351
386
421
456
491
526
561
596
631
666
701
736
771
806
841
876
911
946
981
1016
10
51
1086
11
21
1156
11
91
1226
12
61
1296
13
31
1366
14
01
1436
14
71
1506
15
41
1576
16
11
1646
16
81
1716
17
51
1786
18
21
1856
18
91
1926
19
61
1996
MW
5‐min Time Intervals
SMART‐ISO ‐ Unconstrained Grid ‐ 22‐28 Jul 2010 Wind Buildout 4 ‐ No ramping reserves
Actual Demand (Exc) Simulated (Used) Wind Simulated Storage Power Simulated Fast Power Simulated Slow Power
0
20000
40000
60000
80000
100000
120000
140000
1 36
71
106
141
176
211
246
281
316
351
386
421
456
491
526
561
596
631
666
701
736
771
806
841
876
911
946
981
1016
10
51
1086
11
21
1156
11
91
1226
12
61
1296
13
31
1366
14
01
1436
14
71
1506
15
41
1576
16
11
1646
16
81
1716
17
51
1786
18
21
1856
18
91
1926
19
61
1996
MW
5‐min Time Intervals
SMART‐ISO ‐ Unconstrained Grid ‐ 22‐28 Jul 2010 Wind Buildout 4 ‐ Ramping reserves 9GW
Actual Demand (Exc) Simulated (Used) Wind Simulated Storage Power Simulated Fast Power Simulated Slow Power
Notice that we get uniformly more gas turbine reserves at all points in time (because we asked for it)
Slide 53
Slide 54
Paper available on arXiv:
© 2010 Warren B. Powell Slide 55
Lecture outline
General modeling framework Stochastic lookahead policies The parametric cost function approximation Conclusions
Slide 55
Conclusions:» Weaknesses of stochastic programming (with scenario trees)
• Computationally intensive• It does not ensure robust solutions (needs too many scenarios)• Many approximations are required (e.g. two stage).
» Parametric cost function approximations:• Generalizes standard industry practice• Uses domain knowledge to ensure robust policies across a much wider
range of scenarios.• Resulting models can be implemented using existing commercial
software.
» Parametric CFAs open up an entirely new research directions for stochastic unit commitment:
• Propose new parameterizations to achieve robustness at lowest cost.• Design algorithms (either derivative-based or derivative-free) to optimize
the parameterized policy.• Need to design algorithms for online learning.
Slide 56
Thank you!
http://energysystems.Princeton.edu