Direct Policy Conditioning for reservoir operation
-
Upload
andrea-castelletti -
Category
Engineering
-
view
154 -
download
1
description
Transcript of Direct Policy Conditioning for reservoir operation
Matteo Giuliani1, Andrea Castelletti1,2, Patrick M. Reed3
1 Dipartimento di Elettronica, Informazione, e Bioingegneria, Politecnico di Milano, Milano, Italy 2 Institute of Environmental Engineering ETH-Z, Zurich 3 Department of Civil and Environmental Engineering, Cornell University, Ithaca, NY
IFAC 2014 CAPE TOWN -‐ZA
Modelling and Control of Water Systems
Improving the protection of aquatic ecosystems by dynamically constraining reservoir operation via Direct Policy Conditioning
The fall of the social planner myth?
Stakeholder 1’s utility
Sta
keh
old
er 2
’s u
tility
utopia
REALITY
SOCIAL PLANNER’S PARETO OPTIMAL
dominated
unfeasible
A real world example
Hydropower reservoir
Power plant
Como city
Penstock
River Adda
River Adda
LegendLario
Lario catchment
River
Irrigated area
0 10 20 30 40 505Kilometers
Anghileri, D. et al. Journal of Water Resources Planning and Management, 139(5), 492–500, 2013
LakeComo
LakeComo
r
s 1
s 2
s 3
u 1
u 2
u 3
R2
R1
R2
R1
hydropower plant
irrigated area
H2
H1
H3
H2
H1
H3
q 3
q 2
q 1
q 3
q 2
q 1
s 1
s 2
s 3
u 2
u 3
u 1 m 1
m 2
m 3
(•)
m (•) (•)
(•)
UNCOORDINATED CENTRALIZED
(a) (b)
r
FIG. 3. The model scheme under uncoordinated (left) and centralized (right) man-agement.
23
800 900 1000 1100 1200 1300 1400 1500 1600
460’000
470’000
480’000
490’000
Irrigation deficit [m3/s]2
Hyd
ropo
wer
reve
nue
[eur
o/da
y]
H
ab
C6C5
C4
C3
C2
C1
CO2 CO1 UCC6 C5
C4
C3
C2
C1
UC
LakeComo
LakeComo
r
s 1
s 2
s 3
u 1
u 2
u 3
R2
R1
R2
R1
hydropower plant
irrigated area
H2
H1
H3
H2
H1
H3
q 3
q 2
q 1
q 3
q 2
q 1
s 1
s 2
s 3
u 2
u 3
u 1 m 1
m 2
m 3
(•)
m (•) (•)
(•)
UNCOORDINATED CENTRALIZED
(a) (b)
r
FIG. 3. The model scheme under uncoordinated (left) and centralized (right) man-agement.
23
UNCOORDINATED
CENTRALIZED (SOCIAL PLANNER)
Efficiency vs acceptability: how to trade-off?
acceptability
eff
icie
nc
y
utopia SOCIAL
PLANNER
INDIVIDUALISM
acceptability of the
social planner
efficiency of individualism
Giuliani M. et al., Journal of Water Resources Planning and Management, 2014
coordination mechanism design
Direct Policy Conditioning
an approach to condition the individualistic control policy and push it towards a social welfare equilibrium
Direct Policy Conditioning
an approach to condition the individualistic control policy and push it towards a social welfare equilibrium
PRIMARY obj.
SEC
ON
DA
RY o
bj.
utopia SECONDARY’s
OPTIMUM
PRIMARY’s OPTIMUM
COMPUTE THE SOCIAL PLANNER POLICIES
Direct Policy Conditioning
1
PRIMARY obj.
SEC
ON
DA
RY o
bj.
utopia SECONDARY’s
OPTIMUM
PRIMARY’s OPTIMUM
COMPUTE THE SOCIAL PLANNER POLICIES
Direct Policy Conditioning
GET INSIGHTS ON HOW TO CONDITION THE PRIMARY’S POLICY
1 2
PRIMARY obj.
SEC
ON
DA
RY o
bj.
utopia SECONDARY’s
OPTIMUM
PRIMARY’s OPTIMUM
COMPUTE THE CONSTRAINED
PRIMARY POLICY
COMPUTE THE SOCIAL PLANNER POLICIES
Direct Policy Conditioning
GET INSIGHTS ON HOW TO CONDITION THE PRIMARY’S POLICY
1 2
3 PRIMARY obj.
SEC
ON
DA
RY o
bj.
utopia SECONDARY’s
OPTIMUM
PRIMARY’s OPTIMUM
COMPUTE THE CONSTRAINED
PRIMARY POLICY
GET INSIGHTS ON HOW TO CONDITION THE PRIMARY’S POLICY
Multi-objective optimization using the Direct Policy Search approach
Direct Policy Conditioning
Policy parameters vectors Objectives values
1 2
3 PRIMARY obj.
SEC
ON
DA
RY o
bj.
utopia SECONDARY’s
OPTIMUM
PRIMARY’s OPTIMUM
COMPUTE THE CONSTRAINED
PRIMARY POLICY
Multi-objective optimization using the Direct Policy Search approach
Direct Policy Conditioning
Input Variable Selection of the most relevant parameters in explaining the secondary objectives
Policy parameters vectors Objectives values
Subset of policy parameters to be conditioned
1 2
3 PRIMARY obj.
SEC
ON
DA
RY o
bj.
utopia SECONDARY’s
OPTIMUM
PRIMARY’s OPTIMUM
Single-objective optimization of the primary objective with restricted constraints on the sensitive policy parameters
Multi-objective optimization using the Direct Policy Search approach
Direct Policy Conditioning
Input Variable Selection of the most relevant parameters in explaining the secondary objectives
Policy parameters vectors Objectives values
Subset of policy parameters to be conditioned
1 2
3 PRIMARY obj.
SEC
ON
DA
RY o
bj.
utopia SECONDARY’s
OPTIMUM
PRIMARY’s OPTIMUM
CASE STUDY
The Susquehanna river system
(b)
(a)
atomicpower plant
Baltimore
ChesterFishery andboating
FERC environmentalrequirements
Conowingohydropower plant
Muddy RunfacilityMarietta
station
PennsylvaniaMaryland
lateral inflow
Susquehanna RiverMuddy Run inflow
Lower Susquehanna
River
Maryland
New York
Pennsylvania
Conowingo pondChester
Baltimore
(b)
(a)
atomicpower plant
Baltimore
ChesterFishery andboating
FERC environmentalrequirements
Conowingohydropower plant
Muddy RunfacilityMarietta
station
PennsylvaniaMaryland
lateral inflow
Susquehanna RiverMuddy Run inflow
Lower Susquehanna
River
Maryland
New York
Pennsylvania
Conowingo pondChester
Baltimore
DPC experimental setting
1
SOCIAL PLANNER POLICIES • POLICY: Gaussian Radial Basis function with 2 inputs (level & time) + 4
output (release outputs) + 4 basis functions: 32 parameters • OPTIMIZATION: Borg MOEA parameterized as in Hadka and Reed [2013] • NFE = 1,000,000 per replica • 30 replications to avoid dependence on randomness
1
Social Planner policies Giuliani. M. et al. Water Resources Research, 2014
SOCIAL PLANNER POLICIES • POLICY: Gaussian Radial Basis function with 2 inputs (level & time) + 4
output (release outputs) + 4 basis functions: 32 parameters • OPTIMIZATION: Borg MOEA parameterized as in Hadka and Reed [2013] • NFE = 1,000,000 per replica • 30 replications to avoid dependence on randomness
DPC experimental setting
INPUT VARIABLE SELECTION • Tree based iterative input selection [Galelli and Castelletti, 2013]
1
2
Input Variable Selection
75
50
25
0
expla
ined
varia
nce
b t1 bt
2w42 c t
3 b t3w4
3bt4 w4
4
(a) Selected features and corresponding contribution in explaining the Environment objective
b t1 bt
2w42 c t
3 b t3w4
3bt4 w4
4−1
−0.5
0
0.5
1
lower bound policyreference policyPareto-optimal set
(b) Decision variables selected on the Pareto-optimal set
decis
ion va
riable
x1
x2
x3
u1
Gaussian Radial Basis Function [Giuliani et al. 2014]
b = Basis radius
c = Basis centre
w = Network weights
60% explained variance
Input Variable Selection
Reference p.: the best for the environment
Lower bound p. : current situation
para
met
er v
alue
SOCIAL PLANNER POLICIES • POLICY: Gaussian Radial Basis function with 2 inputs (level & time) + 4
output (release outputs) + 4 basis functions: 32 parameters • OPTIMIZATION: Borg MOEA parameterized as in Hadka and Reed [2013] • NFE = 1,000,000 per replica • 30 replications to avoid dependence on randomness
DPC experimental setting
INPUT VARIABLE SELECTION • Tree based iterative input selection [Galelli and Castelletti, 2013]
CONSTRAINED PRIMARY POLICY • Baseline policy with constraints on 8 policy parameters • Default Borg MOEA parameterization [Hadka and Reed 2013] • NFE = 100,000 per replication • 30 replications to avoid dependence on randomness • Historical horizon 1999 (drought)
2
3
1
DPC policies’ performance
DPC policies’ performance
+18.6 x 106
US$/year
+ 36%
+46%
- 30% but ..
Conclusions
§ Direct Policy Conditioning as a coordination mechanism design
§ Preliminary results seem to be interesting in terms of improved
perfomance of current operation in the Susquehanna rb
§ Weakness in the physical interpretation of the parameters: how to
communicate the conditioning to the dam operator?
§ Sensitivity to the conditioning setting
THANKS
Programmed event supported by the TC
EGU General Assembly, Vienna 12 April—17 April 2015
EGU General Assembly
The EGU General Assembly 2015 will bring togethergeoscientists from all over the world into one meet-ing covering all disciplines of the Earth, Planetary andSpace Sciences. Especially for young scientists theEGU aims to provide a forum to present their workand discuss their ideas with experts in all fields ofgeosciences.In the divisions Energy, Resources and the Environ-ment (ERE) and Hydrological Sciences (HS) the fol-lowing sessions are proposed:
• Design and Operation of Combined Hydro/Wind/SolarPower Generation Systems: Computer Based Controland Optimization;
• Design and Operation of Water Resource Systems:Computer Based Control and Optimization.
Motivation
Many environmental systems have been modified andare still being modified by human intervention. Thisintervention usually takes the form of the constructionof additions to the system intended to change systembehaviour to better serve the needs of society.This implies that these systems and their behaviourare being designed. They are no longer governed bynatural processes alone. Therefore models of both thenatural and the artificial part of the system will beneeded. As the demands placed on water systems bysociety increase and are increasingly in conflict witheach other, it will become harder to define goals forthe modification of these systems and their behaviour.It will also become harder to design systems and oper-ating rules to satisfy these goals.The aim of these sessions is to bring together expertsin the fields of water management, hydro-, solar-, andwind-power, control theory and operations research to
discuss novel methods or novel ways of using tradi-tional methods to define and implement desired beha-viour for environmental systems.
Design and Operation of Water ResourceSystems: Computer Based Control andOptimization
For control theory water systems pose some uniquechallenges because of the presence of large delays andvery limited means of control. In fact for some sys-tems the limits on the size of the change that can beeffected in a given time period necessitate the use offorecasts to anticipate on system behaviour. For oper-ations research the special challenge is the presence ofincommensurable and conflicting optimization targets,the complex network of relations between stakeholdersand the lack of one clear shared motivation amongststakeholders. Moreover, a new awareness of more vari-ability in the climate on longer time scales and rapidsocial changes both pose new challenges for the decisionmaking process. This implies a need for more frequentreconsideration of decisions and a shorter time scale forthe decision process. This process will therefore needfaster models, for instance simplified dynamic modelsof hydrological systems, statistical process emulators,surrogate models (e.g. linear or nonlinear regression)based on data to feed faster optimization algorithms.Currently the following people and institutions are in-volved in the preparation of this session:
• Niels Schütze, Dresden University of Technology,Germany;
• Andrea Castelletti, Politecnico di Milano, Italy;• Francesca Pianosi, University of Bristol, United
Kingdom;• Renata Romanowicz, Institute of Geophysics,
Polish Academy of Sciences, Warszawa;• Ronald van Nooijen and Alla Kolechkina, Delft
University of Technology, Netherlands.
Design and Operation of Combined Hy-dro/Wind/Solar Power Generation Sys-tems: Computer Based Control and Op-timization
In most locations the yield of wind power or solar poweris uncertain. Hydropower seems an attractive means ofproviding backup power and storage of energy for fu-ture use. Combined schemes seem attractive, but willneed automatic control to optimize their yield. Un-certainty about yield and future supply and demandis a key issue for the management of these combinedschemes. They may also need special facilities for in-tegration in the current energy distribution infrastruc-ture.
Currently the following people and institutions are in-volved in the preparation of this session:
• Demetris Koutsoyiannis and Andreas Efstra-tiadis, National Technical University of Athens,Greece;
• Andrea Castelletti, Politecnico di Milano, Italy;
• Burlando Paolo, ETH Zürich, Zwitzerland;
• Patrick Michael Reed, Cornell University, USA;
• Alla Kolechkina and Ronald van Nooijen, DelftUniversity of Technology, Netherlands.
Key dates
• Call for papers for EGU 2015: 15 October 2014
• Deadline for receipt of abstracts: 7 January 2015
• Letter of acceptance to key authors: 23 January2015
• Conference: 12 April to 17 April 2015 in Vienna,Austria
Programmed event supported by the TC
26
thIUGG General Assembly 2015, Earth and Environmental Sciences for Future Generations
Prague, Czech Republic June 22 - July 2, 2015
IAHS Workshop Hw07
Announcement
At the 26th IUGG General Assembly inPrague in 2015 there will be an IAHS work-shop on Control of Water Resource SystemsHw07. The workshop is being organized underthe auspices of the International Commissionon Water Resources Systems (ICWRS).
Motivation
Today it is rare to find a water resource sys-tem where the interaction with society can beignored. Most systems consist of both nat-ural and manmade components and are gov-erned by both natural processes and processeswithin society. The interaction between soci-ety and the natural system is complex. Animportant part of this interaction consists ofour attempts as humans to alter the systembehaviour through the construction and ma-nipulation of structures such as wells, dams,pumps, weirs, gates, sluices and locks. Ina changing world it can no longer be takenfor granted that the operational rules for themanipulation of the manmade components ofthe water resource system will be appropriateover the whole life time of the infrastructure.
This workshop is intended for presentations onthe formulation and adaptation of operationalrules for the automated manipulation of man-made components of water resource systemswith changing boundary conditions, or, lessformally, for presentations on computer con-trol of water resource systems in a world influx.
Convener team
Currently the following people and institu-tions are involved in the preparation of thissession.
• Alla Kolechkina, Delft University ofTechnology, Netherlands
• Ronald van Nooijen, Delft University ofTechnology, Netherlands
• Andrea Castelletti, Politecnico di Mil-ano, Italy;
26
thGeneral Assembly of the Inter-
national Union of Geodesy and Geo-
physics (IUGG)
A better understanding of the way in whichour planet functions and of the effects of our
actions on its behaviour is needed to providefor the needs of future generations.This Scientific Assembly to be held in Praguefrom 22 June to 2 July 2015 will provide anopportunity for scientists from all geophysicaldisciplines and from all countries to meet andexchange knowledge and ideas. The Assemblyalso will also give the participants the oppor-tunity to inform the general public and policymakers.
Key dates for this workshop
• Abstract submission open: September2014
• Deadline for receipt of workshop ab-stracts: 31 January 2015
• Early bird registration deadline : 10April 2015
• Standard fee registration deadline : 15June 2015
• Conference: 22 June to 2 July in Prague,Czech Republic
TC 8.3 meeting – Wed 27 12:00 Dassen Room (Westin)