Power systemsilablri

Ilab METISOptimization of Energy Policies

Olivier Teytaud + Inria-Tao + Artelys

TAO project-team

INRIA Saclay le-de-France

O. Teytaud, Research Fellow,[email protected]://www.lri.fr/~teytaud/

Outline

Ilab Metis, working on power systems

Principles of power system simulationClassical modelReserves constraint

Recourse

Network constraints

This simple model does not solve hydro

It gets more complicated in the future

Alternate tools:Real-time control ?

Games ?

Bilevel problems

Ilab METIS
www.lri.fr/~teytaud/metis.html

Metis = Tao + ArtelysTAO tao.lri.fr, Machine Learning & OptimizationJoint INRIA / CNRS / Univ. Paris-Sud team

12 researchers, 17 PhDs, 3 post-docs, 3 engineers

Artelys www.artelys.com SME - France / US / Canada
- 50 persons==> collaboration through common platform

ActivitiesOptimization (uncertainties, sequential)

Application to power systems

O. Teytaud, Research Fellow,[email protected]://www.lri.fr/~teytaud/

Academic Highlights

Bandit algorithms
Pascal / Nips 06 challenge on dynamic bandits

MCTS for GamesFirst ever win against professional
players in the game of Go

State of the art in MineSweeper

Urban Rivals AI (17 M players)

Planning
Temporal satisficing track, IPC 2011 (International Planning Competition)

Industrial application

Building power systems is expensive
power plants, HVDC links, networks...

Non trivial planning questionsCompromise: should we move solar power to the south and build networks ?

Is a HVDC connection a good idea between X and Y ?

What we do:Simulate the operational level of a given power system (this involves optimization of operational decisions)

Optimize the investments

Planning/controlPluriannual planning: evaluate marginal costs of hydroelectricity

Taking into account stochasticity and uncertainties==> IOMCA (ANR)

High scale investment studies (e.g. Europe+North Africa)Long term (2030 - 2050)

Huge (non-stochastic) uncertainties

Investments: interconnections, storage, smart grids, power plants...==> POST (ADEME)

Moderate scale (Cities, Factories)Master plan optimization

Stochastic uncertainties ==> Citines project (FP7)

Specialization on Power Systems

Example: interconnection studies
(demand levelling, stabilized supply)

The POST project supergrids simulation and optimization

European subregions:

- Case 1 : electric corridor France / Spain / Marocco

- Case 2 : south-west (France/Spain/Italiy/Tunisia/Marocco)

- Case 3 : maghreb Central West Europe

==> towards a European supergrid

Relatedideas in AsiaMature technology:HVDC links(high-voltage direct current)

Investment decisions through simulations

IssuesDemand varying in time, limited previsibility

Transportation introduces constraints

Renewable ==> variability ++

MethodsMarkovian assumptions ==> wrong

Simplified models ==> Model error >> optimization error

Our approachMachine Learning on top of Mathematical Programming

Hybridization reinforcement learning / mathematical programming

Math programmingNearly exact solutions for a simplified problem

High-dimensional constrained action space

But small state space & not anytime

Reinforcement learningUnstable

Small model bias

Small / simple action space

But high dimensional state space & anytime

Who ?

Students + EngineersS. Astete Morales (ph.D. student)

J-J Christophe (engineer)

A. Coutoux (ph.D. Student)

M-L Cauwet (ph.D. Student)

J. Decock (ph.D. student)

J. Liu (ph.D. Student)

D. Lupien Saint-Pierre (post-doc)

Old crocs:M. Schoenauer & M. Sebag (Research Directors)

O. Teytaud (Research Fellow)

Artelys www.artelys.com (most with Ph.D.)
==> many engineers and experts of power systems

Admin. side: A.-C. Lamballe, R. Ronchaud, M. Gilliaud

Many contacts worldwide

Outline



Recourse

Network constraints


Future is (even) more complicated


Games ?

Bilevel problems

The problem in one slide

The assets:I have plenty of (possibly faulty) power plants (each of them located on a network)

I have plenty of (stochastic) electric loads

Total production = total consumption + loss (otherwise many things broken!)

Many constraints on the network

Decisions:Strategic decisions (investments)

Intermediate decisions: Hydro-power = bounded resource

Tactical decisions: which power plant to switch on/off

Let us start with short term

Thermal power plant producing p MW during 1h:productionCost= C x p (fuel cost) + startUpCost (if it was off and p>0) + C2 x p2 - (a quadratic term for fun)

Several time steps, several thermal power plants

pi MW during 1h with power plant #i:productionCost= C x pi + startUpCosti + C2 x pi2 -

==> ok, looks simple: we just optimize the sum of costs.


Principles of power system simulationClassical modelReserves constraint (for adaptivity against uncertain load, faulty power plants, renewable intermittencies) ==> rules of thumb such as short term PP>x MW etc

Recourse

Network constraints




Games ?

Bilevel problems



Recourse (==> when big change, re-optimize with new data ==> fast UC)

Network constraints




Games ?

Bilevel problems



Recourse

Network constraints (N-k style: should be ok if k connections destroyed)




Games ?

Bilevel problems

Outline



Recourse

Network constraints




Games ?

Bilevel problems

Let us see why it is more complicated than that

This optimization problem is termed Unit Commitment (UC).

There is a problem with hydro-power.

Inflows are very stochastic

Water comes for free (nearly), but it is limited. ==> more subtle optimization here

At least this uncertainty must be tackled

How to handle hydro ?

For the moment, we have seen deterministic optimization problems.

Solution 1: Just optimize, assuming that inflows = average inflow value given observed information.

This is so-called Model predictive control (many variants, e.g. using a quantile of the inflow random process).

Not satisfactory if high stochasticity

Solution 2: optimize a policy, not decisions.

Bellman's solution

Let's work on time steps [1,....,365x6] (6 years)

Define V(x,t) = expected cost between time t and T=365*6 if water stock 1 = x1,

water stock 2 = x2,

Water stock d = xd

V(x,T) = easy to compute

Given V(x,t), V(x,t-1) computed by UC

V(x,t) computed Bellman-style

V(x,t+1) is known

V(x,t) = optimal expected cost to go = minimum of: instantaneous cost Ct(a) + future cost V(x',t+1) where x'=future state when choosing action a

It is stochastic! ==> expected value.

So far so good

But V(x,t) is not correctly defined.

It also depends on random processes (load, inflows...) ==> V(random process,x,t)

It also depends on PP status (on / off) ==> we need additional variables in x, not only the stock levels.

More complete picture

Let's work on time steps {1,....,365x6} (6 years)

Define V(a, x,t) = expected cost between time t and T=365*6 if x = all state variables

random processes = a

V(a,x,T) = easy to compute

Given V(a,x,t), V(a,x,t-1) computed by UC:V(a,x,t-1) = min cost(x,decision) + V(a,x',t) with x' = nextState(a,x,t,decision)

So, Bellman-style:

We have plenty of UC problems (one for each V(a,x,t) to be computed).

Usually people rewrite the optimization problems as linear problems or quadratic problems, because otherwise solving is too slow.

The real problem is usually simplified a lot so that it can be solved.

The problem becomes more and more complicated.

Standard tool.

This version is probably run by electricity producers in your country.

We will now see phenomena which are poorly handled by this algorithm.

Outline



Recourse

Network constraints




Games ?

Bilevel problems

New decision variables

Day-ahead adaptivity: some big companies (loads) accept adaptivity contracts:At 5pm, the producer says tomorrow, I multiply prices by 10 (he can do so 22 days per year)

The other days, prices are reduced

With renewable energy, uncertainty increases; the principle above can be generalized:Small time scales, smaller consumers

Other loads: heat networks, fridges, electric vehicles

Summary: models

Simple UC problem = optimizing deterministically.

Advanced version = (spinning) reserves (this is also simplified).

Hydro-power (& other stocks!): in Bellman's solution we must solve plenty of UC problems and take into accounts stocks; this involves computing V(a,x,t) backwards (traditionally Bellman, but you might use your favorite game algorithm).

Investment problem on top of all this.

Still too simple

Ok, we solve plenty of UC for computing the hydro power or other stocks.

Theoretically, solves the problem of uncertainty in inflows (but very expensive)

Many other uncertainties are not negligible:PP directly connected to distribution networks: voltage issues introduce many constraints.

Wind power uncertainty: day-ahead planning often wrong ==> real-time

Long term: how to get the probability that PV units become cheaper than XXX in 2050 ?

Outline



Recourse

Network constraints




Games ?

Bilevel problems

Real-time control, #1

Some power plants have a real-time control: Not enough power ==> implies more power output

Too much power ==> implies less power output ==> P (proportional) control

This is necessary.Power plants to not accept frequency change.So we must regulate frequencies in real-time.

Real-time control #2

Some PPs have a real-time control #1: Not enough power ==> implies more power output


Additional control for market constraints:If flow Spain France too high, then increase power output in Spain (PI control).

Real-time control

Some PPs have a real-time control #1: Not enough power ==> implies more power output


Additional control for market constraints:If flow Spain France too high, then increase power output in Spain (PI control).

Not very satisfactory.When day-ahead market based on wrong wind forecasts, the resources management is very bad:- by law, grid owners (sometimes) must buy wind power- day ahead market orders already decided- the grid owner must find people for buying excess electricity (or destruction!) ==> negative prices

Outline



Recourse

Network constraints




Games ?

Bilevel problems

Looks like a game

Uncertainties = weather, load, wind...

Actions = switching on/off, which power output

Many players; for us, one-player game ==> maximum social welfare policy(politicians and

economist will have to find rules for the political/economical/ecological solution to be implemented...)

Winning = low cost + low pollution

Games ?

Games ?

Looks like a game

Uncertainties = weather, load, wind...

Actions = switching on/off, which power output

Many players; for us, one-player game ==> maximum social welfare policy(politicians and

economist will have to find rules for the political/economical/ecological solution to be implemented...)

Winning = low cost + low pollution

ConsequenceOfderegulation

So the game model for us

Just one player

Who controls all the Pps, all the networks.

He optimizes the social welfare.

At each time step:- observes the state (inflows history, stock levels, PP status...)- makes decisions (switch on/off, produce power)- gets reward (economy / ecology)



Recourse

Network constraints




Games ?

Bilevel problems

Outline

Bilevel problems

Simple version:I have K possible systems: Today's European grid

The same + a big new HVDC connection

The same + X MW of wind power

...

Which one is best ?

I spend T hours, each simulation of one of them needs (e.g.) one hour, how do I choose which system to simulate.

LOOKS LIKE A NICE BANDIT PROBLEM

Non-stationary
bilevel problems

I have K possible systems: Today's European grid

The same + a big new HVDC connection

The same + X MW of wind power

...

In fact, I have to optimize the management of each system. If I optimize during simulations,options become better ==> non-stationarybandit problem.

Power systemsilablri

Technology

Transcript of Power systemsilablri