Entropy Enhanced Covariance Matrix Adaptation Evolution Strategy

(EE_CMAES) Developers:

Main Author: Kartik Pandya, Dept. of Electrical Engg., CSPIT, CHARUSAT, Changa, India Co-Author: Jigar Sarda, Dept. of Electrical Engg., CSPIT, CHARUSAT, Changa, India

1

2018 Grid Optimization Competition Test bed A: Stochastic OPF in presence of renewable energy and controllable loads

2018 IEEE PES General Meeting, August 5-9, 2018, Portland, OR, USA

Table of Contents

• Methodological Approach

• EE method for Optimization

• CMA-ES method for Optimization

• Simulation Results

• References

2

Methodological Approach

• Sequential Combination of two optimization

methods

Entropy Enhanced (EE) Method for exploration.

Covariance Matrix Adaption Evolution Strategy

(CMAES) for exploitation.

3

EE Method for Optimization Cross Entropy method is a versatile heuristic tool for solving

difficult estimation and optimization problems based on

Kullback- Leibler minimization [1].

Cross Entropy method was motivated by Rubinstein , where an

adaptive variance minimization algorithm for estimating

probabilities of rare events for stochastic networks was presented.

4

EE Method • Cross Entropy method involves two iterative phases:

1. Generation of a sample of random data according to a

specified random mechanism.

2. Updating the parameters of the random mechanism, typically

parameters of pdfs, on the basis of the data, to produce a

better sample in the next iteration.

5

EE Method Select 𝜇𝜇0and 𝜎𝜎02, the number of samples per iteration N, the rarity

parameter ρ, the smoothing parameter α, k := 0.

𝑘𝑘 = 𝑘𝑘 + 1 Generate a sample of 𝑋𝑋1,…, 𝑋𝑋𝑁𝑁from the sampling

distribution N(𝜇𝜇𝑘𝑘−1, 𝜎𝜎𝑘𝑘−12 ).

Compute S(𝑋𝑋1), …, S(𝑋𝑋𝑁𝑁) and order the samples from the worst to

the best performing ones, i.e. S(𝑋𝑋1) < .…< S(𝑋𝑋𝑁𝑁).

Compute γ𝑘𝑘 as the ρth quantile of the performance values and select

𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 =ρ*N; let 𝜓𝜓 be the subset from the ordered set of samples that

contains all the samples, i.e., the samples S(X)< γ𝑘𝑘.

6

EE Method For j = 1 to n

𝜇𝜇𝑘𝑘𝑘𝑘= ∑ 𝑋𝑋𝑘𝑘,𝑗𝑗

𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒∈𝜓𝜓 −−−(1) 𝜎𝜎𝑘𝑘𝑘𝑘2 = ∑ (𝑋𝑋𝑒𝑒𝑗𝑗−𝜇𝜇𝑘𝑘𝑗𝑗)2

𝑁𝑁𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒∈𝜓𝜓 ---(2)

End For Apply smoothing

𝜇𝜇𝑘𝑘 = 𝛼𝛼 ∗ 𝜇𝜇𝑘𝑘 + 1 − 𝛼𝛼 ∗ 𝜇𝜇𝑘𝑘−1-------------(3) 𝜎𝜎𝑘𝑘2 = 𝛼𝛼 ∗ 𝜎𝜎𝑘𝑘2 + 1 − 𝛼𝛼 ∗ 𝜎𝜎𝑘𝑘−12 -------------(4)

Until 𝑘𝑘 < 𝑘𝑘𝑀𝑀𝑀𝑀𝑋𝑋 For mean we shall use the same smoothing parameter 𝛼𝛼 (0.5 ≤ 𝛼𝛼 ≤ 0.9). For variance we shall use the dynamic smoothing ,

𝛽𝛽𝑒𝑒 = 𝛽𝛽 − 𝛽𝛽(1 − 1𝑒𝑒)𝑞𝑞-------------------------(5)

𝑞𝑞= Integer (5 ≤ 𝑞𝑞 ≤ 10), 𝛽𝛽= Smoothing constant (0.8 ≤ 𝛽𝛽 ≤ 0.99)

7

CMAES method for Optimization [2] Two main principles for the adaption of parameter of the search

distribution are exploited in the CMAES algorithm.

1. Maximum-likelihood principle, based on the idea to increase the

probability of successful candidate solutions and search steps.

2. Two path of the time evolution of the distribution mean of the

strategy are recorded, called search or evolution paths.

(i) One path is used for covariance matrix adaption procedure

(ii) Second path is used to conduct an additional step-size control.

8

CMAES method • Candidate solution is calculated using following

equation ------- (1) Where, = distribution mean and current favorite solution = step-size = covariance matrix

9

* (0, )ki k k N Cx m σ= +

kmkσ

kC

• The new mean value is computed as --------------(2) Where, 𝑤𝑤𝑒𝑒= recombination weights 𝜆𝜆= number of samples per iteration 𝜇𝜇= 𝜆𝜆/2= number of parents/ points for recombination

• The step-size is updated using cumulative step-size adaption

(CSA), sometimes also denoted as path length control.

10

𝑚𝑚𝑘𝑘+1 = �𝑤𝑤𝑒𝑒

𝜇𝜇

𝑒𝑒=1

𝑥𝑥𝑒𝑒=1:𝜆𝜆 − 𝑚𝑚𝑘𝑘

CMAES method

kσ

• The evolution path is updated using following equation

𝑝𝑝𝜎𝜎 = 1 − 𝑐𝑐𝜎𝜎 ∗ 𝑝𝑝𝜎𝜎 + 1 − (1 − 𝑐𝑐𝜎𝜎)2 ∗ 𝜇𝜇𝑤𝑤 ∗ 𝐶𝐶𝑘𝑘−1/2 ∗ 𝑚𝑚𝑘𝑘+1−𝑚𝑚𝑘𝑘

𝜎𝜎𝑘𝑘---(3)

discount factor complement for discounted variance displacement of m

• New step-size is updated using following equation

𝜎𝜎𝑘𝑘+1 = 𝜎𝜎𝑘𝑘 ∗ 𝑒𝑒𝑥𝑥𝑝𝑝𝑐𝑐𝜎𝜎𝑑𝑑𝜎𝜎

𝑝𝑝𝜎𝜎𝐸𝐸∗ 𝑁𝑁(0,𝐼𝐼)

− 1 ---------------(4)

• The step-size is increased if and only if 𝑝𝑝𝜎𝜎 is larger than the expected value and decreased if it is smaller.

11

CMAES method

• Evolution path is updated using following equation 𝑝𝑝𝑐𝑐 = 1 − 𝑐𝑐𝑐𝑐 ∗ 𝑝𝑝𝑐𝑐 + 1 0,𝛼𝛼 𝑛𝑛 ∗ 𝑝𝑝𝜎𝜎 1 − (1 − 𝑐𝑐𝜎𝜎)2 ∗ 𝜇𝜇𝑤𝑤 ∗

𝑚𝑚𝑘𝑘+1−𝑚𝑚𝑘𝑘𝜎𝜎𝑘𝑘

--------(5)

• Covariance matrix is updated using following equation

𝐶𝐶𝑘𝑘+1 = 1 − 𝑐𝑐1 − 𝑐𝑐𝜇𝜇 + 𝑐𝑐𝑠𝑠 ∗ 𝐶𝐶𝑘𝑘 + 𝑐𝑐1 ∗ 𝑝𝑝𝑐𝑐 ∗ 𝑝𝑝𝑐𝑐𝑇𝑇 + 𝑐𝑐𝜇𝜇 ∗ ∑ 𝑤𝑤𝑒𝑒𝑥𝑥𝑒𝑒:𝜆𝜆−𝑚𝑚𝑘𝑘

𝜎𝜎𝑘𝑘

𝑥𝑥𝑒𝑒:𝜆𝜆−𝑚𝑚𝑘𝑘𝜎𝜎𝑘𝑘

𝑇𝑇𝜇𝜇𝑒𝑒=1 -(6)

12

CMAES method

Test System:- IEEE 57 bus

13

Objective Function:

• Minimize the total fuel cost of traditional generators (buses: 1, 3, 8, 12) plus the expected uncertainty cost for renewable energy generators (buses: 2, 6, 9) plus the compensation cost for controllable loads (buses: 8, 12, 18, 47).

14

Problem Constraints Equality constraints: 1. Power balance equations

Inequality constraints:

1. Nodal Voltages Vi :

ViMin ≤ Vi ≤ Vi

Max , i= 1,2,3……..,NL …..(7)

2. Allowable Branch Power Flows Pij :

PijMin ≤ Pij ≤ Pij

Max , i=1,2,3……..,NB …..(8)

3. Generator Reactive Power Capability QC :

QCiMin ≤ QCi ≤ QCi

Max , i= 1,2,3……..,NC …..(9)

4. Maximum Active Power Output of slack generator PG :

PGi ≤ PGiMax …..(10)

5. Minimum and maximum levels of optimization variable

15

Constraint Handling Method • Select the maximum of the average sum of deviations at

iteration T [3]

16

T-1 T-1 T-1 T-1

t-1 t-1 t-1 t-1t=1 t=1 t=1 t=1

1 1 1 1max ΔP , ΔV , ΔQ , ΔST-1 T-1 T-1 T-1

∑∑ ∑∑ ∑∑ ∑∑

Simulation Results: Test bed 1 • MATLAB 2014a, Intel core i7-2600 CPU with 8.00 GB RAM

• Case Study 1: Stochastic OPF for IEEE 57 Bus System

Considering Wind Energy Generators and Controllable Loads

17

Statistics Case:1

f_best 81430.175

o@fbest 81430.168

g@fbest 0.007

fworst 80594.105

fmean 81382.615

fmedian 81413.841

Simulation Results cont.… • Case study 2: Stochastic OPF for IEEE 57 Bus System

Considering Wind and Solar Energy Generators and Controllable Loads

18

Statistics Case:2

f_best 68522.975

o@fbest 68522.965

g@fbest 0.010

fworst 68861.470

fmean 68519.132

fmedian 68579.969

• Case study 3: Stochastic OPF for IEEE 57 Bus System Considering Wind, Solar and Small-Hydro Generators and Controllable loads

19

Statistics Case:1

f_best 55720.550

o@fbest 55720.547

g@fbest 0.005

fworst 56316.686

fmean 56032.936

fmedian 56043.483

Simulation Results cont.…

• Case study 4: OPF using an Analytical Uncertainty Cost Function for IEEE 57 bus system considering Wind generators and Controllable loads

20

Simulation Results cont.…

Statistics Case:1

f_best 84342.953

o@fbest 84342.950

g@fbest 0.003

fworst 84347.422

fmean 84348.353

fmedian 84347.287

• Case study 5: OPF using an Analytical Uncertainty Cost Function for IEEE 57 bus system considering Wind and Solar generators (Cases 5) and Controllable loads

21

Simulation Results cont.…

Statistics Case:1

f_best 71030.542

o@fbest 71030.540

g@fbest 0.002

fworst 71034.542

fmean 71033.364

fmedian 71033.226

References

1. G Rubinstein, R. Y. (1999), “The cross-entropy method for combinatorial and continuous

optimization”, Methodology and Computing in Applied Probability, 2, 127-190.

2. N. Hansen. (2005 Nov.). The CMA Evolution Strategy: A Tutorial [Online]. Available:

http://www.lri.fr/∼hansen/cmatutorial.pdf

3. V. Miranda and Leonel Carvalho (2014), “DEEPSO Evolutionary Swarms in the OPF

challenge”, [online] Available http://sites.ieee.org/psace-mho/panels-and-competitions-

2014-opf-problems/, pp. 16.

22