von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 1

R.A. Van den Braembussche

von Karman Institute for Fluid Dynamics

Tuning of Optimization Strategies

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 2

Improving Convergence of GA

Performance

Database

Geometry

GA

NSNavier-Stokes

Metafunction

NS, HT, FEA

Predict

Learn

Requirements

Start

FEAStress

analysis

HTHeat

transfer

Parallel computing

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 3

Improving Convergence of GA

Performance

Database

Geometry

GA

NSNavier-Stokes

Metafunction

NS, HT, FEA

Predict

Learn

Requirements

Start

FEAStress

analysis

HTHeat

transfer

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 4

1. Population size N

2. Substring length l

3. Crossover Probability Pc

4. Mutation Probability Pm

5. Number of children ch

Optimal parameter setting

( to accelerate evolution )

Genetic AlgorithmOptimal parameter setting

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 5

%100.minOFOF

OFOFq

REF

GAREF

Genetic Algorithm

Optimal parameter setting

OF defined by test function Tests on 7 and 27

parameter function

GA = non-deterministic

Conclusions based on:5 optimization

Result of given effort

5000 OF evaluations

Six hump camel back test function

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 6

Genetic Algorithm(1) Population size# evaluations = 5000 = N * t

t number of generations

N population size

Small populations Premature convergence

Local optimum

Low number of feasible geometries

Large populations Low number of generations

No evolution

10 < N < 20Population size

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 7

Genetic Algorithm(2) Substring length

l = # of bits / variable

2l values / variable

ε = desired resolution

min max

2log

i ii

i

x xl

Global minimum

Best possible solution

average

OF

xmin xmax

00 01 10 01

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 8

N variables

l bits / variable

L < 3 too low resolution

L > 10 too large design space(slower convergence)

Substring length (# of bits)

Genetic Algorithm (2) Substring length

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 9

Uniform crossover

Swap with probability pc

Genetic AlgorithmCross over00000 11111

00011

# of function evaluations

Single crossover

One random swap / individual

00000 11111

01101

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 10

pm probability a bit is swapped

Mu

tatio

n p

roba

bili

ty

Total string length ( l .N )

Optimal pm

pm =1/(l.n) pm =2/(l.n)

Genetic AlgorithmMutation00000 11111

00011

01011

mutation

Optimal pm

______ pm = 1/(l.N)_ _ _ _ pm = 2/(l.N)

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 11

Genetic AlgorithmNew generation

(n,ch) n best of ch offspring's replace the old population (best individuals can be lost)

(n+ch) n best of (ch offspring's + n old population) replace the old population (elitism)

(n/i+ch) n/i best contribute to new generation

diversity

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 12

Genetic AlgorithmOptimal number of children

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 13

OptimizationConvergence

Performance

Database

Geometry

GA

NSNavier-Stokes

Metafunction

NS, HT, FEA

Predict

Learn

Requirements

Start

FEAStress

analysis

HTHeat

transfer

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 14

MetafunctionType (ANN, RBF) Structure (# hidden l)

RBF 5 hidden neurons

De Jong 2D test function

ANN 2 hidden layers10 hidden neurons

Database # of samplesDistribution of samples

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 15

Database

x x

xx

Systematic scanning

2 values /variable

n variables

full factorial 2n evaluations

7 variables 128 NS evaluations

27 variables 107 NS evaluations

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 16

Databasemerit function

merit function objective function m(x) = f(x) -m dm(x)

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 17

A B C ABC1 + - - +2 - + - +3 - - + +4 + + + +5 + + - -6 + - + -7 - + + -8 - - - -

RunParameters

A B C ABC1 + - - +2 - + - +3 - - + +4 + + + +5 + + - -6 + - + -7 - + + -8 - - - -

RunParameters

Database

Alt: Latin Hypercube – Random selection

3 variables

2 values (+ -) / variable

23 = 8 combinations

1, 2, 3 and 4 : main effect

5, 6, 7 and 8 : interaction

Design Of Experiment

DOE

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 18

DatabaseANN's global error for diffrent number of training samples

919

42

104

275

64

105 110

144

0

20

40

60

80

100

120

140

160

180

200

220

240

260

280

64 32 16 8 8 rand_first 16 rand _first 8 rand_second 8 rand_third 8 rand_fourth

Number of samples

64 32 16 8 8 rand_first 16 rand _first 8 rand_second 8 rand_third 8 rand_fourth

))(()(06,0))((002,0)(001,01 23 AECFFABFECDAR

Random

DOE

64 32 16 8 8a 16 8b 8c 8d

6 parameters

Full factorial =

26 = 64

Err

or

in 6

4 p

oin

ts

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 19

Statistical analysis

2k factorial =64 2 k-2 factorial =16 ))(()(06,0))((002,0)(001,01 23 AECFFABFECDAR k=6

Database

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 20

Statistical analysis

2 k-3 factorial = 8

k=6

2 level variables

(25% and 75% of non dimensional range)

1 central variable

(all variables at 50% of range)

12 to 15 variables 16 runs

16 to 31 variables 32 runs

Database

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 21

MetafunctionANN

n

j

ibjpjiWFTia1

1111 ))()(),(()(

)(1

1)(

xexFT

Learning : define W (weight) and b (bias)N

avi

er

Sto

kes

resu

lts

Ge

om

etr

y &

bo

und

. co

nd.

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 22

MetafunctionKriging

Linear least square approximation

Predicts value and uncertainty

N

i i

j

k

j jj

xfxwxf

functionGausianxZ

functionsregressiong

xZxgxf

1

1

)().()(~

)(

)()(.)(~

Accurate evaluations in regions of high uncertainty

Very time consuming

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 23

MetafunctionRBF

Learning : define W (weight) and b (bias)

Gausian activation function

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 24

Multi-objective optimizationConvergence

von Karman Institute for Fluid DynamicsRTO, AVT 167, October, 2009 25

Genetic AlgorithmGray coding

1 0012 0113 0104 1105 1116 1017 100

Gray coding

Value Code

1 0012 0103 0114 1005 1016 1107 111

Binary coding

Value Code

No real advantage observed