DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 1 Optimization in Dynamic Environments...

1

DE

I/C

ISU

C

Evonet Summer School - Parma© 2003 Ernesto Costa

Optimization in Dynamic Environments

Ernesto Costa

DEI/CISUC

[email protected]

http://www.dei.uc.pt/~ernesto/

2

DE

I/C

ISU

C


Summary

Agents, Problems and Environments

Agents: Natural Selection and Genetics

Problems:Optimization

Environments: Dynamic

Optimization and Dynamic Environments

State of the Art

The Challenge / Problem

3

DE

I/C

ISU

C


Agents, Problems and Environments

Agent

Environment

Problem

Behavior Performance

4

DE

I/C

ISU

C


Agents and Evolutionary Computation

Darwin

Evolution

by Natural selection

Mendel

Genetics and Inheritance

procedure EC t = 0; inicialization P(t); evaluation P(t); while not stop_condition do t = t+1; P1(t) = selection (P(t-1)); P2(t) = op_modification (P1(t)); evaluation (P2(t)); P(t) = combine (P2(t) ,P(t-1));End_do; return_best (P(t));

end_proc.

5

DE

I/C

ISU

C


Problems that:

have a finite number, F, of feasible solutionseach solution has na associate cost, c.goal: a solution f in F that minimizes c

Examples: knapsack minimum spanning tree bin packing set covering vehicle routing ...

Problems: Combinatorial Optimization

6

DE

I/C

ISU

C


W[i], P[i]

w1, p1

w2, p2

w3, p3

.

.

.

wn, pn

Items

0/1 Knapsack

Choose the items that maximize your profit ans such that the total weight is less that some given limit (knapsack capacity)!

1

1

*

max *

n

i ii

n

i ii

x W C

x P

Combinatorial Optimization

Binary representation: vector x=(x1,...,xn)

7

DE

I/C

ISU

C


Minimum Spanning Tree

GivenG=(V,E): a connected weighted undirected graphV={v1, ...,vn}E={e1,...,em}W={w1,...,wm}: weight or cost of each edge

Find a subgraph S of G :S contains all the vertices of GS is connected and contains no cyclesS has minimum cost

A minimum spaning tree (MST)

Combinatorial Optimization (2a)

5

4

4

5 5 4

3 3

33

6

2

8

DE

I/C

ISU

C


-5

-2.5

0

2.5

5-5

-2.5

0

2.5

5

0

20

40

60

80

-5

-2.5

0

2.5

5

Problems: Function Optimization

Rastringin Function

2

1

( ) * ( * (2 ))n

i i ii

rast x n A x A cos x

n=2, A=10

9

DE

I/C

ISU

C


Environment: Dynamic

Changes in the environment:

Restrictions: Knapsack capacity C

Goal: Rastringin Parameter A

Problem Instance: MST #V, #E, W

A different, time dependent, fitness landscape!

10

DE

I/C

ISU

C


Environment: DynamicTypes of dynamics

Discrete vs Continuous

Periodic vs Non-Periodiccycle length

Dimension of changesmall vs big

Predictability of change

11

DE

I/C

ISU

C


Change detectionExplicitly known

Average or best fitness drop

Reevaluating a set of individuals every generation

Keep a model of the environment (model and real ≠)

Does the EA change (e.g. representation)?


Further Aspects

12

DE

I/C

ISU

C


State of the Art

The problemStandard EA

loose diversity (converge to an optimum)No memory of the past

SolutionStart from scratch???

New optimization algorithm (new Agent)Kind of open-ended evolutionUsing past information

DiversityMemory

The challenge!!!

13

DE

I/C

ISU

C


State of the Art

Promoting DiversityHypermutation

Maintaining DiversityAvoid convergence

Random immigrants

Use of MemoryRedundant Representations

Multiploidy

Explicit MemoryInterplay between memory and the evolving population

Multiple PopulationsSelf-Adaptive Memory

14

DE

I/C

ISU

C


The Challenge

Choose a problem

Modify the standard Genetic AlgorithmDiversity mechanisms

Memory mechanism

Make Experiments with (some) previous approaches

Analyse Results

Propose New Solutions

New Results?!

15

DE

I/C

ISU

C


Moving Parabola

2

1

( , ) ( ( ))n

i ii

f x t x t

(0) 0

( ) ( 1)i

i it t s

The problem: Benchmarks

16

DE

I/C

ISU

C


The problem: Benchmarks

Having several peaksPositionHeightWidth

ChangesOne or several parameters

Possible to test different dynamicsA C-version available (Jurgen Branke)A Matlab version (R. Morrison)

Moving Peaks

17

DE

I/C

ISU

C


The problem: Measuring Performance

On-line performanceThe average of the averages so far

Off-line performanceThe average of the best so far

Best-of-generation averages for many runs on the same problem

Question: we want to measure the performance of the EA across the entire range of the fitness landscape dynamics

1

1 T

tt

f fT

* *

1

*1 2

1

max{ , ,..., }

T

tt

t t

f fT

f f f f

* *,

1

11,...,

R

g r gr

f f g GR

18

DE

I/C

ISU

C


The IS is a complex system that includes cells, molecules andorgans that constitutes an identification mechanism capable ofperceiving and combating:

dysfunction of our own cells (infectious self)

action of exogenous infectious microorganisms (infectious non-self)

The IS insures the integrity of the self!

Immune System

Other Ideas

19

DE

I/C

ISU

C


Invasion

Detection

Reaction

Maturation

Memorizing

Immune System

How it works?

Other Ideas

Challenge: can we use it for dynamic environments???

20

DE

I/C

ISU

C


Let’s Work!

ReferencesEvolutionary Optimization in Dynamic Environments, Jürgen Branke, Kluwer Academic Publishers,2002.

Evolutionary Algorithms for Dynamic optimization Problems (EvoDOP 2003) in GECCO 2003, Jürgen Branke (Organizer)

http://www.aifb.uni-karlsruhe.de/~jbr/MovPeaks/

21

DE

I/C

ISU

C


Cycle = 30

60

65

70

75

80

85

90

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281

Generations

Fit

ne

ss

Non-Periodic

60

65

70

75

80

85

90

1 151 301 451 601 751 901 1051 1201 1351 1501 1651 1801 1951

Generations

Fit

nes

s

0/1 Knapsack


Restrictions: changing the knapsack capacity, C

22

DE

I/C

ISU

C


5

4

4

5 5 4

3 3

33

6

2



4

8

117

1 2

26 4

7 9

10

14

8

Problem Instance: different vertices, edges and weights

23

DE

I/C

ISU

C


-5-2.5

02.5

5-5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

-5-2.5

02.5

5-5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

-5-2.5

02.5

5-5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

-5-2.5

02.5

5 -5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

-5-2.5

02.5

5 -5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

-5-2.5

02.5

5 -5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

-5-2.5

02.5

5-5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

-5-2.5

02.5

5-5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

-5-2.5

02.5

5-5

-2.5

0

2.5

5

020406080

-5-2.5

02.5

5

22

1

( , ) 2* ( * (2 ))i i ii

rast x A A x A cos x

A=1..9

Rastringin Function


Goal: different Max

24

DE

I/C

ISU

C



Question: we want to measure the performance of the EA across the entire range of the fitness landscape dynamics

Adapting the offline performance

Moment of changes are known

Using the errorOptimum is known

Current error

Offline error

Dynamic Environments

1,8

2

2,2

2,4

2,6

2,8

3

3,2

1 3 5 7 9 11 13 15 17 19 21

Evaluations

Fit

nes

s Evaluations

Average of best so far

Best

1

1*( )

T

tt

TT

'( )t toptimum t f

25

DE

I/C

ISU

C



Accuracy: recovery capacity

Adaptability: speed of recovery

K= # changes during the runr= # generations between two consecutive changes

Erri,j= difference between current best at generation j after change #i and the optimum

Dynamic Environments

, 11

1 K

ii

Acc ErrK

1

,1 0

1 1K r

i ji j

Ada ErrK r

26

DE

I/C

ISU

C



Formally

Any subgraph S can be represented by a binary vectorx={x1,...xm}, with xi= 1 if ei is in S

If T is the set of all spanning trees in G then the MST isdefined by:

1

min ( ) * |m

i ii

z x w x x T

Combinatorial Optimization (2b)

DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 1 Optimization in Dynamic Environments...

Documents

Transcript of DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 1 Optimization in Dynamic Environments...