A Simulated Annealing Algorithm for Noisy Multi-Objective ...non-dominated solution is high Current...

A Simulated Annealing Algorithm forNoisy Multi-Objective Optimization

Ville Mattila, Kai Virtanen, and Raimo P. HämäläinenSystems Analysis LaboratoryAalto University School of Science

Overview

Noisy multi-objective optimization problemValues of objective functions are uncertain

Techniques for finding non-dominated solutionsFew take into account noiseMostly evolutionary algorithms (EAs), e.g, [Goh&Tan, 2007]

New simulated annealing (SA) algorithmFeatures for handling noise and generating candidatesolutionsPerformance superior to existing EAs in numericaltests

A Simulated Annealing Algorithm for Noisy Multi-ObjectiveOptimization

2/18

Mattila, Virtanen, Hämäläinen

Noisy multi-objective optimizationproblem

minl≤x≤u

[f1(x), . . . , fH(x)]

Values of objective functions fi(x) include uncertainty→ Optimization based on fi(x) + ωi where ωi ∼ N (0, σ2

i )

Decision variable x ∈ <n

Constraints l and u


3/18


General steps of SA inmulti-objective optimization

0. Define performance measure E for candidate solutionsSelect temperature TGenerate a current solution x

1. Generate a candidate solution y from the neighborhood ofx

2. Set x = y with probability min[1, exp

(E(x)−E(y)

T

)]3. Solutions with best values of E into the non-dominated set4. Return to step 1 or stop iteration


4/18


Features of the new SA algorithm

Performance of a solution, EBased on probabilistic dominance

→ Takes into account multiple objectives and noiseUsed previously in evolutionary algorithms [Hughes, 2001],not in SA

Generation of candidate solutionsUsing information on current non-dominated solutions

→ Increase likelihood of obtaining new non-dominatedsolutionsPreviously in deterministic single-objective SA[Sun et al., 2008]


5/18


Probabilistic dominance

Probability that x dominates y

Product of probabilities thatobjective function values aresmaller for x

M samplesf̄i(x) and s2

i (x) sampleaverage and variance

P(x � y) =H∏

i=1

Φ

(f̄i (x)− f̄i (y)√

s2i (x)/M + s2

i (y)/M

)0

0

f1

f 2

x

yf1(x) − f1(y)

f2(x) − f2(y)

Φ cumulative distribution function of standard normal distribution


6/18


Performance of candidate solution E(y)

Sum of probabilities thatcurrent non-dominatedsolutions dominatecandidate solution y

E(y) =∑z∈S

P(z � y)

0

0

f1

f 2

yNon-dominated set S


7/18


Generation of candidate solutions

Use empirical data to samplenew value of decision variable x

Recent values innon-dominated setx(1), . . . , x(m) ∈ [a, b]Feasible neighborhood ofcurrent value [a, b]

Concentrates to regions wherelikelihood of obtaining a newnon-dominated solution is high

Current valueNew value

a x(1) x(2)x(3) x(4)x(5) bx

Probability


8/18


Numerical testing

Reference algorithmMOEA-RF, Multi-Objective EA with Robust FeaturesOutperformed several existing EAs in [Goh&Tan, 2007]

Test problemsZDT1, ZDT4, ZDT6 [Zitzler et al., 2007], FON[Fonseca&Fleming, 1998], KUR [Kursawe, 1991]Four noise levels

Performance measures usedGenerational distanceSpacingMaximum spreadHypervolume ratio


9/18


Non-dominated solutions for ZDT1

Highest noise levelSA algorithm produces solutions closer to actualnon-dominated set

0 10

5

f1

f 2

0 10

5

f1

f 2

� SA algorithm � MOEA-RF, 30 runs


10/18


Generational distance for ZDT1

Distance of solutions to actual non-dominated setValues lower (better) for the SA algorithm

Low noise High noise0

0.05

0.1

0.15

� SA algorithm � MOEA-RF, 30 runs with each noise level


11/18


Spacing for ZDT1

Distance between obtained solutionsValues similar for the algorithms (lower is better)


0.2

0.4



12/18


Maximum spread for ZDT1

Range of objective function values for obtained solutionsValues higher (better) for the SA algorithm

Low noise High noise

0.8

0.9

1



13/18


Hypervolume ratio for ZDT1

Volume dominated by obtained solutionsValues higher (better) for the SA algorithm


0.5

1



14/18


Summary of the numerical tests

Our SA algorithm superior accross test problems andnoise levels

Noise level low highPercentile 0 25 50 75 100 0 25 50 75 100 0 25 50 75 100 0 25 50 75 100

ZDT1ZDT4

GD ZDT6FONKURZDT1ZDT4

S ZDT6FONKURZDT1ZDT4

MS ZDT6FONKURZDT1ZDT4

HV ZDT6FONKUR

� SA algorithm superior � MOEA-RF superior

Conclusions

New SA algorithm for noisy multi-objective optimizationPerformance of solutions based on probabilistic dominanceGeneration of candidate solutions using non-dominated set

Outperformed reference EA in numerical testsComputational requirements comparable to the EASuccessful application: Maintenance scheduling of aircraft[Mattila et al., 2011]Further development

Dynamic sample size for evaluating candidate solutions


16/18


References

C. M. Fonseca and P. J. Fleming. Multiobjective optimization and multiple constrainthandling with evolutionary algorithms - Part II: Application example.IEEE Trans. Syst., Man, and Cybern. A, Syst. Humans, 28(1):38–47, 2000.

C. K. Goh and K. C. Tan. An Investigation on Noisy Environments in EvolutionaryMultiobjective Optimization.IEEE Trans. Evol. Comp., 11(3):354–381, 2007.

E. Hughes. Evolutionary Multi-objective Ranking with Uncertainty and Noise.Proc. 1st Int. Conf. Evol. Multi-Criterion Optim., 329–343, 2001.

F. Kursawe. A variant of evolution strategies for vector optimization.Proc. 1st Workshop on Parallel Problem Solving from Nature, 193–197, 1998.

V. Mattila, K. Virtanen, and R. P. Hämäläinen. Multi-objective simulation-optimization withsimulated annealing and decision analysis – Maintenance scheduling of a fleet of fighteraircraft.Manuscript, 2011.


17/18


References II

S. Sun, F. Zhuge, J. Rosenberg, R. Steiner, G. Rubin, and S. Napel. Learning-enhancedsimulated annealing: Method, evaluation, and application to lung nodule registration.Appl. Intell., 28(1):83-99, 2008.

E. Zitzler, K. Deb, and L. Thiele. Comparison of multiobjective evolutionary algorithms:Empirical results.Evol. Comput., 8(2):173–195, 2000.


18/18


A Simulated Annealing Algorithm for Noisy Multi-Objective ...non-dominated solution is high Current...

Documents

Transcript of A Simulated Annealing Algorithm for Noisy Multi-Objective ...non-dominated solution is high Current...