D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.
Transcript of D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.
![Page 1: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/1.jpg)
DIFFERENTIAL EVOLUTION
By Fakhroddin Noorbehbahani
EA course, Dr. Mirzaee
December, 20101
![Page 2: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/2.jpg)
AGENDA
Preface
Basic Differential Evolution
Difference Vectors
Mutation
Crossover
Selection
General Differential Evolution Algorithm
Control Parameters
Geometrical Illustration
DE/x/y/z
2
![Page 3: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/3.jpg)
AGENDA
Variations to Basic Differential Evolution
Hybrid Differential Evolution Strategies
Population-Based Differential Evolution
Self-Adaptive Differential Evolution
Differential Evolution for Discrete-Valued Problems
Constraint Handling Approaches
Comparison with other algorithms
Applications3
![Page 4: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/4.jpg)
PREFACE
Price and Storn in 1995 Chebychev Polynomial fitting
Problem
3rd place at the First International Contest on
evolutionary Computation (1stICEO) 1996, the best
genetic type of algorithm for solving the real-valued test
function suite.
stochastic, population-based search strategy
Main characteristics
Guide search with distance and direction information from the
current population
original DE strategies for continuous-valued landscapes
4
![Page 5: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/5.jpg)
BASIC DIFFERENTIAL EVOLUTION
mutation is applied first to generate a trial vector, which is then used within the crossover operator to produce one offspring,
mutation step sizes are not sampled from a prior known probability distribution function.
mutation step sizes are influenced by differences between individuals of the current population
5
![Page 6: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/6.jpg)
DIFFERENCE VECTORS
Position of individuals and fitness
Over time, as the search progresses, the distances between
individuals become smaller
The magnitude of the initial distances between individuals is
influenced by the size of the population
Distances between individuals are a very good indication of the
diversity of the current population
Use difference vector to determine the step size
total number of differential perturbations
nv is the number of differentials used
ns is the population size
6
![Page 7: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/7.jpg)
MUTATION
7
![Page 8: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/8.jpg)
CROSSOVER
Binomial crossover Exponential crossover
8
![Page 9: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/9.jpg)
SELECTION
Random Selection To select the individuals from which difference vectors
are calculated. The target vector is either randomly selected or the
best individual is selected
Deterministic Selection To construct the population for the next generation,
the offspring replaces the parent if the fitness of the offspring is better than its parent; otherwise the parent survives to the next generation.
This ensures that the average fitness of the population does not deteriorate. 9
![Page 10: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/10.jpg)
GENERAL DIFFERENTIAL EVOLUTION ALGORITHM
10
![Page 11: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/11.jpg)
11
![Page 12: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/12.jpg)
CONTROL PARAMETERS
population size,ns
scale factor, β
probability of recombination,Pr
12
![Page 13: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/13.jpg)
POPULATION SIZE
The size of the population has a direct influence on the exploration ability of DE algorithms.
The more individuals there are in the population, the more differential vectors are available, and the more directions can be explored
The computational complexity per generation increases with the size of the population.
Empirical studies provide the guideline that ns ≈ 10nx 13
![Page 14: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/14.jpg)
SCALE FACTOR
The scaling factor, β ∈ (0,∞), controls the amplification of the differential variations, (xi2−xi3 ).
The smaller the value of β, the smaller the mutation step sizes Smaller step sizes can be used to explore local
areas. slower convergence Larger values for β facilitate exploration, but may cause
the algorithm to overshoot optima
As the population size increases, the scaling factor should decrease. 14
![Page 15: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/15.jpg)
RECOMBINATION PROBABILITY
This parameter controls the number of elements of the parent, xi(t), that will change.
The higher the probability of recombination, the more variation is introduced in the new population, thereby increasing diversity and increasing exploration.
Increasing pr often results in faster convergence, while decreasing pr increases search robustness
15
![Page 16: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/16.jpg)
GEOMETRICAL ILLUSTRATION
16
![Page 17: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/17.jpg)
GEOMETRICAL ILLUSTRATION
17
![Page 18: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/18.jpg)
GEOMETRICAL ILLUSTRATION
18
![Page 19: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/19.jpg)
EXAMPLE:PEAK FUNCTION
19
![Page 20: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/20.jpg)
GEOMETRICAL ILLUSTRATION
Generation 1: DE’s population and difference vector distributions
20
![Page 21: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/21.jpg)
GEOMETRICAL ILLUSTRATION
Generation 6: The population coalesces around the two main minima
21
![Page 22: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/22.jpg)
GEOMETRICAL ILLUSTRATION
Generation 12: The difference vector distribution contains three main clouds – one for local searches and two for moving between the two main minima.
22
![Page 23: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/23.jpg)
GEOMETRICAL ILLUSTRATION
Generation 16: The population is concentrated on the main minimum
23
![Page 24: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/24.jpg)
GEOMETRICAL ILLUSTRATION
Generation 20: Convergence is imminent. The difference vectors automatically shorten for a fine-grained, local search.
24
![Page 25: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/25.jpg)
GEOMETRICAL ILLUSTRATION
Generation 26: The population has almost converged.
25
![Page 26: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/26.jpg)
GEOMETRICAL ILLUSTRATION
Generation 34: DE finds the global minimum.
26
![Page 27: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/27.jpg)
DE/X/Y/Z
DE/best/1/z
DE/x/nv/z
DE/rand-to-best/nv/z
27
![Page 28: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/28.jpg)
DE/X/Y/Z
DE/current-to-best/1+nv/z
DE/rand/1/bin vs. DE/current-to-best/2/bin DE/rand/1/bin maintains good diversity DE/current-to-best/2/bin shows good convergence
characteristics Dynamically switch between these two
strategies
28
![Page 29: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/29.jpg)
VARIATIONS TO BASIC DIFFERENTIAL EVOLUTION
Hybrid Differential Evolution Strategies Gradient-Based Hybrid Differential Evolution
Acceleration operator : to improve convergence speed Migration operator : to improve ability for escaping
local optima Acceleration operator
uses gradient descent to adjust the best individual toward obtaining a better position if the mutation and crossover operators failed to improve
x(t), replaces the worst individual in the new population, C(t+1).
29
![Page 30: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/30.jpg)
MIGRATION OPERATOR Gradient decent speed up but local minima Migration operator
increase population diversity Generate new individual from best individuals
Applied when diversity is too small i.e.:
30
![Page 31: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/31.jpg)
HYBRID DIFFERENTIAL EVOLUTION WITH ACCELERATION AND MIGRATION
31
![Page 32: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/32.jpg)
EVOLUTIONARY ALGORITHM-BASED HYBRIDS DE reproduction process as a crossover
operator in a simple GA
Rank-Based Crossover Operator for Differential Evolution To select individuals to calculate difference vectors xi1 (t) precedes xi2 (t) if f(xi1(t)) > f(xi2(t)).
32
![Page 33: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/33.jpg)
RANK-BASED MUTATION OPERATOR FOR DIFFERENTIAL EVOLUTION
33
![Page 34: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/34.jpg)
OTHER VARIATIONS TO BASIC DE
Population-Based Differential Evolution Improve exploration by using 2 population set Initialize with ns pairs Rejected individual by selection put in auxiliary pop
Self-Adaptive Differential Evolution Dynamic Parameters
Self-Adaptive Parameters
34
![Page 35: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/35.jpg)
DIFFERENTIAL EVOLUTION FOR DISCRETE-VALUED PROBLEMS
Angle Modulated DE
where x is a single element from a set of evenly separated intervals determined by the required number of bits that need to be generated
35
![Page 36: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/36.jpg)
ANGLE MODULATED DIFFERENTIAL EVOLUTION
36
![Page 37: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/37.jpg)
BINARY DIFFERENTIAL EVOLUTION
37
![Page 38: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/38.jpg)
CONSTRAINT HANDLING APPROACHES
Penalty methods Converting the constrained problem to an
unconstrained problem By changing the selection operator of DE,
infeasible solutions can be rejected
38
![Page 39: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/39.jpg)
COMPARISON WITH GA AND PSO
39
![Page 40: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/40.jpg)
COMPARISON
40
![Page 41: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/41.jpg)
41
![Page 42: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/42.jpg)
42
![Page 43: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/43.jpg)
43
![Page 44: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/44.jpg)
44
![Page 45: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/45.jpg)
APPLICATIONS 1) General Optimization Framework "Mystic" by Mike McKerns,
Caltech. 2) Multiprocessor synthesis. 3) Neural network learning. 4) Chrystallographic characterization. 5) Synthesis of modulators. 6) Heat transfer parameter estimation in a trickle bed reactor. 7) Scenario-Integrated Optimization of Dynamic Systems. 8) Optimal Design of Shell-and-Tube Heat Exchangers. 9) Optimization of an Alkylation Reaction. 10) Optimization of Thermal Cracker Operation. 11) Optimization of Non-Linear Chemical Processes. 12) Optimum planning of cropping patterns. 13) Optimization of Water Pumping System . 14) Optimal Design of Gas Transmission Network . 15) Differential Evolution for Multi-Objective Optimization 16) Physiochemistry of Carbon Materials. 17) Radio Network Design. 18) Reflectivity Curve Simulation.
45
![Page 46: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/46.jpg)
COMMERCIAL SOFT
1) Built in optimizer in MATHEMATICA's function Nminimize (since version 4.2).
2) MATLAB's GA toolbox contains a variant of DE.
3) Digital Filter Design. 4) Diffraction grating design. 5) Electricity market simulation. 6) Auto2Fit. 7) LMS Virtual Lab Optimization. 8) Optimization of optical systems. 9) Finite Element Design. 46
![Page 47: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/47.jpg)
APPLICATION : BUMP PROBLEM
47
![Page 48: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/48.jpg)
APPLICATION : BUMP PROBLEM
48
![Page 49: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/49.jpg)
49
![Page 50: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/50.jpg)
REFERENCES [1] http://www.icsi.berkeley.edu/~storn/code.html [2] Andries P. Engelbrecht ,(2007),Computational Intelligence:
An Introduction, 2nd Edition., ISBN: 978-0-470-03561-0. [3] Price, K.; Storn, R.M.; Lampinen, J.A. (2005).
Differential Evolution: A Practical Approach to Global Optimization. Springer. ISBN 978-3-540-20950-8. http://www.springer.com/computer/theoretical+computer+science/foundations+of+computations/book/978-3-540-20950-8.
[4] Feoktistov, V. (2006). Differential Evolution: In Search of Solutions. Springer. ISBN 978-0-387-36895-5. http://www.springer.com/mathematics/book/978-0-387-36895-5.
[5] J. Vesterstrom and R. Thomson, A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems, Proc. of IEEE Congress on Evolutionary Computation, 2004,pp. 1980–1987.
50
![Page 51: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/51.jpg)
يک سخن
زندگي آن چيزي است که براي تو اتفاق مي افتد،
در حالي که تو سرگرم برنامه ريزي هاي ديگري
هستي
51
![Page 52: D IFFERENTIAL E VOLUTION By Fakhroddin Noorbehbahani EA course, Dr. Mirzaee December, 2010 1.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c415503460f948ef766/html5/thumbnails/52.jpg)
THANKS FOR YOUR ATTENTION
52