Post on 15-Aug-2020
19.11.13
1
Advanced Algorithmics (6EAP) Search and meta-‐heuris0cs
Jaak Vilo 2012 Spring
1 Jaak Vilo http://aima.cs.berkeley.edu/ http://books.google.ee/books/about/Heuristic_Search.html?id=3k5MVjKzBP4C&redir_esc=y
Search
• for what? – a solu@on – the (best possible (approximate?)) solu@on
• from where? – search space (all valid solu@ons or paths)
• under which condi0ons? – compute @me, space, … – constraints, …
Objec@ve func@on
• An op@mal solu@on – what is the measure that we op0mise?
• Any solu0on (sa@sfiability /SAT/ problem) – does the task have a solu@on? – is there a solu@on with objec@ve measure beNer than X?
• Minimal/maximal cost solu@on • A winning move in a game • A (feasible) solu@on with smallest nr of constraint viola@ons (e.g. course @me scheduling)
Search space size?
• Linear (list, binary search, …) • Integer in [i,j] • Real nr in [x,y) • A point in high-‐dimensional space • An assignment of variables (in SAT) • A subset of a larger set • Trees, Graphs • …
The Proposi0onal Sa0sfiability Problem (SAT)
http://www.satcompetition.org/
19.11.13
2
Inputs usually in CNF
(a + b + c) (a’ + b’ + c) (a + b’ + c’) (a’ + b + c’)
Solu@on search space: la_ce for 4 variables (par@al connec@ons)
0000
1111
0001 0010 0100 1000
0111 1011 1101 1110
0011 0101 0110 1010 1001 1100
O(2n)
Tic-‐Tac-‐Toe It is straightforward to write a computer program to play Tic-tac-toe perfectly, to enumerate the 765 essentially different positions (the state space complexity), or the 26,830 possible games up to rotations and reflections (the game tree complexity) on this space (2002?)
TSP, nearest neighbour search
TSP, NN subop@mal
19.11.13
3
Issues: Constraints
• Time, space… – if op@mal cannot be found, approximate
• All kinds of secondary characteris@cs
• Constraints – some@mes finding even a point in the valid search space is hard
19.11.13
4
Examples
• Greedy • A* search • Monte Carlo, Grid • Local search algorithms -‐ Hill-‐climbing, beam, … • Simulated annealing search • Gene@c algorithms (GA, GP) • Differen@al Evolu@ons (DE) • Par@cle Swarm Op@misa@on (PSO) • Ant colony op@misa@on (ACO)
22 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Classes of Search Techniques
Finonacci Newton
Direct methods Indirect methods
Calculus-based techniques
Evolutionary strategies
Centralized Distributed
Parallel
Steady-state Generational
Sequential
Genetic algorithms
Evolutionary algorithms Simulated annealing
Guided random search techniques
Dynamic programming
Enumerative techniques
Search techniques
Greedy
• Always choose the next seemingly best step
• Set Cover – Greedy Approxima@on Algorithm – polynomial-‐@me ρ(n)-‐approxima@on algorithm
• ρ(n) is a logarithmic func@on of set size • n – size of the largest set…
• | C(greedy) | < ln(n) * | C(op@mal) |
Set Cover Problem
source: 91.503 textbook Cormen et al.
SXS F∈
=
X of subsets offamily F
points) of (e.g. set finite X
:),(Instance FX
NP-Complete
CS
SX: XFC
∈
=
⊆
of allcover members whose subset sized-minimum a Find :Problem F
19.11.13
5
Greedy Set Covering Algorithm
source: 91.503 textbook Cormen et al.
Greedy: select set that covers the most uncovered elements
Set Cover Theorem: GREEDY-SET-COVER is a
polynomial-time ρ(n)-approximation algorithm for
Proof: })|:(max{|)( F∈= SSHnρ
selectedsubset th iSi = 1 costs iSselecting
Algorithm runs in time polynomial in n.
Xxcx ∈= element ofcost paid only when x is covered for the first time
)(1
121 −∪∪∪−=
iix SSSSc
assume x is covered for the first time by Si (spread cost evenly across all elements covered for first time by Si )
Number of elements covered for first time by Si
)(1number harmonicth 1
dHi
Hdd
id ==∑
=
H(0)=0
Proof of approxima@on
• hNp://www.cs.dartmouth.edu/~ac/Teach/CS105-‐Winter05/Notes/wan-‐ba-‐notes.pdf
Set Cover (proof con@nued) Theorem: GREEDY-SET-COVER is a
polynomial-time ρ(n)-approximation algorithm for
Proof: (continued)
})|:(max{|)( F∈= SSHnρ
*
∑ ∑∈ ∈
⎟⎠
⎞⎜⎝
⎛≤
CS SxxcC
∑∑ ∑∈∈ ∈
≥⎟⎠
⎞⎜⎝
⎛
Xxx
CS Sxx cc
*
*∑ ∑∈ ∈
⎟⎠
⎞⎜⎝
⎛
CS SxxcCost assigned to
optimal cover:
Each x is in >= 1 S in C*
COVER-SET-GREEDY fromcover be cover optimalan be*Let
CC ∑
∈
=Xx
xcC1 unit is charged at each stage of algorithm
Set Cover (proof con@nued) Theorem: GREEDY-SET-COVER is a
polynomial-time ρ(n)-approximation algorithm for
Proof: (continued)
})|:(max{|)( F∈= SSHnρ
)( SHcSx
x ≤∑∈
How does this relate to harmonic numbers??
)(1number harmonicth 1
dHi
Hdd
id ==∑
=
We’ll show that:
}):(max{* )(*
F∈≤≤ ∑∈
SSHCSHCCS
And then conclude that:
which will finish the proof
F∈Sset any for
19.11.13
6
Local Search
Local search algorithms • In many op@miza@on problems, the path to the goal is irrelevant; the goal state itself is the solu@on
• State space = set of "complete" configura@ons • Find configura@on sa@sfying constraints, e.g., n-‐queens
• In such cases, we can use local search algorithms • keep a single "current" state, try to improve it
Example: n-‐queens
• Put n queens on an n × n board with no two queens on the same row, column, or diagonal
• may get stuck…
Problems
• Cycles – Memorize; Tabu search
• How to transfer valleys with bad choices only…
Tree/Graph search
• order defined by picking a node for expansion
• BFS, DFS
• Random, Best First, … – Best – an evalua@on func@on
19.11.13
7
• Idea: use an evalua0on func0on f(n) for each node – es@mate of "desirability" – Expand most desirable unexpanded node
• Implementa@on: Order the nodes in fringe in decreasing order of desirability Priority queue
• Special cases:
– greedy best-‐first search f(n) = h(n) heuris@c, e.g. es@mate to goal – A* search
A*
• f(n) = g(n) + h(n)
– g(n) – path covered so far in graph – h(n) – es@mated distance from n to goal
A C
D
B
F
20 20 10
15 25
Admissible heuris@cs
• A heuris@c h(n) is admissible if for every node n, h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state from n.
• An admissible heuris@c never overes@mates the cost to reach the goal, i.e., it is op0mis0c
• Example: hSLD(n) (never overes@mates the actual road distance) (SLD – shortest linear distance)
• Theorem: If h(n) is admissible, A* using TREE-SEARCH is
op@mal
Op@mality of A* (proof) • Suppose some subop@mal goal G2 has been generated and is in the fringe.
Let n be an unexpanded node in the fringe such that n is on a shortest path to an op@mal goal G.
• f(G2) = g(G2) since h(G2) = 0 • g(G2) > g(G) since G2 is subop@mal • f(G) = g(G) since h(G) = 0 • f(G2) > f(G) from above
Op@mality of A* (proof) • Suppose some subop@mal goal G2 has been generated and is in the fringe.
Let n be an unexpanded node in the fringe such that n is on a shortest path to an op@mal goal G.
• f(G2) = g(G2) since h(G2) = 0 • g(G2) > g(G) since G2 is subop@mal • f(G) = g(G) since h(G) = 0 • f(G2) > f(G) from above • Hence f(G2) > f(n), and A* will never select G2 for expansion
A* path-‐finder
Better: http://www.youtube.com/watch?v=19h1g22hby8
19.11.13
8
Graph
• A Virtual graph/search space – valid states of Fireen-‐game – Rubik’s cube
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15
Solve
• Which move takes us closer to the solu@on? • Es@mate the goodness of the state
1 2
3 4
5 6 7 8
9 10
11
12
13 14
15
• How many are misplaced? (7)
• How far have they been misplaced? Sum of theore@cal shortest paths to the correct place
• A* search towards a final goal
1 2
3 4
5 6 7 8
9 10
11
12
13 14
15
Search space
The Traveling Salesperson Problem (TSP)
• TSP – op0miza0on variant: • For a given weighted graph G = (V,E,w), find a Hamiltonian cycle in G with minimal weight,
• i.e., find the shortest round-‐trip visiDng each vertex exactly once.
• TSP – decision variant: • For a given weighted graph G = (V,E,w), decide whether a Hamiltonian cycle with minimal weight ≤ b exists in G.
TSP instance: shortest round trip through 532 US ci0es
Search Methods
• Types of search methods: • systema@c ←→ local search • determinis@c ←→ stochas@c • sequen@al ←→ parallel
19.11.13
9
Local Search (LS) Algorithms
• search space S (SAT: set of all complete truth assignments to proposi@onal
variables) • solu0on set (SAT: models of given formula) • neighborhood rela0on (SAT: neighboring variable assignments differ in the truth value of exactly one variable) • evalua0on func0on g : S → R+ (SAT: number of clauses unsa@sfied under given assignment)
⊆⊆
SS ⊆'
SxSN ⊆
Local Search:
• start from ini@al posi@on • itera@vely move from current posi@on to neighboring posi@on
• use evalua@on func@on for guidance Two main classes: • local search on parDal soluDons • local search on complete soluDons
local search on par@al solu@ons
• Order the variables in some order. • Span a tree such that at each level a given value is assigned a value.
• Perform a depth-‐first search. • But, use heuris@cs to guide the search. Choose the best child according to some heuris@cs. (DFS with node ordering)
Local search for partial solutions
DFS
• Once a solu@on has been found (with the first dive into the tree) we can con@nue search the tree with DFS and backtracking.
Construc0on Heuris0cs for par0al solu0ons
• search space: space of par@al solu@ons • search steps: extend par@al solu@ons with assignment for the next element
• solu@on elements are oren ranked according to a greedy evalua@on func@on
19.11.13
10
Nearest Neighbor heuris0c for the TSP:
Nearest neighbor tour through 532 US ci0es
My current best is 27486.199404966355 (nn gives 27766.484757657887)
All the best, Polina
My best is 24839,308924381 (Jaak S)
My new best is 23474 (Oleg) 23297.72476804589
Probably some local minimum near Jaak Sarv's solu@on
19.11.13
11
local search on complete solu@ons
Itera0ve Improvement (Greedy Search):
• ini@alize search at some point of search space • in each step, move from the current search posi@on to a neighboring posi@on with beNer evalua@on func@on value
Itera0ve Improvement for SAT
• ini@aliza@on: randomly chosen, complete truth assignment
• neighborhood: variable assignments are neighbors iff they differ in truth value of one variable
• neighborhood size: O(n) where n = number of variables
• evalua@on func@on: number of clauses unsa@sfied under given assignment
Hill climbing • Choose the neighbor with the largest improvment as the next state
states
f-value
f-value = evaluation(state)
while f-value(state) > f-value(next-best(state)) state := next-best(state)
Hill climbing
function Hill-Climbing(problem) returns a solution state current ← Make-Node(Initial-State[problem]) loop do next ← a highest-valued successor of current if Value[next] < Value[current] then return current current ←next end
19.11.13
12
Problems with local search Typical problems with local search (with hill climbing in par0cular)
• ge_ng stuck in local op@ma • being misguided by evalua@on/objec@ve func@on
Stochas0c Local Search
• randomize ini@aliza@on step • randomize search steps such that subop@mal/worsening steps are allowed
• improved performance & robustness • typically, degree of randomiza@on controlled by noise parameter
Stochas0c Local Search Pros: • for many combinatorial problems more efficient than systema@c search
• easy to implement • easy to parallelize Cons: • oren incomplete (no guarantees for finding exis@ng solu@ons)
• highly stochas@c behavior • oren difficult to analyze theore@cally/empirically
Simple SLS methods
• Random Search (Blind Guessing): • In each step, randomly select one element of the search space.
• (Uninformed) RandomWalk: • In each step, randomly select one of the neighbouring posiDons of the search space and move there.
Random restart hill climbing
states
f-value
f-value = evaluation(state)
Randomized Itera0ve Improvement:
• ini@alize search at some point of search space search steps: • with probability p, move from current search posi@on to a
randomly selected neighboring posi@on • otherwise, move from current search posi@on to neighboring
posi@on with beNer evalua@on func@on value. • Has many varia@ons of how to choose the randomly neighbor,
and how many of them • Example: Take 100 steps in one direc@on (Army mistake
correc@on) – to escape from local op@ma.
19.11.13
13
Search space
• Problem: depending on ini@al state, can get stuck in local maxima
spie98-‐74
General itera@ve Algorithms
– general and “easy” to implement – approxima@on algorithms – must be told when to stop – hill-‐climbing – convergence
spie98-‐75
General itera@ve search • Algorithm
– Ini@alize parameters and data structures – construct ini@al solu@on(s) – Repeat
• Repeat – Generate new solu@on(s) – Select solu@on(s)
• Un@l @me to adapt parameters • Update parameters
– Un@l @me to stop
• End spie98-‐76
Itera@ve search
• Most popular algorithms of this class – Simulated Annealing
• Probabilis@c algorithm inspired by the annealing of metals
– Tabu Search • Meta-‐heuris@c which is a generaliza@on of local search
– Gene0c Algorithms • Probabilis@c algorithm inspired by evolu@onary mechanisms
Hill-‐climbing search
• Problem: depending on ini@al state, can get stuck in local maxima
Simulated annealing Combinatorial search technique inspired by the physical process of annealing [Kirkpatrick et al. 1983, Cerny 1985]
Outline • Select a neighbor at random.
• If beNer than current state go there.
• Otherwise, go there with some probability.
• Probability goes down with @me (similar to
temperature cooling)
19.11.13
14
Simulated annealing hbp://en.wikipedia.org/wiki/Simulated_annealing
• Simulated annealing (SA) is a generic probabilis@c metaheuris@c for the global op@miza@on problem of loca@ng a good approxima@on to the global op@mum of a given func@on in a large search space. It is oren used when the search space is discrete (e.g., all tours that visit a given set of ci@es). For certain problems, simulated annealing may be more effec@ve than exhaus@ve enumera@on — provided that the goal is merely to find an acceptably good solu@on in a fixed amount of @me, rather than the best possible solu@on.
• The name and inspira@on come from annealing in metallurgy, a technique involving hea@ng and controlled cooling of a material to increase the size of its crystals and reduce their defects. The heat causes the atoms to become unstuck from their ini@al posi@ons (a local minimum of the internal energy) and wander randomly through states of higher energy; the slow cooling gives them more chances of finding configura@ons with lower internal energy than the ini@al one.
• By analogy with this physical process, each step of the SA algorithm replaces the current solu@on by a random "nearby" solu@on, chosen with a probability that depends both on the difference between the corresponding func@on values and also on a global parameter T (called the temperature), that is gradually decreased during the process. The dependency is such that the current solu@on changes almost randomly when T is large, but increasingly "downhill" as T goes to zero. The allowance for "uphill" moves poten@ally saves the method from becoming stuck at local op@ma—which are the bane of greedier methods.
• The method was independently described by ScoN Kirkpatrick, C. Daniel GelaN and Mario P. Vecchi in 1983,[1] and by Vlado Černý in 1985.[2] The method is an adapta@on of the Metropolis-‐Has@ngs algorithm, a Monte Carlo method to generate sample states of a thermodynamic system, invented by M.N. Rosenbluth in a paper by N. Metropolis et al. in 1953.[3]
Simulated annealing
eΔE/T
delta = -10 T= 0.1 .. 10,000
Generic choices for annealing schedule
Kirkpatrick, S., C. D. GelaN Jr., M. P. Vecchi, "Op@miza@on by Simulated Annealing",Science, 220, 4598, 671-‐680, 1983.
hNp://fezzik.ucd.ie/msc/cscs/ga/kirkpatrick83op@miza@on.pdf
Simulated Annealing Pseudo code function Simulated-Annealing(problem, schedule) returns
solution state current ← Make-Node(Initial-State[problem]) for t ← 1 to infinity T ← schedule[t] // T goes downwards. if T = 0 then return current next ← Random-Successor(current) ΔE ← f-Value[next] - f-Value[current] if ΔE > 0 then current ← next else current ←next with probability eΔE/T end
19.11.13
15
Example applica0on to the TSP [Johnson & McGeoch 1997]
baseline implementa@on: • start with random ini@al solu@on • use 2-‐exchange neighborhood • simple annealing schedule; à rela@vely poor performance improvements: • look-‐up table for acceptance probabili@es • neighborhood pruning • low-‐temperature starts
Diameter of the search graph • Simulated annealing may be modeled as a random walk on
a search graph, whose ver0ces are all possible states, and whose edges are the candidate moves. An essen0al requirement for the neighbour() func0on is that it must provide a sufficiently short path on this graph from the ini0al state to any state which may be the global op0mum. (In other words, thediameter of the search graph must be small.) In the traveling salesman example above, for instance, the search space for n = 20 ci0es has n! = 2432902008176640000 (2.4quin0llion) states; yet the neighbour generator func0on that swaps two consecu0ve ci0es can get from any state (tour) to any other state in maximum n(n − 1) / 2 = 190 steps.
Summary-‐Simulated Annealing
Simulated Annealing . . . • is historically important • is easy to implement • has interes@ng theore@cal proper@es (convergence), but these are of very limited prac@cal relevance
• achieves good performance oren at the cost of substan@al run-‐@mes
Examples for combinatorial problems:
• finding shortest/cheapest round trips (TSP) • finding models of proposi@onal formulae (SAT)
• planning, scheduling, @me-‐tabling • resource alloca@on • protein structure predic@on • genome sequence assembly
SAT
19.11.13
16
The Proposi0onal Sa0sfiability Problem (SAT)
Tabu Search
• Combinatorial search technique which heavily relies on the use of an explicit memory of the search process [Glover 1989, 1990] to guide search process
• memory typically contains only specific aNributes of previously seen solu@ons
• simple tabu search strategies exploit only short term memory
• more complex tabu search strategies exploit long term memory
Tabu search – exploi0ng short term memory
• in each step, move to best neighboring solu@on although it may be worse than current one
• to avoid cycles, tabu search tries to avoid revisi@ng previously seen solu@ons by basing the memory on aNributes of recently seen solu@ons
• tabu list stores aNributes of the tl most recently visited
• solu@ons; parameter tl is called tabu list length or tabu tenure
• solu@ons which contain tabu aNributes are forbidden
Example: Tabu Search for SAT / MAX-‐SAT
• Neighborhood: assignments which differ in exactly one variable instan@a@on
• Tabu abributes: variables • Tabu criterion: flipping a variable is forbidden for a given number of itera@ons
• Aspira0on criterion: if flipping a tabu variable leads to a beNer solu@on, the variable’s tabu status is overridden
[Hansen & Jaumard 1990; Selman & Kautz 1994]
• Bart Selman, Cornell www-‐verimag.imag.fr/~maler/TCC/selman-‐tcc.ppt Ideas from physics, staDsDcs, combinatorics, algorithmics …
Fundamental challenge: Combinatorial Search Spaces
• Significant progress in the last decade.
• How much?
• For proposi@onal reasoning: • -‐-‐ We went from 100 variables, 200 clauses (early 90’s) • to 1,000,000 vars. and 5,000,000 constraints in • 10 years. Search space: from 10^30 to 10^300,000.
• -‐-‐ Applica@ons: Hardware and Sorware Verifica@on, • Test paNern genera@on, Planning, Protocol Design, • Routers, Timetabling, E-‐Commerce (combinatorial • auc@ons), etc. • •
19.11.13
17
• How can deal with such large combinatorial spaces and • s@ll do a decent job?
• I’ll discuss recent formal insights into • combinatorial search spaces and their • prac@cal implica@ons that makes searching • such ultra-‐large spaces possible.
• Brings together ideas from physics of disordered systems • (spin glasses), combinatorics of random structures, and • algorithms.
• But first, what is BIG?
I.e., ((not x_1) or x_7) ((not x_1) or x_6) etc.
What is BIG?
x_1, x_2, x_3, etc. our Boolean variables (set to True or False)
Set x_1 to False ??
Consider a real-world Boolean Satisfiability (SAT) problem
I.e., (x_177 or x_169 or x_161 or x_153 … x_33 or x_25 or x_17 or x_9 or x_1 or (not x_185)) clauses / constraints are getting more interesting…
10 pages later:
…
Note x_1 …
4000 pages later:
…
Finally, 15,000 pages later:
Current SAT solvers solve this instance in approx. 1 minute!
Combinatorial search space of truth assignments: HOW?
102
Progress SAT Solvers
Instance Posit' 94
ssa2670-136 40,66s
bf1355-638 1805,21s
pret150_25 >3000s
dubois100 >3000s
aim200-2_0-no-1 >3000s
2dlx_..._bug005 >3000s
c6288 >3000s
Grasp' 96
1,2s
0,11s
0,21s
11,85s
0,01s
>3000s
>3000s
Sato' 98
0,95s
0,04s
0,09s
0,08s
0s
>3000s
>3000s
Chaff' 01
0,02s
0,01s
0,01s
0,01s
0s
2,9s
>3000s
Source: Marques Silva 2002
19.11.13
18
• From academically interes@ng to prac@cally relevant.
• We now have regular SAT solver compe@@ons.
• Germany ’89, Dimacs ’93, China ’96, SAT-‐02, SAT-‐03, SAT-‐04, SAT05.
• E.g. at SAT-‐2004 (Vancouver, May 04): • -‐-‐-‐ 35+ solvers submiNed • -‐-‐-‐ 500+ industrial benchmarks • -‐-‐-‐ 50,000+ instances available on the WWW.
• 104
Real-‐World Reasoning Tackling inherent computa@onal complexity
100 200
10K 50K
50K 200K
0.5M 1M
1M 5M
Variables
1030
10301,020
10150,500
1015,050
103010
Wor
st C
ase
com
plex
ity
Car repair diagnosis
Deep space mission control
Chess
Hardware/Software Verification
Multi-Agent Systems
200K 600K
Military Logistics
Seconds until heat death of sun
Protein folding calculation (petaflop-year)
No. of atoms on earth 1047
100 10K 20K 100K 1M
Rules (Constraints) Example domains cast in propositional reasoning system (variables, rules).
• High-Performance Reasoning • Temporal/ uncertainty reasoning • Strategic reasoning/Multi-player
Technology Targets
DARPA Research Program
105 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
“Genetic Algorithms are good at taking large,
potentially huge search spaces and navigating
them, looking for optimal combinations of things, solutions you might not
otherwise find in a lifetime.”
- Salvatore Mangano
Computer Design, May 1995
Genetic Algorithms: A Tutorial
106 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
The Genetic Algorithm
Directed search algorithms based on the mechanics of biological evolution
Developed by John Holland, University of Michigan (1970’s) ♦ To understand the adaptive processes of
natural systems ♦ To design artificial systems software that
retains the robustness of natural systems
107 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
The Genetic Algorithm (cont.)
Provide efficient, effective techniques for optimization and machine learning applications
Widely-used today in business, scientific and engineering circles
108 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Components of a GA
A problem to solve, and ... Encoding technique (gene, chromosome) Initialization procedure (creation) Evaluation function (environment)
Selection of parents (reproduction) Genetic operators (mutation, recombination)
Parameter settings (practice and art)
19.11.13
19
109 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Simple Genetic Algorithm {
initialize population; evaluate population; while TerminationCriteriaNotSatisfied {
select parents for reproduction; perform recombination and mutation; evaluate population;
} }
110 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
The GA Cycle of Reproduction
reproduction
population evaluation
modification
discard
deleted members
parents
children
modified children
evaluated children
Gene0c algorithms
• How to generate the next genera@on. • 1) Selec<on: we select a number of states from the current genera@on. (we can use the fitness func@on in any reasonable way)
• 2) crossover : select 2 states and reproduce a child.
• 3) muta<on: change some of the genues.
Example
8-‐queen example Summary: Gene0c Algorithms
Gene0c Algorithms • use popula@ons, which leads to increased search space explora@on
• allow for a large number of different implementa@on choices
• typically reach best performance when using operators that are based on problem characteris@cs
• achieve good performance on a wide range of problems
19.11.13
20
115 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Classes of Search Techniques
Finonacci Newton
Direct methods Indirect methods
Calculus-based techniques
Evolutionary strategies
Centralized Distributed
Parallel
Steady-state Generational
Sequential
Genetic algorithms
Evolutionary algorithms Simulated annealing
Guided random search techniques
Dynamic programming
Enumerative techniques
Search techniques
A.E. Eiben and J.E. Smith, Introduction to Evolutionary Computing Evolutionary Programming
Example application: evolving checkers players (Fogel’02)
l Neural nets for evaluating future values of moves are evolved
l NNs have fixed structure with 5046 weights, these are evolved + one weight for “kings”
l Representation: – vector of 5046 real numbers for object variables (weights) – vector of 5046 real numbers for σ‘s
l Mutation: – Gaussian, lognormal scheme with σ-first – Plus special mechanism for the kings’ weight
l Population size 15
A.E. Eiben and J.E. Smith, Introduction to Evolutionary Computing Evolutionary Programming
Example application: evolving checkers players (Fogel’02)
l Tournament size q = 5 l Programs (with NN inside) play against other
programs, no human trainer or hard-wired intelligence
l After 840 generation (6 months!) best strategy was tested against humans via Internet
l Program earned “expert class” ranking outperforming 99.61% of all rated players
118 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
The GA Cycle of Reproduction
reproduction
population evaluation
modification
discard
deleted members
parents
children
modified children
evaluated children
119 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Population
Chromosomes could be:
♦ Bit strings (0101 ... 1100) ♦ Real numbers (43.2 -33.1 ... 0.0 89.2) ♦ Permutations of element (E11 E3 E7 ... E1 E15) ♦ Lists of rules (R1 R2 R3 ... R22 R23) ♦ Program elements (genetic programming) ♦ ... any data structure ...
population
120 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Reproduction
reproduction
population
parents
children
Parents are selected at random with selection chances biased in relation to chromosome evaluations.
19.11.13
21
121 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Chromosome Modification
modification children
Modifications are stochastically triggered Operator types are:
♦ Mutation ♦ Crossover (recombination)
modified children
122 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Mutation: Local Modification Before: (1 0 1 1 0 1 1 0)
After: (0 1 1 0 0 1 1 0)
Before: (1.38 -69.4 326.44 0.1)
After: (1.38 -67.5 326.44 0.1)
Causes movement in the search space (local or global)
Restores lost information to the population
123 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Crossover: Recombination
P1 (0 1 1 0 1 0 0 0) (0 1 0 0 1 0 0 0) C1 P2 (1 1 0 1 1 0 1 0) (1 1 1 1 1 0 1 0) C2
Crossover is a critical feature of genetic algorithms:
♦ It greatly accelerates search early in evolution of a population
♦ It leads to effective combination of schemata (subsolutions on different chromosomes)
*
124 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Evaluation
The evaluator decodes a chromosome and
assigns it a fitness measure The evaluator is the only link between a
classical GA and the problem it is solving
evaluation
evaluated children
modified children
125 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Deletion
Generational GA:
entire populations replaced with each iteration Steady-state GA:
a few members replaced each generation
population
discard
discarded members
126 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
An Abstract Example
Distribution of Individuals in Generation 0
Distribution of Individuals in Generation N
19.11.13
22
127 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
A Simple Example
“The Gene is by far the most sophisticated program around.”
- Bill Gates, Business Week, June 27, 1994
128 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
A Simple Example
The Traveling Salesman Problem:
Find a tour of a given set of cities so that ♦ each city is visited only once ♦ the total distance traveled is minimized
129 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Representation
Representation is an ordered list of city numbers known as an order-based GA.
1) London 3) Dunedin 5) Beijing 7) Tokyo 2) Venice 4) Singapore 6) Phoenix 8) Victoria CityList1 (3 5 7 2 1 6 4 8) CityList2 (2 5 7 6 8 1 3 4)
130 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Crossover
Crossover combines inversion and recombination: * * Parent1 (3 5 7 2 1 6 4 8) Parent2 (2 5 7 6 8 1 3 4)
Child (5 8 7 2 1 6 3 4) This operator is called the Order1 crossover.
131 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Mutation involves reordering of the list: * * Before: (5 8 7 2 1 6 3 4) After: (5 8 6 2 1 7 3 4)
Mutation
132 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
TSP Example: 30 Cities
0
20
40
60
80
100
120
0 10 20 30 40 50 60 70 80 90 100
y
x
19.11.13
23
133 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Solution i (Distance = 941)
0
20
40
60
80
100
120
0 10 20 30 40 50 60 70 80 90 100
y
x
TSP30 (Performance = 941)
134 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Solution j(Distance = 800) 44626967786462544250404038213567606040425099
0
20
40
60
80
100
120
0 10 20 30 40 50 60 70 80 90 100
y
x
TSP30 (Performance = 800)
135 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Solution k(Distance = 652)
0
20
40
60
80
100
120
0 10 20 30 40 50 60 70 80 90 100
y
x
TSP30 (Performance = 652)
136 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Best Solution (Distance = 420) 4238352621353273846445860697678716967628494
0
20
40
60
80
100
120
0 10 20 30 40 50 60 70 80 90 100
y
x
TSP30 Solution (Performance = 420)
137 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Overview of Performance
0
200
400
600
800
1000
1200
1400
1600
1800
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Distance
Generations (1000)
TSP30 - Overview of Performance
Best
Worst
Average
138 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Considering the GA Technology “Almost eight years ago ... people at Microsoft wrote
a program [that] uses some genetic things for
finding short code sequences. Windows 2.0
and 3.2, NT, and almost all Microsoft applications products have shipped
with pieces of code created by that system.”
- Nathan Myhrvold, Microsoft Advanced Technology Group, Wired, September 1995
19.11.13
24
139 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Issues for GA Practitioners
Choosing basic implementation issues: ♦ representation ♦ population size, mutation rate, ... ♦ selection, deletion policies ♦ crossover, mutation operators
Termination Criteria Performance, scalability Solution is only as good as the evaluation
function (often hardest part)
140 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Benefits of Genetic Algorithms
Concept is easy to understand Modular, separate from application Supports multi-objective optimization Good for “noisy” environments Always an answer; answer gets better
with time Inherently parallel; easily distributed
141 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Benefits of Genetic Algorithms (cont.)
Many ways to speed up and improve a GA-based application as knowledge about problem domain is gained
Easy to exploit previous or alternate solutions
Flexible building blocks for hybrid applications
Substantial history and range of use
142 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
When to Use a GA
Alternate solutions are too slow or overly complicated
Need an exploratory tool to examine new approaches
Problem is similar to one that has already been successfully solved by using a GA
Want to hybridize with an existing solution Benefits of the GA technology meet key problem
requirements
143 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Some GA Application Types Domain Application TypesControl gas pipeline, pole balancing, missile evasion, pursuit
Design semiconductor layout, aircraft design, keyboardconfiguration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation
Robotics trajectory planning
Machine Learning designing neural networks, improving classificationalgorithms, classifier systems
Signal Processing filter design
Game Playing poker, checkers, prisoner’s dilemma
CombinatorialOptimization
set covering, travelling salesman, routing, bin packing,graph colouring and partitioning
Review 4 main types of Evolu@onary Algorithms
• Gene@c Algorithm -‐ John Holland • Gene@c Programming -‐ John Koza • Evolu@onary Programming -‐ Lawerence Fogel
• Evolu@onary Strategies -‐ Ingo Rechenberg
19.11.13
25
Gene@c Algorithms
• Most widely used • Robust • uses 2 separate spaces
– search space -‐ coded solu@on (genotype) – solu@on space -‐ actual solu@ons (phenotypes)
• Genotypes must be mapped to phenotypes before the quality or fitness of each solu@on can be evaluated
Gene@c Programming • Specialized form of GA • Manipulates a very specific type of solu@on using modified gene@c operators
• Original applica@on was to design computer program
• Now applied in alterna@ve areas eg. Analog Circuits
• Does not make dis@nc@on between search and solu@on space.
• Solu@on represented in very specific hierarchical manner.
Evolu@onary Strategies
• Like GP no dis@nc@on between search and solu@on space
• Individuals are represented as real-‐valued vectors.
• Simple ES – one parent and one child – Child solu@on generated by randomly muta@ng the problem parameters of the parent.
• Suscep@ble to stagna@on at local op@ma
Evolu@onary Strategies (cont’d)
• Slow to converge to op@mal solu@on • More advanced ES
– have pools of parents and children • Unlike GA and GP, ES
– Separates parent individuals from child individuals
– Selects its parent solu@ons determinis@cally
Evolu@onary Programming • Resembles ES, developed independently • Early versions of EP applied to the evolu@on of transi@on
table of finite state machines • One popula@on of solu@ons, reproduc@on is by muta@on only • Like ES operates on the decision variable of the problem
directly (ie Genotype = Phenotype) • Tournament selec@on of parents
– beNer fitness more likely a parent – children generated un@l popula@on doubled in size – everyone evaluated and the half of popula@on with lowest fitness
deleted.
General Idea of Evolu@onary Algorithms
19.11.13
26
CS 478 - Evolutionary Algorithms 151
Genetic Programming
l Evolves more complex structures - programs, Lisp code, neural networks
l Start with random programs of functions and terminals (data structures)
l Execute programs and give each a fitness measure l Use crossover to create new programs, no mutation l Keep best programs l For example, place lisp code in a tree structure,
functions at internal nodes, terminals at leaves, and do crossover at sub-trees - always legal in Lisp
CS 478 - Evolutionary Algorithms 152
153
Evolu@onary design
• Karl Sims Evolved Virtual Creatures (1994) – hNp://www.youtube.com/watch?v=F0OHycypSG8 – hNp://www.karlsims.com/evolved-‐virtual-‐creatures.html – hNp://www.archive.org/details/sims_evolved_virtual_creatures_1994
– hNp://video.google.com/videoplay?docid=7219479512410540649# – course work -‐ 2005
• hbp://rogeralsing.com/2008/12/07/gene0c-‐programming-‐evolu0on-‐of-‐mona-‐lisa/
• hbp://vimeo.com/7074089 155
19.11.13
27
19.11.13
28
19.11.13
29
171 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial
Conclusions
Question: ‘If GAs are so smart, why ain’t they rich?’
Answer: ‘Genetic algorithms are rich - rich in application across a large and growing number of disciplines.’
- David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning
• Games: Spore (2007) hNp://www.ted.com/talks/will_wright_makes_toys_that_make_worlds.html
• hNp://www.gametrailers.com/user-‐movie/spore-‐14min-‐2007-‐demonstra@on/86368
• hNp://eu.spore.com/home.cfm?lang=en
Ant Colony Op@miza@on (ACO)
1991, M. Dorigo proposed the Ant System in his doctoral thesis
Strong or weak pheromone
19.11.13
30
TSP More on ACO
• It can work on dynamic systems, adop@ng to changes in the environment
Differen@al Evolu@on
• Real-‐valued chromosomes • X = ( x1 , x2, x3, … , xn )
19.11.13
31
Robust Regression (Differen@al Evolu@on)
Fi_ng a regression line using minimum median error as a measure.
aX + bY + c = 0 Y = aX + c Find a and c
Differen@al Evolu@on
Fit any polynomial, use mean or median, add MDL based iden@fica@on of the degree of polynomial
An Xn + An-‐1 Xn-‐1 + … + A1 X + A0
19.11.13
32