Lecture 12 Search heuristics - ut · 19.11.13 5 Greedy(SetCovering(Algorithm(source: 91.503...

19.11.13

1

Advanced Algorithmics (6EAP) Search and meta-‐heuris0cs

Jaak Vilo 2012 Spring

1 Jaak Vilo http://aima.cs.berkeley.edu/ http://books.google.ee/books/about/Heuristic_Search.html?id=3k5MVjKzBP4C&redir_esc=y

Search

•  for what? – a solu@on –  the (best possible (approximate?)) solu@on

•  from where? – search space (all valid solu@ons or paths)

•  under which condi0ons? – compute @me, space, … – constraints, …

Objec@ve func@on

•  An op@mal solu@on – what is the measure that we op0mise?

•  Any solu0on (sa@sfiability /SAT/ problem) –  does the task have a solu@on? –  is there a solu@on with objec@ve measure beNer than X?

• Minimal/maximal cost solu@on •  A winning move in a game •  A (feasible) solu@on with smallest nr of constraint viola@ons (e.g. course @me scheduling)

Search space size?

•  Linear (list, binary search, …) •  Integer in [i,j] •  Real nr in [x,y) •  A point in high-‐dimensional space •  An assignment of variables (in SAT) •  A subset of a larger set •  Trees, Graphs •  …

The Proposi0onal Sa0sfiability Problem (SAT)

http://www.satcompetition.org/

19.11.13

2

Inputs usually in CNF

(a + b + c) (a’ + b’ + c) (a + b’ + c’) (a’ + b + c’)

Solu@on search space: la_ce for 4 variables (par@al connec@ons)

0000

1111

0001 0010 0100 1000

0111 1011 1101 1110

0011 0101 0110 1010 1001 1100

O(2n)

Tic-‐Tac-‐Toe It is straightforward to write a computer program to play Tic-tac-toe perfectly, to enumerate the 765 essentially different positions (the state space complexity), or the 26,830 possible games up to rotations and reflections (the game tree complexity) on this space (2002?)

TSP, nearest neighbour search

TSP, NN subop@mal

19.11.13

3

Issues: Constraints

•  Time, space… –  if op@mal cannot be found, approximate

•  All kinds of secondary characteris@cs

•  Constraints – some@mes finding even a point in the valid search space is hard

19.11.13

4

Examples

•  Greedy •  A* search •  Monte Carlo, Grid •  Local search algorithms -‐ Hill-‐climbing, beam, … •  Simulated annealing search •  Gene@c algorithms (GA, GP) •  Differen@al Evolu@ons (DE) •  Par@cle Swarm Op@misa@on (PSO) •  Ant colony op@misa@on (ACO)

22 Wendy Williams Metaheuristic Algorithms Genetic Algorithms: A Tutorial

Classes of Search Techniques

Finonacci Newton

Direct methods Indirect methods

Calculus-based techniques

Evolutionary strategies

Centralized Distributed

Parallel

Steady-state Generational

Sequential

Genetic algorithms

Evolutionary algorithms Simulated annealing

Guided random search techniques

Dynamic programming

Enumerative techniques

Search techniques

Greedy

•  Always choose the next seemingly best step

•  Set Cover – Greedy Approxima@on Algorithm – polynomial-‐@me ρ(n)-‐approxima@on algorithm

•  ρ(n) is a logarithmic func@on of set size •  n – size of the largest set…

•  | C(greedy) | < ln(n) * | C(op@mal) |

Set Cover Problem

source: 91.503 textbook Cormen et al.

SXS F∈

=

X of subsets offamily F

points) of (e.g. set finite X

:),(Instance FX

NP-Complete

CS

SX: XFC

∈

=

⊆

of allcover members whose subset sized-minimum a Find :Problem F

19.11.13

5

Greedy Set Covering Algorithm

source: 91.503 textbook Cormen et al.

Greedy: select set that covers the most uncovered elements

Set Cover Theorem: GREEDY-SET-COVER is a

polynomial-time ρ(n)-approximation algorithm for

Proof: })|:(max{|)( F∈= SSHnρ

selectedsubset th iSi = 1 costs iSselecting

Algorithm runs in time polynomial in n.

Xxcx ∈= element ofcost paid only when x is covered for the first time

)(1

121 −∪∪∪−=

iix SSSSc

assume x is covered for the first time by Si (spread cost evenly across all elements covered for first time by Si )

Number of elements covered for first time by Si

)(1number harmonicth 1

dHi

Hdd

id ==∑

=

H(0)=0

Proof of approxima@on

•  hNp://www.cs.dartmouth.edu/~ac/Teach/CS105-‐Winter05/Notes/wan-‐ba-‐notes.pdf

Set Cover (proof con@nued) Theorem: GREEDY-SET-COVER is a


Proof: (continued)

})|:(max{|)( F∈= SSHnρ

*

∑ ∑∈ ∈

⎟⎠

⎞⎜⎝

⎛≤

CS SxxcC

∑∑ ∑∈∈ ∈

≥⎟⎠

⎞⎜⎝

⎛

Xxx

CS Sxx cc

*

*∑ ∑∈ ∈

⎟⎠

⎞⎜⎝

⎛

CS SxxcCost assigned to

optimal cover:

Each x is in >= 1 S in C*

COVER-SET-GREEDY fromcover be cover optimalan be*Let

CC ∑

∈

=Xx

xcC1 unit is charged at each stage of algorithm

Set Cover (proof con@nued) Theorem: GREEDY-SET-COVER is a


Proof: (continued)

})|:(max{|)( F∈= SSHnρ

)( SHcSx

x ≤∑∈

How does this relate to harmonic numbers??

)(1number harmonicth 1

dHi

Hdd

id ==∑

=

We’ll show that:

}):(max{* )(*

F∈≤≤ ∑∈

SSHCSHCCS

And then conclude that:

which will finish the proof

F∈Sset any for

19.11.13

6

Local Search

Local search algorithms •  In many op@miza@on problems, the path to the goal is irrelevant; the goal state itself is the solu@on

•  State space = set of "complete" configura@ons •  Find configura@on sa@sfying constraints, e.g., n-‐queens

•  In such cases, we can use local search algorithms •  keep a single "current" state, try to improve it

Example: n-‐queens

•  Put n queens on an n × n board with no two queens on the same row, column, or diagonal

•  may get stuck…

Problems

•  Cycles – Memorize; Tabu search

•  How to transfer valleys with bad choices only…

Tree/Graph search

•  order defined by picking a node for expansion

•  BFS, DFS

•  Random, Best First, … – Best – an evalua@on func@on

19.11.13

7

•  Idea: use an evalua0on func0on f(n) for each node –  es@mate of "desirability" –  Expand most desirable unexpanded node

•  Implementa@on: Order the nodes in fringe in decreasing order of desirability Priority queue

•  Special cases:

–  greedy best-‐first search f(n) = h(n) heuris@c, e.g. es@mate to goal –  A* search

A*

•  f(n) = g(n) + h(n)

– g(n) – path covered so far in graph – h(n) – es@mated distance from n to goal

A C

D

B

F

20 20 10

15 25

Admissible heuris@cs

•  A heuris@c h(n) is admissible if for every node n, h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state from n.

•  An admissible heuris@c never overes@mates the cost to reach the goal, i.e., it is op0mis0c

•  Example: hSLD(n) (never overes@mates the actual road distance) (SLD – shortest linear distance)

•  Theorem: If h(n) is admissible, A* using TREE-SEARCH is

op@mal

Op@mality of A* (proof) •  Suppose some subop@mal goal G2 has been generated and is in the fringe.

Let n be an unexpanded node in the fringe such that n is on a shortest path to an op@mal goal G.

•  f(G2) = g(G2) since h(G2) = 0 •  g(G2) > g(G) since G2 is subop@mal •  f(G) = g(G) since h(G) = 0 •  f(G2) > f(G) from above

Op@mality of A* (proof) •  Suppose some subop@mal goal G2 has been generated and is in the fringe.

Let n be an unexpanded node in the fringe such that n is on a shortest path to an op@mal goal G.

•  f(G2) = g(G2) since h(G2) = 0 •  g(G2) > g(G) since G2 is subop@mal •  f(G) = g(G) since h(G) = 0 •  f(G2) > f(G) from above •  Hence f(G2) > f(n), and A* will never select G2 for expansion

A* path-‐finder

Better: http://www.youtube.com/watch?v=19h1g22hby8

19.11.13

8

Graph

•  A Virtual graph/search space – valid states of Fireen-‐game – Rubik’s cube

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15

Solve

•  Which move takes us closer to the solu@on? •  Es@mate the goodness of the state

1 2

3 4

5 6 7 8

9 10

11

12

13 14

15

•  How many are misplaced? (7)

•  How far have they been misplaced? Sum of theore@cal shortest paths to the correct place

•  A* search towards a final goal

1 2

3 4

5 6 7 8

9 10

11

12

13 14

15

Search space

The Traveling Salesperson Problem (TSP)

•  TSP – op0miza0on variant: •  For a given weighted graph G = (V,E,w), find a Hamiltonian cycle in G with minimal weight,

•  i.e., find the shortest round-‐trip visiDng each vertex exactly once.

•  TSP – decision variant: •  For a given weighted graph G = (V,E,w), decide whether a Hamiltonian cycle with minimal weight ≤ b exists in G.

TSP instance: shortest round trip through 532 US ci0es

Search Methods

•  Types of search methods: •  systema@c ←→ local search •  determinis@c ←→ stochas@c •  sequen@al ←→ parallel

19.11.13

9

Local Search (LS) Algorithms

•  search space S (SAT: set of all complete truth assignments to proposi@onal

variables) •  solu0on set (SAT: models of given formula) •  neighborhood rela0on (SAT: neighboring variable assignments differ in the truth value of exactly one variable) •  evalua0on func0on g : S → R+ (SAT: number of clauses unsa@sfied under given assignment)

⊆⊆

SS ⊆'

SxSN ⊆

Local Search:

•  start from ini@al posi@on •  itera@vely move from current posi@on to neighboring posi@on

•  use evalua@on func@on for guidance Two main classes: •  local search on parDal soluDons •  local search on complete soluDons

local search on par@al solu@ons

•  Order the variables in some order. •  Span a tree such that at each level a given value is assigned a value.

•  Perform a depth-‐first search. •  But, use heuris@cs to guide the search. Choose the best child according to some heuris@cs. (DFS with node ordering)

Local search for partial solutions

DFS

•  Once a solu@on has been found (with the first dive into the tree) we can con@nue search the tree with DFS and backtracking.

Construc0on Heuris0cs for par0al solu0ons

•  search space: space of par@al solu@ons •  search steps: extend par@al solu@ons with assignment for the next element

•  solu@on elements are oren ranked according to a greedy evalua@on func@on

19.11.13

10

Nearest Neighbor heuris0c for the TSP:

Nearest neighbor tour through 532 US ci0es

My current best is 27486.199404966355 (nn gives 27766.484757657887)

All the best, Polina

My best is 24839,308924381 (Jaak S)

My new best is 23474 (Oleg) 23297.72476804589

Probably some local minimum near Jaak Sarv's solu@on

19.11.13

11

local search on complete solu@ons

Itera0ve Improvement (Greedy Search):

• ini@alize search at some point of search space • in each step, move from the current search posi@on to a neighboring posi@on with beNer evalua@on func@on value

Itera0ve Improvement for SAT

•  ini@aliza@on: randomly chosen, complete truth assignment

•  neighborhood: variable assignments are neighbors iff they differ in truth value of one variable

•  neighborhood size: O(n) where n = number of variables

•  evalua@on func@on: number of clauses unsa@sfied under given assignment

Hill climbing •  Choose the neighbor with the largest improvment as the next state

states

f-value

f-value = evaluation(state)

while f-value(state) > f-value(next-best(state)) state := next-best(state)

Hill climbing

function Hill-Climbing(problem) returns a solution state current ← Make-Node(Initial-State[problem]) loop do next ← a highest-valued successor of current if Value[next] < Value[current] then return current current ←next end

19.11.13

12

Problems with local search Typical problems with local search (with hill climbing in par0cular)

•  ge_ng stuck in local op@ma •  being misguided by evalua@on/objec@ve func@on

Stochas0c Local Search

•  randomize ini@aliza@on step •  randomize search steps such that subop@mal/worsening steps are allowed

•  improved performance & robustness •  typically, degree of randomiza@on controlled by noise parameter

Stochas0c Local Search Pros: •  for many combinatorial problems more efficient than systema@c search

•  easy to implement •  easy to parallelize Cons: •  oren incomplete (no guarantees for finding exis@ng solu@ons)

•  highly stochas@c behavior •  oren difficult to analyze theore@cally/empirically

Simple SLS methods

•  Random Search (Blind Guessing): •  In each step, randomly select one element of the search space.

•  (Uninformed) RandomWalk: •  In each step, randomly select one of the neighbouring posiDons of the search space and move there.

Random restart hill climbing

states

f-value

f-value = evaluation(state)

Randomized Itera0ve Improvement:

•  ini@alize search at some point of search space search steps: •  with probability p, move from current search posi@on to a

randomly selected neighboring posi@on •  otherwise, move from current search posi@on to neighboring

posi@on with beNer evalua@on func@on value. •  Has many varia@ons of how to choose the randomly neighbor,

and how many of them •  Example: Take 100 steps in one direc@on (Army mistake

correc@on) – to escape from local op@ma.

19.11.13

13

Search space

•  Problem: depending on ini@al state, can get stuck in local maxima

spie98-‐74

General itera@ve Algorithms

– general and “easy” to implement – approxima@on algorithms – must be told when to stop – hill-‐climbing – convergence

spie98-‐75

General itera@ve search •  Algorithm

–  Ini@alize parameters and data structures – construct ini@al solu@on(s) – Repeat

•  Repeat –  Generate new solu@on(s) –  Select solu@on(s)

•  Un@l @me to adapt parameters •  Update parameters

– Un@l @me to stop

•  End spie98-‐76

Itera@ve search

•  Most popular algorithms of this class – Simulated Annealing

•  Probabilis@c algorithm inspired by the annealing of metals

– Tabu Search •  Meta-‐heuris@c which is a generaliza@on of local search

– Gene0c Algorithms •  Probabilis@c algorithm inspired by evolu@onary mechanisms

Hill-‐climbing search

•  Problem: depending on ini@al state, can get stuck in local maxima

Simulated annealing Combinatorial search technique inspired by the physical process of annealing [Kirkpatrick et al. 1983, Cerny 1985]

Outline •  Select a neighbor at random.

•  If beNer than current state go there.

•  Otherwise, go there with some probability.

•  Probability goes down with @me (similar to

temperature cooling)

19.11.13

14

Simulated annealing hbp://en.wikipedia.org/wiki/Simulated_annealing

•  Simulated annealing (SA) is a generic probabilis@c metaheuris@c for the global op@miza@on problem of loca@ng a good approxima@on to the global op@mum of a given func@on in a large search space. It is oren used when the search space is discrete (e.g., all tours that visit a given set of ci@es). For certain problems, simulated annealing may be more effec@ve than exhaus@ve enumera@on — provided that the goal is merely to find an acceptably good solu@on in a fixed amount of @me, rather than the best possible solu@on.

•  The name and inspira@on come from annealing in metallurgy, a technique involving hea@ng and controlled cooling of a material to increase the size of its crystals and reduce their defects. The heat causes the atoms to become unstuck from their ini@al posi@ons (a local minimum of the internal energy) and wander randomly through states of higher energy; the slow cooling gives them more chances of finding configura@ons with lower internal energy than the ini@al one.

•  By analogy with this physical process, each step of the SA algorithm replaces the current solu@on by a random "nearby" solu@on, chosen with a probability that depends both on the difference between the corresponding func@on values and also on a global parameter T (called the temperature), that is gradually decreased during the process. The dependency is such that the current solu@on changes almost randomly when T is large, but increasingly "downhill" as T goes to zero. The allowance for "uphill" moves poten@ally saves the method from becoming stuck at local op@ma—which are the bane of greedier methods.

•  The method was independently described by ScoN Kirkpatrick, C. Daniel GelaN and Mario P. Vecchi in 1983,[1] and by Vlado Černý in 1985.[2] The method is an adapta@on of the Metropolis-‐Has@ngs algorithm, a Monte Carlo method to generate sample states of a thermodynamic system, invented by M.N. Rosenbluth in a paper by N. Metropolis et al. in 1953.[3]

Simulated annealing

eΔE/T

delta = -10 T= 0.1 .. 10,000

Generic choices for annealing schedule

Kirkpatrick, S., C. D. GelaN Jr., M. P. Vecchi, "Op@miza@on by Simulated Annealing",Science, 220, 4598, 671-‐680, 1983.

hNp://fezzik.ucd.ie/msc/cscs/ga/kirkpatrick83op@[email protected]

Simulated Annealing Pseudo code function Simulated-Annealing(problem, schedule) returns

solution state current ← Make-Node(Initial-State[problem]) for t ← 1 to infinity T ← schedule[t] // T goes downwards. if T = 0 then return current next ← Random-Successor(current) ΔE ← f-Value[next] - f-Value[current] if ΔE > 0 then current ← next else current ←next with probability eΔE/T end

19.11.13

15

Example applica0on to the TSP [Johnson & McGeoch 1997]

baseline implementa@on: •  start with random ini@al solu@on •  use 2-‐exchange neighborhood •  simple annealing schedule; à rela@vely poor performance improvements: •  look-‐up table for acceptance probabili@es •  neighborhood pruning •  low-‐temperature starts

Diameter of the search graph •  Simulated annealing may be modeled as a random walk on

a search graph, whose ver0ces are all possible states, and whose edges are the candidate moves. An essen0al requirement for the neighbour() func0on is that it must provide a sufficiently short path on this graph from the ini0al state to any state which may be the global op0mum. (In other words, thediameter of the search graph must be small.) In the traveling salesman example above, for instance, the search space for n = 20 ci0es has n! = 2432902008176640000 (2.4quin0llion) states; yet the neighbour generator func0on that swaps two consecu0ve ci0es can get from any state (tour) to any other state in maximum n(n − 1) / 2 = 190 steps.

Summary-‐Simulated Annealing

Simulated Annealing . . . •  is historically important •  is easy to implement •  has interes@ng theore@cal proper@es (convergence), but these are of very limited prac@cal relevance

•  achieves good performance oren at the cost of substan@al run-‐@mes

Examples for combinatorial problems:

•  finding shortest/cheapest round trips (TSP) •  finding models of proposi@onal formulae (SAT)

•  planning, scheduling, @me-‐tabling •  resource alloca@on •  protein structure predic@on •  genome sequence assembly

SAT

19.11.13

16

The Proposi0onal Sa0sfiability Problem (SAT)

Tabu Search

•  Combinatorial search technique which heavily relies on the use of an explicit memory of the search process [Glover 1989, 1990] to guide search process

•  memory typically contains only specific aNributes of previously seen solu@ons

•  simple tabu search strategies exploit only short term memory

•  more complex tabu search strategies exploit long term memory

Tabu search – exploi0ng short term memory

•  in each step, move to best neighboring solu@on although it may be worse than current one

•  to avoid cycles, tabu search tries to avoid revisi@ng previously seen solu@ons by basing the memory on aNributes of recently seen solu@ons

•  tabu list stores aNributes of the tl most recently visited

•  solu@ons; parameter tl is called tabu list length or tabu tenure

•  solu@ons which contain tabu aNributes are forbidden

Example: Tabu Search for SAT / MAX-‐SAT

•  Neighborhood: assignments which differ in exactly one variable instan@a@on

•  Tabu abributes: variables •  Tabu criterion: flipping a variable is forbidden for a given number of itera@ons

•  Aspira0on criterion: if flipping a tabu variable leads to a beNer solu@on, the variable’s tabu status is overridden

[Hansen & Jaumard 1990; Selman & Kautz 1994]

•  Bart Selman, Cornell www-‐verimag.imag.fr/~maler/TCC/selman-‐tcc.ppt Ideas from physics, staDsDcs, combinatorics, algorithmics …

Fundamental challenge: Combinatorial Search Spaces

•  Significant progress in the last decade.

•  How much?

•  For proposi@onal reasoning: •  -‐-‐ We went from 100 variables, 200 clauses (early 90’s) •  to 1,000,000 vars. and 5,000,000 constraints in •  10 years. Search space: from 10^30 to 10^300,000.

•  -‐-‐ Applica@ons: Hardware and Sorware Verifica@on, •  Test paNern genera@on, Planning, Protocol Design, •  Routers, Timetabling, E-‐Commerce (combinatorial •  auc@ons), etc. •  • 

19.11.13

17

•  How can deal with such large combinatorial spaces and •  s@ll do a decent job?

•  I’ll discuss recent formal insights into •  combinatorial search spaces and their •  prac@cal implica@ons that makes searching •  such ultra-‐large spaces possible.

•  Brings together ideas from physics of disordered systems •  (spin glasses), combinatorics of random structures, and •  algorithms.

•  But first, what is BIG?

I.e., ((not x_1) or x_7) ((not x_1) or x_6) etc.

What is BIG?

x_1, x_2, x_3, etc. our Boolean variables (set to True or False)

Set x_1 to False ??

Consider a real-world Boolean Satisfiability (SAT) problem

I.e., (x_177 or x_169 or x_161 or x_153 … x_33 or x_25 or x_17 or x_9 or x_1 or (not x_185)) clauses / constraints are getting more interesting…

10 pages later:

…

Note x_1 …

4000 pages later:

…

Finally, 15,000 pages later:

Current SAT solvers solve this instance in approx. 1 minute!

Combinatorial search space of truth assignments: HOW?

102

Progress SAT Solvers

Instance Posit' 94

ssa2670-136 40,66s

bf1355-638 1805,21s

pret150_25 >3000s

dubois100 >3000s

aim200-2_0-no-1 >3000s

2dlx_..._bug005 >3000s

c6288 >3000s

Grasp' 96

1,2s

0,11s

0,21s

11,85s

0,01s

>3000s

>3000s

Sato' 98

0,95s

0,04s

0,09s

0,08s

0s

>3000s

>3000s

Chaff' 01

0,02s

0,01s

0,01s

0,01s

0s

2,9s

>3000s

Source: Marques Silva 2002

19.11.13

18

•  From academically interes@ng to prac@cally relevant.

•  We now have regular SAT solver compe@@ons.

•  Germany ’89, Dimacs ’93, China ’96, SAT-‐02, SAT-‐03, SAT-‐04, SAT05.

•  E.g. at SAT-‐2004 (Vancouver, May 04): •  -‐-‐-‐ 35+ solvers submiNed •  -‐-‐-‐ 500+ industrial benchmarks •  -‐-‐-‐ 50,000+ instances available on the WWW.

•  104

Real-‐World Reasoning Tackling inherent computa@onal complexity

100 200

10K 50K

50K 200K

0.5M 1M

1M 5M

Variables

1030

10301,020

10150,500

1015,050

103010

Wor

st C

ase

com

plex

ity

Car repair diagnosis

Deep space mission control

Chess

Hardware/Software Verification

Multi-Agent Systems

200K 600K

Military Logistics

Seconds until heat death of sun

Protein folding calculation (petaflop-year)

No. of atoms on earth 1047

100 10K 20K 100K 1M

Rules (Constraints) Example domains cast in propositional reasoning system (variables, rules).

•  High-Performance Reasoning •  Temporal/ uncertainty reasoning •  Strategic reasoning/Multi-player

Technology Targets

DARPA Research Program


“Genetic Algorithms are good at taking large,

potentially huge search spaces and navigating

them, looking for optimal combinations of things, solutions you might not

otherwise find in a lifetime.”

- Salvatore Mangano

Computer Design, May 1995

Genetic Algorithms: A Tutorial


The Genetic Algorithm

  Directed search algorithms based on the mechanics of biological evolution

  Developed by John Holland, University of Michigan (1970’s) ♦ To understand the adaptive processes of

natural systems ♦ To design artificial systems software that

retains the robustness of natural systems


The Genetic Algorithm (cont.)

  Provide efficient, effective techniques for optimization and machine learning applications

  Widely-used today in business, scientific and engineering circles


Components of a GA

A problem to solve, and ...   Encoding technique (gene, chromosome)   Initialization procedure (creation)   Evaluation function (environment)

  Selection of parents (reproduction)   Genetic operators (mutation, recombination)

  Parameter settings (practice and art)

19.11.13

19


Simple Genetic Algorithm {

initialize population; evaluate population; while TerminationCriteriaNotSatisfied {

select parents for reproduction; perform recombination and mutation; evaluate population;

} }


The GA Cycle of Reproduction

reproduction

population evaluation

modification

discard

deleted members

parents

children

modified children

evaluated children

Gene0c algorithms

•  How to generate the next genera@on. •  1) Selec<on: we select a number of states from the current genera@on. (we can use the fitness func@on in any reasonable way)

•  2) crossover : select 2 states and reproduce a child.

•  3) muta<on: change some of the genues.

Example

8-‐queen example Summary: Gene0c Algorithms

Gene0c Algorithms •  use popula@ons, which leads to increased search space explora@on

•  allow for a large number of different implementa@on choices

•  typically reach best performance when using operators that are based on problem characteris@cs

•  achieve good performance on a wide range of problems

19.11.13

20


Classes of Search Techniques

Finonacci Newton

Direct methods Indirect methods

Calculus-based techniques

Evolutionary strategies

Centralized Distributed

Parallel

Steady-state Generational

Sequential

Genetic algorithms

Evolutionary algorithms Simulated annealing

Guided random search techniques

Dynamic programming

Enumerative techniques

Search techniques

A.E. Eiben and J.E. Smith, Introduction to Evolutionary Computing Evolutionary Programming

Example application: evolving checkers players (Fogel’02)

l  Neural nets for evaluating future values of moves are evolved

l  NNs have fixed structure with 5046 weights, these are evolved + one weight for “kings”

l  Representation: –  vector of 5046 real numbers for object variables (weights) –  vector of 5046 real numbers for σ‘s

l  Mutation: –  Gaussian, lognormal scheme with σ-first –  Plus special mechanism for the kings’ weight

l  Population size 15

A.E. Eiben and J.E. Smith, Introduction to Evolutionary Computing Evolutionary Programming

Example application: evolving checkers players (Fogel’02)

l  Tournament size q = 5 l  Programs (with NN inside) play against other

programs, no human trainer or hard-wired intelligence

l  After 840 generation (6 months!) best strategy was tested against humans via Internet

l  Program earned “expert class” ranking outperforming 99.61% of all rated players


The GA Cycle of Reproduction

reproduction

population evaluation

modification

discard

deleted members

parents

children

modified children

evaluated children


Population

Chromosomes could be:

♦  Bit strings (0101 ... 1100) ♦  Real numbers (43.2 -33.1 ... 0.0 89.2) ♦  Permutations of element (E11 E3 E7 ... E1 E15) ♦  Lists of rules (R1 R2 R3 ... R22 R23) ♦  Program elements (genetic programming) ♦  ... any data structure ...

population


Reproduction

reproduction

population

parents

children

Parents are selected at random with selection chances biased in relation to chromosome evaluations.

19.11.13

21


Chromosome Modification

modification children

  Modifications are stochastically triggered   Operator types are:

♦ Mutation ♦ Crossover (recombination)

modified children


Mutation: Local Modification Before: (1 0 1 1 0 1 1 0)

After: (0 1 1 0 0 1 1 0)

Before: (1.38 -69.4 326.44 0.1)

After: (1.38 -67.5 326.44 0.1)

  Causes movement in the search space (local or global)

  Restores lost information to the population


Crossover: Recombination

P1 (0 1 1 0 1 0 0 0) (0 1 0 0 1 0 0 0) C1 P2 (1 1 0 1 1 0 1 0) (1 1 1 1 1 0 1 0) C2

Crossover is a critical feature of genetic algorithms:

♦  It greatly accelerates search early in evolution of a population

♦  It leads to effective combination of schemata (subsolutions on different chromosomes)

*


Evaluation

  The evaluator decodes a chromosome and

assigns it a fitness measure   The evaluator is the only link between a

classical GA and the problem it is solving

evaluation

evaluated children

modified children


Deletion

  Generational GA:

entire populations replaced with each iteration   Steady-state GA:

a few members replaced each generation

population

discard

discarded members


An Abstract Example

Distribution of Individuals in Generation 0

Distribution of Individuals in Generation N

19.11.13

22


A Simple Example

“The Gene is by far the most sophisticated program around.”

- Bill Gates, Business Week, June 27, 1994


A Simple Example

The Traveling Salesman Problem:

Find a tour of a given set of cities so that ♦ each city is visited only once ♦  the total distance traveled is minimized


Representation

Representation is an ordered list of city numbers known as an order-based GA.

1) London 3) Dunedin 5) Beijing 7) Tokyo 2) Venice 4) Singapore 6) Phoenix 8) Victoria CityList1 (3 5 7 2 1 6 4 8) CityList2 (2 5 7 6 8 1 3 4)


Crossover

Crossover combines inversion and recombination: * * Parent1 (3 5 7 2 1 6 4 8) Parent2 (2 5 7 6 8 1 3 4)

Child (5 8 7 2 1 6 3 4) This operator is called the Order1 crossover.


Mutation involves reordering of the list: * * Before: (5 8 7 2 1 6 3 4) After: (5 8 6 2 1 7 3 4)

Mutation


TSP Example: 30 Cities

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

y

x

19.11.13

23


Solution i (Distance = 941)

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

y

x

TSP30 (Performance = 941)


Solution j(Distance = 800) 44626967786462544250404038213567606040425099

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

y

x



Solution k(Distance = 652)

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

y

x



Best Solution (Distance = 420) 4238352621353273846445860697678716967628494

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

y

x

TSP30 Solution (Performance = 420)


Overview of Performance

0

200

400

600

800

1000

1200

1400

1600

1800

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

Distance

Generations (1000)

TSP30 - Overview of Performance

Best

Worst

Average


Considering the GA Technology “Almost eight years ago ... people at Microsoft wrote

a program [that] uses some genetic things for

finding short code sequences. Windows 2.0

and 3.2, NT, and almost all Microsoft applications products have shipped

with pieces of code created by that system.”

- Nathan Myhrvold, Microsoft Advanced Technology Group, Wired, September 1995

19.11.13

24


Issues for GA Practitioners

  Choosing basic implementation issues: ♦  representation ♦  population size, mutation rate, ... ♦  selection, deletion policies ♦  crossover, mutation operators

  Termination Criteria   Performance, scalability   Solution is only as good as the evaluation

function (often hardest part)


Benefits of Genetic Algorithms

  Concept is easy to understand   Modular, separate from application   Supports multi-objective optimization   Good for “noisy” environments   Always an answer; answer gets better

with time   Inherently parallel; easily distributed


Benefits of Genetic Algorithms (cont.)

  Many ways to speed up and improve a GA-based application as knowledge about problem domain is gained

  Easy to exploit previous or alternate solutions

  Flexible building blocks for hybrid applications

  Substantial history and range of use


When to Use a GA

  Alternate solutions are too slow or overly complicated

  Need an exploratory tool to examine new approaches

  Problem is similar to one that has already been successfully solved by using a GA

  Want to hybridize with an existing solution   Benefits of the GA technology meet key problem

requirements


Some GA Application Types Domain Application TypesControl gas pipeline, pole balancing, missile evasion, pursuit

Design semiconductor layout, aircraft design, keyboardconfiguration, communication networks

Scheduling manufacturing, facility scheduling, resource allocation

Robotics trajectory planning

Machine Learning designing neural networks, improving classificationalgorithms, classifier systems

Signal Processing filter design

Game Playing poker, checkers, prisoner’s dilemma

CombinatorialOptimization

set covering, travelling salesman, routing, bin packing,graph colouring and partitioning

Review 4 main types of Evolu@onary Algorithms

•  Gene@c Algorithm -‐ John Holland •  Gene@c Programming -‐ John Koza •  Evolu@onary Programming -‐ Lawerence Fogel

•  Evolu@onary Strategies -‐ Ingo Rechenberg

19.11.13

25

Gene@c Algorithms

•  Most widely used •  Robust •  uses 2 separate spaces

–  search space -‐ coded solu@on (genotype) –  solu@on space -‐ actual solu@ons (phenotypes)

•  Genotypes must be mapped to phenotypes before the quality or fitness of each solu@on can be evaluated

Gene@c Programming •  Specialized form of GA •  Manipulates a very specific type of solu@on using modified gene@c operators

•  Original applica@on was to design computer program

•  Now applied in alterna@ve areas eg. Analog Circuits

•  Does not make dis@nc@on between search and solu@on space.

•  Solu@on represented in very specific hierarchical manner.

Evolu@onary Strategies

•  Like GP no dis@nc@on between search and solu@on space

•  Individuals are represented as real-‐valued vectors.

•  Simple ES –  one parent and one child –  Child solu@on generated by randomly muta@ng the problem parameters of the parent.

•  Suscep@ble to stagna@on at local op@ma

Evolu@onary Strategies (cont’d)

•  Slow to converge to op@mal solu@on •  More advanced ES

– have pools of parents and children •  Unlike GA and GP, ES

–  Separates parent individuals from child individuals

– Selects its parent solu@ons determinis@cally

Evolu@onary Programming •  Resembles ES, developed independently •  Early versions of EP applied to the evolu@on of transi@on

table of finite state machines •  One popula@on of solu@ons, reproduc@on is by muta@on only •  Like ES operates on the decision variable of the problem

directly (ie Genotype = Phenotype) •  Tournament selec@on of parents

–  beNer fitness more likely a parent –  children generated un@l popula@on doubled in size –  everyone evaluated and the half of popula@on with lowest fitness

deleted.

General Idea of Evolu@onary Algorithms

19.11.13

26

CS 478 - Evolutionary Algorithms 151

Genetic Programming

l  Evolves more complex structures - programs, Lisp code, neural networks

l  Start with random programs of functions and terminals (data structures)

l  Execute programs and give each a fitness measure l  Use crossover to create new programs, no mutation l  Keep best programs l  For example, place lisp code in a tree structure,

functions at internal nodes, terminals at leaves, and do crossover at sub-trees - always legal in Lisp

CS 478 - Evolutionary Algorithms 152

153

Evolu@onary design

•  Karl Sims Evolved Virtual Creatures (1994) –  hNp://www.youtube.com/watch?v=F0OHycypSG8 –  hNp://www.karlsims.com/evolved-‐virtual-‐creatures.html –  hNp://www.archive.org/details/sims_evolved_virtual_creatures_1994

–  hNp://video.google.com/videoplay?docid=7219479512410540649# –  course work -‐ 2005

•  hbp://rogeralsing.com/2008/12/07/gene0c-‐programming-‐evolu0on-‐of-‐mona-‐lisa/

•  hbp://vimeo.com/7074089 155

19.11.13

27

19.11.13

28

19.11.13

29


Conclusions

Question: ‘If GAs are so smart, why ain’t they rich?’

Answer: ‘Genetic algorithms are rich - rich in application across a large and growing number of disciplines.’

- David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning

•  Games: Spore (2007) hNp://www.ted.com/talks/will_wright_makes_toys_that_make_worlds.html

•  hNp://www.gametrailers.com/user-‐movie/spore-‐14min-‐2007-‐demonstra@on/86368

•  hNp://eu.spore.com/home.cfm?lang=en

Ant Colony Op@miza@on (ACO)

1991, M. Dorigo proposed the Ant System in his doctoral thesis

Strong or weak pheromone

19.11.13

30

TSP More on ACO

•  It can work on dynamic systems, adop@ng to changes in the environment

Differen@al Evolu@on

•  Real-‐valued chromosomes •  X = ( x1 , x2, x3, … , xn )

19.11.13

31

Robust Regression (Differen@al Evolu@on)

Fi_ng a regression line using minimum median error as a measure.

aX + bY + c = 0 Y = aX + c Find a and c

Differen@al Evolu@on

Fit any polynomial, use mean or median, add MDL based iden@fica@on of the degree of polynomial

An Xn + An-‐1 Xn-‐1 + … + A1 X + A0

19.11.13

32

Lecture 12 Search heuristics - ut · 19.11.13 5 Greedy(SetCovering(Algorithm(source: 91.503...

Documents

Transcript of Lecture 12 Search heuristics - ut · 19.11.13 5 Greedy(SetCovering(Algorithm(source: 91.503...