1
CS540 February 17, 2015
n Assignment 2: late period until thurs @ noon n Copying code or prose
n If you use publically available code, add a comment to attribute the source
n Do not look at other student’s assignments n Do not take text without adding quotes and do
not use too much quoted material
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Population Based Search
n Ant Colony Optimization n Evolutionary Algorithms
n Evolutionary Strategies n Genetic Algorithms n Genetic Programming
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Ant Colony Optimization (ACO) [Dorigo 1992]
n constructive meta-heuristic that shares limited information between multiple search paths
n Characteristics: n multi-agent/solution exploration (population based), n stochastic, n memory (individual and colony), n adaptive to problem, n implicit solution evaluation via “who gets there first”
n Inspiration: ants follow pheromone paths of other ants to find food. Over time, the best paths have strongest smell, luring more ants.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Inspiration: Ant Behavior
From http://www.nytimes.com/imagepages/2007/11/12/science/2007113_TRAFFIC_GRAPHIC.html CS540, Artificial Intelligence @ Adele Howe, Spring 2015
2
Ant Colony Example 1. An ant (called "blitz") wanders. 2. If it discovers a food source, it returns
~directly to the nest, leaving a pheromone trail;
3. Ants will be inclined to follow, more or less directly, the pheromone trail;
4. Returning to the colony, these ants will strengthen the route;
5. If there are two routes to reach the same food source then, in a given amount of time, the shorter one will be traveled by more ants than the long route;
6. The long route will eventually disappear because pheromones are volatile;
7. Eventually, all the ants have "chosen" the shortest route.
From http://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Ant Colony Simulation
n http://code.kevinworkman.com/Ants/AntsApplet.html
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Core Pseudo-Code for Each Ant 1. initialize 2. until reach solution
1. read local routing table 2. compute P(neighbors | routing-table, memory, constraints,
heuristics) 3. select move based on Ps and constraints 4. update memory for move (pheromone globally and visited arc
locally) 3. die
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
ACO Parameters n when to terminate n how many ants n how to schedule ant activities (ant generation and optionals) n what to keep in memory locally n how to select a next state n how to update memory (e.g., amount as function of solution
quality) n how to “evaporate” pheromone n optionals:
n delayed updating of pheromones on trail? n allow pheromone trail evaporation? n permit off-line pheromone updates (daemon actions)?
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
3
ACO Applications n Scheduling (JSP, FSP…) n Vehicle routing n Quadratic assignment n Image processing… n TSP
http://en.wikipedia.org/wiki/Travelling_salesman_problem
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
CS540 February 19
n A2 will be graded in next week n Work on project!
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Evolutionary Algorithms Overview n Evolutionary algorithms search a population
representing different sample points in the search space.
n Each sample point is represented as a string which can be recombined with other strings to generate new sample points in the space.
n Algorithms are based on biological analogies with “population genetics" and “simulated evolution".
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Why Evolutionary Algorithms? n No Gradient Information Is Needed. These
algorithms do not search along the contours of the function, but rather by hyperplane sampling in Hamming space.
n The Resulting Search is Global. Since they do not hill-climb, they avoid local optima and so can be applied to multimodal functions.
n Potential for Massive Parallelism. Can effectively exploit thousands of processors in parallel.
n They Can Be Hybridized with conventional optimization methods.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
4
How to envision search space
n Local Search? n Evolutionary
Computation?
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Search Spaces
SLS
Current Solution
Neighbor1
Neighbor2
Neighbor3
Neighbor4
Evolutionary Computation 0 0 1 0 0 1 1 0 0 1
1 0 1 0 0 1 1 1 0 0
1 1 1 1 1 0 0 0 0 0
0 1 1 0 1 0 1 1 0 1
0 1 1 0 1 0 0 0 0 0
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Issues: Representation
n Evolutionary model n Genotype: represents information stored in
chromosomes n Phenotype: describes how the individual appears
n Approaches n Indirect: standard data structure, genotype is
mapped to phenotype n Direct or natural: specialized representation using
domain specific search operators
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Issues: Representation
Binary: genotype is encoded as bit strings of length l n Grey codes: similar values have similar
representations n Binary numbers: may be some issue of
precision in encoded values
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
5
Issues: Representation
Nonbinary (integer, continuous, permutation): larger alphabets, real-valued encodings, more natural Arguments against: n tends to have larger search space n there will be fewer explicit hyperplane partitions n the alphabetic characters will not be as well represented in a finite population.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Representation example
n TSP n Genotype? n Phenotype?
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Representation: FSP & JSP n “use an intermediary, encoded representation of schedules that is
amenable to crossover operations, while employing a decoder that always yields legal solutions to the problem.”
n “A complete schedule for the job shop was derived from a list of preferences for each work station, linked to times. A preference list had an initial member- a time at which the list went into effect. The rest of the list was made up of some permutation of the contracts available, plus the elements ‘wait’ and ‘idle’.”
Davis, Lawrence. "Job shop scheduling with genetic algorithms." Proceedings of an International Conference on Genetic Algorithms and Their Applications. Vol. 140. 1985.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Issues: Fitness Function
n Based on the objective function n Allows comparison of different solutions n Domain specific to goals of problem n Single value output: multi-objective must be
combined into single function
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
6
Issues: Fitness Function
n Modifications n Must be fast! May need to be executed
hundreds of thousands of times n Sometimes approximate to achieve speed n Smoothing: replacing fitness with average of
neighbors (useful for plateaus) n Penalty functions: to relax feasibility
requirements when infeasible solutions cannot be removed
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Fitness Function example
n TSP n Objective function? n Fitness function?
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Issues: Initializing Population
Tension between good solutions and diversity (low diversity can lead to quick stagnation or large distance from optimum)
Random: generate n strings uniform randomly, within encoding requirements.
Domain specific: use heuristic method to generate “ok”(greedy) solutions that can be refined.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Initialization example
n TSP n Greedy initialization?
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
7
CS540 February 24
n A2 is partly graded n Take time on the essays… (each essay set
comprises .34*15%=5% of your final grade!) n We’re looking for insight and thought. n Write it as an essay, not direct answers to the
questions. Organize your points. n Do proofread!
n A3: Timetabling using evolutionary computation!
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
PA2 Program
n Significant issues n 28 submitted n 26 passed checkin n 24 produced some output n 16 ran correctly on the trivial instance n 7 programs ran correctly on an instance other
than the trivial one
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Evolution Strategies
n Search method like local search but can be population based
n Vector of continuous variables n Mutates each variable by incrementing value using
randomly generated change with zero mean and set standard distribution… Mutation only to create new solutions
n Survival of fittest between new and previous vectors
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Simplest Evolutionary Strategy (ES)
(1 + 1)-ES n 1 “parent” solution plus n 1 “offspring” solution n Keep the best
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
8
(1+1)-ES Algorithm
1. Create initial solution x 2. while termination criterion is not met do
1. for i = 1 to n do
2. If then
3. Update σ
!xi = xi +σNi (0,1)f !x( ) ≥ f x( )x := !x
N(0,1) indicates a normal distribution with 0 mean and 1 standard deviation.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
(1+1)-ES Update
How to update σ? n Convergence proofs on two simple problems deAine a 1/5 rule
n If ratio is > 1/5, increase σ, else decrease n Keep it Aixed, which is better for escaping local optima
# search steps that find a better solutionall or last t steps
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
(µ+λ)-ES
n µ solutions in population n Generate λ new solutions n Choose the best µ from superset at
each iteration n Can add recombination before mutation n Can have different σ per variable
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Evolution Strategies (cont.)
Self adaptive mutation and rotation n Log-normal distribution for mutation n Adaptive through strategy (σ) and rotation angle (α)
parameters added to chromosome
€
x1,x2,σ1,σ 2,α1,2
Simple Mutations Correlated Mutation via Rotation Figures courtesy of D. Whitley
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
9
Simple Genetic Algorithm (Alg 7 in Rothlauf)
1. Create initial population P with N solutions 2. for i = 1 to N do
1. Calculate 3. while (termination criterion is not met) and (population has not yet converged) do
1. Create M with N individuals selected from P 2. ind = 1 3. repeat
1. if random(0,1) ≤ pc then recombine and and place the offspring in P’ 2. Else copy and to P’ 3. ind = ind + 2
4. until ind > N 5. for i = 1 to N do
1. for j = 1 to l do 1. if random(0,1) ≤ pm then mutate( ) where
6. Calculate where 7. P = P’
xi i ∈ 1,...,N{ }( )
f xi( )
xind ∈M xind+1 ∈Mxind ∈M xind+1 ∈M
x ji xi ∈ "P
f xi( ) xi ∈ "P
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Genetic Algorithm Process
String1 String2 String3 String4 ….. …..
String1 String2 String3 String4 ….. …..
OffspringA OffspringB OffspringC OffspringD
….. …..
Selection (Duplication)
Recombination (Crossover)
Current Generation t
Intermediate Generation t
Next Generation t+1
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
CS540 February 26
n A2 results: look at comments in Checkin and Canvas n A3:
n Check back later today for submitting Part A n Essays: Q2: Start with the pseudocode for the core
algorithm and explain how you set each parameter/design decision in it.
n We will be using Checkin n Strategy for success: read instructions from A2 carefully. After you
tar your code, move it to a fresh directory, un tar it and try running the script on a set of files with different names than before.
n Run on many test cases!
CS540, Artificial Intelligence @ Adele Howe, Spring 2015 CS540, Artificial Intelligence @ Adele Howe, Spring 2015
A2 Results
P1:Slots Program P1:SC Program P2:Slots Program P2:SC Program 4 A 34.5 A 14 A 54.34 J
4 B 34.5 B 14 C 63.4 E
4 C 34.5 F 14 D 72.45 B 4 D 34.5 H 14 H 91.17 I
4 E 34.5 K 14 M 93.46 C 4 F 34.5 L 15 B 115.4 A 4 G 34.5 N 15 E 126.68 D 4 H 34.5 Q 15 F 130.95 F
4 I 35.5 C 15 I 186.54 K
4 J 35.5 D 15 J 273.61 Q
4 K 35.5 R 16 K 308.53 S
4 L 35.75 E 23 S 321.81 L
4 M 35.75 I 98 Q
4 N 37.25 V
4 O 46.25 W
4 P 50.75 U
5 Q 56.75 X
5 R 59.75 O
5 S 60 J
5 T 129 S
10
Simple Genetic Algorithm Selection
1. Create initial population P with N solutions 2. for i = 1 to N do
1. Calculate 3. while (termination criterion is not met) and (population has not yet converged) do
1. Create M with N individuals selected from P 2. ind = 1 3. repeat
1. if random(0,1) ≤ pc then recombine and and place the offspring in P’ 2. Else copy and to P’ 3. ind = ind + 2
4. until ind > N 5. for i = 1 to N do
1. for j = 1 to l do 1. if random(0,1) ≤ pm then mutate( ) where
6. Calculate where 7. P = P’
xi i ∈ 1,...,N{ }( )
f xi( )
xind ∈M xind+1 ∈Mxind ∈M xind+1 ∈M
x ji xi ∈ "P
f xi( ) xi ∈ "P
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Proportionate Selection
n Population is evaluated according to a fitness function.
n Parents are selected for reproduction by ranking according to their relative fitness
ff
i
fi
f jj=1
N
∑
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Proportionate Selection Process Stochastic sampling with replacement Map individuals to space on a roulette wheel
CS540, Artificial Intelligence @ Adele Howe, Spring 2015 From clipartbest.com
More fit individuals are allocated proportionally more space. Spin wheel repeatedly until desired population size is achieved
Population Example, Stochastic Sampling w/Replacement String Fit Space copies
001000000 2.0 .095
101010101 1.9 .186
111110011 1.8 .271
010001100 1.7 .352
111100000 1.6 .429
101000110 1.5 .5
011001110 1.4 .567
001111000 1.3 .629
000110100 1.2 .686
100100011 1.1 .738
010000111 1.0 .786
String Fit Space copies
011001111 0.9 .829
000100110 0.8 .867
110001101 0.7 .9
110000111 0.6 .929
100100100 0.5 .952
011011011 0.4 .971
000111000 0.3 .986
001100100 0.2 .995
100111010 0.1 1.0
010010011 0.0 --
Random #s: .93, .65, .02, .51, .20, .93, .20, .37, .79, .28, .13, .70, .80, .51, .76, .45, .61, .07, .76, .86, .29 CS540, Artificial Intelligence @ Adele Howe, Spring 2015
11
Other Selection Methods
n Tournament Selection: randomly select two strings, place the best into the new population, repeat until intermediate population is full
n Ranking: order individuals by rank rather than fitness value
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Population Example, Tournament Selection String # copies
001000000 0
101010101 1
111110011 2
010001100 3
111100000 4
101000110 5
011001110 6
001111000 7
000110100 8
100100011 9
010000111 10
String Fit copies
011001111 11
000100110 12
110001101 13
110000111 14
100100100 15
011011011 16
000111000 17
001100100 18
100111010 19
010010011 20
Random pairs: (6 4) (16 7) (13 2) (0 17) (4 14) (19 18) (7 19) (2 2) (11 4) (7 2) (7 20) (18 19) (11 1) (20 0) (10 14) (12 15) (14 3) (20 15) (6 1) (12 13) CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Another Fitness Selection Method
n Remainder stochastic sampling n Again map to roulette wheel, but this time add
outer wheel with N evenly spaced pointers. n Spin once to determine all population members
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Population Example, Remainder Stochastic Sampling String Fitness Random copies
001000000 2.0 -- 2
101010101 1.9 0.93 2
111110011 1.8 0.65 2
010001100 1.7 0.02 1
111100000 1.6 0.51 2
101000110 1.5 0.20 1
011001110 1.4 0.93 2
001111000 1.3 0.20 1
000110100 1.2 0.37 1
100100011 1.1 0.79 1
010000111 1.0 -- 1
String Fitness Random copies
011001111 0.9 0.28 1
000100110 0.8 0.13 0
110001101 0.7 0.70 1
110000111 0.6 0.80 1
100100100 0.5 0.51 1
011011011 0.4 0.76 1
000111000 0.3 0.45 0
001100100 0.2 0.61 0
100111010 0.1 0.07 0
010010011 0.0 -- 0
Random #s: .93, .65, .02, .51, .20, .93, .20, .37, .79, .28, .13, .70, .80, .51, .76, .45, .61, .07, .76, .86, .29 CS540, Artificial Intelligence @ Adele Howe, Spring 2015
12
CS540 March 2 n Assignment 3
n Submitted instances that satisfied requirements are posted – lots of test cases! Some very challenging.
n Verifier has been made available to help with your debugging ~cs540/verifier/verifier
n Checkin will be set up tonight. Same instructions and checks as in assignment 2 except for naming as PA3.tar
n ASAP: make sure your solutions are verified and that checkin works for you
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Genetic Algorithm Process
String1 String2 String3 String4 ….. …..
String1 String2 String3 String4 ….. …..
OffspringA OffspringB OffspringC OffspringD
….. …..
Selection (Duplication)
Recombination (Crossover)
Current Generation t
Intermediate Generation t
Next Generation t+1
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Simple Genetic Algorithm Recombination
1. Create initial population P with N solutions 2. for i = 1 to N do
1. Calculate 3. while (termination criterion is not met) and (population has not yet converged) do
1. Create M with N individuals selected from P 2. ind = 1 3. repeat
1. if random(0,1) ≤ pc then recombine and and place the offspring in P’ 2. Else copy and to P’ 3. ind = ind + 2
4. until ind > N 5. for i = 1 to N do
1. for j = 1 to l do 1. if random(0,1) ≤ pm then mutate( ) where
6. Calculate where 7. P = P’
xi i ∈ 1,...,N{ }( )
f xi( )
xind ∈M xind+1 ∈Mxind ∈M xind+1 ∈M
x ji xi ∈ "P
f xi( ) xi ∈ "P
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Reproduction: Recombination/Crossover
Two parents: binary strings representing an encoding of 5 parameters that are used in some optimization problems. 10010101 11011001 01110100 10100101 10000101 xyxyyyxx xyyyyxyx xyyxyyxy yyxyxxxy yxyxyxyx
Recombination occurs as follows: 10010 \ / 10111011001011101 \ / 001010010110000101 xyxyy / \ yxxxyyyyxyxxyyxy / \ xyyyxyxxxyyxyxyxyx
Producing the following offspring: 10010yxxxyyyyxyxxyyxyy001010010110000101 xyxyy10111011001011101xyyyxyxxxyyxyxyxyx
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
13
Desirable Characteristics of Recombination Operators
n Respect: any commonalities of the parents are inherited by the offspring
n Transmission: all components of the offspring must have come from a parent
n Assortment: all components of the offspring must be compatible/feasible
n Ergodicity: can reach any combination from all possible starting points
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Issues: Combination Operators
1 point crossover: pick single crossover point, split strings at this point, recombine
2 point crossover: pick two crossover points, split strings at these points, recombine (think of ring for string)
Uniform crossover: randomly pick each element from one of the two parents
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Crossover Operators: 1 point
0 1
1
0 0
0
1
0 x
x
y
y x
x
y
y
One end is at beginning/end
of string
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Crossover Operators: 2 point
0 1
1
0 0
0
1
0 x
x
y
y x
x
y
y
Both ends can move
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
14
Crossover and Hypercube Paths
1111
0111 1011 1101 1110
0011 0101 0110 1001 1010 1100
0001 0010 0100 1000
0000
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Other Combination Operators
HUX: exactly half of the differing bits are swapped Given parents: 100110101111000010110 101100011100001100001 a new individual is: 100100111110000110010
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
More Combination Operators Reduced Surrogate: Crossover points chosen
within differing bits only 100110101111000010110 101100011100001100001 may become: 100110111100001100001
Domain specific: operations designed to match demands of the domain (e.g., reorder portions for scheduling application)
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Why Might Reduced Surrogate Be Important
n Closely related to HUX n Key idea: look only at portions of strings
that differ 00011111011010011 00010011010010010 How does probability of new string change with reduced surrogate versus 1-‐point crossover?
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
15
Simple Genetic Algorithm Mutation
1. Create initial population P with N solutions 2. for i = 1 to N do
1. Calculate 3. while (termination criterion is not met) and (population has not yet converged) do
1. Create M with N individuals selected from P 2. ind = 1 3. repeat
1. if random(0,1) ≤ pc then recombine and and place the offspring in P’ 2. Else copy and to P’ 3. ind = ind + 2
4. until ind > N 5. for i = 1 to N do
1. for j = 1 to l do 1. if random(0,1) ≤ pm then mutate( ) where
6. Calculate where 7. P = P’
xi i ∈ 1,...,N{ }( )
f xi( )
xind ∈M xind+1 ∈Mxind ∈M xind+1 ∈M
x ji xi ∈ "P
f xi( ) xi ∈ "P
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Mutation
n For each bit in population, mutate with probability
n should be low, typically < .01 n can mean randomly select a value for
the bit or flip the bit
mpmp
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Issues: Which Strings in New Generation
n Replace with offspring n Assumption of canonical GA
n Best of offspring and parents n Alternative view which guarantees always
keep best and puts intensive pressure on population for improvement
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
CS540 March 4 n Assignment 3
n Verifier has been made available to help with your debugging ~cs540/verifier/verifier
n Use Checkin for programs n Use Canvas for essays – need all 3 essays (a few
students sent Q1 with the problem instance)
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
16
Issues: Termination Criteria
n Quality of solution: best individual passes some pre-set threshold
n Time: certain number of generations have been created and tested
n Diminishing returns: improvement over each generation does not exceed some threshold
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Alternatives to the Simple GA Model
n Genitor n CHC
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Genitor (Whitley et al.)
Differences with canonical genetic algorithm: § Reproduction proceeds one individual at a time. § Worst individual in population is replaced by
new offspring. § Fitness is assigned by rank.
§ Steady State GA
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Genitor Algorithm (Whitley)
String1 String2 String3 String4 ….. …..
String n
Parent 1 Parent 2
Child 1 Child 2
String n-1
String1 String2 String3 Child 2 ….. …..
Time t Time t+1
Selection Crossover &
Recombination Insert In
population
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
17
CHC (Eshelman)
Crossover using generation elitist selection
Heterogeneous recombination by incest prevention n Origin of HUX terminology
Cataclysmic mutation, when population starts to converge
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
CHC Process Parent1 Parent2 Parent3 Parent4 …..
Parent n
Child1 Child2 Child3 Child4 …..
Child n
Parent b Child f Child h Parent j …..
Parent m
via Random Selection and HUX
Best n strings from parents and children
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Genetic Algorithms for Scheduling
n Represent solution as permutation of tasks n Requires a schedule builder to convert
solution into schedule and assess objective function. [Indirect search]
n Syswerda’s permutation crossover was used for scheduling.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
GA applied to TSP
n Applet showing simple GA for TSP http://www.obitko.com/tutorials/genetic-
algorithms/tsp-example.php
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
18
Genetic Programming
n Combine and/or parameterize code blocks (functions and terminals) to produce program to solve some problem.
n Follows GA functionality n Solution is often a parse tree of functions
and terminals. n Interior nodes are functions (e.g., “+”, “or”),
control structures (e.g., “if”, “while”) or functions with side effects (e.g., “read”, “print”).
n Terminals are variables and constants. CS540, Artificial Intelligence @ Adele Howe, Spring 2015
GP Steps
1. Initialize population of programs 2. Loop
1. Assess fitness 2. Construct new generation through selection
and n Reproduction, n Mutation or n Crossover
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
GP initialization
n Grow: start with empty tree and iteratively assign nodes to be function or terminal. All nodes at depth kmax are terminals.
n Full: start with empty tree, randomly add functions up to depth kmax-1, terminals at depth kmax.
n Ramped-half-and-half: divide population into (kmax-1) parts, half of each is created by grow and other by full, depth of nodes in ith part is varied from 2 to kmax
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
GP: Assessing Fitness
n Use program to solve problem n Quality of solution n Efficiency of solution
n May require simulation, e.g., UAV
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
19
GP: Representation
Domain Dependent! n E.g., Simple Lisp
functions (math, cons, list, append…)
+
^
*
-
a b
3 c a
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
GP: Mutation
n Substitute one function (terminal) for another
n Substitute one subtree for another
+
^
/
-
a b
3 c d
+
^
*
-
a b
3 a sqrt
c
*
d
Animation: http://www.genetic-programming.com/mutation.gif CS540, Artificial Intelligence @ Adele Howe, Spring 2015
GP: Crossover +
^ *
-
a b
3 a sqrt
c *
d
+ ^
* -
a b
3 a abs
b *
d
+ ^ -
3 a sqrt
c *
d
-
a abs
cb *
d Animation: http://www.genetic-programming.com/crossover.gif
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
GP for Software Repair (Forrest et al. GECCO 2009)
Given n C source code (represented as abstract syntax trees), n a negative test case that exercises the fault to be
repaired (leading to weighted paths through the tree), n positive test cases that encode the required behavior of
the program Use GenProg to create better/fixed code
n selects a subset (top ranked 50%) of programs based on fitness (sum of passed test cases), and
n modifies the programs with mutation and crossover.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
20
The Zune Bug
days is # of days since Jan 1, 1980. When days is the last day of a leap year, the program enters an infinite loop.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
void zunebug(int days) { int year = 1980; while (days > 365) { if (isLeapYear(year)){ if (days > 366) { days -= 366; year += 1; } else{ } } else { days -= 365; year += 1; } } printf("current year is %d\n", year); }
Repairing the Zune Bug
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
void zunebug(int days) { int year = 1980; while (days > 365) { if (isLeapYear(year)){ if (days > 366) { days -= 366; year += 1; } else{ } } else { days -= 365; year += 1; } } printf("current year is %d\n", year); }
void zunebug(int days) { int year = 1980; while (days > 365) { if (isLeapYear(year)){ if (days > 366) { //days -= 366; //GP deletes year += 1; } else{ } days -= 366; //GP inserts } else { days -= 365; year += 1; } }
GenProg Movie
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
http://dijkstra.cs.virginia.edu/genprog/#videos
Underlying Theory: Hyperplane Sampling
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
21
Another View of Hyperplane Sampling
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Population Example for Hyperplane Sampling String Fitness Random copies
001##...## 2.0 -- 2
101##...## 1.9 0.93 2
111##...## 1.8 0.65 2
010##...## 1.7 0.02 1
111##...## 1.6 0.51 2
101##...## 1.5 0.20 1
011##...## 1.4 0.93 2
001##…## 1.3 0.20 1
000##…## 1.2 0.37 1
100##…## 1.1 0.79 1
010##…## 1.0 -- 1
String Fitness Random copies
011##…## 0.9 0.28 1
000##…## 0.8 0.13 0
110##…## 0.7 0.70 1
110##…## 0.6 0.80 1
100##…## 0.5 0.51 1
011##…## 0.4 0.76 1
000##…## 0.3 0.45 0
001##…## 0.2 0.61 0
100##…## 0.1 0.07 0
010##…## 0.0 -- 0
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Some Schemata and Fitness Values Schema Mean Count Expect Observe 101**…* 1.70 2 3.4 3
111**…* 1.70 2 3.4 4
1*1**…* 1.70 4 6.8 7
**01*…* 1.38 5 6.9 6
***1*…* 1.30 10 13.0 14
**11*…* 1.22 5 6.1 8
11***…* 1.175 4 4.7 6
001**…* 1.166 3 3.5 3
1****…* 1.089 9 9.8 11
0*1**…* 1.033 6 6.2 7
10***…* 1.020 5 5.1 5
**1**…* 1.010 10 10.1 12
*****…* 1.000 21 21.0 21 CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Hyperplane Deception Since genetic algorithms are driven by hyperplane sampling a misleading problem can be constructed as follows. f(0**) > f(1**) f(*0*) > f(*1*) f(**0) > f(**1) f(00*) > f(01*), f(10*), f(11*) f(0*0) > f(0*1), f(1*0), f(1*1) f(*00) > f(*01), f(*10), f(*11) BUT f(111) > f(000) where f(x) gives the average fitness of all strings in the hyperplane slice represented by x.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
22
Fitness Landscape Analysis
n A fitness landscape (X,f,d) of a problem instance consists of a set of solutions X, an objective function f and a distance measure d.
n We can analyze problems based on characteristics derived from this definition and determine (roughly) problem difficulty.
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Locality
n How well distances correspond to differences in fitness between solutions
n High locality if nearby solutions have similar fitness
n Low locality problems are difficult for local search
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Fitness-Distance Correlation
n Metric of locality
n c is the covariance, <> are means. n Positive correlation (i.e., better is closer) is easy,
uncorrelated is difficult, negative correlation is misleading.
ρFDC =cfd
σ f( )σ dopt( )where
c fd =1m
fi − f( )i=1
m
∑ di,opt − dopt( )
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Big Valley Topology Original observation
from (Boese, Kahng & Muddu 1994)
Figure: benchmark FlowShop scheduling problem from Watson et al. 1999
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
23
Ruggedness n Statistical property on relation of objective values to intra-
solution distances n Random walk correlation between solutions separated by s
steps
xi is the solution at step i n Correlation length…high means smooth, easy for guided
search
r s( ) =f xi( ) f xi+s( ) − f 2
f 2 − f 2
lcorr = −1
ln r 1( )( )
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
No Free Lunch…
n Wolpert and Macready, 1997 “For any algorithm, any elevated
performance over one class of problems is exactly paid for in performance over another class.”
n Proven for finite optimization problems solved with deterministic “black-box” algorithms
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
NFL Theorem I For any pair of algorithms a1 and a2: f is an optimization problem, m is number of distinct
points sampled by the algorithm, y is the cost value, d is the best cost value over the sample.
Meaning… average performance for an algorithm over all problems is the same as for any other algorithm, assuming equal sampling size.
€
P(dmy | f ,m,a1) =
f∑ P(dm
y | f ,m,a2)f∑
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Implications of NFL
n Emphasizes need for different algorithms n Does not preclude superior performance on
some interesting class(es) of problem!
n However, if no problem knowledge is used, then P(f) is essentially uniform.
€
P(dmy |m,a) = P(
f∑ dm
y | f ,m,a)P( f )
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
24
Implications (cont.)
n Structure of problem must be understood and used to inform choice of search.
n Knowledge of cost function properties is folded into P(f) or
n “performance of an algorithm is determined by … how aligned is with the problems “
€
P(dmy |m,a) =
v d my ,m,a •
p
€
p
€
p
€
v d my ,m,a
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Comparing Performance
n Assume we can construct a histogram of cost values produced by a run of an algorithm , then use these to gauge performance: n Over all cost functions:
n Average of n for the random algorithm
n Given f and m, % of algorithms with
€
c
€
P min c ( ) > ε | f ,m,a( )
€
P min c ( ) > ε | f ,m,a( )
€
min c > ε( )
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Minimax & Fixed f
n NFL focuses on average performance. n One algorithm may be much better than another on a
subset of f (the other is a little better on remaining to even out average).[Minimax]
n May occur when samples intersect
n If f is fixed, observing performance until time t does not indicate what will happen later. n Intelligent choice requires knowledge of both f and a
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Phase Transitions
Cheeseman, Kanefsky & Taylor (1991) observed that:
n The hardest problems in an NP-complete class (e.g., TSP, SAT) were at the boundary between feasibility and infeasibility.
n Problems that are clearly feasible (or not) are easy to solve (or easy to recognize there’s no solution).
n “phase transition” depends on identifying a key scale up parameter (e.g., constraints per variable, standard deviation of costs)
0
20
40
60
80
100
120
140
0 5 10 15
Feasible Not
CS540, Artificial Intelligence @ Adele Howe, Spring 2015
Top Related