Post on 15-Jan-2016
Genetic Algorithms
Chapter 9, Complexity: A Guided Tour
Evolution by Natural Selection
Charles Darwin
Evolution by Natural Selection
Charles Darwin
• Organisms inherit traits from parents
Evolution by Natural Selection
Charles Darwin
• Organisms inherit traits from parents
• Traits are inherited with some variation, via mutation and sexual recombination
Evolution by Natural Selection
Charles Darwin
• Organisms inherit traits from parents
• Traits are inherited with some variation, via mutation and sexual recombination
• Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring.
Evolution by Natural Selection
Charles Darwin
• Organisms inherit traits from parents
• Traits are inherited with some variation, via mutation and sexual recombination
• Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring.
• This way traits producing adapted individuals spread in the population
Evolution by Natural Selection
• Organisms inherit traits from parents
• Traits are inherited with some variation, via mutation and sexual recombination
• Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring.
• This way traits producing adapted individuals spread in the population
in computers
Computer (e.g., programs)
Charles Darwin
Evolution by Natural Selection
• Organisms inherit traits from parents
• Traits are inherited with some variation, via mutation and sexual recombination
• Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring.
• This way traits producing adapted individuals spread in the population
in computers
Computer (e.g., programs)
Charles Darwin
John Holland
Evolution by Natural Selection
• Organisms inherit traits from parents
• Traits are inherited with some variation, via mutation and sexual recombination
• Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring.
• This way traits producing adapted individuals spread in the population
in computers
Computer (e.g., programs)
Charles Darwin
John Holland
Genetic Algorithms (GAs)
Some real-world uses of genetic algorithms
Some real-world uses of genetic algorithms
• Used by GE to automate parts of aircraft design
Some real-world uses of genetic algorithms
• Used by GE to automate parts of aircraft design
• Used by pharmaceutical companies to discover new drugs
Some real-world uses of genetic algorithms
• Used by GE to automate parts of aircraft design
• Used by pharmaceutical companies to discover new drugs
• Used by the London Stock Exchange to automatically detect fraudulent trades
Some real-world uses of genetic algorithms
• Used by GE to automate parts of aircraft design
• Used by pharmaceutical companies to discover new drugs
• Used by the London Stock Exchange to automatically detect fraudulent trades
• Used to generate realistic computer animation in the movies Lord of the Rings: The Return of the King and Troy
Some real-world uses of genetic algorithms
• Used by GE to automate parts of aircraft design
• Used by pharmaceutical companies to discover new drugs
• Used by the London Stock Exchange to automatically detect fraudulent trades
• Used to generate realistic computer animation in the movies Lord of the Rings: The Return of the King and Troy
• Used to model and understand evolution in nature!
Example: Evolving Strategies for Robby the
Robot
Example: Evolving Strategies for Robby the
RobotInput: Contents of N, S, E, W,
C(Current)
Example: Evolving Strategies for Robby the
RobotInput: Contents of N, S, E, W,
C(Current)
Possible actions:
Move N
Move S
Move E
Move W
Move random
Stay put
Try to pick up can
Example: Evolving Strategies for Robby the
RobotInput: Contents of N, S, E, W,
C(Current)
Possible actions:
Move N
Move S
Move E
Move W
Move random
Stay put
Try to pick up can
Rewards/Penalties (points):
Picks up can: 10
Tries to pick up can on empty site: -1
Crashes into wall: -5
Example Strategy
Encoding a Strategy
Code: MoveNorth = 0MoveSouth = 1MoveEast = 2MoveWest = 3StayPut = 4PickUpCan = 5MoveRandom = 6
Encoding a Strategy
Code: MoveNorth = 0MoveSouth = 1MoveEast = 2MoveWest = 3StayPut = 4PickUpCan = 5MoveRandom = 6
Encoding a Strategy
0
Code: MoveNorth = 0MoveSouth = 1MoveEast = 2MoveWest = 3StayPut = 4PickUpCan = 5MoveRandom = 6
Encoding a Strategy
0 2
Code: MoveNorth = 0MoveSouth = 1MoveEast = 2MoveWest = 3StayPut = 4PickUpCan = 5MoveRandom = 6
Encoding a Strategy
0 2 6
Code: MoveNorth = 0MoveSouth = 1MoveEast = 2MoveWest = 3StayPut = 4PickUpCan = 5MoveRandom = 6
Encoding a Strategy
0 2 6 5 . . . 3 . . . 4
Code: MoveNorth = 0MoveSouth = 1MoveEast = 2MoveWest = 3StayPut = 4PickUpCan = 5MoveRandom = 6
Encoding a Strategy
0 2 6 5 . . . 3 . . . 4
243 values
Code: MoveNorth = 0MoveSouth = 1MoveEast = 2MoveWest = 3StayPut = 4PickUpCan = 5MoveRandom = 6
Genetic algorithm for evolving strategies for Robby
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
Random Initial Population
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy, calculate fitness (average reward minus penalties earned on random environments)
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy, calculate fitness (average reward minus penalties earned on random environments)
3. The strategies pair up and create offspring via “sexual recombination” with random mutations ― the fitter the parents, the more offspring they create.
Parent 1:
Parent 2:
Parent 1:
Parent 2:
Parent 1:
Parent 2:
Child:
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy, calculate fitness (average reward minus penalties earned on random environments)
3. The strategies pair up and create offspring via “sexual recombination” with random mutations ― the fitter the parents, the more offspring they create.
4. Keep going back to step 2 until a good-enough strategy is found!
My hand-designed strategy:
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
“Otherwise, if there is a can in one of the
adjacent sites, move to that site.”
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
“Otherwise, if there is a can in one of the
adjacent sites, move to that site.”
“Otherwise, choose a random direction to
move in.”
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
“Otherwise, if there is a can in one of the
adjacent sites, move to that site.”
“Otherwise, choose a random direction to
move in.”Average fitness of this strategy: 346
(out of max possible 500)
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
“Otherwise, if there is a can in one of the
adjacent sites, move to that site.”
“Otherwise, choose a random direction to
move in.”Average fitness of this strategy: 346
(out of max possible 500)
Average fitness of GA evolved strategy: 486
(out of max possible 500)
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
“Otherwise, if there is a can in one of the
adjacent sites, move to that site.”
“Otherwise, choose a random direction to
move in.”Average fitness of this strategy: 346
(out of max possible 500)
Average fitness of GA evolved strategy: 486
(out of max possible 500)???
One Run of the Genetic Algorithm
Bes
t fit
ness
in p
opul
atio
n
Generation number
Generation 1
Best average score = 81
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 0 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 1 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 2 Score: 5
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 2 Score: 5
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 3 Score: 10
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 3 Score: 10
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 4 Score: 15
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 4 Score: 15
Generation 10
Best average score = 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 0 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 1 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 2 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 3 Score: 0
Generation 200
Fitness = 240
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 0 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 1 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 2 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 3 Score: 10
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 4 Score: 10
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 5 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 6 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 7 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 8 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 9 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 10 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 11 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 12 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 13 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 14 Score: 30
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 15 Score: 30
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 16 Score: 40
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 17 Score: 40
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 18 Score: 50
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 19 Score: 50
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 20 Score: 60
Generation 1000
Fitness = 492
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 0 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 1 Score: 0
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 2 Score: 10
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 3 Score: 10
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 4 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 5 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 6 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 7 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 8 Score: 20
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 9 Score: 30
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 10 Score: 30
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 11 Score: 40
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 12 Score: 40
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 13 Score: 50
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 14 Score: 50
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 15 Score: 60
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 16 Score: 60
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 17 Score: 70
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Time: 18 Score: 70
Why Did The GA’s Strategy Outperform Mine?
My Strategy
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
The GA’s Evolved Strategy
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7 8 9
Principles of Evolution Seen in Genetic Algorithms
Principles of Evolution Seen in Genetic Algorithms
• Natural selection works!
Principles of Evolution Seen in Genetic Algorithms
• Natural selection works!
• Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation
Principles of Evolution Seen in Genetic Algorithms
• Natural selection works!
• Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation
Bes
t fit
ness
in p
opul
atio
n
Generation number
Principles of Evolution Seen in Genetic Algorithms
• Natural selection works!
• Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation
• Exaptation is common
Principles of Evolution Seen in Genetic Algorithms
• Natural selection works!
• Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation
• Exaptation is common
• Co-evolution speeds up innovation
Principles of Evolution Seen in Genetic Algorithms
• Natural selection works!
• Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation
• Exaptation is common
• Co-evolution speeds up innovation
• Dynamics and results of evolution are unpredictable and hard to analyze