Evolutionary Learning
description
Transcript of Evolutionary Learning
![Page 1: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/1.jpg)
Evolutionary Learning
![Page 2: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/2.jpg)
Some Examples of Biologically Inspired AI
• Neural networks
• Evolutionary computation (e.g., genetic algorithms)
• Immune-system-inspired computer/network security
• Insect-colony optimization (ants, bees, etc.)
• Slime-mould path-finding
• Swarm intelligence (e.g., decentralized robots)
![Page 3: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/3.jpg)
Evolution by Natural Selection
• Organisms inherit traits from parents
• Traits are inherited with some variation, via mutation and sexual recombination
• Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring.
• This way traits producing adapted individuals spread in the population
in computers
Computer (e.g., programs)
John HollandGenetic Algorithms
Charles Darwin, 1809−1882
![Page 4: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/4.jpg)
![Page 5: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/5.jpg)
Some Real-World Examples of Genetic Algorithms
![Page 6: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/6.jpg)
http://resources.altair.com/altairadmin/images/resource_library/graphics-en-US/htc07_boeing.jpg
Designing parts of aircraft (GE and Boeing)
![Page 7: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/7.jpg)
http://idesign.ucsc.edu/papers/lohn_gptp05.pdf
Spacecraft antenna design (NASA)
![Page 8: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/8.jpg)
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=398998
Assembly line scheduling (John Deere Co.)
![Page 9: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/9.jpg)
http://www.nature.com/nrd/journal/v7/n8/fig_tab/nrd2615_F4.html
Automated drug discovery (several companies)
![Page 10: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/10.jpg)
Fraud detection (credit cards, financial trading)
![Page 11: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/11.jpg)
http://www.eurekalert.org/features/doe/images/danl-lag080702.1.jpg
Automated analysis of satellite images (Los Alamos National Lab)
![Page 12: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/12.jpg)
http://www.wired.com/wired/archive/12.01/stuntbots.html
Generation of realistic computer animation (Lord of the Rings: The Return of the King and Troy)
![Page 13: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/13.jpg)
Genetic Algorithm Example:
Evolving a Control Program for a Virtual “Robot”
![Page 14: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/14.jpg)
Herbert: The Soda Can Collecting Robot(Connell, Brooks, Ning, 1988)
http://cyberneticzoo.com/?p=5516
![Page 15: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/15.jpg)
Herbert: The Soda Can Collecting Robot(Connell, Brooks, Ning, 1988)
http://cyberneticzoo.com/?p=5516
Robby: The Virtual Soda Can Collecting Robot
(Mitchell, 2009)
![Page 16: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/16.jpg)
Robby’s World
Soda can
![Page 17: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/17.jpg)
What Robby Can See and DoInput: Contents of North, South, East, West,
Current
Possible actions:
Move N
Move S
Move E
Move W
Move random
Stay put
Try to pick up can
Rewards/Penalties (points):
Picks up can: 10
Tries to pick up can on empty site: -1
Crashes into wall: -5
Robby’s Score: Sum of rewards/penalties
![Page 18: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/18.jpg)
Goal: Use a genetic algorithm to evolve a control program (i.e., strategy) for Robby.
![Page 19: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/19.jpg)
One Example Strategy
1234...
243
![Page 20: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/20.jpg)
Encoding a Strategy
1234...
243
![Page 21: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/21.jpg)
0 2 6 5 . . . 3 . . . 4
Code: MoveNorth = 0MoveSouth = 1MoveEast = 2MoveWest = 3StayPut = 4PickUpCan = 5MoveRandom = 6
1234...
243
![Page 22: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/22.jpg)
Question: How many possible strategies are there in our representation?
Goal: Have GA search intelligently in this vast space for a good strategy
0 2 6 5 . . . 3 . . . 4
243 values
7 possible actions for each position: 7 × 7 × 7 × ... × 7
![Page 23: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/23.jpg)
Genetic algorithm for evolving strategies
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy, calculate fitness (average reward minus penalties earned on random environments)
3. The strategies pair up and create offspring via crossover with random mutations ― the fitter the parents, the more offspring they create.
4. Keep going back to step 2 until a good-enough strategy is found (or for a set number of generations)
![Page 24: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/24.jpg)
Robby’s fitness functionCalculate_Fitness (Robby) { Total_Reward = 0 ; Average_Reward = 0 ‘ For i = 1 to NUM_ENVIRONMENTS {
generate_random_environment( ); /* .5 probability * to place can at * each site */
For j = 1 to NUM_MOVES_PER_ENVIRONMENT {Total_Reward = Total_Reward + perform_action(Robby);
} }
Fitness = Total_Reward / NUM_ENVIRONMENTS; return(Fitness);}
![Page 25: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/25.jpg)
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
![Page 26: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/26.jpg)
Random Initial Population
![Page 27: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/27.jpg)
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy in the population, calculate fitness (average reward minus penalties earned on random environments)
![Page 28: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/28.jpg)
Fitness = Average final score from N moves on each of M random environments
![Page 29: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/29.jpg)
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy in the population, calculate fitness (average reward minus penalties earned on random environments)
3. Strategies are selected according to fitness to become parents. (See code for choice of selection methods.)
![Page 30: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/30.jpg)
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy in the population, calculate fitness (average reward minus penalties earned on random environments)
3. Strategies are selected according to fitness to become parents. (See code for choice of selection methods.)
4. The parents pair up and create offspring via crossover with random mutations.
![Page 31: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/31.jpg)
Parent 1:
Parent 2:
![Page 32: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/32.jpg)
Parent 1:
Parent 2:
![Page 33: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/33.jpg)
Parent 1:
Parent 2:
Child:
![Page 34: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/34.jpg)
Parent 1:
Parent 2:
Child:Mutate to “0”
Mutate to “4”
![Page 35: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/35.jpg)
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy in the population, calculate fitness (average reward minus penalties earned on random environments)
3. Strategies are selected according to fitness to become parents. (See code for choice of selection methods.)
4. The parents pair up and create offspring via crossover with random mutations.
5. The offspring are placed in the new population and the old population dies.
![Page 36: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/36.jpg)
Genetic algorithm for evolving strategies for Robby
1. Generate 200 random strategies (i.e., programs for controlling Robby)
2. For each strategy in the population, calculate fitness (average reward minus penalties earned on random environments)
3. Strategies are selected according to fitness to become parents. (See code for choice of selection methods.)
4. The parents pair up and create offspring via crossover with random mutations.
5. The offspring are placed in the new population and the old population dies.
6. Keep going back to step 2 until a good-enough strategy is found!
![Page 37: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/37.jpg)
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
“Otherwise, if there is a can in one of the adjacent sites,
move to that site.”
“Otherwise, choose a random direction to move in,
avoiding walls.”
![Page 38: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/38.jpg)
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
“Otherwise, if there is a can in one of the adjacent sites,
move to that site.”
“Otherwise, choose a random direction to move in,
avoiding walls ”Average fitness of this strategy:
346 (out of max possible 500)
![Page 39: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/39.jpg)
My hand-designed strategy:
“If there is a can in the current site, pick it up.”
“Otherwise, if there is a can in one of the adjacent sites,
move to that site.”
“Otherwise, choose a random direction to move in,
avoiding walls.”Average fitness of this strategy:
346 (out of max possible 500)
Average fitness of GA evolved strategy: 486
(out of max possible 500)
![Page 40: Evolutionary Learning](https://reader035.fdocuments.us/reader035/viewer/2022062520/56816371550346895dd44d8b/html5/thumbnails/40.jpg)
One Run of the Genetic AlgorithmB
est f
itnes
s in
pop
ulat
ion
Generation number