pure::variants User's Guide€¦ · pure::variants User's Guide ... 1
Evolving Quick Learners -...
Transcript of Evolving Quick Learners -...
![Page 1: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/1.jpg)
Evolving Quick Learners: Novel Initialization Strategies for Markov Brains
Douglas Kirkpatrick and Tyler Derr
![Page 2: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/2.jpg)
OverviewBackground
Markov Brains
Pre-Evolution & Multi-Task Evolution
Problem
Project Description
Pre-Evolution
Evolving your Ancestors
Results
![Page 3: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/3.jpg)
Background
![Page 4: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/4.jpg)
Markov Brains 9 Input Nodes3 OutputNodes
4 Hidden Nodes
3 Gates
![Page 5: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/5.jpg)
Markov Brains
Idea: We are evolving the logic gates that wire up a set of nodes, thus forming a network
● Genotype: ○ Variable Length 2k to 20k○ Each loci can be an integer value from 0 to 255○ Start codons along genome mark where gates should be read from (similar to
genes in DNA)● Phenotype:
○ A set of N nodes (input, hidden, and output)○ A set of logic gates connecting the nodes (in our case deterministic gates)
![Page 6: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/6.jpg)
ProblemBrains are not quick to evolve
100s of millions of years to evolve in humans
100,000 to 1,000,000 or more generations to evolve in Markov Brains
Can we better initialize the Markov Brains so that they evolve faster?
![Page 7: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/7.jpg)
Pre-EvolutionIf I’m good at hockey, my one cousin is good at football, and a second cousin is good at basketball, it’s likely that my ancestors had some athletic ability
If there’s another relative out there, it’s likely they’re good at rugby or soccer
Similar idea applied to Markov Brains
![Page 8: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/8.jpg)
Multi-Task EvolutionMulti-Task Learning: attempting to learn more than one task/problem at a time
Ex. Learning to drive a car and a boat in the same day
This can lead to better and more generalized understanding ...why?...because it allows the learner to see the commonality (i.e. the underlying tasks) and use them better towards the main tasks
Ex. Learning the general concept of steering left/right will greatly improve one’s ability to drive both a car and a boat
![Page 9: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/9.jpg)
Approach
![Page 10: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/10.jpg)
Approach 2 Experiments
1. Pre-Evolution 2. Multi-Task Evolution
![Page 11: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/11.jpg)
Pre-Evolution ExperimentCore world - Red/Blue Berry
Variants:
- Green (poison) berry - Orange (superfruit) berry- Random Walls in environment
![Page 12: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/12.jpg)
Berry World
![Page 13: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/13.jpg)
World Variants Used 5 Training Worlds:
1. Poison2. Superfruit3. Superfruit & Poison 4. Random Walls & Poison5. Random Walls & Special Food
1 Testing World:
Original Red/Blue Berry World (no enhancements)
![Page 14: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/14.jpg)
Experimental Structure
EVOLVE
EVOLVE
10,000 Generations
Best
1000 GenerationsX100
Ancestor
Results!
![Page 15: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/15.jpg)
SetupMarkov Brains w/ 24 nodes
1000 Generations / 500 Brains for Training Runs
- 20 of each Training world
10,000 Generations / 100 Brains for Testing Runs
Testing Runs compared with Equivalent Randomly Seeded Runs
55 Replicates
![Page 16: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/16.jpg)
For each p in P● Let p have children and grandchildren where they select mates based on
how good at playing chess they are ● chessBest = fitness of best chess playing descendent of p● Let p go back in time (have children…) but this time select mates based
on how good at playing go they are● goBest = fitness of best go playing descendent of p● Fitnessp= avg(chessBest,goBest)
P = selection&mutation(P)
Real World AnalogyExperiment: Evolve your ancestors
P = {random set of people}repeat
until(stoppingCondition)
...kinda sorta ...no, not really
(i.e. when P is finally worthy to be your ancestors)
![Page 17: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/17.jpg)
If you picked A It’s possible, but good luck!
If you picked B
A B(random set of ancestors) || (the evolved ancestors P)?
Question:If your goal in life is to be the best checkers player ever
Would you rather come from
Remember P was evolved to have descendants that are excellent at playing chess and go.
Experiment: Evolve your ancestorsReal World Analogy
Image Source: https://www.floridamemory.com/items/show/134382
(again, most likely not so real)
This could be you!
![Page 18: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/18.jpg)
Evolve P*0 where
Fitness =
Experiment: Evolve your ancestorsInitialize
Random PopP0={p1,p2,...pn}
Evolve P0 where
Fitness =
Best FitnessFound
Initialize
P*0 = Pfinal
Fitness =
Evolve P0 where
Best FitnessFound
Compare?? >=< ??
![Page 19: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/19.jpg)
Problems Used: Random Mapping WorldGenerate(numInputBits, numOutputBits)
Lists all possible input values and randomly selects a corresponding output
Ex. Generate(3,2)000 | 00001 | 00010 | 11011 | 01100 | 11101 | 10110 | 00111 | 10
Fitness = % output correctly predicted based on input values
Image Source: http://2new4.fjcdn.com/pictures/Sparta+chess+my+style+of+chess+_511e16_5692709.jpg
![Page 20: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/20.jpg)
SetupMarkov Brains w/ 16 nodes
Each world: 8 input & 3 output bits (i.e. 256 inputs each with 8 possible outputs)
Training: Multi-Task Evolution on (K-1) worlds - 5,000 Generations & popSize=100
● Fitness of each individual based on evolving 10 mutated copies of it for 25 generations in each (K-1) worlds
Testing: Single-Task Evolution on world K - 10,000 Generations & popSize=100
![Page 21: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/21.jpg)
Results
![Page 22: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/22.jpg)
Relevant XKCD
![Page 23: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/23.jpg)
Results
![Page 24: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/24.jpg)
Shifted
![Page 25: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/25.jpg)
Results
![Page 26: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/26.jpg)
Results
![Page 27: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/27.jpg)
Conclusion Not the stunning success we had hoped for
Experiment 1:● Did not let the training evolution run long enough (similar results to random)
Experiment 2:● The Random Mapping problems were just dummy problems for POC● Need to either modify or use different problems that do have some underlying
similarity to make use of the multi-task evolution initialization● Allowing the ancestors to evolve descendants as a means to evaluate its own
fitness is unneeded -> simply evaluate the fitness of the given individual as is
![Page 28: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/28.jpg)
Future WorkFix the World Setup
- Equivalence for Berry World- Different distribution for Random World
- Ranging the similarity between them
Try Different Worlds
![Page 29: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/29.jpg)
Evolve P*0 where
Fitness =
Future WorkInitialize
Random PopP0={p1,p2,...pn}
Evolve P0 where
Fitness =
Best FitnessFound
Initialize
P*0 = Pfinal
Fitness =
Evolve P0 where
Best FitnessFound
Compare?? >=< ??
![Page 30: Evolving Quick Learners - cse.msu.educse.msu.edu/~derrtyle/downloads/TylerDerr_CSE848_Presentation.pdf · - Random Walls in environment . Berry World . World Variants Used 5 Training](https://reader036.fdocuments.us/reader036/viewer/2022071023/5fd78c60a504f947073edede/html5/thumbnails/30.jpg)
Thank You. Questions?