Spacetime Constraints Revisited

Joe Marks J. Thomas Ngo

Using genetic algorithms to find solutions to spacetime constraint problems in 2D

Difficulties with SC problems

• Multimodality– exponential number of possible solutions

• Search space discontinuities– small changes in actuators leads to large

changes in trajectory

Why Genetic Algorithms?

• Traditional SC algorithms– Initial trajectory needed to work on

• Initial trajectory is done by human hand - “coarse”

• Initial trajectory is locally optimized

• Genetic Algorithms– Discovers solutions “from scratch”

– Cover global optimization problem

– Can find “novel” solutions that were not imagined

Overview of GA in SC problems

• All trajectories are described as behaviors

• Behaviors generated by parametric algorithms (nothing smart here)

• Evolutionary computation chooses behavior parameters to find optimal solutions– can find multiple effective solutions

• Better solutions (genomes) bred further

• Worse solutions bred-out of gene pool

GA algorithm for 2D simulation(Ngo and Marks)

• Dynamics module– simulates a physics driven world for testing

creatures

• Behavior module– generates behavior using parameterized

algorithm

• Search module– uses GA to choose values for parameters– generates near-optimal solutions

Dynamics Module

• Most CPU intensive part of this algorithm

• Simulating simplified physics– Based on work by Hahn– rate of change of internal degrees of creature

controlled, rather than calculating torques– friction and slippage simulated upon contact

Behavior Module

• Behaviors generated by stimulus-response (SR) control algorithm– causes instinctive reflexes to conditions

• conditions are stimulus functions over senses

• reflexes are responses to conditions

– avoids traditional use of forces– behaviors selected in search module– no learning or planning algorithms involved

Stimulus functions• Scalar functions based on sense variables

– state of joint angles– force between body endpoints and floor– vertical velocity of center of mass– height from floor of center of mass

• Contain parameters that are determined by GA-based search module

• Stimulus functions exhibit sensitive regions– locus of points for which function is positive– important notion during mutation

Response

• Change in shape in reaction to a condition– Conducted on highest valued positive stimulus

function– Change of creature’s actual shape to a target shape– Change kept smooth by damped motion equations

• Target shape may change during a response– Creature must respond to another stimulus

• Response is active for several time steps

Search Module

• Use of GA to pick near optimal behaviors

• Parallel processing of solutions– each processor handles one genome solution

per generation

do parallelRandomize genome

end dofor generation = 1 to number_of_generations

do parallelEvaluate genomeSelect mate from another processorCross genome with mateMutate genome

end doend for

Randomization

• Initial parameters chosen at random

• Favorable initial gene pool formed– hill climbing algorithm used to find good initial

solutions to populate gene pool– each solution mutated, re-evaluated four times

• mutation skewed towards multi-step solutions

– best of five solutions on each processor used• final population is non-random, skewed

Evaluation

• One of the most important aspects of GA

• Net horizontal distance covered by center of mass in given time– Sometimes this encourages “cheating”– e.g., leap head-first rather than walking

• To encourage walking, modify criteria– e.g., net horizontal distance covered by

midpoint between two feet - i.e., use your feet!

Mate Selection

• Only local mating permitted– maintains diversity, handles multimodality– mate chosen on a random walk of 10 steps

• if better solution than self found, mate is chosen

• otherwise, current solution stays single

– produces large local colonies of good genes• larger colonies use up more processing power

• Convergence - when one colony dominates

Crossover• Linear crossover not seen as meaningful

– certain parameters should migrate as groups

• More structured, specialized crossover used– two (of ten) SR pairs taken from self– six (of ten) SR pairs taken from mate– one SR pair created with stimuli and response taken

from each parent, respectively– one SR pair created with numbers taken by random

from parents

Mutation

• Mutation algorithm tailored for SR method• One SR pair in genome is subject to creep

– each parameter in pair is slightly changed

• One SR pair is randomized from scratch– but one corner of sensitive regions of new

stimulus function coincides with original sense-space trajectory

• keeps new functions from either dominating or not having any effect.

Results• Hardware, software used

– 4,096 processor Thinking Machines CM-2– Code written in C*

• Creatures used– 5 rod figures; 50 time steps; 100 generations

• Five-rod Fred

• Mr. Star-man

• Beryl Biped– importance of adapting criteria

• 30 to 60 minutes of computation used