2011 0006.Advanced Evolutionary

7/23/2019 2011 0006.Advanced Evolutionary

http://slidepdf.com/reader/full/2011-0006advanced-evolutionary 1/76

AdvancedAdvanced

ExamplesExamples

and Ideasand Ideas



Three Layer EvolutionaryApproach

EvolveBehavior

s

EvolveMotions

EvolvePerceptio

ns

Global perceptions, possibly

encoded such as “narrow

Corridor” or “beautiful

Princess”

Local perceptions, such as

“bald head” or “long beard”

Behaviors such as “go forward

until you find a wall, else turnrandomly right or left

Encoded behaviors

or internal statesime intervals

!otions as timed

se"uences of encoded

actions, for instance

#$#$LL



Evolve in hierarchyEvolve in hierarchy

1. Together or separately

2. Feedback ro! !odel or ro! real"orld

#. First evolve !otions and encode the!.

$. Then evolve behaviors.

%. Finally develop perceptions.

& you seea beautiulprincessgo to her

and bo"

& yousee a

dragon

escape

'o to the endo the corridorand then look

or ood



Evolve in hierarchyEvolve in hierarchy

avoidobstacles

E(ecuteopti!al!otions

)ave energy

Look orenergy

sources inadvance

E(ecuteactions thatyou en*oy

%hat if robot li&es to play

soccer and sees the ball

but is low on energy'



+pti!i,ing a !otion

ParkingParking

a Truck a Truck



$ind the control

Solving this

analytically would bevery difficult



(uestion) *ow to represent the

chromosomes'

*ere you see several snapshots of a

“movie” about par&ing a truc&, stages ofthe solution process+



t is ti!e u



Another e(a!ple

LearningLearningObstacleObstacleAvoidingAvoiding



)i!ilar to Braitenberg -ehicle but has sensors



ho"

/o" "ouldyou representchro!oso!es0

esignrossovers0

-nput and output

data are some

form of !. logic

/+ #obot can move freely but

has to avoid obstacles0+ his can be li&e the lowest

level of behaviors in

subsumption or other

behavioral architecture for

all your robots

e!e! er e goa " en



he &ey

to

success

is often in

fitnessfunction

e!e! er e goa " enyou create the 4tness

unction



Number of collisions

Ti!e olearning

%hen you

train longer

you decreasethe number of

collisions



ApplicatioApplications andns andProblemsProblems



'eneral 'A)che!a



Evolutionary Methods

+pti!i,ation proble!s5 1 )ingle ob*ective opti!i,ation proble!s

1 Multi6+b*ective opti!i,ation Proble!s



)earch Proble!s 7Path search8

+pti!al !ulti6robot coordination

Multi6task opti!i,ation

+pti!al !otion planning o robot ar!s 7Tra*ectory planning

o !anipulators 8 Motion opti!i,ation 7opti!i,ation o controller para!eters 6

!orphology in di9erent control sche!as8 1 P& 7P&8

1 Fu,,y

1:eural 1 /ybrid 7neuro6u,,y8

Path planning and tracking 7!obile robots8

+pti!al !otion planning o robot ar!s 1 Tra*ectory planning o !anipulators

Vision – computational optimization

!ore e2amples of problems in which we use

evolutionary algorithms and similar methods+



Evolutionary Algorith!s 6 3elated techni;ues5 1 Ant colony opti!i,ation 7A+8 1 Particle s"ar! opti!i,ation 1 i9erential evolution

1 Me!etic algorith! 7MA8 1 )i!ulated annealing 1 )tochastic opti!i,ation 1 Tabu search 1 3eactive search opti!i,ation 73)+8

1 /ar!ony search 7/)8 1 :on6Tree 'enetic progra!!ing 7:T 'P8 1 Arti4cial &!!une )yste!s 7A&)8 1 Bacteriological Algorith!s 7BA8

%hat are these “other

algorithm”'

/+ 3ou can try them

in your homewor&

/ if G4 or GP is

too easy for you+

0+ 5sing them gives

you higher

possibility ofcreating a

successful

superior method

for a new problem



'A6operators )election

1 3oulette 1 Tourna!ent 1 )tochastic sa!pling 1 3ank based selection

1 Bolt,!ann selection 1 :onlinnear ranking selection

rossover 1 +ne point

1 Multiple points Mutation

3ead in Au(iliary )lides

about these !ethods.+r invent your o"noperators or your

proble!.



<our design para!eters tobe decided

'enotype length

1 Fi(ed length genotype

1 -ariable6length genotype

Population

1 Fi(ed population

1 -ariable population

1 )pecies inside population

1 'eo!etrical separation



ra"backs o 'A

ti!e6consu!ing "hen dealing "ith alarge population

pre!ature convergence

ealing "ith !ultiple ob*ective proble!s

Solutions

6iches -slands

Pareto approach

7thers



More e(a!ples o using 'Ain robotics

TraectoryTraectoryPlanningPlanningProblemsProblems



'A and Tra*ectory Planning

1. 'A techni;ues or robot ar! to identiy theopti!al tra*ectory based on !ini!u! *oint tor;uere;uire!ents 7P. 'arg and M. =u!ar> 2??28

2. path planning !ethod based on a 'A "hile

adopting the direct kine!atics and the inversedyna!ics 7Pires and Machado> 2???8#. point6to6point tra*ectory planning o @e(ible

redundant robot !anipulator 7F3M8 in *oint space7). '. <ue et al.> 2??28

$. point6to6point tra*ectory planning or a #6link7redundant8 robot ar!> ob*ective unction is to!ini!i,ing traveling ti!e and space 7=a,e!>Mahdi> 2??8

Pro8ects last years

i l h i



+pti!al path generation orobot !anipulators

1. ontrol )che!a

2. 3obotic ar! kine!atic !odel#. ontroller type

$. +b*ective unction 6 opti!al path

%. +pti!i,ation algorith! 7!ethod8. 'A use s!ooth operators and avoidssharp *u!ps in the para!eter values.



Adaptive !ontrol "chemaAdaptive !ontrol "chema Track ontrol errorunction bet"een outputs o a real syste! and!athe!atical !odel

Chat "e opti!i,e0 Chich para!eters !ust be opti!i,ed0 /o" !any ob*ectives 7single ob*ective or

!ultiob*ective80

ollision ree0 7/o" to !odel collision in 'A08



Three oin #anipulatorThree oin #anipulator

A three6*oint robotic !anipulator syste! hasthree inputs and three outputs.

The inputs are the tor;ues applied to the *ointsand the outputs are the velocities o the *oints

:o ripples



For n6+F "e "ill have n inputs u i> iD1n> 7ui θi8

ontroller 1 P& 7P&8 1 :eural net"ork 7!ultilayer perceptron> recurrent

::> 3BF based ::8 1 Fu,,y

1 :euro6Fu,,y 7hybrid8

Design of roboticDesign of robotic

controllerscontrollers



::5 Ce !ust to adapt the "eights and eventually the bias

The chro!oso!e5

Adapt the "eights k ijw

Use of Neural NetworksUse of Neural Networks



$%&&' LO(I!$%&&' LO(I! Fu,,y Logic

Aggregation o rules

deu,,i4cation

ree6o6obstacles "orkspace 7Mucientes> et. al> 2??G8

"all6ollo"ing behavior in a !obile robot

1 1 1 1:i i i i i i i i i

N N M M R if x is A and x is A then y is B and y is B= =

Learning $%&&' LO(I!Learning $%&&' LO(I!



Learning o u,,y rule6based controllers Find a rule or the syste!

)tep 15 evaluate populationH)tep 25 eli!inate bad rules and 4ll up populationH)tep #5 scale the 4tness valuesH

)tep $5 repeat :& iterations or )tep $ to )tep I )tep %5 select the individuals o the populationH )tep 5 crossover and !utate the individualsH )tep G5 evaluate populationH )tep 5 eli!inate bad rules and 4ll up populationH

)tep I5 scale the 4tness values. )tep 1?5 Add the best rule to the 4nal rule set.

)tep 115 Penali,e the selected rule. )tep 125 & the stop conditions are not ul4lled go to

)tep 1

Learning $%&&' LO(I!Learning $%&&' LO(I!!ontrollers!ontrollers



Encoding u,,y controls

The chro!oso!e encode the rules5

)n is constant in this application but it can be also variable tobe opti!i,ed

"all6ollo"ing behavior o the robot

1 the robot is e(ploring an unkno"n area

1 !oving bet"een t"o points in a !ap

3e;uire!ents

1 !aintain a suitable distance ro! the "all that is being ollo"ed

1 to !ove at a high velocity "henever the layout o theenviron!ent is per!itting

1avoid sharp !ove!ents 7progressive turns and changes invelocity8

1 1 2 2( , , , , , )i i i i i i i

NV NV C b c b c b c= =



Path6based robot behaviors

The re;uire!ents are JencodedK inniverses o discourse and precisions o thevariables 1 right6hand distance 738

1 the distances ;uotient 78> based on let6handdistance

1 +rientation

1 linear velocity o the robot 7L-8

1 Linear acceleration

1 Angular velocity

Path o the robot 7si!ulated environ!ents8

Fast reliable no har! to



Fast> reliable> no har! torobot or to environ!ent

This is useul or out

P) 'uide 3obot1. o not har!

hu!ans

2. o not har! robot



Fi(ed points5 the desired artesian path Pt is given the

proble! is to 4nd the set o *oint paths Pθ in order to

!ini!i,e the cu!ulative error bet"een desire and real pathduring tra*ectory

Pk is the kine!atic !odel

Free end points case

1 1

2 1 2

( , ) ( , ) ( ) j j

N N N

t r

i j i

E P i j P i j E i

− −

= = =

= − =∑ ∑ ∑

( )r k P F P θ =

!inimi9e the

cumulative

error

$ind the set

of 8oint

paths, ne2tsmooth it



Ceighted 'lobal Fitness

4tness unction 7!ini!i,ation8

'lobal 4tness5 Linear unction o individual ob*ectives

Fot e(cessive driving 7su! o all !a(i!u! tor;ues8> ; the

total *oint traveling distance o the !anipulator> c 6 total

artesian tra*ectory length> t T 6 total consu!ed ti!e or robot

!otion

Penalty unction

Population initiali,ation 7probability distribution8

1 3ando! unior!

1 'aussian

1 2 3 4total ot q c T

F f f f t β β β β = + + +

1

1 F

E =

+



e(a!ple

)rug)rug)elivery)eliveryProblemProblem

rug delivery using !icrorobots 7Tao et al



rug delivery using !icrorobots 7Tao> et. al>2??%8

1. 7'A8based area coverage approach or robot pathplanning.

2. ra"backs o !ost currently available drug delivery!ethods are that the drug target area> delivery a!ount>

and 1 release speed are hard to be precisely controlled.

#. &t is very diNcult or i!possible to eli!inate side e9ects.$. +pen issues

1.actively control the delivery process

2.Access to appropriate areas that cannot be reached usingtraditional devices%. urrent &ssues

1.+n6line path planning 7solve une(pected obstacles proble!82.+pti!al path planning 7eNciency> path planning8



!icrocontroller is used to guide the robot !ove!ent 'A6based approach uses 4ne grid cell

deco!position or area coverage Because the robot "ill !ove cell by cell> the start

point o chro!oso!es has to be changeddyna!ically "henever the robot reaches the centero a cell

The end point o a chro!oso!e is not 4(ed andneeds to be deter!ined by applying 'A operators.

The robots !ay !ove ro! the center o a cell to its

ad*acent cells along directions. so!e obstacles are unkno"n beore drug delivery

7the robot discover these obstacles during the!otion8



E(pandable chro!oso!es

eleting the path

rossover operator



:e" !utation operators 1 Travel urther

1 elete

1 3everse delete 1 )tretch

1 )hortcut

The algorith! keep !ind the visitednodes

E(tension to operational research0



+ther applications using evolutionaryalgorith!s

Autonomous mobile robot navigation 6Path planning using ant colony opti!i,ationand u,,y cost unction evaluation 7'arcia> et.al> 2??I8.

Legged 3obots and Evolutionary esign +pti!al path and gait generations 7Pratihar>

ebb> and 'osh> 2??28 ?O1 absence orpresence o rule

si(6legged robot collision-free coordination of multiple

robots ( Peng and Akela> 2??%8

Chat i you "ant to opti!i,e t"o



Chat i you "ant to opti!i,e t"opara!eters at the sa!e ti!e0

ParetoPareto

Optimi*atiOptimi*ationon

Pareto



)e+nitions

ophelimity

noun econo!ic>Econo!icsatisaction. Theability to please

EvolutionaryMethods

Chat is better



Ce "ant to opti!i,e both unctions 1 and

2

Chat is betterthis or this0

Biob*ective



Biob*ective !eans t"oob*ectives

to reach

Ce have ( and y> t"o ob*ectives here

Pareto solutions for

different algorithms

ParetoPareto

FrontFront



Pareto frontPareto front

The single ob*ective opti!isation proble! 7)+P8conduct to a !ini!i,ation 7or !a(i!i,ation8 oone cost unction> less or !ore co!ple(> that isa single ob*ective is taken into account.

onversely> the !ulti6ob*ective opti!i,ationproble! takes into account t"o or !oreob*ective that has to be !ini!i,ed 7or!a(i!i,ed8 si!ultaneously.

)o!e ob*ectives can be in co!petition> so asi!ultaneous !ini!i,ation is not possible> butonly a trade6o9 a!ong the!. )o!e ti!e> the nu!ber o ob*ectives can be high>

like 1 ob*ectives or !ore that !ake the !ulti6ob*ective opti!i,ation proble! 7M+P8 andinteresting and challenging area o research

E(a!ple o Pareto +pti!i,ation



E(a!ple o Pareto +pti!i,ationo t"o para!eters

Optimi*ationOptimi*ationof Airplaneof Airplane

,ings,ings



T"o ob*ectives5 T"o ob*ectives5 Ma(i!i,e lit> and



T"o ob*ectives5 T"o ob*ectives5 Ma(i!i,e lit> and !ini!i,e drag

Q &n !ost o the

design spacethe red !ethodis better thanthe blue !ethod

Q &t is good touse !any Pareto!ethods and!odiypara!eters

1. Ce opti!i,e !anypara!eters



Multi6Paretopara!eters>

2. Ce !ay s"itchbet"een subsets othe!.

#. )ubsets can havet"o ele!ents each.



Three-Three-

dimensionaldimensional

#inimi*ation#inimi*ationProblemProblem



ParetoPareto

$ront$ront

'eneral !ultiob*ective'eneral !ultiob*ective



'eneral !ultiob*ective'eneral !ultiob*ectiveopti!i,ation proble!opti!i,ation proble!

The !ultiob*ective opti!i,ationproble! could be generallyor!ulated as !ini!i,ation o vector

ob*ectives Rt7(8 sub*ect to a nu!bero constraints and bounds5

P t ti l tP t ti l t



Pareto6opti!al setPareto6opti!al set

1. &n the case o co!peting ob*ectives a trade6o9 isinvolved such a proble! usually has no uni;uesolution.

1 &nstead> "e can ad!it a set o solutions> e;ually valid non6do!inated as a set o alternative solutions kno"n asPareto6opti!al set

2. &n "hat ollo"s "e assu!e "ithout loss o generalitythat all the unction ob*ectives !ust be !ini!i,ed.

1 & "e have a !a(i!i,ation case f "e si!ply !ini!i,e theunction -f.

#. For any t"o points that are usually na!ed candidatesolutions -1>-2∈Ω> -1 do!inates -2 in the Paretosense 7P6do!inance8 i and only i the ollo"ingcondition hold

∈

∈

,...,),()(

,...,),()(

121

121

1

1

joneleast at for V f V f

iV f V f

j j

ii

The Pareto setThe Pareto set



The Pareto set The Pareto set

1. The Pareto set is the set o P+ 7Pareto6+pti!al8 solution in design do!ain and thePareto Front 7PF8 is the set o P+ solutions inthe ob*ective do!ain.

2. The !ost popular "ay to solving the M+P7Multi +b*ective +pti!i,ation Proble!8 is toreduce the !ini!i,ation proble! to a scalaror! by aggregating the ob*ectives in"eighted su!> "ith the su! o "eightsconstant5

#. The "eighted su! !ethod has a seriousdra"back> the !ethod usually ail in thecase o nonconve( PF.

∑

n

i

i

n

i

iit w ! w ! 11

1,min

E(a!ple o a clear picture



E(a!ple o a clear pictureo Pareto points



:ice properties

1. 'A can provide an elegant solution or tradeo9 a!ong di9erent !ini!i,ation o cost unctionor each variable versus total cost or othervariable.

2. :on6conve( solutions

#. J&!!igrantsK> possible solution or *u!p ro!

local !ini!a.

$. ealing "ith !any variables 7e.g. 1 variables8



Multi63obots

Pareto opti!al !ulti6robot coordination "ithacceleration constraints 7Rung and 'hrist>2??8

1. collection o robots sharing a co!!onenviron!ent

2. each robot constrained to !ove on a road!apin its con4guration space

#. each robot "ishes to travel to a goal "hile

opti!i,ing elapsed ti!e considering vector6valued 7Pareto8 opti!a

$. all illegal or collision sets are re!oved.

!onclusions!onclusions



!onclusions!onclusions

1. 'A is not a universal panacea to opti!i,ationproble!s.

2. Coding the problem into a genotype is the mostimportant challenge!

#. The best selection sche!a o individuals or crossoveroperator is diNcult to be chosen apriori 7tourna!entselection see!s to be !ore pro!ising8

$. A nu!ber o para!eters are deter!ined e!pirically51. )i,e o population2. pc and p! even oten values inspired ro! biology are

given

#. +ther para!eters in hybrid or !ore sophisticated 'A



'ood properties

1. +ne o the !ost i!portant ele!ent in the design o adecoder6based evolutionary algorith! is its genotypicrepresentation.

2. The genotype6decoder pair !ust e(hibit eNciency> locality>and heritability to enable e9ective evolutionary search

3. locality, and heritability 5

1. s!all changes in genotypes should correspond to s!all

changes in the solutions they represent>and

1. solutions generated by crossover should co!bineeatures o their parents



SourcesSources

ragos Arotaritei



examplexampl

ee

2011 0006.Advanced Evolutionary

Documents

Transcript of 2011 0006.Advanced Evolutionary