Evolving Multimodal Behavior Through Modular...

54
eee n m mee me

Transcript of Evolving Multimodal Behavior Through Modular...

Page 1: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

ee e

n

m m e e me

Page 2: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

n e

n y y y ye e e m

¨m ys ys e ey

n? m e ?

¨n ym y y y

¨y s s y

¨y y y ys e y

¨ s y

¨ s ee e y

Ø e )))))))

Page 3: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

n s y e s s m ss ysy

n n e s em y

e s s y e �y e y ��??

n e e e s e s

e s e y

g m n y g

Page 4: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

syy y y

n y y y s

e s ys e y m s s y

n? e s e s e

y e e e ?

Page 5: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e

n

n

¨? y ?

¨? ?

n y

n y ys ss y

n y ye yy sy e y

Page 6: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

? y ?n g eey s y

¨ s s s y y e ysy

¨ m ss ss ysy

n m s e e g

n e s e e ys e

¨ m ? e e e s e e

¨ s � � s e y s � s �

n s s m s e

n s e s em y m y e y y

¨ em s e y e y

n n g m s y e

¨ ys e y s s ysy

Page 7: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

emn e s em e y y s y s e ys y

¨ n e s y s

n y s e y

¨ ? ys e s s s s e y

n m s e y y m s y

¨ m ess e ys y e y e s y

s ys s e n y

)y )))))

n s yy

e s yy

Page 8: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

? e ?

n n sy s y

¨ ysy y ys y

¨ s e yy ys y

¨ y e s y y

n e

¨ ee e y s ee e e ys e

¨ hhy ? s e y m y e

n m y y

¨ms y y y e ys y

Page 9: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y ys n s y

n y ss e y

¨ y ey s e s y ee e

¨ s m e s e s e y

n y ss s y

¨ y em e e m s s

¨ h sy s �s em� s y

y y y )y ess e y)

¨ y y e s e

y y s y

¨ n s y y s

e e y

2 )

m y y

Page 10: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e

n

n

n y

¨n

n )y e )

¨ ee e s g

n hh )y e )

n y ys ss y

n y ye yy sy e y

Page 11: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y y e n

n h e s y s n n sy

n y e e m

n h e s e y

n e y e y )s y yy )

n e y e

y

s yy e

)es n s em m m 2))))

eeeeeeeeeeeeee eeeeeeeeeeeeee eeeeeeee

Page 12: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

n s y

n y e m ss my

n y ss e s s y

) y) )e ) )y s e )

)es y e 2)))))me ss 2)))s 2)))s 2))2)

n y

e y

Page 13: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

se y ee e es g ) )))))

eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee

,eeeeeeeeee,eeeeeee,

eeeeeeheeeeehee,eeeeeee

seeeee,,eeeeseeeeeeseeeeseh

ssssssssss ssssddssdsdd ssdss sf lssfll

)(

s.s. sd dssdss lll ssdslsds

sdsssll oldsss ss

:ssss dssdss dsssdlssd-nsd

)((,,1{ ..

ldd )((,,1{ 1.

s.s. , dsssdlsss

ssdss?dssdsls ldd ssls

gsslsdsdg slls -

tslls tlslgs -

:ssssssssss oss sosglss eslgsds

xyFyAx

FA

FA

uvni

uvni

uvuv

ii

ii

rfrrr

K

K

rfrrr

Î$$Î"

ÛÍ

>Î$

³Î"

Û

Page 14: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

n s m h e s nn)y e s 2))))

n s y g nP

n hy )´ e yy ) ´ y g nP ´

n y e s s y s È ´ y g 2n

n n s s y g n s y s y s È ´

Page 15: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

h e h y )me ss 2))))

n yy¨ s m e � ym� ee e y

¨ g e y � � ee e y

n �� � � ym� s y s y¨ ymy s yy �s y y y� e ) e )

¨ ? y m e

n eee e y e e

n n e y y e

? eee e y

Page 16: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

hh h se n y y e

¨e e ems e e y s

ld) slss ss lss l ( sodldlssssd sses-pssd :

gsdsdlsssdsd sssds sssssssss dsdlllsssd ssdlgs :

gsdsdlsssdsd sssds lssdlgs sf lssdlgs ossg ssd-gsssds :

)( 11

t

t

t

tttt

r

tx

tr

rxrr

a

a--

-+¬

Page 17: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e

n

n

n y

n y ys ss y

¨ ms y s y y

¨g s es e )e y e )

¨ s m )e y e )

¨ y )e y e )

¨ ys es ) )

n y ye yy sy e y

? y y

s y

ss ms y s ys

y y?

Page 18: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y y s ysyn ysy e e e s m y

¨ y ss e

n ysy m e

¨s y e ysys e y y e

n y y e e e ys e

¨s ysy e s m y s ?

Page 19: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y y s ysyn ysy e e e s m y

¨ y ss e

n ysy m e

¨s y e ysys e y y e

n y y e e e ys e

¨s ysy e s m y s ?

g s es e

s m

n s y

y

g

Page 20: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e

n

n

n y

n y ys ss y

¨ ms y s y y

¨g s es e

¨ s m

¨ y

¨ ys es

n y ye yy sy e y

n y ysy

n ss e m

n e

y ee

¨ s s y

?

Page 21: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

g s es e )me ss 2)))s 2))2)

n g y y y e

n e y ys e e

¨s e s e es s y

n e e

n e y s e m e e

n y yy e ys

¨y e

¨s

¨m m

Page 22: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

n e s s ysy

¨ y

¨ ss e m

n m y e y

¨ ys

¨ s

n y s

¨

n ysy y s y

n e sy e ys

n e s

n e s es

g s es e e y y

Page 23: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e

n

n

n y

n y ys ss y

¨ ms y s y y

¨g s es e

¨ s m

¨ y

¨ ys es

n y ye yy sy e y

n y ysy

n ms ss e m

n e

y ee

¨ ? ss ?

Page 24: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

s m )me ss 2)))s 2))2)

n g y y y e

¨ n s y n y y s s ye s s y y

¨ n m s e y y y

n y y s e ss e m

¨ ys y es e s

n y

¨y e

n m y

¨s

¨m m

Page 25: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

s m e y yn m s y ms ys s s y s m

¨ y s e

n s s s y

¨ ys y y s

n ) ) s s y s m

n )e) sy

¨ s y y

¨ e s

¨ h ss e y

n e es

e

n m e y y

y y e y

Page 26: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e

n

n

n y

n y ys ss y

¨ ms y s y y

¨g s es e

¨ s m

¨ y

¨ ys es

n y ye yy sy e y

n e ysy

n hh

n e

y ee

¨ n s e s

Page 27: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y )me ss 2))))

n g y y y ss

n e y y y y

¨ e s m y e

¨ m e y y

¨ m e y y

n ess y s y ysy e

n y y

¨ y e

¨ s

¨ m m

y y

esy

e �y e

Page 28: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y e y yn hh s s y s nmhssnn

n e y e

¨ e y s es

¨ y e y

¨

n y ss

y y ee yys

Page 29: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e

n

n

n y

n y ys ss y

¨ ms y s y y

¨g s es e

¨ s m

¨ y

¨ ys es

n y ye yy sy e y

n ysy

n me

n y s y

Page 30: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

ys es n y y e y ee

n y yy es ss ¨ e y s s y s

n ss m ¨ es s y e y

n h yy s g ye em¨ s y e

¨ s y y

¨ y y ) s s s )

n n s y e¨ s m ym y

n g g y¨ y g

Page 31: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

ys e s

n y y e e y

¨ e y y

¨ y y s s y

¨ ?

n s y y e m e ?

¨ yy e y y

¨ ys y y e

n y e e s s e y

n y s ye y s

Page 32: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y ? s es n y y m y

¨ h e y 2 g )))2

¨ n y h ´ e e 2)))s ´ e e y 2)))s s 2)))

¨ e s e e y ´ e e y 2)))s m e s 2)))s 2))3

¨ s s s m e y e e y ´ e e y 2)))

n me y s y s y e y n e yy ¨ ´ g ggm e ey ? ´ ny g s 2)))

¨ n s e sy ? ´ h 2)))

¨ s y m es g y s 2)))

¨ sy m e y ns ´ n 2)))

¨ y e y yy g s 2))2

n es ys h y y y s y es ¨ h e y 22222222 222))2)222222222222222222222222)222222 22222222 2222

¨ sy m e y m s y s 2)))s 22222222 222))2)22222

¨ n s e sy m yy ´ m yy 2))2

¨ s y m es g y 22)2) 22222222222

n es ys h y y y s y h y y¨ n y ? s s s 2)))

¨ e m y h ´ y 2))2

¨ sy m e y n m ´ y 2))3

Page 33: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y e n n ys em y s )ynh 2))2)

n n y s e s y y y

n e y ss s e e e

n es e y s

e s s e

e s e

s

e s

Page 34: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

m syn m ys

¨ s y ss m e

¨ s y s ss e s s

n y ye y y s s e y

¨ ys ys )e)s )y)

s ys y )y)

n y

e y

Page 35: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e e s ys es ? y s e m y y

Page 36: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y s y s y

e ys ss ye y

y s e m y y hy

Page 37: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e e

y sye sy m y

y s e m y y hy

Page 38: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e s y y

e e

sye sy y

s m ))e s

y s e m y y hy

Page 39: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e s y y

e sm

e e

e y sy y

)ee s

y s e m y y hy

Page 40: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

g h e e s n y ss e y s ss y

e e s

y y

e

e sm

s

ys

Page 41: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

g h e e s y e y

n ee y y y e e

¨ y ysy e

n m e ys y y s s

n s e y e e y y e

n y y y

¨s m ym e y y s y

¨? y ye y y m

¨ e y es y y

n es y s s y

n ? sy es ye s ysm y y

Page 42: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

g h e e s y e y n h y y ye

¨m e s y y m

n e m )e)y y

n s y y s ym?

¨ ym y s

¨? s y s yy

¨y ye y ss

n ? ye y s y

¨ s y e s

¨ hh e s s e

Page 43: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

s e y ys es ? y s e m y y

Page 44: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

n sy ? hh

n s y s y y s y

n hh s y y s s e y s e m

Page 45: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

m y s y

y s e m y y hy

e s y y e sm

e e

Page 46: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

s y y s

y s y

y s e m y y hy

s e se

y

Page 47: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

g h s e y n y ss e y s ss y

yy

s e se

e

m

Page 48: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y s y e ? s

ss2222222s)2)22)

0)2220002222s)2)22) 002222292909g g 22)22t2))222222)2222))22

02s03)2)s))) yh y ms e e e y 2))3

0)s)))32s0)) yh y msh

922092402920 yh22)22t2))222222)2222))22

33s)2)))s))) yhh y e y s 2))2

)3s)0)30s)3) yhsyee e s 2))2

3)s)e)))s))3g g h sy sys e e e y 2)))

))se0))0s)))g g h s e e e y 2)))

s2X2G0ee22 0es2s222)22)22)2)

yh s e s y y s e e y es y

ys s s s 3))) y sys s s e s y e

s y ye y eme y g y s s y ys s

ye ys

Page 49: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

e

n

n

n y

n y ys ss y

n y ye yy sy e y

Page 50: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y ye yy

n n y y y e y y y

¨ sy s y s y y

¨ hh sy s s s y

n e y y e s y s

n y ss e

¨ ? s s y e y s y s y )) ee)

¨ e y y e e m s

¨ m y s y

n ys ys e y y

Page 51: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

g ? s

n h e m y

¨ nyy ?

n e y y y ee yy

¨? ee e y y ? hh y s

¨ y ms m s

n e s y

¨h y es

n my e y

¨h s e ey

Page 52: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y e y

n y y e ys y

¨s m s ss e y y ee yys

n y y e ys y y

¨m s y ys y y s s e y n ys y sy s

¨ y y y e e e

¨ hh s y e

n e y y ys es y s yy s y e ys y

Page 53: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

y e y

n y e y

¨ n s ms y s ys y y

n ny s n s

¨ms y y y s y ys y

n e yy

¨ sy s e y

n e ys y e e ysy

¨ hh e y ye y e y y m

n s y e s y s ee e ss e

Page 54: Evolving Multimodal Behavior Through Modular …nn.cs.utexas.edu/downloads/slides/schrum.phd14.pdfcomponent behaviors Flying Nesting Foraging Multimodal Assistants n Consider all the

? y y?