Evolving Spatiotemporal Coordination in a ModularRobotic System

Mikhail Prokopenko�, VadimGerasimov

�andIvanTanev

��CSIRO InformationandCommunicationTechnologyCentre,Lockedbag17,NorthRyde,

NSW1670,Australia�Departmentof InformationSystemsDesign,DoshishaUniversity, 1-3Miyakodani,Tatara,

Kyotanabe,Kyoto610-0321,Japanmikhail.prokopenko@csiro.au

Abstract. In thispaperwepresentanovel information-theoreticmeasureof spa-tiotemporalcoordinationin amodularroboticsystem,anduseit asafitnessfunc-tion in evolving thesystem.This approachexemplifiesa new methodologyfor-malizing co-evolution in multi-agentadaptive systems:information-driven evo-lutionarydesign.Themethodologyattemptsto link togetherdifferentaspectsofinformationtransferinvolved in adaptive systems,andsuggeststo approximatedirecttask-specificfitnessfunctionswith intrinsicselectionpressures.In particu-lar, theinformation-theoreticmeasureof coordinationemployedin thiswork esti-matesthegeneralizedcorrelationentropy

� � andthegeneralizedexcessentropy� � computedover a multivariatetime seriesof actuators’states.Thesimulatedmodularrobotic systemevolved accordingto the new measureexhibits regularlocomotionandperformswell in challengingterrains.

1 Introduction

Innovations in distributed sensorand actuatortechnologies,as well as advancesinmulti-agentcontroltheoryandstudiesof self-organization,supportrapidgrowth in ap-plicationsof complex adaptive multi-agentsystems(MAS), suchasmodularrobotics,multi-robotteams,self-assembly, etc.In particular, modularrobotsbuilt of severalsim-ilar building blocks(modules)becomemoreandmoreattractive dueto high versatilityin their shapes,locomotionmodes,tasks,andmanipulationabilities [3,26,22,21,7].Thismulti-facetedversatilityincreasesrobustness,adaptability, andscalabilityrequiredin practicalsystems,rangingfrom searchandrescueto spaceexploration.Thesere-quirementsareachievedthroughadistributionof sensing,actuationandcomputationalcapabilitiesthroughoutthe MAS suchasa modularrobotic system.This distributionformsa complex multi-agentnetwork, enablingthedesiredresponsesto self-organizewithin thesystem,without centralcontrol.However, themainchallengewith develop-ing aself-organizingMAS is adesignmethodologyfor systematicallyinter-connectinga setof globalsystem-level tasks,functions,etc.with localizedsensors,behaviors,andactuators.

In this paperwe further develop sucha methodologyoriginally sketchedin [14],aiming at formalizing “tasklessadaptation”of co-evolving multiple agents(roboticmodules,network nodes,swarm elements,etc.).The co-evolution canbe achieved in

two ways: via task-specificobjectives or via genericintrinsic selectioncriteria. Thegeneric information-theoreticcriteria may vary in their emphasis:for example,wemayfocusonmaximizationof informationtransferin perception-actionloops[11,12];minimizationof heterogeneityin agentstates,measuredwith thevarianceof the rule-space’s entropy [25,17] or Boltzmannentropy in swarm-bots’states[1]; stability ofmulti-agenthierarchies[17]; efficiency of computation(computationalcomplexity); ef-ficiency of communicationtopologies[15,16]; efficiency of locomotionanddistributedactuation[14,6,22,21], etc. The solutionsobtainedby information-driven evolutioncan be judgedby their degreeof approximationof direct evolutionary computation,wherethelatterusestask-specificobjectivesanddependsonhand-craftingfitnessfunc-tionsby humandesigners.A goodapproximationwill indicatethat thechosencriteriacapturetheinformation-theoreticcoreof selectionpressures.Themaintheme,however,is thatdifferentselectioncriteriaincorporateinformationtransferwithin specificchan-nels,andselectingsomeof thesechannelsandnot theotherswouldguideinformation-drivenevolutionarydesign.

Following [14] we applyherean information-theoreticmeasureof spatiotemporalcoordinationin amodularroboticsystemto anevolutionof asufficiently simplesystem:a modular limbless,wheellesssnake-like robot (Snakebot) [22,21] without sensors.Theonly designgoalof Snakebot’s evolution, reportedby Tanev andhis colleagues,isfastestlocomotion.Our immediategoal is information-theoreticapproximationof thisdirectevolution.Specifically, we constructmeasuresof spatiotemporalcoordinationofdistributed actuatorsusedby a Snakebot in locomotion.The measuresare basedonthegeneralizedcorrelationentropy � � (a lower boundof Kolmogorov-Sinaientropy)andits excessentropy � � computedover a multivariatetime seriesof actuators’states.The experimentsreportedby [14] confirmedthat maximal coordinationis achievedsynchronouslywith fastestlocomotion.In thispaperwereplacethedirectmeasurewiththe information-theoreticmeasureof spatiotemporalcoordination,and usethe latterexclusively in evolving theSnakebot.

The following Sectionplacesthis methodologyin thecontext of previous studies,describestheproposedmeasures,andpresentsresults,followedby conclusions.

2 Information Transfer as an Intrinsic Selection Pressure

An exampleof anintrinsicselectionpressureis theacquisitionof informationfrom theenvironment:thereis evidencethat pushingthe informationflow to the information-theoreticlimit (i.e.,maximizationof informationtransfer)cangive riseto intricatebe-havior, inducea necessarystructurein the system,andultimately adaptively reshapethesystem[11,12]. Thecentralhypothesisof Klyubin et al. is that thereexists “a lo-cal anduniversalutility functionwhich mayhelp individualssurvive andhencespeedup evolution by makingthe fitnesslandscapesmoother”,while adaptingto morphol-ogyandecologicalniche.Theproposedgeneralutility function,empowerment, couplestheagent’s sensorsandactuatorsvia theenvironment.Empowermentis theperceivedamountof influenceor control theagenthasover world, andcanbeseenastheagentspotentialto changetheworld. It canbemeasuredvia theamountof Shannoninforma-tion thattheagentcan“inject into” its sensorthroughtheenvironment,affectingfuture

actionsandfutureperceptions.Suchaperception-actionloopdefinestheagent’sactua-tion channel,and,technically, empowermentis definedasthecapacityof thisactuationchannel:the maximummutual informationfor the channelover all possibledistribu-tionsof thetransmittedsignal.“The moreof theinformationcanbemadeto appearinthe sensor, the morecontrol or influencethe agenthasover its sensor”— this is themainmotivationfor this localanduniversalutility function[12].

Heterogeneityin agentstatesis anothergenericpressurerelatedto intrinsic coordi-nationandself-organization.For example,it wasmeasuredwith thevarianceof therule-space’sentropy [25] andappliedto evolve thespatiotemporal stabilityof multi-cellularpatternsin asensor/communicationnetwork embeddedwithin aself-monitoringimpactsensingtest-bedof anaerospacevehicle[9,17,24]. Thestudyof spatiotemporalstabil-ity in evolving impact boundaries— continuouslyconnectedmulti-cellular circuits,self-organizing in presenceof cell failuresandconnectivity disruptionsarounddam-agedareas— employs both task-dependentgraph-theoreticandgenericinformation-theoreticmeasuresin separatingchaotic regimesfrom ordereddynamics.The task-dependentmeasurecapturedtheimpactboundary’s connectivity in termsof thesizeofthe averageconnectedboundaryfragment— an analogueof a largestconnectedsub-graphandits standarddeviationover time.Theintrinsic information-theoreticmeasurecapturedthediversityof transitionrulesinvokedby thenetwork cellsduringanimpactboundaryformation,usingtheShannonentropy of therules’ frequency distribution:

���� ��������� � � �� �������� �

��!

where� is thesystemsize(thetotalnumberof cells),and� �

is thenumberof timesthetransition" wasusedattime # acrossthesystem.Bothmeasuresconcurredin identifyingcomplex dynamics,pointingto thesamephasetransitionbetweenchaosandorder, forparticularregionsin a parameter-space.Theentropy

��� �canalsobeinterpretedas

thejoint statetransitionentropy��$ $ &% � � , where

$ is thestateof thecell at time #

[17]. Thisopensaway to considerinformationtransfer'(�$ �) $ &% � ���*��$ �,+-��$ &% � �.�/��$ $ &% � � within thechannelbetweenacell anditself at thenext time-step.

An investigation of Baldassarreet al. [1], characterizedcoordinatedmotion in aswarmcollectiveasaself-organizedactivity, andmeasuredtheincreasingorganizationof the groupon the basisof Boltzmannentropy. In particular, the emergentcommondirectionof motion,with the chassisorientationsof the robotsspatiallyaligned,wasobserved to allow thegroupto achieve high coordination.Baldassarreet al. proposeda methodto capturethespatialalignmentvia Boltzmannentropy by dividing thestatespaceof the elementsof the systeminto cells (e.g.,cells of 02143 each,correspondingto chassisorientations),measuringthe numberof elementsin eachcell for a givenmacrostate5 , computingthe number6 � of microstatesthat compose5 , andcalcu-lating Boltzmannentropy of the macrostateas � � �87 � ��9 6 ��: , where

7is a scaling

constant.This constantis setto the inverseof themaximumentropy which is equaltothe entropy of the macrostatewhereall the elementsareequallydistributedover the

cells.Theresultsindicatethat“independentlyof thesizeof thegroup,thedisorganiza-tion of thegroupinitially decreaseswith anincreasingrate,thentendsto decreasewitha decreasingrate,andfinally reachesa null valuewhenall the robotshave the sameorientation”[1].

In thiswork, weadvancefrom apurelyspatialcharacterization(suchasBoltzmannentropy of amacrostatedistributingchassisorientationsoverthecells)to aspatiotempo-ral measure.Theentropy measureproposedin ourwork is intendednotonly to capturespatialalignmentof differentmodules,but alsoto accountfor temporaldependenciesamongthem,suchas travelling or standingwaves in multi-segmentchainsobservedby Ijspeertet al.. Importantly, weplanto focusonchannelswhereinformationtransfercontributesto aselectionpressure.

We refer hereto onemoreexampleof a selectionpressure— efficiency of com-municationtopologies— which canbeinterpretedasin termsof informationtransfer.Onefeasibleaveragemeasureof acomplex network’sheterogeneityis givenby theen-tropy of a network definedthroughthe link distribution. The latter canbedefinedviathesimpledegreedistribution— theprobability ;=< of having anodewith

7links. Sim-

ilarly, onecancapturetheaverageuncertaintyof thenetwork asawhole,usingthejointentropy basedon thejoint probabilityof connectedpairs ;><@? <BA . Ultimately, theamountof correlationbetweennodesin thegraphcanbecalculatedvia themutualinformationmeasure,theinformationtransfer[19], as

'( ; ) ;DC �E�� ; �F�/� ;HG ;DC �E� �� < � � ��<4A � � ; <@? < A>I JLK ; <@? < A; < ; < ANMThereviewedexampleshighlight thepossiblerole of informationtransferin guid-

ing selectionof efficientperception-actionloops,spatiotemporallystablemulti-cellularpatterns,andwell-connectednetwork topologies.We intendto demonstratethat spa-tiotemporalcoordinationin a modularroboticsystemcanalsobecapturedasinforma-tion transfer, andapplysuchameasureto thesystem’sevolution.

Beforepresentingour approach,we briefly review somestudiesof therelationbe-tweenlocomotionand rhythmic inter-modularcoordination.Dorigo [7] describesanexperimentin swarmrobotics(SWARM-BOT) which alsocomplementsstandardself-reconfigurabilitywith task-dependentcooperation.Small autonomousmobile robots(s-bots)aggregateinto specificshapesenablingthe collective structure(a swarm-bot)to performfunctionsbeyondcapabilitiesof a singlemodule.Theswarm-botformsasa resultof self-organization“ratherthanvia a globaltemplateandis expectedto moveas a whole and reconfigurealong the way when needed”[7]. One basicability of aswarm-bot,immediatelyrelevantto ourresearch,is coordinatedmotionemergingwhentheconstituentindependently-controlledmodulescoordinatetheir actionsin choosinga commondirection of motion. Our focus is on how much locomotioncan be “pat-terned” in an aggregatedstructure.Regardlessof an environment(aquatic,terrestrialor aerial),locomotionis achieved by applyingforcesgeneratedby the rhythmic con-tractionof musclesattachedto limbs, wings,fins, etc.Typically, a locomotorygait isefficient whenall the involved musclescontractandextendwith the samefrequencyin differentphases.For example,Yim et al. [26] investigateda snake-like (serpentine)sinusoidgait, whereforwardmotionis essentiallyachievedby propagatingawaveform

Fig. 1: Sideview of theSnakebot. Fig. 2: Topview of theSnakebot.

travelling down thelengthof thechain.Tanev andhiscolleagues[22,21] demonstratedemergenceof side-windinglocomotionwith superiorspeedcharacteristicsfor thegivenmorphologyaswell asadaptabilityto challengingterrainandpartialdamage.

3 Spatiotemporal Coordination of Actuators

Snakebot is simulatedasa setof identical sphericalmorphologicalsegments,linkedtogethervia universaljoints.All joints featureidenticalanglelimits, andeachjoint hastwo attachedactuators.In the initial standstillpositionof Snakebot,the rotationaxesof the actuatorsareorientedvertically (vertical actuator)andhorizontally(horizontalactuator).Theseactuatorsperformrotationof the joint in the horizontalandverticalplanesrespectively. No anisotropicfriction betweenthe morphologicalsegmentsandthesurfaceis considered.OpenDynamicsEngine(ODE) waschosento provide a re-alistic simulationof themechanicsof Snakebot.Given this representation,the taskofdesigningthe fastestSnakebot locomotioncan be rephrasedas developing temporalpatternsof desiredturning anglesof horizontaland vertical actuatorsfor eachjoint,maximizingtheoverallspeed.Previousexperimentsof evolvablelocomotiongaitswithfitnessmeasuredaseithervelocity in any directionor velocity in forwarddirection[22]indicatedthatside-windinglocomotion— locomotionpredominantlyperpendiculartothe long axisof Snakebot(Figures1 and2) — providessuperiorspeedcharacteristicsfor theconsideredmorphology. Theactuatorsstates(horizontalandverticalturningan-gles)areconstrainedby the interactionsbetweensegmentsandtheterrain.Theactualturninganglesprovideanunderlyingtimeseriesfor our information-theoreticanalysis:horizontalturninganglesOQP

� SR andverticalturninganglesOUT

� QR at time # , where" is the

actuatorindex,$

is the numberof joints, V*WX"YW $ , and Z is the consideredtimeinterval, V*W[#\WXZ . Sincewe dealwith actualratherthan ideal turning angles,theunderlyingdynamicsin thephase-spacemayincludebothperiodicandchaoticorbits.

Weintendto estimate“irregularity” for eachof themultivariatetimeseriesOQP� R andOQT

� QR . Eachof thesetime series,henceforthdenotedfor generality OQ]

� QR , containsboth

spatialandtemporalpatterns,andminimizing theirregularityoverbothspaceandtimedimensionsshouldideally uncover the extent of spatiotemporalcoordinationamongactuatorstates.

For any given actuator" , a simplecharacterisationof the “regularity” of the timeseriesOQ] R is providedby theauto-correlationfunction.However, theauto-correlationis limited to measuringonly lineardependencies.We considerinsteada moregeneralapproach.Oneclassicalmeasureis theKolmogorov-Sinai(KS) entropy, alsoknown asmetric entropy [13]: it is a measurefor the rateat which informationaboutthe stateof the systemis lost in the courseof time. In other words, it is an entropy per unittime, anentropy rateor entropy density. Supposethat the ^ � dimensionalphasespaceis partitionedinto boxesof size_a` . Let ; � bdc c c � eSfhg bethejoint probabilitythatatrajectoryis in box "Qi at time j , in box " � at time kl# , M M M , andin box " `�m � at time

^ � V � kl# , wherekl# is the time interval betweenmeasurementson thestateof thesystem(in our case,we mayassumekn# � V , andomit thelimit kl#po�j in thefollowing definitions).TheKS entropy is definedby� ��� I q rsut i I q r` twv V^xkn# �� bdc c c � eUfhg ;

� bdc c c � eSfhg I yz; � bdc c c � eSfhg (1)

andmoreprecisely, asa supremumof � on all possiblepartitions.This definitionhasbeengeneralizedto theorder-{ Renyi entropies�-| [18]:� | �}� I q r~ t i I q�rs�t i I q r` tDv V^xkl# { � V ��I y �� bdc c c � eUfhg ; |

� b c c c � eSfhg M (2)

It is well-known that � � j in anorderedsystem,� is infinite in arandomsystem,and� is a positive constantin a deterministicchaoticsystem.Grassberger andProcaccia[10] consideredthe correlationentropy � � in particular, and capitalizedon the fact����� � in establishinga sufficient condition for chaos� �*� j . Their algorithmestimatestheentropy rate � � for a univariatetime series.For our analysiswe needtointroducea spatialdimensionacrossmultiple Snakebot’s actuators.An estimateof thespatiotemporalentropy densitycanbeobtainedas� ��� I q r`B� tDv I q r`Q� twv V^2� V^ ���� `4� ? `d�u� �

Q�p ^2� ^ �S� I y � Q�p ^2� ^ �U� (3)

where�p ^ � ^ � are“patterns”of spatialsize ^ � andtime length ^ [2]. Our objective,

anestimateof spatiotemporalgeneralizedcorrelationentropy, canbeobtainedas� � ��� I q r`B� twv I q r`d� twv V^2� V^ I y ���� `4� ? `d��� �� Q�p ^2� ^ �S� M (4)

In achieving thisobjective,wefollow Grassberger-Procacciamethod[10] of computingcorrelationintegrals,but usethemultivariatetime serieswith

$actuators(joints) andZ timestepsin thefollowing approximation:

� ` � ` �� �$ Z _ ��� I y�� ` � ` � &$ Z _ �� `4� � `d� % � � &$ Z _ � + I y�� ` � ` � �$ Z _ �� � `4� % � ��`Q� �$ Z _ � (5)

wherecorrelationintegralsaregeneralizedas

� `4��`Q� �$ Z _ ��� V Z � V � Z �$�� V ��$��� � � � �� � � ��� � � �

�� � � ��� _ ���Q�*�� �\���� ��� M (6)

Here � is the Heaviside function (equalto j for negative argumentand V otherwise),and the vectors

���� and� �� containelementsof the observed time series OQ]

� R for

eachactuator(the spatialdimension),“converting” or “reconstructing”the dynami-cal information in two-dimensionaldatato information in the ^ � ^ -dimensionalem-beddingspace[20]. More precisely, we usespatiotemporaldelayvectors

� ¡ �8&¢z ¡ ¢  u£=¤¡ ¢  u£N¥¡ MBM4M ¢  u£§¦�¨u©p¤¡ �, whoseelementsaretime-delayvectors

¢z ¡ �ª ]�< ]

�< % � ]

�< % � M4M4M ]

�< % `Q��m � � , andthespatialindex " is fixed[14]. Thenorm

�d���� �-� �� � is thedistancebetweenthevectorsin the ^2�u^ -dimensionalspace,e.g.,themaximumnorm:�Q�*�� �\���� �«� `4��m �rl¬U­® � i `d� m �rl¬U­¯ � i ] � % ®

� % ¯ � ]� % ®� % ¯ �

Putsimply, correlationintegral � ` � ` � �$ Z _ � computesthefractionof pairsof vectorsin the ^ � ^ -dimensionalembeddingspacethatareseparatedby a distancelessthanorequalto _ . In order to eliminateauto-correlationeffects, the vectorsin equation(6)shouldbe chosento satisfy G � ��° G �²± , for an integer ± , and G � �´³ G �[µ , for aninteger µ , in orderto excludeauto-correlationeffectsamongtemporallyclosedelaysor closelycoupledsegments[23]. The standardtemporaldelayreconstruction[20] isrecoveredby setting2� � V [4].

Thecorrelationentropy � � (thegeneralizedentropy rate)measurestheirregularityor unpredictabilityof the system.A complementaryquantity is the excessentropy �[8, 5] — it may be viewed asa measureof the apparentmemoryor structurein thesystem.The generalizedexcessentropy � � is definedby consideringhow the finite-template(finite-delayandfinite-extent)entropy rateestimates� `4��`Q�� �$ Z _ � (equation(5)), converge to their asymptoticvalues� � (equation(4)). It is estimatedfor a fixedspatialextent ¶\� andagiventime range¶ as:

� � ¶/� ¶ $ Z _ �E��· ��` � � �

· ��` � � �

� `4��`Q�� �$ Z _ �.� � � � M (7)

For regularlocomotiontheasymptoticvaluesshouldbezero(while non-zeroentropieswould indicatenon-periodicity, i.e. deterministicchaos).It wasshown that theexcessentropy alsomeasurestheamountof historicalinformationstoredin thepresentthatiscommunicatedto the future [5,8]. In otherwords,it canberepresentedasasymptoticmutualinformationbetweentwo adjacent � ^ -dimensionalhalf-planesI q r`4� ? `Q� twv '(&�*�©«¦�¸ ) ���¹ �E�I q�r` � ? ` � twv '(U&¢.�©«¦ ¸ ¢z� £>¤©«¦ ¸ ¢.� £N¥©º¦ ¸ M4MBM ¢.� £N¦ ¨ ©p¤©«¦ ¸ � ) &¢��¹ ¢ � £>¤¹ ¢ � £N¥¹ MBM4M ¢ � £§¦ ¨ ©p¤¹ �S�where

¢  ¡ �´ ]�< ]�< % � ]

�< % � M4M4M ]

�< % `Q��m � � . This alternative representationestablishes

that the proposedmeasuremay estimateinformation transferwithin the spaceof ac-tuators:themoreinformationbetweenthespatiotemporalpastandthespatiotemporalfutureis transferred,themorecoordinationis achieved.If � �»³ in thelastexpression,thetransferis purelybetweenthetemporalpastandthetemporalfuture.Otherwise,if�Y¼��³ , weareconcernedwith how muchinformationcontainedin thepastof onegroupof actuatorsis injectedinto thefutureof anothergroupof actuators.

Whendealingwith non-zeroentropy rates� � , onemay considerrelativeexcessentropy: ½ � ¶ � ¶ $ Z _ �E��· ��

`4� � �· ��`d� � �

� `B��`d�� �$ Z _ �.� � �� � +-¾ M (8)

where¾

is asmallconstant(e.g.,¾«� j M jÀ¿ ), balancingtherelativeexcessentropy

½ � forvery small entropy rates� � . The relative excessentropy

½ � attemptsto “reward” thestructure(coupling)in thelocomotionand“penalise”its non-regularity.

4 Results

In this sectionwe presentexperimentalresultsof Snakebot’s evolution basedon esti-matesof theexcessentropy � � (equation(7)) andtherelativeexcessentropy

½ � (equa-tion (8)). TheGeneticProgramming(GP)techniquesemployedin theevolutionarede-scribedelsewhere[22,21]. In particular, thegenotypeis associatedwith two algebraicexpressions,whichrepresentthetemporalpatternsof desiredturninganglesof boththehorizontalandverticalactuatorsof eachmorphologicalsegment.Becauselocomotiongaits,by definition,areperiodical,we includetheperiodicfunctions ÁSq y and ÂuJ(Á in thefunction setof GP in additionto the basicalgebraicfunctions.The selectionis basedon a binary tournamentwith selectionratio of j M V andreproductionratio of j M Ã . Themutationoperatoris therandomsubtreemutationwith ratio of j M jÀV . Snakebotsevolvewithin a populationof ÄLjÀj individuals,andthebestperformersareselectedaccordingto theexcessentropy values,over anumberof generations.

Figures3 and4 contrast(for verticalactuators)actualanglesusedby thefirst off-springandthefinal generation.Similarly, Figures5 and6 contrastthespatiotemporalcorrelationentropiesproducedby the first offspring and the evolved solution. It canbeeasilyobservedthatmoreregularangledynamicsof theevolvedsolutionmanifestsitself asmoresignificantexcessentropy. Figures7 and8 show typical fitnessgrowthtowardshigher excessentropiesestimatedas � � (equation(7)) and the relative ex-cessentropies

½ � (equation(8)), for two differentexperiments.It shouldbenotedthattherearewell-coordinatedSnakebotswhicharemoving notasquickly astheSnakebotsevolvedaccordingto thedirectvelocity-basedmeasure,i.e. thesetof fastsolutionsiscontainedwithin the setof well-coordinatedsolutions.This meansthat the obtainedapproximationof thedirectfitnessfunctionby theinformation-theoreticselectionpres-suretowardsregularity is soundbut not complete.

In certaincircumstances,a fitnessfunction rewardingcoordinationmay be moresuitablethan a direct velocity-basedmeasure:a Snakebot trappedby obstaclesmayneedto employ a locomotiongait with highly coordinatedactuatorsbut near-zeroabso-lutevelocity. In fact,theobtainedsolutionsexhibit reasonablerobustnessto challengingterrains,trading-off somevelocity for resilienceto obstacles.In particular, theevolvedSnakebot shown in Figure 9 is able to traverseraggedterrainswith obstaclesthreetimesashigh asthesegmentdiameter, move througha narrow corridor (only twice aswideasthesegmentdiameter),andovercomevariousextendedbarriers.In addition,theSnakebotis robustto failuresof individual segments:e.g.,it is ableto moveevenwhenevery third segmentis completelyincapacitated,albeit with only a half of the normal

2040

6080

100120

140160

180

2

4

6

8

10

12

14

−40

−30

−20

−10

0

10

20

30

40

TimeActuators

Ver

tical

Ang

le

Fig. 3: First offspring:actuatorangles.

2040

6080

100120

140160

180

2

4

6

8

10

12

14

−50

0

50

TimeActuators

Ver

tical

Ang

le

Fig. 4: Evolvedsolution:actuatorangles.

2 4 6 8 10 20 300

0.5

1

1.5

2

2.5

3

3.5

Time DelaysActuators

Cor

rela

tion

Ent

ropy

Fig. 5: First offspring:correlationentropy.

2 4 6 8 1020

30

0

0.5

1

1.5

2

2.5

Time DelaysActuators

Cor

rela

tion

Ent

ropy

Fig. 6: Evolvedsolution:correlationentropy.

speed.Interestinglyenough,the relative excessentropy is increasedin partially dam-agedSnakebots,astheamountof transferredinformationin thecoupledlocomotionhasto increase.Moreover, thereappearsto bea strongcorrelationbetweenthenumberofdamaged(evenlyspread)segmentsÅ andtheresultingrelativeexcessentropy

½ ��ºÆÈÇ Å ,wherethe coefficient Ç of the linear fit is approximatelyequalto the relative excessentropy of a non-damagedSnakebot

½ i � . This observationopensa way for Snakebot’sself-diagnosticsandadaptation:the run-timevalueof

½ � may identify the numberofdamagedsegments,enablingamoreappropriateresponse.

5 Conclusions

We modelleda specificsteptowardsa theoryof information-driven evolutionaryde-sign, using information-theoreticmeasuresof spatiotemporalcoordinationin a mod-

14

16

18

20

22

24

0 5 10 15 20 25 30 35 40

Exc

ess

Ent

ropy

Generation

Fig. 7: Snakebot fitnessover time: the bestperformer in eachgeneration,using excessentropy.

5

10

15

20

25

30

0 10 20 30 40 50

Rel

ativ

e E

xces

s E

ntro

pyÉ

Generation

Fig. 8: Snakebot fitnessover time: the bestperformerin eachgeneration,using relativeexcessentropy.

ular robotic system(Snakebot).Thesemeasuresestimatethe generalizedcorrelationentropies� � computedover a time seriesof actuators’statesandthe spatiotemporalexcessentropies� � . As expected,increasedcoordinationof actuatorsis achieved byagentswith fasterlocomotion.However, thesetof fastsolutionsis asubsetof thesetofwell-coordinatedsolutions.A morepreciseapproximationof fastlocomotionis a sub-jectof futurework. In parallel,weareinvestigatingothertasksadaptationto whichmayrequirea high degreeof actuators’coordination: e.g.,ruggedterraintraversal,energy-efficient locomotion,etc.Both directionsessentiallyrequireidentificationof channelsthroughwhich the informationtransferamongsystem’s componentsis optimized.Webelieve that developmentof adequateinformation-theoreticcriteria, suchasthe mea-sureof spatiotemporalcoordinationof distributedactuators,will contribute to designguidelinesfor co-evolving multi-agentsystems.

Acknowledgements. Thethird authorwassupportedin partby theNationalInsti-tuteof InformationandCommunicationsTechnologyof Japan.

References

1. G.Baldassarre,D. Parisi,andS.Nolfi. Measuringcoordinationasentropy decreasein groupsof linkedsimulatedrobots.Preprint,2005.

2. G. Boffetta,M. Cencini,M. Falcioni,andA. Vulpiani. Predictability:a way to characterizecomplexity. PhysicsReports, 356:367–474,2002.

3. H. Bojinov, A. Casal,andT. Hogg. Multiagentcontrolof self-reconfigurablerobots.Artifi-cial Intelligence, 142:99–120,2002.

4. R. Carretero-Gonzalez,S. Ê rstavik, andJ.Stark. Quasidiagonalapproachto theestimationof lyapunov spectrafor spatiotemporalsystemsfrom multivariatetime series. PhysRev EStatPhysPlasmasFluids RelatInterdiscipTopics, 62(5)Pt A:6429–6439,2000.

5. J.P. CrutchfieldandD.P. Feldman.Regularitiesunseen,randomnessobserved:Theentropyconvergencehierarchy. Chaos, 15:25–54,2003.

6. R. Der, U. Steinmetz,andF. Pasemann.Homeokinesis- a new principle to backup evolu-tion with learning. ConcurrentSystemsEngineeringSeries, 55: Computat.IntelligenceforModelling,Control,andAutomation:43–47,1999.

Fig. 9: Snakebotnegotiatinga terrainwith obstacles.

7. M. Dorigo. Swarm-bot:An experimentin swarmrobotics.In P. ArabshahiandA. Martinoli,editors,Proceedingsof SIS2005– 2005IEEE SwarmIntelligenceSymposium, pages192–200.IEEEPress,2005.

8. D.P. FeldmanandJ.P. Crutchfield. Structuralinformationin two-dimensionalpatterns:En-tropy convergenceandexcessentropy. PhysicalReview E, 67,051104,2003.

9. M. Foreman,M. Prokopenko, andP. Wang. Phasetransitionsin self-organisingsensornet-works. In W. Banzhaf,T. Christaller, P. Dittrich, J.T. KIm, andJ.Ziegler, editors,Advancesin Artificial Life - Proceedingsof the 7th EuropeanConferenceon Artificial Life (ECAL),volume2801of LectureNotesin Artificial Intelligence. SpringerVerlag,2003.

10. P. GrassbergerandI. Procaccia.Estimationof thekolmogorov entropy from achaoticsignal.Phys.Review A, 28(4):2591,1983.

11. A. S. Klyubin, D. Polani,andC. L. Nehaniv. Organizationof the informationflow in theperception-actionloopof evolvedagents.In Proceedingsof 2004NASA/DoDConferenceonEvolvableHardware, pages177–180.IEEEComputerSociety, 2004.

12. A. S. Klyubin, D. Polani,andC. L. Nehaniv. All elsebeingequalbeempowered. In M.S.Capcarrere,A.A. Freitas,P.J. Bentley, C.G. Johnson,andJ. Timmis, editors,AdvancesinArtificial Life, 8thEuropeanConference, ECAL2005,Canterbury, UK, September5-9,2005,Proceedings, volume3630of LNCS, pages744–753.Springer, 2005.

13. A.N. Kolmogorov. Entropy perunit time asa metric invariantof automorphisms.DokladyAkademiiNaukSSSR, 124:754–755,1959.

14. M. Prokopenko, G. Gerasimov, andI. Tanev. Measuringspatiotemporalcoordinationin amodularroboticsystem.In L.M. Rocha,M. Bedau,D. Floreano,R. Goldstone,A. Vespig-nani,andL. Yaeger, editors,Proceedingsof Artificial Life X. in press,2006.

15. M. Prokopenko,P. Wang,M. Foreman,P. Valencia,D. Price,andG.Poulton.Onconnectivityof reconfigurableimpactnetworksin agelessaerospacevehicles.RoboticsandAutonomousSystems, 53:36–58,2005.

16. M. Prokopenko, P. Wang, and D. Price. Complexity metrics for self-monitoringimpactsensingnetworks. In J. Lohn, D. Gwaltney, G. Hornby, R. Zebulum, D. Keymeulen,andA. Stoica,editors,Proceedingsof 2005NASA/DoDConferenceonEvolvableHardware(EH-05), pages239–246.IEEEComputerSociety, 2005.

17. M. Prokopenko, P. Wang,P. Valencia,D. Price,M. Foreman,andA. Farmer. Self-organizinghierarchiesin sensorandcommunicationnetworks. Artificial Life, 11:407–426,2005.

18. A. Renyi. Probability theory. North-Holland,1970.19. R.V. Sole andS. Valverde. Informationtheoryof complex networks:On evolution andar-

chitecturalconstraints.In E. Ben-Naim,H. Frauenfelder, andZ. Toroczkai,editors,ComplexNetworks, volume650of LectureNotesin Physics, pages189–210.Springer-Verlag,2004.

20. F. Takens. Detectingstrangeattractorsin turbulence. Dynamicalsystemsand turbulence,898:366,1981.

21. I. Tanev. Learnedmutationstrategiesin geneticprogrammingfor evolution andadaptationof simulatedsnakebot.GeneticEvolutionaryComputation- Gecco2005,Proc., pages687–694,2005.

22. I. Tanev, T. Ray, andA. Buller. Automatedevolutionarydesign,robustness,andadaptationof sidewinding locomotionof asimulatedsnake-likerobot. IEEETransactionsOnRobotics,21:632–645,2005.

23. J. Theiler. Spuriousdimensionfrom correlationalgorithmsappliedto limited time-seriesdata.PhysicalReview A, 34(3):2427–2432,1986.

24. P. WangandM. Prokopenko. Evolvablerecovery membranesin self-monitoringaerospacevehicles. In S. Schaal,A. Ijspeert,A. Billard, S. Vijayakumar, J. Hallam,andJ.-A. Meyer,editors,From Animalsto AnimatsVIII - Proceedingsof the 8th InternationalConferenceon theSimulationof AdaptiveBehaviour, LosAngeles, pages509–518.Cambridge,MA: ABradfordBook/MIT Press,2004.

25. A. Wuensche.Classifyingcellular automataautomatically:Finding gliders,filtering, andrelatingspace-timepatterns,attractorbasins,andthez parameter. Complexity, 4(3):47–66,1999.

26. M. Yim, K. Roufas,D. Duff, Y. Zhang,C. Eldershaw, andS. Homans. Modular reconfig-urablerobotsin spaceapplications.AutonomousRobots, 14:225–237,2003.