Individual and Cooperative Behaviors Representation

Based on Petri Nets

Nuno Pedro Pedrosa Rodrigues

Dissertation to obtain the master degree in Electrical and

Computer Engineering

Jury

President: Professor Carlos Jorge Ferreira Silvestre

Supervisor: Professor Pedro Manuel Urbano Almeida Lima

Vogal: Professor Carlos Baptista Cardeira

December, 2010

I

Acknowledgements

The moments that marked my academic journey are many and the number of people that I shared

them with is even greater. To them I can only express my profound gratitude for all the good memories

I take with me as I leave IST.

To my family, my parents and my sister for all the love and support that they have given me, without it I

probably wouldn’t have been able to deal with all the obstacles and challenges that life throws at us.

There were moments that I wasn’t the easiest person to get along with, especially, when the work

seamed never ending and time seemed nonexistent.

To my grandparents, that, not having had the opportunity to study themselves, always had a lot of

pride in their grandson and were always ready to give their support and help in any way they could.

To my childhood friend, Marques, that has also been a constant companion during my academic

formation. Many were the adventures that we lived together since we left our hometown and came to

Lisbon to study. All the best, maybe we will work again together in the future.

To all the friends that I made in Lisbon and all the friends that I left in Figueira da Foz, that were

always a refuge from all the craziness of the big city.

To Professor Pedro Lima, that inducted me in this wonderful world of robotic soccer and was my

supervisor during the completion of this thesis. A special thanks for all the support, availability and

above all the patience and understanding in allowing me the time to deal with personal issues outside

the scope of work.

Finally, to the entire SocRob team for all the help and companionship of these last couple of years. A

special acknowledgement to Marco Barbosa, João Estilita, João Messias and Aamir Ahmad, for all the

moments lived in Castelo Branco and in the RoboCup. There were some stressful moments,

especially in between games, while trying to get the robots ready for their next game but it was a fun

experience in which I learned a lot. Thank you all.

II

Resumo

Esta dissertação de mestrado teve como objectivo desenvolver, testar e analisar comportamentos

individuais e relacionais no âmbito do projecto SocRob. Estes comportamentos são modelados

usando Redes de Petri onde os lugares representam papéis, comportamentos, acções e predicados.

Estes ultimos compõem um modelo lógico do mundo através de mensagens trocadas entre os robôs

e através de informação obtida por parte dos sensores de cada robô. Os eventos são representados

por transições.

Foram introduzidos comportamentos individuais para situações de jogo que anteriormente não eram

contempladas, nomeadamente comportamentos defensivos em situação de falta, situação de “bola no

chão” e procura da posição da bola. Todos os outros comportamentos individuais foram optimizados

de forma a garantir um maior sucesso na sua execução e para garantir o comprimento das regras. Ao

nivel dos comportamentos relacionais, nomeadamente numa situação de passe dinâmico, foi

introduzido um novo modelo de compromisso mais generalista, permitindo o seu uso em mais

situações do que as apresentadas neste trabalho (passe curto e passe longo).

Os comportamentos criados podem ser compostos com modelos que representam as acções e o

ambiente, obtendo-se uma rede final sobre a qual é efectuada uma análise quantitativa fazendo uso

das Cadeias de Markov. Os modelos do ambiente podem incluir eventos controláveis por parte dos

nossos agentes mas também eventos não controláveis que modelam o impacto de outros fenómenos

físicos ou outros agentes.

Todos os comportamentos foram testados no simulador Webots e nos robôs reais.

Palavras Chave: Redes de Petri, Futebol Robótico, Comportamentos Relacionais, Análise

Quantitativa.

III

Abstract

The objective of this thesis is to develop, test and analyse individual and cooperative behaviors within

the framework of the SocRob project. These behaviors are modelled using Petri Nets where the

places represent roles, behaviors, actions or predicates. The latter compose a world model using

information collected by each robot sensors or from messages received from other robots. Events are

represented by transitions.

Individual behaviors were introduced for situations that weren’t contemplated before, such as

defensive behavior when the game is in a foul situation, dropped ball situation and search for the ball.

Other behaviors were also optimized to increase the success rate when executed and to keep up with

changes in the RoboCup Middle Size League rules. Regarding cooperative behaviors a new model for

commitment was introduced that can be used in more situations than the ones described in this work

(shortpass, longpass).

Every behavior developed can be composed with models of the actions and the environment in order

to obtain a single model in which a quantitative analysis can be performed by using Markov Chains.

Environment models use not only events controllable by the robots, but also uncontrollable events that

represent the world physics and/or the impact of other agents in the world. This is a simplified

representation of the surrounding world, nevertheless important conclusions can be obtained

concerning quantitative performance of the robot team.

All the developed behaviors were tested in the simulator (Webots) and the real robots.

Keywords: Petri Nets, Robotic Soccer, Individual Behavior, Relational Behaviors, Quantitative

Analysis, Commitment Establishment

IV

Index

1 Introduction ........................................................................................................................... 1

1.1 Motivation ..................................................................................................................................... 1 1.2 Problem Specification ................................................................................................................... 1 1.3 Thesis Outline ............................................................................................................................... 2

2. Theoretical Concepts ............................................................................................................ 3

2.1 Petri Nets ....................................................................................................................................... 3

2.1.1 Formal Definition ...................................................................................................... 3

2.1.2 Firing Rule................................................................................................................. 3

2.1.3 Generalized Stochastic Petri Nets ............................................................................. 4

2.1.4 Composition .............................................................................................................. 6

2.1.5 Properties ................................................................................................................... 6

2.3 Robotic Plans Based on Petri Nets ................................................................................................ 7 2.4 Theory of Joint Commitment ........................................................................................................ 8

2.4.1 Formal Definition ...................................................................................................... 8

2.5 Relational Behaviors ..................................................................................................................... 9

2.5.1 Commitment Establishment ...................................................................................... 9

2.5.2 Commitment Management ...................................................................................... 11

2.5.3 Communication ....................................................................................................... 12

2.5.4 Synchronization ....................................................................................................... 13

2.6 Environment and Action Models ................................................................................................ 13

3. Plans for Cooperative Tasks .............................................................................................. 17

3.1 MeRMaID ................................................................................................................................... 17 3.1.1 Middleware and Architecture .................................................................................. 17

3.1.2 Petri Nets Framework.............................................................................................. 18

3.2 Organizational Behaviors ............................................................................................................ 20 3.2.1 TeamOrganizer ........................................................................................................ 20

3.2.2 BehaviorCoordinator ............................................................................................... 23

3.3 Individual Behaviors ................................................................................................................... 28 3.4 Relational Behaviors ................................................................................................................... 36

3.4.1 Static Pass ................................................................................................................ 36

3.4.2 Dynamic Pass .......................................................................................................... 41

4 Experimental Platform and Methodology ........................................................................ 47

4.1 ISocRob Team ............................................................................................................................. 47 4.1.1 Robots...................................................................................................................... 47

4.1.2 Research Topics ....................................................................................................... 48

4.1.3 Webots Simulator .................................................................................................... 48

4.2 Experimental Methodology......................................................................................................... 48 4.2.1 Tests in the Simulator and Real Robots ................................................................... 49

V

4.2.2 Analysis Performed ................................................................................................. 49

5 Results and Analysis ............................................................................................................ 57

5.1 Webots Simulation and Real Robots ........................................................................................... 57 5.2 Theoretical Analysis .................................................................................................................... 58 5.3 Conclusions ................................................................................................................................. 60

6 Conclusion and Future Work ............................................................................................. 61

Bibliography ........................................................................................................................... 63

Appendix A – Petri Nets Developed ...................................................................................... 65

A.1- TacticBase ................................................................................................................................. 65 A.2- RoleAttacker ............................................................................................................................. 66 A.3- RoleDefender ............................................................................................................................ 67 A.4- RoleSupporter ........................................................................................................................... 68 A.5- Macro CommitmentEstablishmentDynamicPassAttacker ........................................................ 69 A.6- Macro CommitmentEstablishmentDynamicPassSupporter ...................................................... 70 A.7- Macro FindFoulPosition ........................................................................................................... 71 A.8- ReceiveFoul .............................................................................................................................. 72 A.9- TakeFoul ................................................................................................................................... 73 A.10- BehaviorBaseAttack ............................................................................................................... 74 A.11- BehaviorBaseDefend .............................................................................................................. 75 A.12- BehaviorBaseSupport ............................................................................................................. 75 A.13- BehaviorDynamicPassReceiver .............................................................................................. 76 A.14- BehaviorDynamicPassTaker ................................................................................................... 77 A.15- BehaviorFindBallInCorner ..................................................................................................... 77 A.16- BehaviorFoulDefend ............................................................................................................... 78 A.17- BehaviorFoulReceiver ............................................................................................................ 79 A.18- BehaviorFoulTaker ................................................................................................................. 79 A.19- BehaviorFoulTakerAlone ........................................................................................................ 80 A.20- BehaviorPenaltyTaker ............................................................................................................. 81

Appendix B – Models used in analysis ................................................................................. 83

B.1- Environment Models ................................................................................................................. 83

B.1.1- BallMovement ....................................................................................................... 83

B.1.2- BallPosition ........................................................................................................... 83

B.1.3- DynamicPassConditionsMet ................................................................................. 84

B.1.4- DynamicPassEnded ............................................................................................... 84

B.1.5- HasBall .................................................................................................................. 84

B.1.6- PartnerReady2Receive .......................................................................................... 85

B.1.7- PassDone ............................................................................................................... 85

B.1.8- R1CloseToBall ...................................................................................................... 85

B.1.9- R1HasBall ............................................................................................................. 85

B.1.10- R1IsTurnedToPartner .......................................................................................... 85

B.1.11- R1Position ........................................................................................................... 85

B.1.12- R2CloseToBall .................................................................................................... 86

B.1.13- R2HasBall ........................................................................................................... 86

VI

B.1.14- R2IsAtReceivePosition ....................................................................................... 86

B.1.15- R2Position ........................................................................................................... 86

B.1.16- StopDynamicPass ................................................................................................ 87

B.2- Action Models ........................................................................................................................... 87

B.2.1- R1CatchBall .......................................................................................................... 87

B.2.2- R1Kick .................................................................................................................. 87

B.2.3- R1MoveAwayFromBall ........................................................................................ 88

B.2.4- R1TurnToPartner ................................................................................................... 88

B.2.5- R2CatchBall .......................................................................................................... 88

B.2.6- R2Move2ReceivePosition ..................................................................................... 89

B.2.7- SendDynamicPassEnded ....................................................................................... 89

B.2.8- SendPassDone ....................................................................................................... 89

B.2.9- SendReady2Receive .............................................................................................. 89

B.2.10- RXMove2Ball ..................................................................................................... 90

VII

Acronyms list

BC – BehaviorCordinator

BE - BehaviorExecutor

ETPN - Exponential Timed Petri Nets

FIFA - Fédération Internationale de Football Association

GSPN - Generalized Stochastic Petri Nets

MeRMaID - Multiple-Robot Middleware for Intelligent Decision-making

MSL - Middle size league

PNP - Petri Nets Plans

SocRob - Society of Robots /soccer robots

SPN - Stochastic Petri Nets

TO - Team Organizer

VIII

List of Figures

Figure 1: PN marking before T0 fires Figure 2: PN marking after

after T0 fires .............................................................................................................................. 3

Figure 3: Generalised stochastic PN with Immediate (TI1, TI2, TI3) .............................. 5

Figure 4: Robotic plan ............................................................................................................. 8

Figure 6: Commitment management .................................................................................. 12

Figure 7: Synchronization example .................................................................................... 13

Figure 8: Petri Net Model of a free rolling ball................................................................... 14

Figure 9: General action model ........................................................................................... 15

Figure 10: Architecture design ............................................................................................. 17

Figure 11: Robot position when defending against an opponent free or goal kick ..... 32

Figure 12: Robot position when defending against a corner kick for the opponent .... 33

Figure 13: Robot position when defending against a throw in ....................................... 33

Figure 14: of a behavior that moves the robot to a position ........................................... 36

Figure 15: Dynamic pass in the RoleAttacker ................................................................... 42

Figure 16: Dynamic pass in the RoleSupporter ................................................................ 44

Figure 17: Robot used in the project .................................................................................. 47

Figure 18: Webots field ......................................................................................................... 48

Figure 19: Behavior of an attacker ...................................................................................... 50

Figure 20: Ball Position ......................................................................................................... 51

Figure 21: Robot position ..................................................................................................... 51

Figure 22: HasBall model ..................................................................................................... 51

Figure 23: ShootOpportunity model .................................................................................... 51

Figure 24: InterceptBall action model ................................................................................. 52

Figure 25: Drible2Goal action model .................................................................................. 52

Figure 26: Drible2Score action model ................................................................................ 53

Figure 27: Kick action model ............................................................................................... 53

Figure 28: Dynamic Pass on the BehaviorCoordinator level .......................................... 54

Figure 29: Field Positions where the robots and the ball can be ................................... 54

Figure 30: Results for Case 1 when a big amount of time passes ................................ 59

Figure 31: Results for Case 1 when a small amount of time passes ............................ 59

Figure 32: probability for the receiver to catch the ball in a short and long pass ........ 60

IX

List of Tables

Table 1: TacticBase ............................................................................................................... 23

Table 2: RoleAttacker ............................................................................................................ 24

Table 3: RoleDefender .......................................................................................................... 26

Table 4: RoleSupporter ......................................................................................................... 27

Table 5: FindFoulPosition ..................................................................................................... 28

Table 6: BehaviorBaseAttack ............................................................................................... 29

Table 7: BehaviorBaseDefend ............................................................................................. 30

Table 8: BehaviorBaseSupport ............................................................................................ 31

Table 9: BehaviorFoulDefend .............................................................................................. 33

Table 10: BehaviorFoulTakerAlone ..................................................................................... 34

Table 11: BehaviorPenaltyTaker .......................................................................................... 35

Table 12: BehaviorFindBallInCorner ................................................................................... 35

Table 13: TakeFoul ................................................................................................................ 37

Table 14: ReceiveFoul .......................................................................................................... 38

Table 15: BehaviorFoulReceiver ......................................................................................... 40

Table 16: Short Pass ............................................................................................................. 41

Table 17: Long Pass .............................................................................................................. 42

Table 18: BehaviorDynamicPassReceiver......................................................................... 45

Table 19: BehaviorDynamicPassTaker............................................................................... 45

X

1

1 Introduction

1.1 Motivation

In 1997 an international robotics competition, called RoboCup, was founded with the aim to develop

autonomous soccer robots in order to promote research and education in the field of artificial

intelligence. The name RoboCup is an abbreviation of the competition's full name, "Robot Soccer

World Cup", but there are many other stages of the competition such as "Search and Rescue" and

"Robot Dancing".

The main goal of the project is to develop, by mid-21st century, a team of fully autonomous

humanoid soccer robots that can win against the winner of the most recent World Cup, while

complying with the official rules of FIFA. For this purpose, every year, the RoboCup Federation

changes the rules in order to bring them closer and closer to the FIFA rules and instigate the

continuous progress of the technology.

The SocRob project [16], an acronym for Society of Robots or Soccer Robots was created in 1997

at the Intelligent Systems Laboratory of the Institute for Systems and Robotics at the pole of the

Instituto Superior Técnico (ISR / IST),Technical University of Lisbon, with its primary research focus

on applications involving cooperative robotics and multi-agent systems. The ISocRob team is the

project's case study on soccer robots, and has regularly participated in RoboCup Middle-Size League

since 1998.

The aim of this thesis is to develop, test and analyse individual and cooperative behaviors based

on Petri Nets. These behaviors should be implemented in the soccer robots of the ISocrRob team and

should respond to as many game situations as possible and act accordingly to accomplish the main

objective of scoring goals while trying to prevent the opposite team from scoring.

In all situations, MSL rules ([10]) have to be met.

1.2 Problem Specification

This thesis extends previous work ([2],[3],[4]) accomplished within ISR/IST SocRob project. However,

the response to some of the game situations offered a small range of options and several game

situations were not yet taken into account such as defensive behaviors for when the game is in a foul

situation, be it for us or the other team, dropped ball situation and searching for the ball in foul

situations and even during the game play. Other individual behaviors needed to be optimized and

changed in order to accomplish a higher success rate when performed by the real robots and to keep

up with changes in the rules.

Regarding cooperative behaviors, the commitment model for the static pass (foul situation) can be

kept with minimal changes, however, the model for performing the dynamic pass needed to be

completely redone in order to be more effective and have the potential to be used in more situations

than the ones described in this work. Also, the old conditions used to perform a dynamic pass were

very restrictive so the pass was rarely performed. This was a problem due to the fact that when the

2

game is in a dropped ball situation the pass is mandatory. This called for a simple pass that, in a

dropped ball situation, could be performed with the maximum security (short pass). On the other hand,

there was also a need to have a flashier pass to be performed when a more restrictive set of

conditions occur.

Another important objective was to make use of the framework in [14] and apply it in real situations

that could occur in a competition setting. This lead to the creation of actions and environment models,

based in Petri Nets, that can be composed with the robot behavior in order to obtain a single model in

which a quantitative analysis can be performed using Markov Chains. Environment models model, not

only controllable events by our agents, but also uncontrollable events that represent the world physics

and/or the impact of other agents in the world. The environment models will prove to be important for a

priori verification and validation of behaviors. It further allows to determine undesired behaviours and

possible unexpected occurrences, providing means for simulation and decision making based on

prediction.

1.3 Thesis Outline

This thesis is organized in six chapters. Chapter 2 contains all the theoretical concepts needed to

tackle the objectives of this work, namely Petri Nets, including their formal definition, firing rule,

Stochastic Petri Nets, composition of Petri Nets to obtain a single model for analysis and finally the

different types of analysis. Other important concepts are mentioned, like Joint Commitment theory as

well as the methodology introduced in previous work ([2] , [3]) to achieve a commitment between two

robots in order to perform a joint task.

Chapter 3 gives a description of the middleware (MeRMaID) used in the SocRob and of how Petri

Nets are used in this context. Also, in this chapter, a detailed description of all the Petri Nets currently

in use on the project is given since, an high number of new behaviors were created and the ones that

came from previously work were completely redone in order to increase their success rate when

performed in the real robots and to keep up with changes in the rule book for RoboCup.

Chapter 4 gives a description of the IsocRob team. This includes information about the robots, their

hardware and software and the simulator Webots. The more relevant experiences performed in the

simulator and the real robots are enumerated, described and the objectives for each one given. Also, a

description of the game situations is given intended for analysis as well as a description of the process

of creating the environment and action models that will be composed with the behaviors.

Chapter 5 shows the results of the experiments performed in Chapter 4 for the simulation in the

Webots, real robots and for the quantitative analysis.

Finally, chapter 6 includes the conclusion and future work that can be performed related to this

theme.

Appendix A encompasses all the organizational, individual and cooperative behaviors as well as

the Petri Nets to perform a commitment for a dynamic pass.

Appendix B encompasses all the environment and action models needed for the analysis of the

short and long pass.

3

2. Theoretical Concepts

2.1 Petri Nets

Petri Nets were invented by Carl Adam Petri ([20]), in August 1939 when he was only 13, to describe

chemical processes. Twenty three years later, in 1962, he formally presented the concept of Petri Net

in his doctoral dissertation at the Technical University of Darmstadt, Germany. Tadeo Murata [8] gives

an introduction to the Petri nets by presenting them as a graphical and mathematical tool, which allows

to describe and study systems that are concurrent, asynchronous, distributed, parallel,

nondeterministic, and/or stochastic.

As a graphical tool it can be used similarly to block diagrams, flowcharts or state machines. They

are useful when one wants to describe or analyze the evolution of a dynamical system driven by

events and with a discrete state space. Throughout this dissertation this capability will be used in order

to describe the protocols used.

As a mathematical tool it can be used to extract a system of algebraic equations or other

mathematical models that allow the analysis of the system modelled by the Petri Net.

2.1.1 Formal Definition

A marked ordinary Petri Net is a five-tuple PN = (P, T, A, w, M0) [8], where:

• P = (p1, p2, … , pn) – represents a finite set of places;

• T = (t1, t2, … , tm) – represents a finite set of transitions. In general, m≠n.

• A ⊂ (P x T ) ∪ (T x P ) – represents the set of arc connections from place to transition and

transition to place respectively;

• w : A → N – weight function associated with each arc;

• M0 – initial marking of the PN.

2.1.2 Firing Rule

This rule is composed of the following three consecutive steps:

1. A transition is ready to fire, or it is active, if all its input places are marked with at least w(p, t)

tokens, where w (p, t) denotes the weight associated with the arc linking the place of entry to the

transition.

2. An active transition may or may not fire depending on whether the event associated with the

transition occurs or not.

3. If a transition t fires, it removes w(p, t) tokens from its input places, and adds w(t, p) tokens to

each output place of t.

Figure 1: PN marking before T0 fires Figure 2: PN marking after T0 fires

4

2.1.3 Generalized Stochastic Petri Nets

Classical Petri nets are useful for performing a qualitative or logical evaluation of the systems

properties, however, in order to perform a quantitative analysis it is necessary to introduce the concept

of time in the PN definition. For this purpose, Generalized Stochastic Petri Nets (GSPN) are adequate.

These transitions belong to two distinct classes:

Immediate transitions that fire as soon as they are active;

Timed transitions with a stochastic delay, which represents the time it takes to fire the transition

from the moment that it is active. This delay is associated with a random variable with exponential

distribution:

o 1

[ ]E X

;

o 2

1[ ]Var X

;

o t( ; ) ef x .

In cases where there are conflicts, i.e., there is more than an immediate transition ready to fire, the

Generalized Stochastic Petri Nets allows the use of Random Switches, where for each transition in

conflict a probability distribution is associated that determines which should fire first. These Random

Switches can be static or dynamic, depending on whether they depend on the PN marking or not.

In [9] a formal definition of the Generalized Stochastic Petri Nets is presented, here we find that a

GSPN is an eight-tuple (P, T, A, w, M0, R, S), where:

• P = (p1, p2, . . . , pn) - represents a finite set of places;

• T = (TE, TI) - represents a finite set of transitions, that can be immediate transition (TI) or

exponential transitions (TE);

• A ⊂ (P x T) ∪ (T x P) – represents the set of arc connections from place to transition and

transition to place respectively;

• w : A → N - weight function associated with each arc;

• M0 - initial marking of the PN places;

• R – function from the set of transitions TE to the set of real numbers, R(tEj) = λj, where λj is

called the firing rate of tEj ;

• S – set of Random Switches which associates probability distributions to subsets of conflicting

immediate transitions. These random switches can be static (invariant to the marking of the net) or

dynamic (dependent on the marking of the net).

As represented in Figure 3, immediate transitions are represented by a black line while the

exponential transitions are represented by a rectangle.

5

Figure 3: Generalised stochastic PN with Immediate (TI1, TI2, TI3)

and exponential transition (TE1, TE2).

Stochastic (exponential) transitions, once enabled, fire only when an exponentially distributed

time dj has elapsed. When two or more stochastic transitions are enabled, the firing probability

depends on the enabled transitions rate. If 1E ET t , . . . , t

n is the set of enabled stochastic

transitions for a given marking, the firing probability of each transition is given by

Consider the GSPN model depicted in Figure 3. In this example, tE1 and tE2 are exponentially

timed transitions while tI1, tI2 and tI3 are immediate transitions with associated weights. Initially tE1

and tE2 are enabled, since p1 has a token, and either one will fire after an exponentially distributed

time with rate λ1 and λ2 has elapsed, respectively. The firing probability of either transition is given by

The token flows from p1 to p2 and, since tI1 is an immediate transition, it will immediately flow from

p2 to p3, reaching marking M3 = [0, 0, 1]. In this marking tI2 and tI3 form a set of conflicting

transitions, whereas only one will fire, according to the following probabilities:

If tI3 fires, the marking remains the same, if tI2 is fires, the net returns to the initial marking.

6

The GSPN marking is a Markov process with a discrete state space given by the reachability graph

of the net for an initial marking ([8] , [9]). A Continuous Time Markov Chain (CTMC) and/or the

corresponding Embedded Markov Chain (EMC) can be obtained from the marking process, and both

the transition rate matrix (CTMC) and the transition probability matrix (EMC) can be computed by

using the firing rates of the exponential timed transitions and the probabilities associated with the

random switches. This allows us to perform quantitative analysis of the stationary and transient

properties of any given GSPN.

2.1.4 Composition

A major advantage of Petri nets is their modularity. The various models that constitute the entire model

can be adjusted separately and combined only when needed, for example, when trying to perform the

analysis of the full model. In [14] a thoroughly explanation can be found on the method of composing

the various models so only the more important considerations are mention here such as:

1. Every predicate place with the same label is the same place;

2. Every action place is a different place regardless of the label;

3. Every transition is different regardless of the label.

To perform the composition of all the models used in the quantitative analysis of this work the

algorithms described in [14] were used so the focus of this work is to create environment models and

action models based on that methodology described.

2.1.5 Properties

The type of analysis that can be undertaken using Generalized Stochastic Petri nets (GSPN) can be

divided into two major groups: qualitative and quantitative. In the first case, time is of no importance,

only the structure of Petri net is of interest, while in the latter case, stochastic time is used in the

analysis, providing information on the performance of the Petri net. Here, the GSPN marking is a

Markov process with a discrete state space given by the reachability graph of the net for an initial

marking.

Next, a list is given with the qualitative properties that are more relevant in the case study of soccer

robots. For a more extensive description on the properties of Petri nets one should consult [8].

Boundedness - A Petri net is k-bounded or simply bounded if the number of tokens in each

place is less than or equal to k for any marking from the initial marking, e.g. M (p) ≤ k for any

place p or any marking M ∈ R (M0), where R(M0) is the set of all the markings reachable from

the initial marking M0;

Safety - A Petri net is safe if it is 1-bounded;

Liveness and Deadlock – Given a Petri net with an initial marking M0, a transition tj is alive if,

for all the markings M(i), there is a sequence of transitions that fire starting in M (i) that causes

tj to fire. The Petri net is said to be live if all transitions are live. This means that a live Petri net

7

has no deadlocks, whatever the firing sequence. Therefore, a deadlock occurs when a

marking is reached where there aren’t any more transitions that can be enabled;

Reachability- determines if a given state is reachable. One way to examine whether a given

marking is reachable is through the reachability graph, which represents all markings

reachable from the initial marking.

Quantitative properties are related to the temporal transitions of the PN and with the net

performance over time. There are several properties that can be analyzed but they all fall into two

types of analysis: transient and steady state. Next, a description of these two types of quantitative

analysis is given:

Transient: properties are related to the evolution of the PN over time before a steady state can

be achieved. These include properties such as the average time it takes to reach a state with

a certain probability;

Steady state: properties in which enough time has passed to achieve a steady state. This

steady state can provide information about the probability of achieving a particular state, the

average time spent in a state, among others.

2.3 Robotic Plans Based on Petri Nets

In [12] is described a formalism for modeling robotic tasks in Petri Nets, which should consist of

different layers with different degrees of abstraction.

Predicate places are introduced, consisting of a logical evaluation, true or false. A predicate place

in a Petri net is marked with one token when the value of the logical evaluation of the predicate is true.

For example, the predicate SeeBall is marked only when the robot can see the ball and when the

robot can’t see the ball a new predicate place is introduced consisting of the negation of the predicate

represented by the predicate ¬ SeeBall. This is the lowest level of abstraction.

In order for each action to function properly, one must ensure that transition preconditions are met

as well as conditions for execution. For example, the action CatchBall has as precondition the

predicate SeeBall and as execution condition the predicate ¬HasBall. It is also necessary to define the

post-conditions that occur when an action ends. The action CatchBall has Hasball as post-condition

when it ends successfully and ¬SeeBall when it ends unsuccessfully..

Finally, the robotic plan consists of a network of actions, linked through their pre-conditions and

final conditions. For example, when the robot is in possession of the ball, in which case the action

CatchBall ends successfully, it must initiate action DribbleToGoal. The post-condition of the former,

HasBall, is the precondition of the latter. This is the highest level of abstraction.

As an example of a robotic plan, observe the Petri net depicted in Figure 4.

8

Figure 4: Robotic plan

2.4 Theory of Joint Commitment

In [1], Cohen and Levesque introduced the theory of joint commitment, where a formalism is defined

for relational behaviors in a team of multiple agents.

When a group of agents share the same environment it can be advantageous and even essential

to act jointly on that same environment. It is therefore necessary to introduce a set of formalities and

protocols, to ensure that throughout the conduct of each of the elements, all of them should always act

in accordance with each other and always in cooperation.

In this reference, an example is given of two drivers on the highway, in which one needs the other

to get to a certain place and where several undesirable situations can happen if a commitment

protocol is not met between them, such as what to do when they cease to be in visual range of each

other, or what is the destiny that each wants to achieve.

2.4.1 Formal Definition

In [1], three different types of objectives are defined:

Persistent Goal:

An agent has a persistent goal to achieve p relative to q if the following conditions are met:

• The agent believes that p is momentarily false;

• The agent wants p to become true;

• It is assumed as truth (and the agent knows this) that the second condition will persist until the

agent comes to believe that p is true, or that will never be true, or that q is false.

Low Priority Objective:

An agent has a goal of low priority concerning q in regards to a team that believes in p if the following

9

conditions are met:

• The agent has a standard objective in order to believe p, i.e., the agent still does not believe that p

is true but has p as a goal that will become true;

• The agent believes that p is true, will never be true, or is irrelevant (i.e., q is false), but has a goal

that says that the state of p is believed jointly by all team members.

Relational Persistent Goal:

A team of agents has a persistent relational goal in regards to q to achieve p if:

• All the agents believe that p is momentarily false;

• Everybody wants p to become true;

• Being true (and being common knowledge) that until everyone agrees that p is true or that p will

never be true, or that q is false, all agents will continue to believe that each one has a low priority in

regard to q.

Summarizing, the establishment of a commitment toward a goal begins with the acquisition of a

persistent goal by each individual agent involved. Nevertheless, this occurrence is not sufficient for the

commitment to be established. The mere existence of a joint persistent goal only ensures that each

robot will pursue its goal until it has reached it or until it realizes that the goal is unattainable, without

guaranteeing that if the goal becomes invalid the agent will inform its teammates. So, the need to

create another type of objective arises, one known as a low-priority objective. In this case the agents

only leave the commitment if they are all in agreement about the success or failure of the assignment,

pledging to inform the companions of the decision reached. This creates the concept of global

persistent goal.

2.5 Relational Behaviors

A relational behavior takes place when two or more robots act in cooperation when executing a

behavior that intends to achieve a common goal. The steps and protocols that are followed in the

beginning, during and at the end of a relational behaviour are presented below.

2.5.1 Commitment Establishment

As one can imagine, a key aspect in performing a relational behavior is the communication between

the robots. In [2] and [3] a communication protocol was defined where the robots are designated as R1

and R2, acronyms of robot1 and robot2.

The protocol comprises the following steps:

1. R1 sends a request to other robots of his team indicating the intention of holding a given

relational behavior and the need for a partner. The robot advances to a state where it awaits a

response from his teammates during a given time interval;

2. In the presence of a request for commitment, R2 considers if it has conditions to undertake the

10

commitment in question. If it can, R2 sends a message telling R1 that it is able to enter a commitment;

3. When the time interval mentioned in 1 expires, R1 checks for responses from his teammates

and if a response is found, R1 proceeds to a new state where he analyzes the responses. In case that

R1 didn’t receive answers, it ceases the attempt of establishing a commitment;

4. If R1 obtained valid responses from his teammates, R1 selects the most adequate partner for the

desired behavior and sends a message to confirm the selected partner (R2) and a message of

rejection to the outcasts. Then, R1 advances to the state in which it awaits for a message indicating

that R2 has established a commitment with success.

5. On the side of R2 teammate, one of two possible situations will occur:

• Receive rejection: In this case the robot will terminate the establishment of

commitment;

• Receive confirmation: In this situation, the robot sends a message of commited to the partner

(R1), indicating that it is committed to achieve the common goal.

6. R1 waits, during a fixed time interval, for the commitment message by R2. If this does not arrive

during this time interval, R1 cancels the commitment. On the other hand, if it receives the commited

message of committed by R2, R1 also sends a commited message to R2.

7. Finally, in the same way as R1, R2 also waits for a fixed time for the message committed from

R1. If it receives during that time interval, R2 advances to the state R1Commited otherwise terminates

the commitment, and sends the messages to R1 that the commitment wasn’t successful.

The timers mentioned in 3, 5 and 6 and 7 are used in order to avoid deadlock situations that might

arise. For example, if R2 is at point 5, waiting for Message Confirmation or rejection, and if the

communication support can no longer be usable, without the use of a timer, R2 would be forever

waiting for a message from R1.

Upon the execution of this protocol, if both robots have reached a state in which they are

committed, and have information that the other robot is also committed, the relational behavior can be

initialized. This communication protocol was implemented successfully on a team of soccer robots in

order to perform cooperative behaviors for scoring a foul, resulting in a static pass ([3] and [2]).

11

2.5.2 Commitment Management

After establishing the commitment and starting the relational behavior, it is necessary to ensure the

proper functioning of both. As can be noted in Section 2.4, a robot must abandon the behavior when

the goal is reached, or when it is impossible to achieve. For that, commitment managers are used.

In Figure 5, is given an example of how commitment managers operate.

In this case (Figure 5) the commitment managers are:

• Commitment <PartnerName> - when this place has a token, it means that the partner robot is

committed;

• EndCommitment <MyName> - when this place has a token, it means the robot doesn’t have the

proper conditions to continue the commitment. Once it finishes the commitment, it will change from

place Committed to place NotCommited;

• NotCommited <PartnerName> - when this place has a token, the partner robot isn’t committed

so if the robot is performing a relational behavior it should stop performing it and stop the commitment.

For different types of relationship behaviors, different types of protocols and commitment

management may exist. For example, in a case in which a relational behavior is being conducted

between three robots, and one of them sends a message that it is not committed, instead of the other

two cancelling their relational behavior and commitment they can still maintain the commitment

between them and execute a different set of actions, if doable, in order to achieve the joint goal.

Figure 5: Commitment managers

12

2.5.3 Communication

In Figure 5, the PN represents two robots, R1 and R2. Since these are autonomous robots, it is

necessary to separate the PN into two, each one representing one robot, as is depicted in Figure 6.

Figure 6: Commitment management

The transition <PartnerName>StoppedCommitment fires when the event with the same name is

launched. This happens when the partner robot completes his commitment, and it represents a

communication made by one robot to another indicating that the commitment ended.

For example, when R1 has a token in place EndCommitmentR1, the event R1StoppedCommitment

should be launched from the PN for R2, secured through a medium of communication.

Through the commitment managers and the communication method described above it is possible

to manage a commitment between two autonomous agents.

In the described case, a robot knows in what state the other is, through messages that are sent to

him. If, for example, a robot wants to get information on where the other is, the only way to learn that

information is through messages sent between them. This type of communication is called explicitly

because it requires that both robots transmit messages between themselves in order for them to be

informed about the state in which each one is in.

By contrast, there is also implicit communication. The main difference between implicit and explicit

communication, is that the evaluation of the other robot state is achieved by observations performed

by the robot sensors without any type of communications. For example, in the case above, a robot

knows where the other is by only using one camera and image processing algorithms. Presently, this

approach is still very limited, but with the evolution of technology and image understanding methods, it

is expected that it might be implemented in the future.

13

2.5.4 Synchronization

Another aspect to consider when performing a cooperative task is the synchronization

between robots. Synchronization is required to ensure that the robots are running

tasks in line between them. For this, take in account PN of Figure 7.

Figure 7: Synchronization example

From Figure 7 it is easy to conclude that in order for R1 to execute action Pa1 it needs to know if

R2 has finished action Pa2. In turn, after finishing Pa2, R2 will be in a state where it expects that R1

communicates when Pa1 has ended in order to complete the action Pb2. Places R2FinishedPa2 and

R1FinishedPa1 ensure that synchronization is being fulfilled.

Without ensuring this synchronization, the results of relational behaviors would not be

the desired ones.

2.6 Environment and Action Models

Environment Models

In [14] a framework is described for modelling, for instance, state changes that might occur due to

actions performed by other robots/agents or even physics. A discrete set of relevant states is

abstracted from the actual environment. Naturally these models will not fully model the environment,

but an abstraction of it.

The environment abstraction is achieved by discretising the world using logic predicates. As such,

environment models consist on GSPNs with predicate places.

An example is given in [14] of an environment model consisting of a free rolling ball, in which, due

to friction on the floor, it is expected that the ball will stop after some time. To model this process using

a GSPN model under the given framework, one must first discretise it, such that we can describe it

through the use of logic predicates. In Figure 8, we consider that the ball could be moving fast, slowly

14

or be stopped, and that the ball will, with time, pass from the fastest movement to the stopped state. It

also includes several transitions with different rates, associated with the same state change, in this

case the ball slows down depending on the weather conditions.

Figure 8: Petri Net Model of a free rolling ball [Reprinted from [14]]

If, for instance, one also wanted to model the fact that some other agent could increase the ball

speed, one could add transitions in the opposite direction, albeit with different associated rates,

considering the probability of that occurrence.

Action Models

The action models represent what is actually performed by the robot and how it is suppose to act. In

[14] a framework, that allows the creation of these models, is described in-depth.

An action is mainly described by the effects it causes on the environment and the conditions that

need to be met for the effects to take place. In logical terms, the action properties can be partitioned in

the following sets:

• Running-conditions: Conditions that need to be met for the action to be able to produce changes

in the world;

• Effects: Composed of Success Effects and Failure Effects, reflect the impact an action has on the

world;

-Success-effects: These are the effects associated with the success of the action. These include

the desired-effects of the action plus additionally intermediate effects that might occur in order to

achieve success;

– Failure-effects: These effects are the undesired ones, which might happen as a direct result of

running the given action.

The general model of an action is depicted in Figure 9. Although immediate transitions were not

included in this model, transitions can be either stochastic timed or immediate. Only stochastic timed

transitions were included in the Figure, since generally an action does not lead to immediate world

15

changes. However in Chapter 4 we will have cases where this occurs.

On this model not only the desired-effects of the action are represented, but also intermediary and

failure effects, all in a probabilistic approach through the use of stochastic transitions. Note that

enabling an action does not necessarily imply that any of its transitions will fire, due once again to their

stochastic nature.

Having the running-conditions as input places of all transitions, models the fact that the action can

only cause any impact on the environment if these conditions are met.

Figure 9: General action model

[Reprinted from [14]]

16

17

3. Plans for Cooperative Tasks

3.1 MeRMaID

3.1.1 Middleware and Architecture

In [5] and [6] details are provided on the development of MeRMaID (Multiple-Robot Middleware for

Intelligent Decision-making) that aims to provide an environment of high level, simplified and

systematic for programming and development of behaviors leading to an easily maintained code.

The architecture is described through Figure 10.

Figure 10: Architecture design

[Reprinted from [5]]

As can be seen in Figure 10, MeRMaID’s architecture is divided into several blocks that will be

presented below.

Atlas: Block responsible for managing the interaction of the robot with the environment, feel and

act. It comprises the following sub-blocks:

o Devices: Hardware that interacts with physical-world devices which can be of perception

18

(e.g. camera) or action (e.g. motors);

o Sensors: process information from the devices (e.g. Odometry, ball position, etc.);

o Information fusion: merges information from multiple sensors to improve accuracy and

extract high level information (e.g. using odometry with information taken from vision to

improve self-localization);

o Primitive actions: basic actions that cannot be decomposed into simpler actions. It

usually consists of simple calculations, followed by conveying the information to the

devices (e.g., shoot, move to the ball);

o Navigation Primitives: navigation algorithm that, given the current posture and the

desired posture, calculates the speed needed for the robot to move to the desired position

while avoiding obstacles.

Wisdom: the block where the relevant information about the world like the robot's posture, ball

position, among others, is maintained and managed. This information can be obtained through the

sensors, through messages received from other robots of the team or through the referee box.

Cortex: block where the decision occurs of which behaviors and actions should be implemented,

based on information stored in the block Wisdom. The cortex also connects with the block

Communication to communicate with the other robots in order to perform relational behaviors.

Cortex can be divided into three levels:

o Team Organizer: selects what role the robot must have in concurrence with the team

tactic (e.g. attacking, defence, or support);

o Behavior Coordinator: select behaviors to be execute within the selected role;

o Behavior Executor: selects a set of primitive actions so as to perform a behavior.

Communication: Block which manages the transmission of messages between robots and the

referee box.

3.1.2 Petri Nets Framework

Within the cortex, roles, behaviors and actions can be programmed through a variety of tools such as

state machines, fuzzy logic algorithms or Petri nets. For the purposes of this work, we are interested in

the Petri nets that are implemented through a framework developed on MeRMaID with the name of

PetriNetExecutor. As already mentioned, the cortex is divided into three levels, they are the Team

Organizer, Behavior Coordinator and Behavior Executor and, following the development of new Petri

Nets, a set of rules exits in order to classify the type of place and transition using a set of prefixes.

Thus we find the following prefixes on the PN for places:

action: a place with this prefix represents the execution of a role, a behavior or an action

depending on the level where that prefix is used. If we are at the Team Organizer level, a

19

place with this prefix represents the execution of a role. If we are on the Behavior Coordinator

level it represents the execution of a behavior and if we are at the Behavior Executor level it

represents the execution of a primitive action;

predicate: a place with this prefix represents an assessment/evaluation of the environment. If

the evaluation is true, the place has a token and if it is false, the place with the additional prefix

predicate.NOT_ has a token. Take for example the evaluation of the fact that the robot sees

the ball or not through predicate. SeeBall. If the evaluation is true, the predicate

predicate.SeeBall has a token, however, if the evaluation is false, the place with a token will be

predicate.NOT_SeeBall. Since this place represent an evaluation of the world and its outcome

is achieved internally, the framework that runs the PN cannot directly change the value by

inserting or removing a token in a place of this type. Thus, places with the prefix predicate are

simultaneously input and output places;

macro: as the name implies, a place with this prefix is a macro that encapsulates a PN which

leads to reduced complexity and allows for a better understanding by the user of what is being

implemented. This PN is a PN that contains the special places and GOAL_REACHED

GOAL_NOT_REACHED representing the success or failure (respectively) of the macro. In the

PN where the macro is built there should be also transitions with the same name that allows

the evolution of the PN depending of the macros’ success or failure;

no prefix: places that are just internal memories.

Transitions also contain the following prefixes:

T#: immediate transitions that are associated with predicate changes that fire as soon as they

have been activated;

event: transitions with this prefix fire only when they are active and the associated event has

occurred. These transitions are only used when we have a primitive action that sends a

message to another robot and in order to guarantee that the message was sent before the PN

is allowed to evolve, the primitive action also sends an internal event that that will be

associated with the next transition. This way the action only loses the token when the

message was sent;

GOAL_REACHED: transition that fires only when the associated macro has a token in the

place GOAL_REACHED;

20

GOAL_NOT_REACHED: transition that fires only when the associated macro has a token in

the place GOAL_NOT_REACHED.

In previous work, all the PN where initialized when we ran the framework and were kept running in

background, being constantly actualized depending of the evaluation of the environment. However this

lead to a huge increase in complexity in regards to modelling the PN nets and it didn’t allow us to

make use of the advantages of having the three levels (TeamOrganizer, BehaviorCoordinator and

BehaviorExecutor). When, for example, a PN on the BehaviorCoordinator level selected a PN on the

BehaviorExecutor that is deactivated by a transition controlled by a predicate, this same pair

transition/predicate had to be present in all the places in BehaviorExecutor PN in order to guarantee

that the PN reached the initial marking for when it was called again.

Another problem in having all the PN running simultaneously was that we could have a PN that, for

its task, worked perfectly fine when tested isolated from the rest of the PNs but when it was ran

simultaneously with all the rest, could enter a livelock, crashing the entire framework. This was

changed during the course of this work and now, the PN only runs when called by a higher level,

starting always from the initial marking.

This way when modelling a new PN, the person working on it doesn’t have to concern itself in

ensuring that the PN evolves to its initial marking when the PN is disabled by the higher level and

doesn’t need to be concerned with the fact that the PN can enter a livelock state because of another

PN. As a result of this change, the complexity of modelling a PN decreased, the PetriNetExecutor runs

faster and is more robust against modelling errors.

3.2 Organizational Behaviors

In [2], [3] and [4] relational and individual behaviors were developed that made up the team tactics,

however, there were many game situations that were not taken into account and therefore the plans

for this situations had to be created. Also, the existing plans had to be completely remodelled

because, as explained in 3.1.2, the way the framework dealt with the PN processing changed and with

it the way that PNs are modelled. Due to this fact, there is a need for an in-depth description of all the

Petri Nets currently in use in the project.

3.2.1 TeamOrganizer

The PN TacticBase runs in the TeamOrganizer level and, as mentioned in 3.1.1, this is the level where

the choice between the possible roles the robot can have is made, thus defining the team tactics.

In this particular tactic the robot starts, for safety reasons, as a Defender and depending on the

evaluation of the predicates responsible for the attribution of a role (ShouldIGo, ShouldISupport and

21

ShouldIDefend), the PN evolves, selecting the correct role in concurrence with the conditions assigned

for its execution.

For each place, representing a role, there are output transitions that connect to each one of the

other roles and input transitions from those same roles which allow the robots to move from one role to

the other.

From all the roles places, there are also transitions that fire when the predicate NOT_IsRunning

has a token which indicates that the motors are not on and, therefore the robot will not be assigned a

role. When this happens, the robot will execute RoleStop and this marking will be maintained until

predicate IsRunning is evaluated as true which will result in the selection of RoleDefender and from

here the PN will evolve normally depending on whether the conditions for each role are met.

As mentioned, this tactic selects three possible roles that depend on the evaluation of predicates.

They are described below.

Attacker: The main objective of this role is to actively try to acquire the ball and score a goal. An

Attacker is always the one to take a foul or the one to execute a dynamic pass to another robot. On

the other hand, the Attacker never searches for the bar nor defends the goal.

This role is selected only when the evaluation of predicate ShouldIGo is TRUE for which the only

prerequisite is that the robot is the closest one to the ball as long as he is not simultaneously the

goalkeeper. Therefore, this predicate uses the evaluation of the predicate ClosestToBall that collects

the information of the distance from the ball of all the robots, except the goalkeeper, and calculates

which one is the closest to the ball.

The evaluation to analyse if the robot is the Attacker is performed continuously during the game in

order to allow the change of roles in a dynamic way, however, during a static or dynamic pass, this

evaluation is only done in the beginning and the roles are fixed during the execution of these passes.

During a static pass, the robots move around the ball to position themselves in the right place and,

during this movement, a situation could arise in which they get “stuck” in a movement that has them

changing roles constantly. If, before a static pass (foul situation), the robot has to search for the ball,

the role is fixed before the search and when the ball is found, a new evaluation is made and fixed until

the completion of the static pass.

During a dynamic pass we only want the pass to be over and the roles to change when the robot

that is to receive the ball, gets it and the commitment is broken. If the roles were not fixed, when the

ball passed the point where the receiver becomes the one closest to the ball and, therefore, the

Attacker, the roles would be exchanged and the instruction to break the commitment would be lost in

the process because when a new role is selected, it always starts in the initial marking.

During a game situation the team can have:

One Attacker – one of the robots is the closer to the ball;

22

No Attackers - none of the robots know the ball’s position.

Defender: The main goal of this role is to defend the goal without engaging the ball. Furthermore, a

Defender can also look for the position of a foul when nobody sees the ball.

This role is only selected when the predicate ShouldIDefend is TRUE. In order for this to happen,

the robot has to be one of the two robots closest to the team own goal. This evaluation is performed by

the predicate AmIDefender that receives the position of every robot in order to proceed with the

evaluation. However, being in this position is not enough for the robot to be a defender, it is also

necessary for the robot to not be the closest nor the second closest to the ball nor the goalkeeper.

As with the attacker, the robot role is fixed during a foul situation, during the search for the ball and

during a pass in order for the robot to be able to get to the position that he should occupy. During a

dynamic pass the defender doesn’t have to have the role fixed because he isn’t part of the pass. If

more cooperative behaviors are created that include the Defender, the role should be fixed for those

situations, given the reasons already mentioned.

During the course of a game, depending on the position of the robots in relation to the ball and the

field, the team can have:

Two Defenders – the two robots closest to own goal are not the closest nor the second closest

to the ball and, therefore are the Defenders;

One Defender – one of the robots closest to own goal is also the closest or second closest to

the ball so it will be an Attacker or Supporter, respectively;

No Defenders - the two robots closest to own goal are also the closest and second closest to

the ball and therefore, the Attacker and Supporter, respectively.

Supporter: This role is the most complex one. A Supporter is always the one that receives a static

pass or a dynamic pass and it can search for the ball during a foul situation or when the game is

running. It can also defend the goal if the team doesn’t have the ball and even engage the ball if the

Attacker isn’t able to intercept it. When the Attacker has the ball, the Supporter tries to find a good

position to receive a pass.

The role is selected if the predicate ShouldISupport is evaluated as TRUE. This occurs when the

robot is the second closest to the ball, evaluated by the predicate SecondClosestToBall, or if the robot

isn’t already a Defender or Attacker.

The situation in which this role is fixed is the same as for the attacker.

During the course of the game the number of supporters can be:

One Supporter;

Two Supporters;

23

Three Supporters.

Whenever the robot is not a Defender or an Attacker, it is a Supporter so the number of Supporters

depend of the number of robots that are performing the other roles. This allows for a lot of flexibility

and dynamism in selecting the roles, allowing the team to respond very well to the various game

situations that can occur.

Stop: This role, as mentioned above, is only selected if the motors are not on. In this situation the

other robots won’t use this robot’s data when calculation things like which robot is the closest to the

ball. This should only be a primitive action and not a role but because of the three levels in the Cortex

a primitive action can’t be called in this level.

Table 1. summarizes all the info pertaining to this PN. In appendix A.1 this PN can be consulted.

Table 1: TacticBase

Name TacticBase

Type Organizational Behavior - Tactic

Objective Responsible for the team tactic by selecting the

role that the robot should perform

Roles RoleAttacker, RoleDefender, RoleSupporter,

RoleStop

Predicates ShouldIGo, ShouldIDefend, ShouldISupport,

IsRunning

MeRMaID Level TeamOrganizer

Appendix A.1

3.2.2 BehaviorCoordinator

After selecting the role for each robot, it is time to choose the behavior they should have in each game

situation. These choices are made on this decision level since in each role, mentioned above, we have

Behaviors and macros, which are also composed by Behaviors. The activation of each of these

behaviors is of the responsibility of the new conditions, which are expressed through the

implementation of new predicates.

RoleAttacker: when this role is selected the initial marking of the PN net is a token in the place

action.BehaviorStop. From this point it is selected a set of behaviors that depend on the situation in

which the game is in. These situations are directly determined by what the referee box sends to the

team and, before the game status changes, the signal that the game is stopped is always given in

order for the robot to return to the initial marking.

24

During the execution of RoleAttacker the following game situations can occur:

Game running

If the robot is receiving the signal that the game has started, predicate GameStarted TRUE, the

individual behavior used to attack (BehaviorBaseAttacker) will be performed until the game is stopped

or until the ball is in the opponent goal area (robot can’t breach this area). When this happens the

robot will go to the edge of the penalty area until the ball leaves the opponent goal area or the game is

stopped.

Dynamic Pass

While performing the individual behavior BehaviorBaseAttack a situation in which a dynamic pass

can be performed can occur, however, this will be explained in more detail in section 3.4 when

cooperative behaviors are addressed.

Dropped Ball

If the robot receives a signal that we are in a dropped ball situation, predicate DroppedBall will be

TRUE and the robot will perform the Dropped Ball behavior until the dropped ball situation ends.

Foul Situation

If the robot receives information that the game is in a foul situation, predicate FoulSituation, which

is composed by an aggregate of predicates for each type of fouls, will be TRUE and the behavior to

defend against a foul (BehaviorFoulDefend) will be selected. From this point, two situations can occur:

Foul for the other team - the robot will stay in this behaviour and defend against the foul until

predicate FoulSituationEnded becomes TRUE which indicates that the foul ended (predicates

OurFoulTaken, OppFoulTaken and FoulTimeExpiring);

Foul for our team - since the robot is the Attacker and, therefore the closest to the ball, the

robot will be the one that will perform the static pass so predicate ShouldITakeFoul will be

TRUE and macro TakeFoul will be performed which will select the behaviors needed to take a

foul.

In this role there is no need to perform a search for the ball because this role is only selected if the

robot is already seeing the ball.

Table 2. summarizes all the info pertaining to this PN. In appendix A.2 this PN can be consulted.

Table 2: RoleAttacker

Name RoleAttacker

Type Orgazinational Behavior - Role

Objective Selects a set of Behaviors that the robot has to

perform for each game situation when it is the

25

Attacker

Behaviors BehaviorStop, BehaviorFoulDefend,

BehaviorDroppedBall, BehaviorBaseAttack,

BehaviorMove2OppPenaltyArea,

BehaviorDynamicPassTaker

Macros TakeFoul, CommitmentBreakingBC,

CommitmentEstablishmentDynamicPassAtacker

MeRMaID Level BehaviorCoordinator

Appendix A.2

RoleDefender: Similar to the RoleAttacker, when this role is selected, the initial marking is a token in

BehaviorStop and, whenever the game status changes, the PN net returns to this marking because

the referee box sends the signal that the game is stopped in between changes of the game status.

During the execution of RoleDefender the following game situations can occur:

Game running

When the game is started, predicate GameIsStarted will be TRUE and the individual behaviour use

to defend (BehaviorBaseDefend) will be performed until the game status changes and the referee box

sends game is stopped.

Dynamic pass

Robots with this role are not involved in the dynamic pass.

Dropped Ball:

If the game status is of a dropped ball situation then predicate DroppedBall is TRUE and

BehaviorDroppedBall is performed, however, from this position two things can happen:

Robot is not selected to find the ball - it simply keeps on performing this behaviour;

Robot is selected to find the ball - macro FindFoulPosition is performed in which a set of

behaviors will be selected to find the ball position. Two situations can occur in this case:

o Goal reached - robot finds the ball and BehaviorDroppedBall is selected again to send

the robot to the correct position in relation to the ball;

o Goal not reached - robot doesn’t find ball before the game is started and the role

reverts back to its initial marking and performs BehaviorStop.

Foul situation

If we are in a foul situation predicate FoulSituation is TRUE and the individual behaviour to defend

against a foul (FoulDefend) is selected and, similarly as with the dropped ball situation, two situations

can occur:

26

Robot is not selected to find the ball - it simply keeps on performing this behaviour until

predicate FoulSituationEnded becomes TRUE;

Robot is selected to find the ball - macro FindFoulPosition is performed in which a set of

behaviour will be selected to find the ball position. Two situations can occur in this case:

o Goal reached - robot finds the ball and FoulDefend is selected again to defend against

the foul;

o Goal not reached - robot doesn’t find ball before the game is started and the role

reverts back to its initial marking and performs BehaviorStop.

If the foul is for our team, all the above can happen but when the robot finds the position of the ball, if it

is determined that the robot will be involved in the pass, this role will be deselected and role Attacker

or Supporter will be selected by TeamOrganizer. If it is determined that the robot will not be involved in

the static pass, the individual behaviour to defend against a foul is performed.

Table 3. summarizes all the info pertaining to this PN. In appendix A.3 this PN can be consulted.

Table 3: RoleDefender

Name

Type Organizational Behavior - Role

Objective Selects a set of Behaviors that the robot has to

perform for each game situation when it is the

Defender

Behaviors BehaviorFoulDefend, BehaviorStop,

BehaviorBaseDefend, BehaviorDroppedBall

Macro FindFoulPosition

MeRMaID Level BehaviorCoordinator

Appendix A.3

RoleSupporter: Aside from the fact that this role has a section devoted for the dynamic pass and

the static pass, the role is identical in all aspects to the RoleDefender. Note that in between changes of

the status of the game a signal of game is stopped is given so the PN returns to the initial marking of

BehaviorStop.

During the execution of RoleDefender the following game situations can occur:

Game running

If the information coming from the referee box is that the game has started, the individual behaviour

to support (BehaviorBaseSupport) is selected until the game is stopped again.

27

Dynamic pass

While the individual behavior BehaviorBaseSupport is being performed, a dynamic pass can

occur. The case of a dynamic pass will be explored and explained in depth in section 3.4.

Dropped ball and foul situation for opponent team

If the game status is a dropped ball or a foul for the other team the procedure is the same as in the

RoleDefender and the behaviors, macro to find the ball and predicates involved are the same.

Foul situation for own team

If it is a foul for our team, when the token is in BehaviorFoulDefend, it can be determinate by

predicate ShouldIReceiveFoul that the robot is going to participate, as the receiver, in a static pass

and therefore, macro ReceiveFoul is selected and performed until the goal is reached.

If the robot isn’t going to be involved in the static pass, the role performs the individual behaviour to

defend against a foul (BehaviorFoulDefend) until the predicate FoulSituationEnded becomes TRUE.

Table 4. summarizes all the info pertaining to this PN. In appendix A.4 this PN can be consulted.

Table 4: RoleSupporter

Name

Type Orgazinational Behavior - Role

Objective Selects a set of Behaviors that the robot has to

perform for each game situation when it is the

Supporter

Behaviors BehaviorFoulDefend, BehaviorDroppedBall,

BehaviorStop, BehaviorBaseSupport,

BehaviorDynamicPassReceiver,

Macros Receive Foul, FindFoulPosition,

CommitmentEstablishmentDynamicPassSupporter,

CommitmentBreakingBC

MeRMaID

Level

BehaviorCoordinator

Appendix A.4

Macro FindFoulPosition: This macro is selected inside RoleDefender and RoleSupporter and has

the objective of finding the ball in a foul situation, be it for our team or for the opponent team.

Only the robots in which predicate ShouldIFindFoulPosition is TRUE will have this macro selected.

28

The initial marking for this PN is a token in the place BehaviorStop and from this point a choice

between several behaviors will be done, depending of the type of the foul:

kickoff for our team, BehaviorMove2MidField will be selected;

corner kick for our team or for the opponent team, BehaviorFindBallInCorner will be selected;

dropped ball, BehaviorSearchDroppedBallPoints will be selected;

goal kick for our team, BehaviorMove2OwnPenalty will be selected;

free kick and throw in, there is no search because it would take a lot of time to search all the

possible positions in which the ball could be found and there would be the risk of leaving the

goal unprotected. In these cases, the roles don’t select this macro and only

BehaviorFoulDefend is performed. Also, during the game it is rare the situation in which these

two fouls occur without any robot being able to see the ball.

The goal of this macro will be reached if the robot can see the ball before the game is started

again. On the other hand, the goal is not reached if the robot can’t find the ball before the game is

started or the game is stopped which indicates that the foul situation ended.

Table 5. summarizes all the info pertaining to this PN. In appendix A.7 this PN can be consulted.

Table 5: FindFoulPosition

Name

Type Organizational Behavior - Macro

Objective Selects a set of behaviors with the goal of

finding the ball during several foul situations

Behaviors BehaviorStop,

BehaviorMove20wnPenaltyArea,

BehaviorSearchDroppedBallPoints,

BehaviorFindBallInCorner,

BehaviorMove2Midfield,

MeRMaID Level BehaviorCoordinator

Appendix A.7

3.3 Individual Behaviors

BehaviorBaseAttacker

This behavior was designed to be run by the attacker while the game is started and its aim is to score

a goal.

As a prerequisite to the success of any behavior, the robot must know the location of the ball. This

29

evaluation is done using the predicate SeeBall and, if it is negative, the robot performs the primitive

action SearchBallAllOver which searches the ball along the field. Note that this action shouldn’t be

needed here because only the attacker performs this behavior and if the robot is the attacker, it

automatically sees the ball, however, it’s important to have this action in order to test the behavior

individually without running all the other levels.

If the robot sees the ball the sequence of actions needed to score a goal are performed. Firstly, the

action Move2DribblePosition is performed, which puts the robot close to the ball, facing the opponent

goal. When predicate IsAtDribblePosition becomes TRUE, the desired effect was reached, and action

CatchBall is performed until predicate HasBall becomes TRUE.

After CatchBall succeeds in catching the ball, action Dribble2Goal is executed, whose aim is to

move the robot towards the opposing goal, deviating from opponents that are in the way. From this

point, two things can happen, depending on whether the robot has the kicker operational (predicate

KickerOn) or not, that are described bellow:

Kicker not operational - the robot performs Dribble2Goal until the robot is at the opponent

Penalty Area. When this happens the robot performs the action kick that will, since it doesn’t

have a kicker, give a final push to the ball;

Kicker operational - when the robot is near the opponent goal, the robot will choose between

two options to approach the goal:

o Drible2Score, in which the robot deviates from the hurdles and aligns itself with the goal;

o Aim2Score in which the robot describes a diagonal in front of the goal.

In either case the objective is for the predicate ShootingOpportunity to be TRUE in order for

the robot to select the action Kick. However, if the robot reaches the penalty area without finding

a goal opportunity the robot will kick the ball, nevertheless, because it can’t invade the goal

area.

If at any time of the behavior the robot loses the ball it performs action Move2DribblePosition again

and the cycle repeats itself.

Table 6. summarizes all the info pertaining to this PN. In appendix A.10 this PN can be consulted.

Table 6: BehaviorBaseAttack

Name BehaviorBaseAttack

Type Individual Behavior

Objective Selects a set of actions with the main objective

of scoring a goal

Actions SearchBallAllOver, Move2DribblePosition,

CatchBall, Dribble2Goal, Kick,

30

Move20ppPenaltyArea, Dribble2Score,

Aim2Score

MeRMaID Level BehaviorExecutor

Appendix A.10

BehaviorBaseDefend

This behavior defines what the defender executes while the game is started.

If the robot does not see the ball, it is sent to a predefined position, in front of his goal, by executing

action Move2DefenderPosition. When the robot reaches this position it executes the action

ShearchBallDefend, which causes the robot to describe a movement similar to a trapeze, in front of

the goal.

When the robot can see the ball, primitive action CoverGoal is performed and the robot covers the

goal by minimizing the shot’s angle.

Table 7. summarizes all the info pertaining to this PN. In appendix A.11 this PN can be consulted.

Table 7: BehaviorBaseDefend

Name BehaviorBaseDefend

Type Individual Behavior

Objective Selects a set of actions with the goal of

defending.

actions Move2DefenderPosition, SearchBallDefend,

CoverGoal

MeRMaID Level BehaviorExecutor

Appendix A.11

BehaviorBaseSupport

This behavior defines the set of actions that the robots with the role of support should run during the

game, when the game is started.

If the robot cannot locate the ball, it executes the action SearchBallAllOver that aims to cover the

whole field in order to find the position of the ball. If the robot sees the ball there are several actions

that can be executed depending on whether the attacker has the ball or not. These are described

below:

Attacker holds the ball in his possession - the support will follow the ball from a distance of two

meters and, thus, it will be in a position to retrieve the ball if the attacker loses it. If we want the

31

team to perform a long pass during the game the variable in the World info called

WeWantLongPass should be put with the value TRUE which will result in the predicate

ShouldFindPassPosition to be true and the robot to not just follow the ball but to actively try to

find a good position for the pass to occur by performing action FindPassPosition.

Attacker does not hold the ball - the supporter will execute action CoverGoalFollowingBall

which causes the robot to cover the goal to minimize the angle of the opponent's shot while

maintaining a two meters distance to the ball. Should the ball get too close to the team's goal

without the attacker being able to retrieve it, the Supporter will engage the ball and try to catch

it. This is done in order to try minimizing the risk of conceding a goal.

Table 8. summarizes all the info pertaining to this PN. In appendix A.12 this PN can be consulted.

Table 8: BehaviorBaseSupport

Name BehaviorBaseSupport

Type Individual Behavior

Objective Selects a set of actions with the goal of

supporting. These actions can be to defend

without engaging the ball, to defend engaging

the ball, search for the ball or find a good

position for a dynamic pass

action FindPassPosition,FollowBallOnDistance,

SearchBallRandomly, CoverGoalFollowingBall,

Move2Ball, CatchBall

MeRMaID Level BehaviorExecutor

Appendix A.12

BehaviorFoulDefend

This individual behavior is executed by the robots that are not involved in the replacement of the ball

on the field when the foul is for our team and by all team members when the foul is in favor of the

opponent.

Depending on the type of foul and the number of robots that are still running, if the robot sees the

ball this behavior sends the robots to a position to better defend the goal and if not, it send the robots

to their start positions, which are different for each robot and are defined in WorldInfo.

The initial marking of this behavior is a token in action Stop and from this point the robot will select

and execute different actions depending on the foul type.

If the foul situation is a free kick or a goal kick, action Move2FreeKickDefendPosition is executed

if the robot sees the ball or Move2StartPosition is executed if it doesn’t.

32

In Figure 11 (a) we can see the result of Move2FreeKickDefendPosition when the foul is not close

to the penalty area and in (b) what happens when the foul is close to penalty area. In Figure 11 (b) we

can see that only the attacker enters the penalty area but not the goal area while all the other robots

move along the edge of the penalty area in order to find a position that is at the minimal distance

allowed in the rules from the ball without breaching the penalty area. In order for the positions to be

assigned to each robot, they have to communicate to each other which role they are running in order

for the calculations of what positions they are going to be sent. The attacker always goes to a position

that is between the ball and the goal at the minimal distance allowed from the ball or until reaching the

edge of the goal area. The other robots go to a position slight to the side of the attacker if they are far

from the penalty area, Figure 11 (a) , or if they are in risk of entering the penalty area they move along

the penalty area edge, Figure 11 (b).

If the foul situation is a corner kick, action Move2CornerKickDefendPosition is selected if the robot

sees the ball and action Move2StartPosition if not.

In Figure 12 is shown the distribution that the robots will have if they see the ball during a corner for

the opponent. Note that if the corner was for our team the only difference would be that the robot 2

would be the one taking the corner while all the other robots would be in the positions as shown in

Figure 13.

If the foul situation is thrown in, action Move2ThrowInDefendPosition is selected if the robot sees

the ball and action Move2StartPosition if not.

In Figure 14 is shown the distribution that the robots will have if they see the ball during a thrown in

for the opponent. Only the closest to the ball and second closest to the ball will be sent to a position

that depends on the ball position, as shown in Figure 13. All the others are sent to their start positions.

Finally, if the foul situation is a kick off the robots are simply sent to their start position.

(a) (b)

Figure 11: Robot position when defending against an opponent free or goal kick (a) Far from the penalty area (b) Close to the penalty area

33

Figure 12: Robot position when defending against a corner kick for the opponent

Figure 13: Robot position when defending against a throw in

Table 9. summarizes all the info pertaining to this PN. In appendix A.16 this PN can be consulted.

Table 9: BehaviorFoulDefend

Name BehaviorFoulDefend

Type Individual Behavior

Objective Selects a set of actions that send the robot to

the right positions in order to defend against

the foul. These positions will be in relation to

the ball position and will depend on the foul

type as well of the role that each robot has. If

the robot doesn’t see the ball, the robot will

move to its start position.

34

Action Stop,

Move2FreeKickDefendPosition,

Move2StartPosition,

Move2ThrowInDefendPosition,

Move2CornerKickDefendPosition

MeRMaID Level BehaviorExecutor

Appendix A.16

BehaviorFoulTakerAlone

This behavior is executed when a commitment for the static pass wasn’t successful and therefore the

attacker has to take the foul alone.

In this behavior, the attacker moves towards the ball until it is close to it. Next, the robot will turn

towards the opponent goal and when it is facing the goal the robot stops and waits for the referee box

to give the signal for the start of the game. Upon receiving this signal, the robot catches and kicks the

ball hard towards the opponent goal.

In the end an action (SendFoulTaken) sends a message to all the robots in order to signalize the

end of the behavior, allowing the PNs to evolve from a foul situation.

Table 10. summarizes all the info pertaining to this PN. In appendix A.19 this PN can be consulted.

Table 10: BehaviorFoulTakerAlone

Name BehaviorFoulTakerAlone

Type Individual Behavior

Objective Behavior in which a foul is marked by a single

robot

action Move2Ball, TurnAroundBallOppGoal, Stop,

CatchBall, Kick, SendFoulTaken

MeRMaID Level BehaviorExecutor

Appendix A.19

35

BehaviorPenaltyTaker

This behavior is designed to respect the rules of scoring a penalty, as was stipulated in the RoboCup

rule book.

The robot moves to the midfield and when it reaches this position, it waits for the referee box to

give the signal that the game is started. After this signal is received, the robot moves to the opponent

penalty area until being able to see the ball, therefore, allowing the robot to move towards the ball until

it is close enough to catch the ball.

After catching the ball the robot executes an action that guarantees that it his facing the goal and,

when it is, the robot executes action kick to shoot towards the goal. Note that if the mechanism to grab

the ball was better, instead of just turning to the opponent goal, another primitive could be used with

the objective of finding a good position to shoot the ball, where the keeper couldn’t reach it as easily.

Finally action an action is performed (SendPenaltTaken) that informs all the robots of the end of the

behavior.

Table 11. summarizes all the info pertaining to this PN. In appendix A.20 this PN can be consulted.

Table 11: BehaviorPenaltyTaker

Name BehaviorPenaltyTaker

Type Individual Behavior

Objective Behavior in which the robot scores a penalty

action Move2Midfield, Stop, Move2OppPenaltyArea,

Move2Ball, TurnAroundBallToOppGoal,

SendPenaltyTaker, Kick, CatchBall

MeRMaID Level BehaviorExecutor

Appendix A.20

BehaviorFindBallInCorner

This behavior is selected when the robot has to find the ball in one of the corners. For this purpose,

the robot will move from the left corner to the right corner until it finds the ball or the behavior is

unselected.

Table 12. summarizes all the info pertaining to this PN. In appendix A.15 this PN can be consulted.

Table 12: BehaviorFindBallInCorner

Name BehaviorFindBallInCorner

Type Individual behavior

Objective Find the ball in the corners

36

action Stop,

SearchOppLeftCorner,

SearchOppRightCorner,

SearchOwnLeftCorner,

SearchOwnRightCorner

MeRMaID Level BehaviorExecutor

Appendix A.15

BehaviorMove2Position

All behaviors that have the objective of moving the robot to a position follow the structure of Figure 14.

The robot runs an action that sends it to the desired position and when it reaches it the robot runs

action Stop.

Figure 14: of a behavior that moves the robot to a position

Other Behaviors

All behaviors not mentioned here only have a place that selects an action with the same name as the

behavior. This happens because of the three levels. Sometimes when we are in the

BehaviorCordinator level there’s a need to select only an action and not a full behavior, however,

since the formalism is to be followed, behaviors with only an action must be created with the same

name as the behavior.

3.4 Relational Behaviors

3.4.1 Static Pass

In [2], [3] and [4] Petri Nets were developed with the aim of modeling the behavior to take and receive

a foul, for the total set of possible foul types. However, the method used, modeled the set of

instructions needed as roles which was not formally correct since the only roles the robots should

have are attacker, supporter and defender. Due to this, macros TakeFoul and FoulReceiver were

created and, as mentioned in 3.2, called inside the roles RoleAttacker and RoleSupporter respectively,

as well as the predicates needed to select these macros.

37

Note that these macros include not only the establishment of a commitment between two robots as

well as the behavior that the robot has during the relational behavior but also the individual behavior

the robot performs when the commitment fails. Individual behaviors were already explained in 3.3.,

however, the macros, TakeFoul and FoulReceiver, that select them are only explained here because

they are also responsible for the selection of the PNs performed during a relational behavior which are,

inevitably, connected with the individual behaviors that have to be performed when the commitment

fails.

The behaviors, needed to be performed by the taker and receiver, were also created from scratch

as well as the actions and predicates involved. This had the objective of contemplating all types of foul,

optimize the changes of success and to take into account the fact that now the PNs always start in the

initial marking when called by a higher level.

These new macros and behavior are described below.

Macro TakeFoul

Macro TakeFoul is a part of the BehaviorCoordinator and it selects all the behaviors that a robot

performs during a foul, when it is the one responsible to take the foul

This PN has, as an initial marking, a token in the place BehaviorStop. From this point, it

distinguishes between the fact that we are in a situation of penalty or another type of foul.

If we have a penalty situation the behavior BehaviorPenaltyTaker is executed in order to take the

penalty and it ends its execution if the predicate GameIsStopped is TRUE or if predicate PenaltyTaken

is TRUE.

If we are in the presence of any of the other types of foul, the robot tries to set a commitment by

executing macro CommitmentEstablishmentStaticPass_Request. If this endeavor is a success, the

transition GOAL_REACHED will fire and the behavior BehaviorFoulTaker will be executed, i.e., the

relational behavior is performed. However, if the commitment fails, the individual behavior

BehaviorFoulTakerAlone is executed instead. The relational behavior only ends when predicate

OurFoulTaken (foul was marked successfully) or predicate StopFoulCommitment (received

GameIsStoped or something failed during the relational behavior, e.g., the robots lost communication

with each other) becomes TRUE.

In the end, the commitment will be broken by macro CommitmentBreakingBC and if the relational

behavior was a success the macro ends but if not, the individual behavior is performed instead. When

the individual behavior is performed it only ends when one of the following predicates becomes true:

FoulTimeExpiring, GameIsStopped and OurFoulTaken.

Table 13 summarizes all the info pertaining to this PN. In appendix A.9 this PN can be consulted.

Table 13: TakeFoul

Name TakeFoul

38

Type Organizational Behavior - Macro

Objective Selecting the behaviors responsible for marking

a foul. If the commitment is a success the

relational behavior is executed but if it fails, an

individual behavior is executed instead

Behaviors

BehaviorStop,

BehaviorFoulTaker, BehaviorSendFoulFinished, BehaviorFoulTakerAlone

Macros CommitmentEstablishmentStaticPass_Request,

CommitmentBreakingBC

MeRMaID Level BehaviorCoordinator

Appendix A.9

Macro ReceiveFoul

Macro TakeFoul is also a part of the BehaviorCoordinator and it selects all the behaviors that a robot

performs during a foul, when it is the one responsible to receive the foul.

This PN has, as an initial marking, a token in the place BehaviorStop and, initially, it is also made

the differentiation between the fact that we can be in a penalty situation or other type of foul.

If we have a penalty situation the behavior BehaviorStop is executed because only the robot that is

the taker will be performing a penalty, all the other robots don’t need to do a thing but wait until the

penalty is over.

If we are in the presence of any other type of fault, the robot tries to form a commitment by

executing macro CommitmentEstablishmentStaticPass_Accept. If this endeavor is a success the

transition GOAL_REACHED will fire and the behavior BehaviorFoulReceiver will be executed, i.e., the

relational behavior is performed, however, if the commitment fails, the individual behavior

BehaviorFoulDefend is executed instead.

From this point the macro evolves the same way as for the macro TakeFoul, the only difference is

that the individual behavior ends when predicate GameIsStarted or GameIsStopped has the value

TRUE. This may seem strange but note that until this point the game status indicated the type of foul

so both of these predicates were false.

Table 14 summarizes all the info pertaining to this PN. In appendix A.8 this PN can be consulted.

Table 14: ReceiveFoul

Name ReceiveFoul

Type Organizational Behavior - Macro

39

Objective Selecting the behaviors responsible for

receiving a foul. If the commitment is a success

the relational behavior is executed but if it fails,

an individual behavior is executed instead

Behaviors

BehaviorStop,

BehaviorFoulReceiver,

BahaviorFoulDefend,

BehaviorSendFoulFinished

Macros CommitmentEstablishmentStaticPass_Accept,

CommitmentBreakingBC

MeRMaID Level BehaviorCoordinator

Appendix A.8

The macros used for starting a commitment, CommitmentEstablishmentStaticPass_Accept and

CommitmentEstablishmentStaticPass_Request were not changed from what was done in [2] and [3]

apart from a slight detail in CommitmentEstablishmentStaticPass_Request. Because the search of the

ball was introduced when we are in a foul situation and the robot doesn’t see the ball, when the robot

sends a request the previous time that the robot spent waiting for a response might not be enough

because the other robot, that will be the receiver, might still be looking for the ball. Therefore this time

was increased, however, as one might imagine, it is not feasible for the robot to be spend a big

amount of time waiting around without doing nothing so, a predicate OnlyRobotRunning, that has the

value true if the robot is the only field robot that is functional, was created. If after sending a request

the robot verifies that it is the only robot on the field there is no need to wait for a reply so it concludes

that the commitment was not a success. Apart from this, the robot stops waiting for a response when

the game is started because, if the other robot didn’t reached the ball in time there’s no point in waiting

more because there is only ten seconds for the foul be marked when the referee sends game is

started.

The individual behaviors used in the previous macro were already explained in section 3.3 so the

only thing left to explain are the relational behaviors that each robot performs. Note that the way the

synchronism points are done is different from what was used in [2] and [3] because they are now

controlled by pares of action/predicate in which the action changes a variable in the block wisdom,

specifically in WorldInfo, to true. This variable is constantly being sent to other robots through the

Block Communications and the other robot will receive this value in their Block Communications and

then a predicate will evaluate this information. It should be noted that the action that changes the

variable in WorldInfo should be performed only once and as such should also generate an event to

inform the Petri net that the variable has been changed in order for the token to be removed from the

action. The pair action/predicate responsible for the synchronism points are respectively

40

SendReady2Receive/Ready2ReceiveFoul and SendFoulTaken/OurFoulTaken.

A description of the behaviors executed during a relational behavior to perform a static pass is

given below.

BehaviorFoulReceiver

When this behaviour is selected the robot is the one that will be receiving the ball in a static pass so

the first action executed is Move2ReceiveFoulPosition that puts the robot between the middle of the

field and the ball at the minimal allowed distance from the ball. This condition allows for the robot to

always be in a good position to receive the ball whatever the type of foul, which negates the need to

create a different primitive action for each type of foul as was the intention demonstrated in [2] and [3].

When the robot is at the position desired, predicate IsAtReceiveFoulPosition will be TRUE, and the

robot executes action SendReady2Receive, first synchronism point.

At this point the robot will only execute action Stop until it receives a positive evaluation from

predicate OurFoulTaken, second point of synchronism. When this evaluation is true, the robot will

move to the ball using action Move2Ball. Note in appendix A.17 that there is a transition after this

action (T4) that is only there in order for the behavior to reach the initial marking again so that this

behavior can be tested without running the higher levels.

Table 15 summarizes all the info pertaining to this PN. In appendix A.17 this PN can be consulted.

Table 15: BehaviorFoulReceiver

Name BehaviorFoulReceiver

Type Individual behavior in a relational situation

Objective Selecting the actions needed to receive the ball

in a foul situation

Behaviors BehaviorStop,

BehaviorFoulTaker,

BehaviorFoulTakerAlone,

BehaviorSendFoulFineshed,

BehaviorPenaltyTaker

MeRMaID Level BehaviorExecutor

Appendix A.17

BehaviorFoulTaker

When this behavior is selected the robot knows it will be the one to take the foul so the first action

41

executed is Move2TakeFoulPosition that moves the robot close to the ball and facing the center of the

field because, as explained before, is where the receiver will be. As for the receiver, this allows for this

single primitive action to be valid for all the foul situations.

When, predicate IsAtTakeFoulPosition is TRUE the robot will execute action Stop and wait for the

signal of the referee box indicating that the game has started and for confirmation from the receiver

that it his ready to receive, in the form of the predicate Ready2ReceiveFoul.

When this confirmation is received the robot will use action CatchBall and then action Kick2Foul to

perform the pass. Note that this action should be regulated when the kickers are operational to

maximize the success of the pass.

Finally the robot executes action SendFoulTaken to inform internally and the other robots that the

foul was taken. Note in appendix A.18 that transition T6 is only there so that the behavior can reach

the initial marking so that it can be tested without running the higher levels.

3.4.2 Dynamic Pass

In [4] was introduced a dynamic pass, however it was very restricted, being rarely performed because

the conditions needed for its execution seldom occurred. Therefore the dynamic was completely

remodelled, nevertheless, note that in terms of communication the exact same procedure is used as

described in [4].

Two types of pass were created in order to contemplate two different game situations. Their

specifications can be found in table 16 and table 17:

Table 16: Short Pass

Robot that initialises the pass Attacker

Objective When we have a dropped ball situation, the rules stipulate that

before scoring a goal, a pass must be performed. Since this is

mandatory, the objective of this pass is maximize the success rate

of the pass

Description The robot that will receive the ball positions itself in front of the

attacker and then the attacker gets out of the way, leaving the ball

behind. Since the ball is stationary the receiver won’t have any

problem in capturing it and dribbling to the opponent goal.

Pre-conditions to request Last was a dropped ball

Robot has the ball

Conditions to accept Robot is the second closest to the ball

42

Table 17: Long Pass

Robot that initialises the pass Supporter

Objective Perform an spontaneous dynamic pass during the game when the

conditions necessary are met

Description When the ball is our possession the supporter will try to find a good

position to receive a pass. When this is achieved and if the attacker

agrees that the pass is a good solution a dynamic pass is done.

The attacker only turns to the direction of the supporter and passes

the ball

Pre-conditions to request Ball in our possession

Is in a good position to receive a pass with no obstacles in the

direction of the goal or obstacles between him and the attacker.

Conditions to accept Attacker has the ball

There are obstacles in the direction of the goal

All the PNs involved in these relational behaviours are explained in-depth below.

RoleAttacker

As mention in 3.2, when providing a description of the role RoleAttacker, when the robot is performing

the individual behavior BehaviorBaseAttack there is a chance that a dynamic pass can be performed.

In Figure 15 is shown the part of this role that pertains to the dynamic pass.

Figure 15: Dynamic pass in the RoleAttacker

43

From Figure 15 we can ascertain that while the robot is executing the individual behavior, in this

case BehaviorBaseAttack, the procedure needed to perform a dynamic pass is running in parallel.

The memory place DynamicPassControl guaranties that the attempt to perform a dynamic pass is

only done after a previous attempt ended. Without this place the number of tokens in the place macro.

CommitmentEstablishmentDynamicPassAttacker would increase without control.

In order for the robot to attempt to perform a commitment one of two things must occur:

Robot received a request from another robot to perform a dynamic pass, predicate

DynamicPassRequest TRUE;

Conditions for the robot to initialize a pass with another robot are met, predicate

DynamicPassConditionsOK TRUE. Note that the conditions for a given pass changes

depending of which pass we want to perform.

If the commitment is not a success, the memory place DynamicPassControl gets a token again and

the cycle begins anew. On the other hand, if the commitment is a success, the robot stops performing

the individual behavior and starts executing the relational behavior, in this case DynamicPassTaker.

The relational behavior only ends when the pass was a success, predicate DynamicPassEnded

TRUE, or if the pass had to interrupted, StopDynamicPass TRUE, which happens when the game was

Stopped, the time allotted for the pass has expired or if the conditions to perform the pass are no

longer met.

If the pass is a success, the robot that was the receiver will have the ball but if the pass was

interrupted the robot that should pass the ball will keep the ball.

Finally, macro CommitmentBreakingBC is executed in order to break the commitment and the

individual behavior is performed again.

RoleSuporter

Role support will also need to contain the structure needed to perform the dynamic pass since the

supporter will be the one to receive the ball. From Figure 16 we can ascertain that the protocol used in

the supporter is exactly the same as in the attacker. The only thing that changes is the individual and

relational behaviors as well as the macro to perform the commitment.

Note that this formalism works not only for a dynamic pass but also for any type of relational

behavior that we choose to implement in the robots. Also, if we wanted to include more robots in the

relational behavior, the only thing needed was to add this structure to the role RoleDefender and

change the conditions to request and accept a relational behavior.

44

Figure 16: Dynamic pass in the RoleSupporter

Macro CommitmentEstablishmentDynamicPassAttacker

Macro CommitmentEstablishmentDynamicPassSupporter

These two macros are responsible for creating a commitment for a dynamic pass and they both follow

closely the establishment of a commitment as described in 2.5.1 with the following differences:

Removal of existing timers described in point 1 and 3. This allows R1 to stay in the position in

which the request is sent, provided that the conditions for a dynamic pass continue favorable.

There are no messages of confirmation sent. Rejection messages are sent in the same step has

the message of commitment.

Each macro used for establishing a commitment has the procedure to request a commitment and

to accept a commitment because each robot can be the one requesting a commitment or the one

accepting a commitment, depending on the relational behavior. Note in appendix A.5 and A.6 that

the macros are the same the only thing that changes is the fact that for the attacker, in the

presence of an short pass, he’s the one doing the request and in the case of a long pass it will be

the one accepting the pass and vice versa for the supporter.

In the end of establishing a commitment there is an action that informs the robots what pass is

being performed because the PNs executed by the taker and the receiver are the same and what

will determinate what actions will be performed, are the predicate DoingShortPass and predicate

DoingLongPass.

BehaviorDynamicPassReceiver

When this behaviour is selected, the robot knows that it will receive a dynamic pass therefore the first

action executed is Move2DynamicPassReceive. This position is different if the robot is performing a

short pass or a long pass, nevertheless, as soon that this position is achieved, action

45

SendReady2Receive is performed in order for the partner to know that he can pass the ball. First point

of synchronism.

When the predicate PasDone is TRUE, second point of synchronism, the robot moves towards the

ball with action Move2Ball and when is close to it, predicate ColseToBall TRUE, action CatchBall is

executed. Finally, when the robot catches the ball, action SendDybnamicPassEnded is performed in

order to inform all the robots that the pass ended, third point of synchronism.

Table 18 summarizes all the info pertaining to this PN. In appendix A.13 this PN can be consulted.

Table 18: BehaviorDynamicPassReceiver

Name BehaviorDynamicPassReceiver

Type Individual behavior in a relational situation

Objective Selecting the actions needed to receive the ball

in a dynamic pass

action Move2DynamicPassReceivePosition, Stop,

SendReady2Receive, Move2Ball, CatchBall,

SendDynamicPassEnded

MeRMaID Level BehaviorExecutor

Appendix A.13

BehaviorDynamicPassTaker

When this behavior is selected the robot knows that it will perform a pass so the first action executed

is TurnAroundBallToPartner since it is the job of the receiver to go the receiving position. If the robot is

turned to the partner, it executes action Stop and wais for confirmation that the partner is ready to

receive, predicate PartnerReady2Receive TRUE, however, while waiting the robot might lose the ball

so actions Move2Ball and Catch ball is also included in the plan. When predicate

PartnerReady2Receive is TRUE, first synchronism point, the kick the ball and the send pass done if it

is a long pass or it will execute MoveAwayFromBall and then send pass done if it is a short pass.

Table 19 summarizes all the info pertaining to this PN. In appendix A.14 this PN can be consulted.

Table 19: BehaviorDynamicPassTaker

Name BehaviorDynamicPassTaker

Type Individual behavior in a relational situation

Objective Selecting the actions needed to pass the ball in

46

a dynamic pass situation

action Move2Ball, CatchBall,

TurnAroundBallToPartner, Stop, Kick,

SendPassDone, MoveAwayFromBall

MeRMaID Level BehaviorExecutor

Appendix A.14

47

4 Experimental Platform and Methodology

4.1 ISocRob Team

4.1.1 Robots

In [11] an in-depth description of team IsocRob is provided and in Figure 17, the omnidirectional

robotic soccer platform currently used by the ISocRob team is shown. This platform was developed

jointly between ISR/IST and the Portuguese SME IdMind. The following are the most relevant details

regarding the capabilities of its actuators and sensors:

Actuators:

Each of the robot's three Swedish wheels is actuated by a MAXON DC motor (model

RE35/118776), through a MAXON gear (model 203118) with a reduction of 21:1, providing the

robotic platform with a maximum translational speed of approximately 3.5 m/s and maximum

rotational speed of 20rad/s;

In order to kick the ball, an electromagnetic strength controlled kicker is used;

To aid in ball dribbling, a rolling drum is present near the kicker, with controllable rolling speed and

elevation.

Sensors:

The robot's vision system is based on an AVT Marlin F-033C firewire camera, which is equipped

with a fish-eye lens providing a field-of-view of 185°, facing downwards. This dioptric system

endows the robot with omnidirectional vision, capable of detecting relevant objects (such as the

ball and other robots) at a distance of up to 5 m. This particular setup is also less sensitive to

vibrations caused by the robot's motion than the previously used catadioptric system;

Each of the robot's motors is coupled to a 500 CPR encoder for motor control and odometry;

An AnalogDevices rate-gyro (XRS300EB) is present to improve self-localization.

In this robotic platform, the software architecture runs on a NEC FS900 laptop, equipped with a

Centrino 1.6GHz CPU and 512Mb of RAM.

The description of the architecture was already provided in 3.1

Figure 17: Robot used in the project

48

4.1.2 Research Topics

The are several important research topics being performed under the SocRob project but the major

topics are:

MeRMaID Software Architecture;

Motion Control;

Self-Localization;

3D Ball Tracking;

Cooperative Object Localization;

Individual and Relational Behaviors, the focus of this thesis.

4.1.3 Webots Simulator

The robots aren’t always available for testing the behaviors and some behaviors, special the

organizational ones as well as the relational behavior are difficult to test in the real robots due to the

small dimension of the available field. Therefore, there is a need to use a simulator that can represent

the soccer field and the robots of the IsocRob team. For this the simulator Webots is used. In Figure

18 is shown the environment used to test things in the simulator.

Figure 18: Webots field

4.2 Experimental Methodology

In section 3.2, 3.3 and 3.4 all the PNs created during the course of this work were described in-depth.

All of them were tested in the simulator as well as in the real robots and general conclusions on these

tests will be presented in Chapter 6. However, since the focus of this work was to develop dynamic

49

passes and optimize the existing static passes, a more in-depth description of each of the tests

performed on them will be given in 4.2.1 with the results given in Chapter 5.

The analysis performed in the PNs were twofold. First, an evaluation on a possible behavior for the

attacker is performed with the objective of describing the procedure to determine how to obtain the

primitive action that has more impact in the overall goal of a behavior and, therefore, improve it.

Second, a direct comparison is made between the two passes in order to conclude which has a higher

probability of success. These two analyses are described in 3.4.2 and the results given chapter 5.

4.2.1 Tests in the Simulator and Real Robots

The situations tested in the simulator and the real robots can be divided in:

[Test01] – In this test, every possible foul is tested in the simulator and then in the real robots by

putting the robots in an arbitrary position and the ball in the correct position, depending of the type of

foul. First the referee box sends a signal that the game is stopped, then the type of foul to be tested

and finally the game is started;

[Test02] – In this test the goal is to perform a short pass. For this, the ball is put in a dropped ball

point, robots are located in a position in which they can’t see the ball to test simultaneously if the

correct robots are selected to search for the ball and if they can find it. The referee first gives a signal

of game stopped, followed by dropped ball and finally by game is started;

[Test03] - In this test the goal is to perform a long pass. For this, the ball is put in an arbitrary

position and the robots in a position from where they can’t see the ball. The referee just gives the

signal of game started.

4.2.2 Analysis Performed

As mentioned above two different situations were analysed in which the objectives were different. Both

were transient analysis. This analysis will be called Case 1 and Case 2 and will be described below.

Case 1

In this analysis, the goal is to have a way to determine which primitive action is more important for the

overall impact of one behavior. For that, the PN of Figure 19 was created with the objective of scoring

a goal in the opponent goal and the analysis that was performed was a transient analysis.

The behavior and all the environment and action models used in this case will be the same,

however, three tests will be performed:

1. With standard values for all stochastic transitions;

2. With InterceptBall improved;

3. With Dribble2Goal improved.

By comparing the results of these three tests, we will be able to ascertain which primitive has a

bigger impact in the probability of scoring a goal in opponent goal and, therefore, which primitive

action should be improved in the project.

50

This PN doesn’t exist in the project and therefore the values used in the stochastic transitions are

arbitrary but not unrealistic. In [14] is shown that the results obtained from this type of analysis are

identical to the ones obtained by using real values taken from the simulation so an a priori analysis

can be performed without having to test the behavior in the real robots beforehand and from it

important knowledge can be obtained.

Figure 19: Behavior of an attacker

In order to perform a transient analysis, the PN of Figure 19 has to be composed with the

environment models and with the action models. Figures 20 through 23 represent the models needed

for the environment and Figures 24 through 27 the action models. Note that in order to model the

improvement in the actions InterceptBall and Dribble2Goal only a small number of stochastic

transitions will have their rates changed. These transitions are identified when the environment and

action models are described.

Since the objective is only to ascertain the impact of InterceptBall and Dribble2Goal, the

environment models don’t have to be very complex, only needing to take in account the aspects that

represent the success of each primitive action in order for them to be improved.

In Figure 20 a model is given for the ball position. The field is divided in five positions and the ball

can travel between them by action of the environment or other agents, represented by the stochastic

transitions. However, we consider that while the robot has the ball in its position, only him can affect

the ball and, therefore, in the ball position model the transitions can only fire when the robot doesn’t

have the ball. Note that when the ball reaches the position BallOppGoal or BallOwnBall a goal was

scored and the ball can’t exit this position.

51

Figure 20: Ball Position

Figure 21 represents the position the robot can occupy in the field. It is consider that the robot

always knows its position so no stochastic transitions are needed.

Figure 21: Robot position

Figure 22 represents HasBall model. At any given time the robot can lose the ball and this is what

this model represents. Note that when improving the Dribble2Goal the rate of T1 should be decreased

in order to represent the fact that the robot will lose the ball less often.

Figure 22: HasBall model

Figure 23 represents ShootOpportunity model. After the action Dribble2Score finds a good spot to

shoot the ball, this spot can be lost before the robot has the chance to kick and, the model represents

this.

Figure 23: ShootOpportunity model

Figure 24 represents the action model for action InterceptBall. Basically the model makes the robot

position predicates change towards the ball position predicate that is true (s1 through s6), as long as

the robot sees the ball. When the robot is in the same position as the ball, it will catch the ball after a

delay (s7, s8, s9). Note that when we are improving this action only s7, s8 and s9 will have the value

changed since we consider that the robot already moves to the ball at full speed and only the part of

52

grabbing the ball will be improved.

Figure 24: InterceptBall action model

Figure 25 represents the action model of Dribble2Goal and what happens is that the robot takes

the ball from its current position to near the opponent goal. Note that when improving this action none

of the transitions will be changed because it is consider that the robot already moves the ball at

maximum speed so the improvement of this action is model only in the environment model HasBall.

Figure 25: Drible2Goal action model

Figure 26 represents the action model of Dribble2Score in which the robot tries to find a good

position to shoot the ball towards the goal.

53

Figure 26: Drible2Score action model

Figure 27 represents the action model of Kick. In this model it is modelled the fact that after the kick

the ball can end up in any part of the field.

Figure 27: Kick action model

To perform this analysis the predicates, in the composed PN, that have a token are BallMidField,

RobotNearOwnGoal and SeeBall. All the other predicates have a token in their negation.

Case 2

In this analysis the goal is to perform a transient analysis on both types of passes and determine

which one has the biggest probability of performing the pass with success. For that the PN in Figure

28 was obtained, representing the structure of relational behaviour in the BehaviorCoordinator level.

This PN will be composed with the environment models and action models as well as with

BehaviorDynamicPassReceiver and BehaviorDynamicPassTaker that can be found in appendix A.13

and A.14 and were described in 3.4.2. Note that in Figure 28 action.BehaviorDynamicPass

representes both BehaviorDynamicPassReceiver and BehaviorDynamicPassTaker running in parallel.

In this analysis the part of the commitment is not important so after the macros there are stochastic

transitions that represent the time that the macros spent running performing and breaking the

commitment.

54

Figure 28: Dynamic Pass on the BehaviorCoordinator level

All the environment and action models for both the passes are the same, the only that is different is

the initial marking of the composed PN used for analysis. Also, in order for the analysis to be more

accurate, the complexity of the models was increased when compared with the models in Case 1.

In this case it was consider that the field is divided in six positions as demonstrated in Figure 29.

Figure 29: Field Positions where the robots and the ball can be

In appendix B.1 all the environment models are given.

In appendix B.1.1 we have the model for BallMovement that represents the fact that the ball can be

moving or stopped because of the action of the environment or other agents. However, it is consider

that when the robot has the ball only him affects the ball movement, therefore, predicate NOT_HasBall

is added in this model as a condition for the change in movement to occur.

In appendix B.1.2 we have the model for ball position. This model is similarly to the one in Figure

20, however, since there are six positions in which the ball can be, the complexity of the model

increases a lot as it is easily observed.

In appendix B.1.3 we have the model for the predicate DynamicPassConditionsMet. For the long

pass, this predicate is true when robot2 is at R2X1Y3 position. For the short pass, this predicate is

55

true when robot1 has the ball. Note that for the short past, last dropped ball should also be a condition

and for the long pass it should be evaluated if the robot is in a good position to receive the pass and

not in a specific position, however, these conditions were not contemplated because it would increase

the complexity of the action and environment models.

Since we have two robots, we will have one robot position model for each one of them. These

models can be found in appendix B.1.11 for robot1 and appendix B.1.15 for robot2. Both models will

be similar the only difference is that robot1 has a seven position that will be used when running the

action MoveAwayFromBall.

Similarly, there will be different models for Hasball and CloseToBall for each robot. These can be

found in appendix B.1.8, B.1.9 for robot1 and B.1.12 and B.1.13. These are equivalent has the one

used for Hasball in Case 1.

As mentioned, for each robot there is a model for HasBall, however in the BallMovement and

BallPosition models there is a predicate NOT_HasBall as a condition for the transitions to occur so the

a model for this predicate has to be modelled. In appendix B.1.5 we can observe that if one of the

robots has the ball, predicate HasBall is TRUE and if none of the robots has the ball, predicate

HasBall is FALSE.

Predicate StopDynamicPass is modelled in appendix B.1.16 and this predicate will be TRUE only

when the time to perform a pass has ended. This is represented by the stochastic transitions in this

PN.

Finally, in appendix B.1.10 we have R1IsTurnedToPartner that changes the value of this predicate

to FALSE when a given time has passed, and in appendix B.1.14 we have a PN that changes the

value of this predicate when the robot2 is not in the receiving position for each type of pass.

In appendix B.2 all the environment models are given.

In appendix B.2.1 we have the model for action R1CatchBall that is equal to the model for action

R2CatchBall in appendix B.2.5. The actions have as running conditions R1/R2CloseToBall and the

time to spent in catching the ball will depend on whether the ball is moving or not.

In appendix B.2.2 we have the model for action R1Kick that is similar to the one shown in Figure 27.

The difference is in the position the ball can go and the fact that there is also an immediate transition

that models the fact that after the kick, if the ball was stopped, it is moving now. Note that it was only

considered that the ball can only go to the three positions in front of the robot and not all the positions

in which the field was divided.

In appendix B.2.3 we have the model for action R1MoveAwayFromBall that sends the robot from

any of the six positions in which the field was divided to one outside that division (position XY).

In appendix B.2.4 we have action R1TurnToPartner in which predicate IsTurnedToPartner is put as

TRUE after a delay that represent the time the robot spens turning to the partner.

In appendix B.2.6 the action R2Move2ReceivePosition is shown. In this model the robot moves

from the position that it is to the position in which it will receive the pass. When this position is reached

56

an immediate transition will change the value of predicate IsAtReceivePosition to TRUE.

Finally in appendix B.2.10 we have the action model for RXMove2Ball that is used by both robots.

The robot running this action will move from its position to the position where the ball is. When the

position is reached predicate CloseToBall will become TRUE after a delay that depends on whether

the ball is moving or stopped.

As mentioned before, the communications used for the synchronism between the receiver and the

taker are done by a pair action/predicate. In appendix B.2.7, B.2.8 and B.2.9 we have the models for

the actions of these pairs. The only thing that this action does is to change the value of each predicate

pair associated with each action to TRUE. In appendix B.1.4, B.1.6 and B.1.7 we have the models of

the predicates of these pairs. The only thing that these models do is to change the predicates values

back to False after a small amount of time has elapsed.

Since the same models are used for both types of pass, the transitions can be arbitrary as long as

they aren’t unrealistic due to the fact that what we want to conclude is how much better the probability

of one pass is in relation to the other.

In order to test the short pass, the composed PN will have the following predicates with a token:

BallX2Y1, R1X2Y2, R2X1Y3, DoingShortPass, DynamicConditionsMet, GameIsStarted,

HasBall,R1HasBall, R1CloseToBall, SeeBall, BallStopped.

Note that if the predicate was not mention in the list above, the negation of the predicate is the one

that has a token.

In order to test the long pass, the composed PN will have the following predicates with a token:

BallX1Y1, R1X1Y1, R2X1Y3, DoingLongPass, DynamicConditionsMet, GameIsStarted,

HasBall,R1HasBall, R1CloseToBall, SeeBall, BallStopped.

Note that if the predicate was not mention in the list above, the negation of the predicate is the one

that has a token.

57

5 Results and Analysis

5.1 Webots Simulation and Real Robots

All the PNs described in section 3.2, 3.3 and 3.4 were tested in the Webots and in the real robots as

well as [Test01], [Test02] and [Test03]. Videos of all of this can be found in

http://socrob.isr.ist.utl.pt/videos/behaviors/behaviors2011.html.

In these tests it was possible to ascertain that all the Organizational behaviors perform correctly

when tested in the simulator since the communications, the ball localization and self-localization don’t

have failures. However, when these are tested in the real robots, the robot doesn’t always select the

right behaviors because of failures in these systems. Nevertheless, there is no way around this

problem unless to try to make this systems the more robust possible since all that can be done in the

cortex and predicate manager is already being done (hysteresis and using only information from the

other robots that is not older than one second).

About the behaviors in the BehaviorExecutor level, the same type of errors occurs in the real robots

because of failures on those same systems. Primitives that need the robot to grab the ball are the

ones that present most failures because currently there is no mechanism to grab the ball. The other

primitives perform correctly.

By testing all game situations that can happen in a game, no deadlock or livelock are detected and

the robots present enough behaviors to interact in all situations.

In particular, when [Test01], defined in section 4.2.1, is tested in the simulator if the robot is not

seeing the ball, the correct two robots that have to find it are always selected and the ball is always

found if given enough time to search possible positions with macro FindFoulPosition. Note that for a

free kick and a thrown in the robot never searches the ball because the possibilities on where the ball

could be are many and by experience, in competitions, when the this two types of foul occur, almost

always a robot sees the ball. After finding the ball, if the foul is for the opponent BehaviorFoulDefend is

performed and the robots defend against the foul correctly. On the other hand, if the foul is for us

one robot will be the Taker and the other the Receiver. While performing this pass, we see that the

place where the pass doesn’t go smoothly is when the receiver tries catching the ball which

demonstrates that a better primitive action to intercept the ball has to be developed for when the ball is

moving.

If [Test01] is performed in the real robots we find that sometimes the robots that should search for

the ball are not selected. This happens because of failures in self-localization which is the base to

select what are robots that are going to find the ball. If the robots find the ball then everything works

fine unless there is major failure in ball localization. When the receiver is trying to catch the ball it has

more difficulty than in the simulator.

Noting more can be observed from this test.

For [Test02] everything works fine in the simulation, even catching the ball by the receiver,

because the ball is stationary.

58

In the real robots, the usual errors might occur. The robots that should look for the ball during

dropped ball might not be selected because self-localization and when the attacker catches the ball

and tries to perform the commitment with the supporter, failures in communications might prevent it. In

this case, the commitment doesn’t occur and the robots will continue their individual behaviors.

Nevertheless, failures in communications don’t occur often.

After performing the commitment and as the robots execute their cooperative behaviors, several

errors can occur. When the attacker tries to turn to its partner, he might face the wrong direction

because the position that he received from the partner was wrong, because the self-localization was

not correct. The other problem is while performing action TurnArounBallToPartner, the robot tends to

lose the ball and as a result has to catch it again. Nevertheless if the robot manages to get to the right

position, the receiver easily catches the ball because the taker leaves the ball stationary in front of the

receiver.

For [Test03] the same problems occur that were already observed in [Test02] plus the fact that,

when the ball is passed to the receiver, the ball will have movement which will cause a lot of trouble

for the robot that has to catch it.

5.2 Theoretical Analysis

The composing of the PNs for Case 1 and Case 2 to be analysed was performed using the algorithms

described in [14] and the actual analyses was performed using TimeNet [19].

In Case 1 the goal was to see if by improving one primitive action at a time it would be possible for us

to identify the one that contributes more to the overall goal of the behavior. This will be a transient

analysis and what is measured is the probability of having a token in predicate BallOppGoal, that

represents a goal for our team.

In Figure 29 and 30 the results for the three situations analysed are shown. In blue we see the

probability of scoring a goal with the base values for the transitions. In green we see the results for

when the InterceptBall is improved by around 50% and finally in magenta we see the results of when

the Dribble to goal is improved 50%.

59

Figure 30: Results for Case 1 when a big amount of time passes

Figure 31: Results for Case 1 when a small amount of time passes

From Figure 29 we can see that improving the primitive InterceptBall the probability of scoring a goal

in the opponent goal is bigger than when action Dribble2Goal is improved. With Figure 30 we see that

this probability increases much faster when the InterceptBall is improved.

In Case 2 the goal was to do a direct comparison between the short pass and long pass. For that a

transient analysis was done to determine the probability of having a token in predicate R2HasBall

which indicates that the pass was a success.

60

Figure 32: probability for the receiver to catch the ball in a short and long pass

5.3 Conclusions

By comparing the results from the simulator with the test on the real robots, we can conclude that the

main reason for a behaviour to fail is failures in the self-localization, ball localization and

communications and not the logical PN model. The only way of improving the success is to make

these systems more robust or trying to make the predicates more reliable even though in most of the

predicates, hysteresis are being used and information from other robots older than one second is not

accepted.

By testing in the real robots the behaviour we can reach the conclusion that the actions that depend

on the mechanism to grab the ball are actions that the robot has most trouble performing and a close

second are the primitive actions that try to move to the ball in particular when the ball has movement.

These are important areas that need to be improved in order for the ISocRob team be more

competitive.

The theoretical analyses performed on the Petri nets allowed for important information to be obtained.

With Case 1 we see that improving the action InterceptBall improves the behaviour a bit more than

that an improvement in Dribble2Goal does. An evaluation like this can be performed for any behavior

with the objective of detecting the primitive actions that are more critical for the overall objective.

With Case 2 we can conclude that the short pass is safer that the long pass like we had already

conclude by performing [Test02] and [Test03] in the real robots. Since a dynamic pass has to be

performed after a dropped ball situation, the option of using short pass is the best one because it has

a bigger probability of success.

61

6 Conclusion and Future Work

The objectives proposed for this dissertation were met. All the situations that can arise during a

competitive match are now contemplated and the PN modelled allow for the correct response by the

robots. From the results obtained in the simulator and real robots we can conclude that all the

behaviors are correctly modelled and that problems in their execution arise when communications,

self-localization and ball localization fail. To improve the success rate of all the behaviors it is

necessary to improve these systems in order for them to be more robust. Another way to tackle this

issue is to improve the predicates, since it’s them that will evaluate the state of the world. However,

this is difficult because it is almost impossible to predict and correct errors at this level that arise from

the mention systems.

The execution of behaviors is more stable and robust and, currently, the only thing that should be

done in the cortex is to have more than one type of behavior for each situation that arises during the

game. At this time there is only one possible behavior that is selected for each game situation and it

would be interesting to see the robot performing different things each time the same situation occurred

(e.g. take a foul with receiver in different ways). Nevertheless, the complexity of each behavior

shouldn’t be increased until communication, self-localization and localization aren’t more stable. Apart

from this tests in the real robots showed that time should be invested in the mechanism to catch the

ball and keep it controlled as well as time in develop the primitive action InterceptBall. At these point,

it’s is safe to say that the range of what can be implemented in the PN, when taking in account the

limitations of the ISocRon team, has been stretched to the limit and behaviors with a higher complexity

level should only be implemented in the simulator.

Apart from this, a new methodology was implemented to be used in a dynamic pass, however, it

can be used in any type of relational behaviour that we want to perform during the game. For that, only

the preconditions to request a commitment and the conditions to accept that same commitment

change as well as the behaviour performed during the joint task. In the sphere of the robotic soccer

the main goal for performing a relational behaviour is a pass between two robots, and for this reason

two dynamic passes were created. One that intends to be safer and another that is more of real pass

in the true sense of the word, but also has a bigger failure rate.

This methodology for relational behaviors can be extended to include more than two robots when

trying to perform a join task, and to other domains.

Other important objective was to make use of the framework develop in other work [14] and apply it

in real situations that could occur in a competition setting. This led to the creation of actions and

environment models based in Petri Nets that are composed with the Behavior that runs in the robot in

order to obtain a single model in which a quantitative analyses can be performed using Markov

Chains. With the analysis, relevant information can be obtain by performing an a priori analysis without

having the need of executing the PN in the simulator or in the real robots.

Regarding the three levels in the cortex, the formalism should be changed and instead of only

having the prefix action to execute roles, behaviors or primitive actions, depending on which level they

are called, three new prefixes should be introduced. These should be role, behavior and macro. This

way the prefix action would only be used to execute primitive actions and for roles, behaviors and

62

macros, the new prefixes would be uses. With this, more flexibility could be achieved in the Cortex and

problems experienced in cases like with RoleStop, that should not be a role but a primitive action,

would not occur.

On a final note, it is import to stress the fact that while the way communications are implemented in

the project is the best way of communication robot information like the robot posture but in terms of

the communication needed to perform a commitment, is not a good solution and, therefore, steps

should be taken to improve this point.

63

Bibliography

[1] P. R. Cohen and H. J. Levesque. Teamwork. Nous, 35:487-512, 1991

[2] João Gonçalo Delgado Torres, Comportamentos Relacionais Para Robots Futebolistas: Análise de

Incerteza, Dissertação para obtenção do Grau de Mestre em Engenharia Electrotécnica e de

Computadores, Instituto Superior Técnico, 2007

[3] Tiago Miguel Baltazar Coelho de Moura Antunes, Comportamentos Relacionais Para Robots

Futebolistas: Análise Lógica, Dissertação para obtenção do Grau de Mestre em Engenharia

Electrotécnica e de Computadores, Instituto Superior Técnico, 2007

[4] Artur Manuel Costa Pinto Prego de Faria, Desenvolvimento e Análise de Comportamentos

Relacionais em Robots Futebolistas,Dissertação para obtenção do Grau de Mestre em Engenharia

Electrotécnica e de Computadores, Istituto Superior Técnico 2008

[5] Nelson Miguel Silva Ramos, Arquitectura de Comportamentos para Equipa Multi-Robot com

Coordenação por Arbitragem, Dissertação para obtenção do Grau de Mestre em Mestrado em

Engenharia Informática e de Computadores, Instituto Superior Técnico, 2007

[6] Marco Alexandre Fagulha Barbosa, Development Environment for a Multi-Robot Team, Dissertação

para obtenção do Grau de Mestre em Engenharia Informática e de Computadores, Instituto

Superior Técnico, 2007

[7] Vittorio Amos Ziparo and Luca Iocchi, Petri Net Plans, 2006

[8] T. Murata, Petri Nets: Properties, Analysis and Applications, Proc. of the IEEE, Vol. 77, No. 4, pp

541-580. 1989.

[9] N. Viswanadham and Y. Narahari., Performance Modeling of Automated Manufacturing Systems,

Prentice Hall, New Jersey, 1992, 487-512

[10] MSL Technical Committee, Middle Size Robot League Rules and Regulations for 2010, 2010

[11] Pedro Lima, João Messias, João Santos, João Estilita, Marco Barbosa, Aamir Ahmad, João

Carreira, ISocRob 2009 Team Description Paper, Instituto Superior Técnico, 2009

[12] Hugo Costelha e Pedro Lima, Modelling, Analysis and Execution of Robotic Tasks using Petri

Nets, Proc. of IROS - IEEE, 2007

[13] PNP: A Formal Model for Representation and Execution of Multi-Robot Plans

64

[14] Hugo Filipe Costelha de Castro, Robotic Tasks Modelling and Analysis Based on Discrete Event Systems, Dissertation to obtain the Doctorate Degree in Electrical and Computer Engineering, 2010

[15] WEBOTS, http://www.cyberbotics.com/

[16] http://socrob.isr.ist.utl.pt/

[17] http://www.robocup.org/

[18] PIPE, http://pipe2.sourceforge.net

[19] TimeNet, http://www.tu-ilmenau.de/fakia/8086.html

[20] http://en.wikipedia.org/wiki/Petri_nets

65

Appendix A – Petri Nets Developed

A.1- TacticBase

66

A.2- RoleAttacker

67

A.3- RoleDefender

68

A.4- RoleSupporter

69

A.5- Macro CommitmentEstablishmentDynamicPassAttacker

70

A.6- Macro CommitmentEstablishmentDynamicPassSupporter

71

A.7- Macro FindFoulPosition

72

A.8- ReceiveFoul

73

A.9- TakeFoul

74

A.10- BehaviorBaseAttack

75

A.11- BehaviorBaseDefend

A.12- BehaviorBaseSupport

76

A.13- BehaviorDynamicPassReceiver

77

A.14- BehaviorDynamicPassTaker

A.15- BehaviorFindBallInCorner

78

A.16- BehaviorFoulDefend

79

A.17- BehaviorFoulReceiver

A.18- BehaviorFoulTaker

80

A.19- BehaviorFoulTakerAlone

81

A.20- BehaviorPenaltyTaker

82

83

Appendix B – Models used in analysis

B.1- Environment Models

B.1.1- BallMovement

B.1.2- BallPosition

84

B.1.3- DynamicPassConditionsMet

B.1.4- DynamicPassEnded

B.1.5- HasBall

85

B.1.6- PartnerReady2Receive

B.1.7- PassDone

B.1.8- R1CloseToBall

B.1.9- R1HasBall

B.1.10- R1IsTurnedToPartner

B.1.11- R1Position

86

B.1.12- R2CloseToBall

B.1.13- R2HasBall

B.1.14- R2IsAtReceivePosition

B.1.15- R2Position

87

B.1.16- StopDynamicPass

B.2- Action Models

B.2.1- R1CatchBall

B.2.2- R1Kick

88

B.2.3- R1MoveAwayFromBall

B.2.4- R1TurnToPartner

B.2.5- R2CatchBall

89

B.2.6- R2Move2ReceivePosition

B.2.7- SendDynamicPassEnded

B.2.8- SendPassDone

B.2.9- SendReady2Receive

90

B.2.10- RXMove2Ball