Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in...

75
Swarm Intelligence – W10: Division of Labor I: Threshold-Based Algorithms

Transcript of Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in...

Page 1: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Swarm Intelligence – W10:Division of Labor I:

Threshold-Based Algorithms

Page 2: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Outline

• Division of labor in Natural Systems– Motivation from social insects– Biologically plausible models

• Threshold-based algorithms– Principles and foundations– An example of non-robotic application

• Division of labor in Artificial Systems– Motivation from mobile robotics– Taxonomy of approaches

• Threshold-based algorithm applied to multi-robot coordination– Examples

Page 3: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Division of Labor in Natural Systems:

Motivation and Overview From Social Insects

Page 4: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

• The most obvious sign of the division of labor is the existence of castes.

• We distinguish between three kinds of castes: physical, temporal (temporal polyethism) and behavioral.

• The individuals belonging to different castes are usually specialized for the performance of a series of precise tasks.

The Division of Labor and its Control

The control of task allocation

Page 5: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending
Page 6: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Physical Castes (Wilson, E. O., 1976)

In Pheidoleguilelmimuelleri the minors show ten times as many different basic behaviors as the majors

0 02 04 06

Self-groomingMinor worker

Dealate queenMale

Carry or roll eggCarry or roll larvaFeed larva solidsCarry or roll pupa

Assist eclosion of adult

Minor workerDealate queen

MaleForage

Lay odor t railFeed inside nest

Agression (drag or at tack)Carry dead larva or pupa

Feed on larva or pupaLick wall of nestAntennal tipping

Guard nest entrance

M ino r

0 02 0 4 06

M a jo r

Behavioral repertoires of majors and minors

Average fraction of time spent in a given activity/behavior

Page 7: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Temporal Polyethism

Cleaning cells

Tending brood

Tending Queen

Eating pollen

Feeding & grooming nestmates

Ventilating nest

Shaping comb

Storing nectar

Packing pollen

Foraging

Patrolling

Resting

1 4 7 10 13 16 19 22 25

20

20

20

20

40

40

40

10

10

10

10

10

10

10

10

40

6080

Perc

enta

ge o

f tim

e sp

ent i

n ea

ch a

ctiv

ity

Age of bee (days)

Behavioral changes in worker bees as a function of age

Young individuals work on internal tasks (brood care and nest maintenance). Older individuals forage for food and defend the nest.

Page 8: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Behavioral Castes

From D. Gordon, “Ants at Work”, 1999

Allocation of the dailyactivities in a colony of desert harverster ants (Portal, AZ)

Page 9: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

• The number of individuals performing different tasks and the nature of the tasks to be done are subject to constant change in the course of the life of a colony.

• The proportions of workers performing the different tasks variesin response to internal or environmental perturbations.

• This is true under certain conditions even when hard morphological differences exist or irreversible aging processestake place.

Flexibility of social roles

The Division of Labor and its Control

→ The division of labor in social insects must flexible.

Page 10: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

How does the colony control the proportions of individuals assigned to each task, given that no individual possesses any global representation of the needs of the colony?

• The flexibility of the division of labor depends on the behavioral flexibility of the workers.

• A mechanism arising from the concept of a response threshold allows the production of this flexibility. Or at least it represents a biologically plausible model for explainingsuch division of labor facts in social insect colonies.

Flexibility of social roles

The Division of Labor and its Control

Page 11: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

How is flexibility implementedat the level of the individual?

Page 12: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

The Division of Labor and the Flexibility of Social Roles

The control of task allocation

3

1

2 ?

Task 1 Task 2 Task 3

How is dynamic task allocation achieved?

Page 13: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

σi1σi1σi1

σi1σi1σi1

σi1σi1σi1

3

1

2

The control of task allocation explained with a fixed-threshold model

The Division of Labor and the Flexibility of Social Roles

The lower the threshold, the lower can be the stimulus for achieving a given response; respectively, the lower the threshold, the higher will be the response of an individual for a given stimulus.

Page 14: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

An Example of Control of the Division of Labor Implying the Existence of Fixed Response

Thresholds

Page 15: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

© Guy Theraulaz

Page 16: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

0

1

2

3

4

5

6

7

8

9

0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1

Fraction of majors

Pheidole pubiventris

Self-grooming

Social behavior

The idea of a response threshold

Beh

avio

ral A

cts/

Maj

or/H

our

Beh

avio

ral A

cts/

Maj

or/H

our

0

1

2

3

4

5

6

7

8

9

10

0,0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0

Fraction of majors

P. guilelmimuelleri

Self-grooming

Social behavior

The Division of Labor and its Control

Page 17: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

A Macroscopic Model based on Fixed Thresholds for 1

Task and 2 Castes

Page 18: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Using a Model for Explaining Wilson’s Results and more …

• Macroscopic model, continuous time• Response to stimuli based on fixed thresholds• No detailed microscopic mechanisms for stimuli

perception and answer incorporated in the model • Assumptions: nonspatial model, i.e. equiprobable

exposure of individuals to the stimuli associated with the task and stimulus homogeneously distributed overspace

• 1 task (e.g., social behavior), 2 castes (e.g., minor andmajor)

Page 19: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Task 1Given Task

θj

Task 1

θi

An example of a response threshold

0

0,25

0,5

0,75

1

0,1 1 10 100

Stimulus

Majors

MinorsT(

s) R

espo

nse

Pro

babi

lity

T (s) = sn

sn + θinθi

s : intensity of the stimulus associated with the taskθi : response threshold of caste i to the taskn : nonlinearity parameter, e.g. n = 2

θminors θmajors

The Division of Labor and its Control

n=2

Given Task

Page 20: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Properties and Parameters of the Threshold-Based Response Function

Different θ for n = 10 Different n for θ = 50

Page 21: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Deterministic vs. Probabilistic Responses

Deterministic response for θ = 50 and various levels of Gaussian noise (1000 runs)

Deterministic with noise ( σ = 9) vs. probabilistic response (n = 10) for θ = 50 (1000 runs)

Page 22: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Note on the Macroscopic Model and its Variables

Average time spent by an individual i carrying out task j

Average number of active individuals in caste i carrying out task j

xij

1 individual = 1 castecaste i = set of individuals with the same threshold

nij, Ni

Mainly in the course book

Mainly in the lecture slides

Concerned Variable

Note 1: depending on the significance of the variable some model parameter might slightly change their meaning (per unit step, per individual, etc.)

Note 2: at the macroscopic level we do not deal with details, just need statistical meaningful state variables; SI has among its ingredients multiple interactions; that can be achieved with a lot of individuals or a few individuals interacting a lot over time and repeated runs

Note 3: see also week 8 lecture slides and Lerman et al, SAB 2004 paper

Page 23: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Fixed Threshold Model with1 Task and 2 Castes

Two castes of individuals in the colony (physical, behavioral, or age-based castes);each individual can perform the task or doing nothing. Note ni and Ni swapped in comparison to your book!

n1 = mean number individuals of type 1 engaged in carrying out the taskn2 = mean number of individuals of type 2 engaged in carrying out the task

x1 = n1 / N1 : mean fraction of individuals of type 1 engaged in carrying out the task (or average fraction of time/number of actions spent in carrying out the task)x2 = n2 / N2 : mean fraction of individuals of type 2 engaged in carrying out the task (or average fraction of time/number of actions spent in carrying out the task)

1-x1: fraction of individuals of type 1 not performing the task (or average fraction of time/number of actions spent in not carrying out the task)1-x2: fraction of individuals of type 2 not performing the task (or average fraction of time/number of actions spent in not carrying out the task )

N1 = number of individuals of type 1N2 = number of individuals of type 2

N1 + N2 = N = total number of individuals in the colony

Page 24: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Dynamics of the fraction of active individuals in each caste:

s : intensity of the stimuli associated with the taskθ i : response threshold of caste i to the taskra : rate of task abandoning (i.e. probability per infinitesimal time intreval that an

active individual abandons the task on which it is engaged for moving to idle)1/ra : average time spent by an individual in working on a task before abandoning it

= (1-x1) - ra x1s2

s2 + θ12

= (1-x2) - ra x2

s2

s2 + θ22

∂t x1

∂t x2

Fixed Threshold Model with1 Task and 2 Castes

dtdxxt =∂Note:

Page 25: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Dynamics of demand associated with the task

∂t s = δ – (n1 + n2)α

N

δ : rate of stimulus increase (demand common to both castes)α : normalized effectiveness rate for the individual contribution on the task

(in this case an active individual of type 1 contribute in the same way as an active individual of type 2 when performing the task)

Fixed Threshold Model with1 Task and 2 Castes

Page 26: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

0

20

40

60

80

0 0,25 0,5 0,75 1

Fraction of majors

N=1000

N=100

N=10

Number of acts per major as a function of the fraction of majors for different sizes of colony

Parametersof the simulation

N = 10, 100, 1000θ1 = 8, θ2 = 1α = 3δ = 1ra = 0.2

Num

ber o

f act

s p

er m

ajor

durin

g th

e s

imul

atio

n

Fixed Threshold Model with1 Task and 2 Castes

Page 27: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

0

2

4

6

8

10

0

25

50

75

0 0,25 0,5 0,75 1

Fraction of majors

simulation N=100

Pheidole pubiventris

Pheidole guilelmimuelleri

simulation N=10

Comparison between the macroscopic model and experimental results

Num

ber o

f act

s pe

r maj

ordu

ring

the

sim

ulat

ion

Num

ber o

f act

s pe

r maj

ordu

ring

the

real

exp

erim

ent

Parametersof the simulation

N = 10, 100θ1 = 8, θ2 = 1α = 3δ = 1ra = 0.2

Fixed Threshold Model with1 Task and 2 Castes

Page 28: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Variable Threshold Model for Controlling the Division

of Labor

Page 29: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

3

1

2

σi1σi1σi1

σi1σi1σi1

σi1σi1σi1

+-+-+-

+-+-+-

+-+-+-

The Division of Labor and the Flexibility of Social Roles

The control of task allocation explained with a variable-threshold model

The lower the threshold, the lower can be the stimulus for achieving a given response; respectively, the lower the threshold, the higher will be the response of an individual for a given stimulus.

Page 30: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

© Guy Theraulaz

Polist Wasps: again an Interesting Example

Page 31: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Origins of the Division of Labor in the Polist Wasps

Polists : primitive eusocial species

• Colonies usually contain only a small number of individuals (ca 20).

• These species do not show morphological differences between castes in the adult stages, nor any control or physiological determination of the role an individual will play in the colony as an adult.

• Individual behavior is very flexible; all individuals are able to perform the whole range of tasks which determine the survival of the colony.

• The integration and coordination of individual activities is achieved through the interactions which occur between the members of the colony, and between the members of the colony and the local environment.

Page 32: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

The role of learning in the differentiation of activities

• The state of the brood triggers the activities of foraging and feeding the larvae.

• In adults there are different response thresholds and these thresholds vary as a consequence of the individuals’ activities.

• Within an individual, the act of setting out on a foraging task has the effect of lowering the response threshold for larval stimulation.

• There is therefore a self-exciting process (positive reinforcement/feedback mechanims) bringing about the specialization of individuals which have carried out foraging tasks; this process can be described by means of a variable threshold model.

• Specialists can be created from generalists

Origins of the Division of Labor in the Polist Wasps

Page 33: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Properties of Mechanisms Arising from Adaptive Response Thresholds

Page 34: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

i : caste index [1 …m] s : intensity of stimuli associated with the taskθi : response threshold of individual i to the task: θi ∈ [θmin,θmax ]ξ : incremental learning parameter (the threshold of an individual carrying

out a task is reduced by ξ) ϕ : incremental forgetting parameter (the threshold of an individual not

carrying out a task increases by ϕ)

Individual behavioral algorithm – Fixed-Step In-Line Search Algorithm

θi θi - ξ when i performs the task→

θi θi + ϕ when i does not perform the task→

T (s) = s2

s2 + θi2θi

Parameters of individuals

Variable Threshold Model with1 Task and m Castes

Page 35: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Note from Lecture of Week 9

• An adaptive in-line Search algorithm could be used (e.g. that presented in lecture 9 for the stick pulling experiment) as well for tuning the individual thresholds

• A diversity and specialization analysis based on social entropy and its correlation with swarm performance could be used also in this case as it was used for the stick-pulling experiment.

Page 36: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

YES

NO

Given Task

θi

θi

−ξ

θi

Description of the algorithm

∂t xi = Tθi (s)(1-xi) - ra xi

Average duration = 1/ra

Execute task

ra: abandoning rate (as before for fixed thresholds)

Variable Threshold Model with1 Task and m Castes

System of DE:

Page 37: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

∂t s = δ – ( ni)α

N

Example of dynamics of the overall stimulus (demand) associated with a task

Σi= 1

m

:

δ : rate of stimulus increaseα : normalized effectiveness rate for the individual contribution to the task (in

this case all the active individual belonging to different castes contributein the same way)

ni: number of active individual belonging to caste i

Variable Threshold Model with1 Task and m Castes

Page 38: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Example with 6 Castes (θ1 … θ6)

Thresholds’ evolution

Model Parametersθi = [1 …1000], initial random distributionα = 3δ = 1ra = 0.2ξ = 10φ = 1

Red and blue caste lowerthe threshold -> specialists(note started from low thresholds)

Page 39: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Example with 6 Castes (θ1 … θ6)

Evolution of the demand associated to the stimulus s

Page 40: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Evolution of the fraction of active individuals in each caste(demand profile overlapped)

Example with 6 castes (θ1 … θ6)

High fraction of active red and blue individuals

Xi

Xi

Page 41: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Light gray specialistremoved (corresponds to the red specialist in the previous example)

Example with 6 Castes and one Caste Removed at t = 150

Page 42: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

The Control of the Division of Labor in a variable Threshold

Model with 2 Tasks

Page 43: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Variable Threshold Modelwith 2 Tasks and m Castes

sj : intensity of the stimulus associated with the task jθij : response threshold of the caste i to task j, θij ∈ [θmin,θmax ]ξ : incremental learning parameter (the threshold of an individual carrying out

task is reduced by ξ )ϕ : incremental forgetting parameter (the threshold of an individual not carrying out a task

increases by ϕ)

Individual behavioral algorithm

θij θij - ξ when caste i performs task j→

θij θij + ϕ when caste i does not perform task j→

T (sj) = sj

2

sj2 + θij

2θij

Variable and parameters

Page 44: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Parameters of the demand associated with a task

∂t sj = δj – ( nij )αj

N

δj : rate of stimulus increase associated with a task jαj: normalized effectiveness rate for the individual contribution to the task j (in this case

all the active individual belonging to different castes contribute in the same way)m : total number of castes in the colonynij : number of active individuals belonging to caste i and carrying out the task j

Dynamics of the demand associated with a task

Σi = 1

m

Variable Threshold Modelwith 2 Tasks and m Castes

Page 45: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Dynamics of response thresholds

Task 2

Res

pons

e th

resh

old

Res

pons

e th

resh

old

Task 1

0

250

500

750

1000

0 1000 2000 3000

time

caste 5

caste 4

caste 3

caste 2

caste 1

Variable Threshold Modelwith 2 Tasks and m Castes

0

250

500

750

1000

0 1000 2000 3000

time

caste 5

caste 4

caste 3

caste 2

caste 1

Page 46: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

0,00

0,25

0,50

0,75

1,00

0 1000 2000 3000

time

Prop

ortio

n of

act

ive

indi

vidu

als

Prop

ortio

n of

act

ive

indi

vidu

als

Task 2Task 1

Dynamics of the proportion of active individuals per caste involved in each task

0,00

0,25

0,50

0,75

1,00

0 1000 2000 3000

time

Variable Threshold Modelwith 2 Tasks and m Castes

Page 47: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

A (Non Robotic) Example of Application of a Variable

Threshold Model to a Problem of Adaptive Task Allocation

Page 48: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

• Simulations are carried out on a grid with 5X5 zones

• 4 neighboring zones (no differentiation) are taken into account in the calculations; the boundary conditions are periodic (wrap around)

• 5 agents

The case of an express mail company

Application to a Problem of Adaptive Task Allocation

• Different task = different zone -> demand specific to each of the zones! Each agent has 5x5=25 different thresholds!

Page 49: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

• At each iteration, the demand is increased by 50 units in each of 5 randomly selected zones.

• The agents are consulted in random order, and each agent i computes its probability Ti,j of responding to the demand coming from each zone j. Response is reactive (based on a stimulus and a threshold function) but probabilistic in this case.

• If no agent has responded within 5 consultations, the next iteration begins.

• When an agent responds to a demand, it will be unavailable for a time proportional to the distance between its current position and the zone to which it is moving.

• When an agent moves to a zone, the demand associated with that zone remains at 0 while it is there.

Simulation details

Application to a Problem of Adaptive Task Allocation

Page 50: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Each time an agent decides to search for a letter in a zone j :

θi,j θi,j- ξ0→

θi,n(j) θi,n(j)- ξ1→

θi,k θi,k+ ϕ, for k≠ j , k ∉ {n(j)}→

θij : response threshold of agent i to a demand coming from zone jθi,n(j) : response threshold of agent i to a demand coming from zones adjacent to zone j{n(j)} : the set of zones adjacent to jξ0 : learning coefficient associated with zone jξ1 : learning coefficient associated with zones adjacent to jϕ : forgetting coefficient associated with other zones

Updating the agents’ response thresholds

Application to a Problem of Adaptive Task Allocation

Page 51: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Tij : probability that an individual i, located in zone z(i) will respond to a demand sj in zone j

θi ∈ [θmin,θmax] : response threshold of agent i to a demand coming from zone jdz(i),j : distance between z(i) and jα ≥ 0, β ≥ 0 : modulation parameters

The threshold function

Tij(sj)= sj + αθij + βdz(i),j

sj2

2 2 2

Application to a Problem of Adaptive Task Allocation

Ex.1 β = 0 → standard threshold function; the higher threshold the higher needs to be thestimulus in order to respond (e.g. interest/laziness of the dispatcher for a given zone)Ex.2 α = 0 → response based only on the distance cost between demand in zone j and current zone z(i)

Page 52: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

threshold

demand

specialist removal

Page 53: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Division of Labor in Artificial Systems:

Motivation and Overview from Mobile Robotics

Page 54: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Tasks for Multirobot Teams

• Mapping and exploration• Target tracking• Carrying objects• Playing team games• Construction• Inspection

Caloud et al; 1990

CMPack; 2002

Page 55: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Division of Labor

• Who does what? How many do what?– how many robots scan a room? – which robots track this object? – who plays goalie? – how many inspect each structure?

Page 56: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Description of Tasks

Retrievedifferentobjects

There areT types

of objects

Object oj is of type Toat (xj, yj)

There areno objectsof type to

No divisionof labor

Task: Each type toHow many types of robots?How many robots per type?

Task: Each object ojWhich robot to which object?

Task allocation problem and solutiondepend heavily on task description

!

Fine Coarse

Page 57: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Taxonomy of Approaches

Centralized Distributed

Self-organizedHybridIntentionalFully Centralized

Centralized Allocation

market threshold...

Page 58: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Taxonomy of Approaches

Centralized– potentially optimal– intractable– slow response– single pt-of-failure

Distributed– poor solutions– fast– responsive– robust

markets

Page 59: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Threshold-Based Multi-Robot Coordination:

Examples with one Task and One or Multiple Castes

Page 60: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

• 1 task: foraging (and maintaining nest reserves)

• Multiple castes (# of castes = # of robots): each of the robots is endowed with a different threshold

• Robot states: either active (foraging) or inactive (in the nest)

• Foraging demand associated with a central stimulus: maintaining a virtual nest energy above a given level

• Foraging stimulus perceivable only in the nest; deterministic robot response

• Solution without com compared with primitive com (tandem recruitment)

The First Attempt to Transport a Threshold-Based Macroscopic Model to a Multi-Unit Embedded

System (Krieger and Billeter, 2000)

Page 61: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Krieger and Billeter (2000)

Page 62: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Krieger and Billeter (2000)

Example with 6 robots

Page 63: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Theoretical contribution

+ Fixed threshold algorithm verified in a real robot experiment+ Experimental sound results (10 runs per experiment, up to 12 robots, stat tests)- High investment in manpower (2.5 man/year) and hardware- No effort in exploiting the system noise in order to reduce individual

complexity; adapting macro-to-micro mechanisms to artificial platform- No effort to overcome unbalanced workload

- No systematic simulations, no modeling: isolated experiment.- No systematic study on threshold distribution and noise influence on response

Autonomous robotics contribution

- The experiment does not add any additional information to a simple macroscopic model since no effort on clarifying potential microscopic mechanisms

- No quantitative link between artificial (robots) and natural (ants) system

Social insect contribution (robots as a model for insects)

Krieger and Billeter (2000)

Page 64: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Threshold-Based Control of Aggregation Activity (Agassounon and Martinoli, 2001)

Special type of aggregation: linear structure building (more on week 13)

Initial situation Final situationParking lot

Page 65: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Possible metric: average cluster size (20 seeds)

1 and 5 robots 10 robots

Saturation phase: all seeds in a single cluster or in the robots’ grippers

Performance without Threshold-Based Activity Regulation

Page 66: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

• Q: can we regulate the robot system activity in a fully distributed way so that robots the number of individuals active during the aggregation process is matched with aggregation demand?

• A: yes, using a threshold-based algorithm!

• Key motivations: – Evolution of manipulation sites: at the beginning there are several manipulation

sites, work in parallel positive; the more the aggregation/building process progresses the less manipulation sites there are, the more competition (interference) for the same manipulation sites there is.

– End criterion: a power-efficient building system should stop working when the task is accomplished

– Increasing the final cluster size: at the end all the seeds should belong to the single cluster (only those on the ground count for the aggregation metrics)

– Designing a truly distributed threshold-based algorithm (no supervisor!)

Distributed Activity Regulation of Aggregation?

Page 67: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

• 1 task: aggregation → 1 threshold per robot • How many different thresholds, threshold distribution?• Ideas (minimizing complexity, maximizing robustness/interchangeability):

– Robots have the same capabilities, no reason to have different threshold– Probabilistic response even with a single threshold will suffice to regulate the activity;

not all the robots stop at the same time, when one drop the work, direct influence on aggregation demand

– How do we implement: deterministic response + noise = probabilistic response →exploit local perception based on on-board sensors as noise generator!

• Chosen stimulus: time needed to find a seed to manipulate; the larger the time, the lower the stimulus associated with the aggregation demand

• Special case of demand evolution: it does not increase automatically but stay constant if nothing is done. Initial condition: s(0) = S0 and δ = 0 (instead of s(0) = 0 δ > 0 as in the previous examples) → switching mechanism asymmetric: active → idle possible; idle → active not possible.

• Algorithm extensible with a random wake up time/…; customized to aggregation demand without major seed reinsertion

Implementation of the Threshold-Based Algorithm

Page 68: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Average cluster size Number of active robots

No more saturation:growing phase beyond 10-seeds single cluster

Performance with Threshold-Based Activity Regulation20 seeds, threshold for abandoning the arena= 25 min, 10 robots

Page 69: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

How good does this work?

Advantages– easy to determine, unique threshold for the all team;– robot homogenous (also controller): on average good load distribution– optimal value determined as a function of the system feature (number of

robots/speed/number of seeds/arena surface etc. ) → quantitative models (micro and macro) can help to find the optimal threshold

Drawbacks– fixed unique threshold = single parameter encoding the whole dynamics of the

experiment → algorithm extremely robust but system operation point might be sub-optimal in case of major environmental changes (e.g., double the aggregation area) or system changes (e.g., half of the robots fail)

– parameter difficult to tune analytically if noise distribution non parametric (e.g. no assumption on the distribution possible)

Page 70: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Example of Performance LandscapeThreshold optimization with macroscopic model (numerical integration)

Optimal threshold = 27 min for a given cluster size at a given time

Robots stop to early

Robots still keep working and keep the seedsIn their grippers

Page 71: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

• PrFT: Threshold-based algorithm with private demand estimation and fixedthresholds; algorithms of the previous slides

• PrVT: variable threshold; not continuously adapting but calibration phase, then fixed threshold → result in a heterogeneous, multi-caste group

• PuFT: fixed threshold but demand estimated via wireless sharing among all members (average)

• PuVT: variable threshold (calibration) + demand estimation shared

• Arena 1: standard arena (that of previous slides), 80x80 cm, 20 seeds• Arena 2: larger arena, 178x178 cm, 20 seeds• Arena 3: standard arena, 20 seeds, 5 seeds added after 2 hours into the

aggregation process

Robots are not Ants: Can we exploit Wireless Com for Demand Estimation?

NOTE: com only used for stimulus estimation sharing; no central decision making as in market-based approaches!!!

[Agassounon and Martinoli, AAMAS, 2002]

Page 72: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

In bold the lowest Integrated Cost (within 1 std dev):

197.2 ± 5.9310.8 ± 8.8227.4 ± 4.8W/o WA134.2 ± 9.1227.2 ± 9.4141.3 ± 5.2PuVT122.4 ± 6.4337.6± 10.7138.2 ± 6.9PuFT152.2 ± 8.7231.9 ± 10.7155.1 ± 8.0PrVT154.5 ± 7.9324.9 ± 10.8138.9 ± 7.0PrFT

Arena3Arena2Arena1Algorithm

2 2

2

cos

(2)

( , , ) γ ( ) γ ( )

γ ( ) t t t x opt t y opt t

z opt t

tF x y z X x Y y

Z z

= − + −

+ −

Note: W/o WA = without work allocation (no activity control)

xt = average cluster size at time step t; Xopt = 20yt = the average number of clusters at time step t; Yopt = 1zt = average number of active workers at time step t; Zopt = 0 γx, γy, γz = weighting parameters

Performance Comparison of Different Variants of the Algorithm

Page 73: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Conclusions

Page 74: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Take Home Messages• Existence of several castes in natural societies (age-

based, behavioral, morphological) are evident signs of division of labor

• Two different classes of threshold-based macroscopic models: fixed and variable thresholds

• Threshold-based models explain well experimental biological data experiments with social insects

• Examples of computational and embedded applications (division of labor in robotic swarms) are described

• From macroscopic models to microscopic mechanisms: a challenging and not trivial step when applied to embedded systems

Page 75: Swarm Intelligence – W10: Division of Labor I: Threshold-Based … · 2008-11-21 · Mainly in the course book Mainly in the lecture slides Concerned Variable Note 1: depending

Additional Literature – Week 10Papers• Bonabeau E., Theraulaz G., and Deneubourg J.-L., “Fixed Response Thresholds and

the Regulation of Division of Labour in Insect Societies”. Bulletin of Mathematical Biology, 1998, Vol. 60, pp.753-807.

• Theraulaz G., Bonabeau E., and Deneubourg J.-L., “Response Threshold Reinforcement and Division of Labour in Insect Societies”. Proc. of the Royal Society of London Series B, 1998, Vol. 265, pp. 327-332.

• Pacala S. W., Gordon D. M., and Godfray H. C. J. 1996. Effects of Social Group Size on Information Transfer and Task Allocation, Evolutionary Ecology, 10: 127-165.

• Krieger M. J. B. and Billeter J.-B., “The Call of Duty: Self-Organised Task Allocation in a Population of up to Twelve Mobile Robots”. Robotics and Autonomous Systems, 2000, Vol. 30, No. 1-2, pp. 65-84.

• Krieger, M., Billeter, J.-B. & Keller, L. , “Ant-like task allocation and recruitment in cooperative robots”, Nature, 406: 992–995, 2000.

• Agassounon W. and Martinoli A., "Efficiency and Robustness of Threshold-Based Distributed Allocation Algorithms in Multi-Agent Systems". Proc. of the First Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems, July 2002, Bologna, Italy, pp. 1090-1097.

• Agassounon W., Martinoli A., and Easton K., “Macroscopic Modeling of Aggregation Experiments using Embodied Agents in Teams of Constant and Time-Varying Sizes”. Autonomous Robots, special issue on Swarm Robotics, Dorigo M. and Sahin E., editors, 17(2-3): 163-192, 2004.