An Introduction to Modeling Techniques and Levels, and ... · Modeling the Immune System – W8 An...
Transcript of An Introduction to Modeling Techniques and Levels, and ... · Modeling the Immune System – W8 An...
Modeling the Immune System – W8
An Introduction to Modeling Techniques and Levels, and
Ordinary Differential Equations
Modeling: why and what ?• The grand challenge:
Constitute a computer model that represents all we know about the immune system, which can be – stored– searched– compared – and analyzed
• We are not yet there, but you may help one step
http://research.microsoft.com/towards2020science/
Modeling
Individual behaviorsand local interactions
Global structuresand collective
decisions
• Modeling to understand microscopic to macroscopic transformation
• Modeling as interface to artificial systems
Ideas forartificialsystems
The Crucial Role of Modeling
Goal of the Modeling Module• Understand what a model • Master the concepts underlying the most
frequently used models• Know how to simulate them• Learn by example how they were used• Get a feeling of the limitations of the
current modeling techniques and their experimental validation
Contents of the Modeling Module• Introduction
– Swarm-intelligent systems– Modeling levels and techniques– Examples from animal and robotic populations
• ODE models (Ordinary Differential Equations)– Deterministic– Based on Rob de Boer’s lecture notes– Apply it to Annotated Paper DePillis et al
Contents of the Modeling Module• Stochastic macroscopic models
– Markov chain models – random– Based on Jean-Yves Le Boudec’s lecture notes– Apply it to Annotate Paper Fraser et al
• Stochastic microscopic models – Cellular Automata as example– Based on research papers
Outline of This Lecture
• Swarm-intelligent systems– Mechanisms– Comparison between natural societies and IS– Modeling levels and techniques– Comparison between robotic societies and IS
• An introduction to ODE models– Logistic equation– Steady state analysis– Phase plots
Is the Immune System a Swarm-Intelligent System?
(Examples from Social Insects)
A Typical SI System: Ant Colonies• Natural system
– social insect societies• Unit coordination
– distributed control + environmental template– individual autonomy– self-organization
• Communication– direct local communication (peer-to-peer)– indirect communication through signs in the
environment (stigmergy) • Scalability
– A few to milions of units• Robustness
– redundancy– balance exploitation/exploration– individual simplicity
Immune System: a SI System?• Natural system
– immune system• Unit coordination
– distributed control + environmental template (e.g. chemical gradients, different organs)
– cell autonomy (e.g. decisional, perception-to action, energetic)
– self-organization• Communication
– direct local communication: cell-to-cell contact– indirect communication: e.g., cytokines
• Scalability– Up to hundreds of billions of units. Lower
bound?• Robustness
– redundancy– balance exploitation/exploration (e.g., antibody
production)– individual simplicity (in comparison to the
whole system level)
Neutrophils
Mast cellsMacrophages
Cells:
B-Cells
T-CellsDC-Cells
Molecules:- Acute phase proteins- Cytokines- Chemokines- Antibodies- …
B
T
…
Two Key Ideas in Swarm-Intelligent Systems
1. Self-Organization
2. Stigmergy
Self-Organization
• Set of dynamical mechanisms whereby structure appears at the global level as the result of interactions among lower-level components
• The rules specifying the interactions among the system's constituent units are executed on the basis of purely local information, without reference to the global pattern, which is an emergent property of the system rather than a property imposed upon the system by an external ordering influence
Basic Ingredients of Self-Organization
• Multiple interactions• Randomness• Positive feedback
– E.g., recruitment of other cells• Negative feedback
– E.g., limited number of available cells
Characteristics of Natural Self-Organized Systems
• Creation of spatio-temporal structures– E.g., AC: foraging trails, nest architectures, clusters of objects, ...;
IS?
• Multistability(i.e., possible co-existence of several stable states)– E.g., ants exploit only one of two identical food sources, build a
cluster in one of the many possible locations, ...; IS?
• Existence of bifurcations when some parameters change– E.g., termites move from a non-coordinated to a coordinated phase
only if their density is higher than a threshold value; IS?
StigmergyGrassé P. P., 1959
• “La coordination des taches, la regulation des constructions ne dependent pas directement des oeuvriers, mais des constructions elles-memes. L’ouvrier ne dirige pas son travail, il est guidé par lui. C’est à cette stimulation d’un type particulier que nous donnonsle nom du STIGMERGIE (stigma, piqure; ergon, travail, oeuvre = oeuvre stimulante).”
• [“The coordination of tasks and the regulation of constructions does not depend directly on the workers, but on the constructions themselves. The worker does not direct his work, but is guided by it. It is to this special form of stimulation that we give the name STIGMERGY (stigma, sting; ergon, work, product of labor = stimulating product of labor).”]
It defines a class of mechanisms exploited by social insects to coordinate and control their activity via indirect interactions.
Stigmergic mechanisms can be classified in two different categories: • quantitative (or continuous) stigmergy• qualitative (or discrete) stigmergy
Stimulus
Answer
S1
R1
S2
R2
S3
R3
time
S 4
R4
S 5
R5
Stop
Definition
Stigmergy
→ probably the most widely used in the IS
An Example from Ant Colonies
Nest
Food Source
Symmetric Bridge Experiment(Deneubourg, 1989)
• Stigmergy?
• Self-organization?– Spatio-temporal structures?– Multi-stability?– System bifurcation?
• Ingredients of SO:• Multiple interactions?• Randomness?• Positive feedback?• Negative feedback?
Bridge with two Branchesof the Same LengthExperimental Results
= ( k + A i ) n
PA( k + A i ) n + ( k + B i ) n
= 1 - PB
A i : number of ants having chosen branch A
B i : number of ants having chosen branch B
Microscopic Model(Deneubourg 1990)
Probabilistic choice of an agent at the bridge’s bifurcation points; Montecarlo simulation
PA and PB : probability for the ant i+1 to pick up the branch A or B respectively
n (model parameter): degree of nonlinearityk (model parameter): degree of attraction of a unmarked branch
Ai
Aii PifB
PifBB
≤>+
=+ δδ1[1
Ai
Aii PifA
PifAA
>≤+
=+ δδ1[1
iBA ii =+ δ = uniform random variable on [0,1]
0
0,25
0,5
0,75
1
1 10 100
k=1 n=5
k=1 n=2
k=1 n=1
P APPAA
Parameters of the Choice Function
• The higher is n and the faster is the selection of one of the branches (sharper curve); n high corresponds to high exploitation
• The greater k, the higher the attractivity of a unmarked branch and therefore the higher is the probability of agents of making random choices (i.e. not based on pheromones concentration deposited by other ants); k high corresponds to high exploration
0
0,25
0,5
0,75
1
1 10 100
k=10 n=5
k=10 n=2
k=10 n=1
P APPAA
Number of ants havingchosen branch A
Number of ants havingchosen branch A
k highk low
Bridge with two Branchesof the Same Length
Model vs. Experiments
40
50
60
70
80
90
100
0 500 1000 1500 200040
50
60
70
80
90
100
0 500 1000 1500 2000
experiment
model
Total number of ants having traversed the bridge
% o
f ant
pas
sage
s on
the
dom
inan
t bra
nch
Parameters that fit experimental data:n = 2k = 20
An Example of Varying the Environmental Conditions
• Two branches (A and B) differing in their length (length ratio r) connect nest and food source
• Test for the optimization capabilities of ants
© J.-L. Deneubourg
Asymmetric Bridge Experiment(Deneubourg, 1989)
Food Source
Nest
• The previous simple microscopic model based on Montecarlo techniques should be modified to take into account distance/traveling time corresponding to the geometry of the asymmetric bridge
• This is an alternative option at the microscopic level: multi-agent system, more computationally intensive (agent trajectory simulated but no embodiment).
Asymmetric Bridge –Microscopic Modeling
© Marco Dorigo, ULB, 1999
An Example from the IS?
• Idea of the bridge experiment: – Selected lab experiment (i.e. in vitro) vs. real
environment (i.e. in vivo)• What’s the “ bridge experiment” for the IS?
– Has been done?– What are the difference in experimental
conditions? – What are the technical challenges in gathering
data?
Similarities between an Ant Colony and the Immune System
• Swarm-intelligent system (self-organization, stigmergy, distributed control, individual autonomy etc.)
• Multi-caste, inter-individual diversity comparable within the same caste
• Individuals endowed with sensors, actuators, information processing capabilities
• Mobility
– IS: more “hierarchical” structure (e.g., 3 barrier lines), more morphologically different cells (hardware specialization)
– AC: mostly passive environment – open loop or relatively slow closed loop between environment and collective system (e.g. evolution over generations); IS: active environment (body), relatively fast closed-loop
– IS: individual intelligence level low (no learning? no internal representations?) AC: navigation, learning, etc.
– IS: very high numbers; AC: high numbers– IS: population reproduction (birth/death) crucial; AC: most
of studies (particularly the modeling one) during life span– Experimental techniques different; data for modeling also
differentConsequences at the modeling level (tools, resulting models)?
Differences between an Ant Colony and the Immune System
Modeling Levels and Techniques
(Examples from Swarm Robotics)
• Even if individual node control deterministic, the interaction with the environment/teammates in the real-world is noisy and barely predictable → probabilistic models
• Swarm-intelligent systems exploit self-organization as main coordination mechanism: among key ingredients of self-organization (see week 1 and 2) there is randomness, an ingredient at the core of the exploration-exploitation balance of these systems. Coordinated collective behavior base on self-organization is statistically predictable using appropriate probabilisticmodels!
Rationale for Probabilistic Modelling –Swarm-Intelligent Real-Time Systems
• Different levels, different system parameterscertain low-level parameters (e.g., body morphology, sensor characteristics, trajectories ) can be captured in an explicit and more accurate way with microscopic models; others (e.g., birth numbers, death numbers, number of cells of a given species) can be captured also at higher level
• Different levels, different generalization powerthe higher the abstraction, the better the generalizing power (e.g., other experimental constraints, other class of experiments, outline common fundamental blocks)
• Different levels, different computational costquantitatively accurate models have often to be solved numerically: the higher the abstraction, the lower the computational cost
Benefits of Multi-Level Modelling
Modeling Levels
Target swarm system: multiple units
ModelsMacroscopic models: single representation for the swarm
Microscopic models: unit represented individually • S&A-based simulator• Multi-agent system
• Point simulator• …
• Rate equations• Master equations
• Stochastic reaction networks• …
Abs
trac
tion
Expe
rim
enta
l tim
e
Multi-Level Modeling Methodology
Ss SaSs SaSs Sa
∑ ∑′ ′
′−′′=n n
nnn tNtnnWtNtnnW
dttdN )(),|()(),|()(
Ss Sa
Target system (physical reality): info on controller, S&A, communication, morphology and environmental features
Microscopic – Module-based: intra-robot (e.g., S&A, transceiver) and environment (e.g., physics) details reproduced faithfully
Microscopic – Agent-based: multi-agent models, only relevant robot features captured, 1 agent = 1 robot
Macroscopic: rate equations, mean field approach, whole swarm
Com
mon
met
rics
Microscopic Level– Individual-centered (e.g., cell-centered in IS); accurate at the
individual level; multi-agent model– One can ask questions about diversity, including each individual
different from each other– States are representing for instance: current behavior, position,
orientation, etc.– Might still be represented as Markov process in continuous time
but state space might not be countable (e.g., continuous spatialspace); of course, in simulation spatiotemporal discretizationshappen according to machine resolution/simulation settings which results in anyway extremely large number of states
– Usually computational rather than mathematical models– Computational time scales at least linearly with the number of
units– Multiple runs needed for statistical significance on the
population average behavior
Macroscopic Level– Population-centered (e.g., IS-centered, for a specific aspect);
accurate at the population level; single abstraction model– One can ask questions about diversity on pre-establishing castes
at the price of major extension of models (e.g., new set of ODE for each caste)
– States are representing for instance: number of current individuals of a given type, in a given behavior, etc.
– Markov process in continuous time, countable space – Usually mathematical, numerically solvable models (simplified
models can be analytically tractable)– Computational time is independent of the number of units in the
system– Depending on the implementation, population average behavior
can be predicted (rate equations, ODE) or absolute numbers (stochastic network models)
Modeling Assumptions
Modeling Assumptions 1. Probabilistic FSM description for environment and
multi-agent system; arbitrary state granularity2. Semi-Markovian properties: the system future state
is a function of the current state (and possibly of the amount of time spent in it)
3. Nonspatial metrics for swarm performance4. Mean field approach (well-mixed system): mean
spatial distribution of agents over multiple runs assumed to be homogeneous, as they were randomly hopping on the arena
5. Linear superposition of object/robot detection areas(sparse object/robot distribution; no overcrowding, no detection areas overlapped)
Assumptions 1 and 2• We work with states
pin poutTx
Sx
Sx: state xTx: duration of state xpin, pout: probabilities to entry and leave state x
• States can characterize both robots and the environment
Assumptions 3,4,5• Trajectories and object location do not count -> 1D
Montecarlo simulation• N objects of type i -> N x (prob. to encounter i)
O1O1
O2
O2O2
2D physical space 1D probability space
O2
O1
Free space
From Microscopic to Macroscopic Models:
Theoretical Background
Microscopic Level
∑ ∑′ ′
∆+′−′∆+=
−∆+=∆
n ntnptnttnptnptnttnp
tnpttnptnp),(),|,(),'(),|,(
),(),(),(
p(n,t) = probability of an agent to be in the state n at time tIf Markov properties fulfilled (neglect distributions):
inflow outflow
Probability the agent was in a given state n’
Transition probability
ttnttnptnnW
t ∆′∆+
=′→∆
),|,(lim);|(
0Transition rate
Sum over all possible states n’ the agent can be in
Macroscopic Level
∑ ∑′ ′
′ ′−′=n n
nnn tNtnnWtNtnnW
dttdN )(),|()(),|()( Rate Equation
(time-continuous)
inflow outflown, n’ = states of the agents (all possible states at each instant)Nn = average fraction (or absolute number) of agents in state n at time tW = transition rates (linear, nonlinear);
∑ ∑′ ′
′ ′−′+=+n n
nnnn kTNkTnnTWkTNkTnnTWkTNTkN )(),|()(),|()())1((
Time-discrete version. k = iteration index, T time step (often left out)
Left and right side of the equation: averaging over the total number of agents, dividing by ∆t, limit ∆t → 0; neglect distributions of the stochastic variables (mean field approach):
Time Discretization: The Engineering Receipt
1. Assess what’s the time resolution needed for your swarm performance metrics
2. Consider unit-to-unit interaction essence: probabilistic/deterministic, asynchronicity role, …
3. Choose whenever possible the most computationally efficient model: time-discrete less computationally expensive than emulation of continuity (e.g. Runge-Kutta, etc.); in our systems/metrics there is no evidence of decreased prediction accuracy
4. Advantage of time-discrete models: a single common sampling rate can be defined among different abstraction levelsOften not even a tradeoff: just use the appropriate
instrument for the appropriate problem!
Time-discrete vs. time-continuous models:
Model Parameters• Gray-box approach: a priori information about the system
is exploited – multi-agent system– # of agents– technological and environmental constraints
• Models should not only explain but have also predictive power: the mapping between model parameters and design choices should be as explicit as possible (the higher the abstraction level the more difficult it is)
• Two types of parameters for micro-AB and macro: – State durations (e.g., interaction times with objects)– State-to-state transition probabilities (e.g., encountering
probabilities)
Linear Example:Wandering and Obstacle
Avoidance
A Simple Linear Model
© Nikolaus Correll 2006
Example: search (moving forwards) and obstacle avoidance
A simple Example
Nonspatiality& microscopiccharacterizationDeterministic
robot’s flowchart
Search Avoidance
Start
Obstacle?YN
Search Avoid., τa
Start
Obstacle?pa
ps
1-pa
Probabilistic agent’s flowchart
Ss Sa
pa
τa
ps
PFSM (Markov Chain)
Linear Model – Probabilistic Delay
Search Avoidance, Ta
Ta = mean obstacle avoidance durationpa = probability of moving to obstacle av.ps = probability of resuming searchNs = average # robots in searchNa= average # robots in obstacle avoidanceN0 = # robots used in the experimentk = 0,1, … (iteration index)
Ns(k+1) =
Na(k+1) =
Ns(k)
N0 – Ns(k+1)
ps=1/Ta
+ psNa(k)- paNs(k)
pa
Ns(0) = N0 ; Na(0) = 0
Linear Model – Deterministic Delay
Search Avoidance, Ta
Ta = mean obstacle avoidance durationpa = probability moving to obstacle avoidanceNs = average # robots in searchNa= average # robots in obstacle avoidanceN0 = # robots used in the experimentk = 0,1, … (iteration index)
Ns(k+1) =
Na(k+1) =
Ns(k)
N0 – Ns(k+1)
1
+ paNs(k-Ta)- paNs(k)
pa
! Ns(k) = Na(k) = 0 for all k<0 !Ns(0) = N0 ; Na(0) = 0
Linear Model – Sample Results
Micro-AB to macro comparison(same robot density but wall surfacebecome smaller with bigger arenas)
Micro-AB to micro-MB comparison(different controllers, static scenarios, allocentric measures)
Na*/N0
Steady State Analysis• Nn(k+1) = Nn(k) for all states
n of the system → Nn*
• Note 1: equivalent to differential equation of dNn/dt = 0
• Note 2: for time-delayed equations easier to perform the steady-state analysis in the Z-space but in t-space also ok (see IJRR-04)
• For our linear example (time-delay option):
aas Tp
NN+
=1
0*
Group size
Ex.: normalized mean number of robots in search mode at steady state as a function of time for obstacle avoidance
aa
aaa Tp
TpNN+
=1
0*
aas Tp
NN+
=1
0*
Nonlinear Example –Stick-Pulling
A Case Study: Stick-Pulling
Proximity sensors
Arm elevationsensor
Physical Set-Up Collaboration via indirect communication
• 2-6 robots• 4 sticks• 40 cm radius arena
IR reflectiveband
Systematic Experiments
Real robots Module-based model
•[Martinoli and Mondada, ISER, 1995]•[Ijspeert et al., AR, 2001]
Experimental and Realistic Simulation Results
• Real robots (3 runs) and realistic simulations (10 runs)• System bifurcation as a function of #robots/#sticks
Nrobots > Nsticks
Nrobots ≤ Nsticks
Geometric Probabilities
sgg
sg
aww
rR
arr
ass
pRp
ppAAp
NppAApAAp
=
==
−===
2
1
0
/)1(
//
Aa = surface of the whole arena
From Reality to Abstraction
Deterministic robot’s flowchart
Probabilistic agent’sflowchart
Markov Chain (PFSM)Nonspatiality& microscopiccharacterization
Full Macroscopic Model
• 6 states: 5 DE + 1 cons. EQ• Ti,Ta,Td,Tc ≠ 0; Τxyz = Τx + Τy + Τz• TSL= Shift Left duration• [Martinoli et al., IJRR, 2004]
)()()()()()()();()()(])()([)()1(
22
121
iasRaswcdascdagcascag
cgasacgagsRwggss
TkNpTkNpTkNTkTkNTkTkNTkTkkNppkkkNkN
−+−+−−∆+−−∆+−Γ−∆+++∆+∆−=+
For instance, for the average number of robots in searching mode:
∏−
−−=
−=Γ
=∆
−−=∆
SL
SLg
Tk
TTkjsgSL
ggg
dggg
jNpTk
kNpk
kNkNMpk
)](1[);(
)()(
)]()([)(
2
22
011
with time-varying coefficients (nonlinear coupling):
Swarm Performance Metric
C(k) = pg2Ns(k-Tca)Ng(k-Tca)
e
T
k
T
kCe
∑== 0
t
)( (k)C
: mean # of collaborations at iteration k
: mean collaboration rate over Te
Collaboration rate: # of sticks per time unit
Results (Standard Arena)
Micro-MB (10 runs)Micro-AB (100 runs)Macro (1 run)
Discrepancies due to continuous vs. discrete quantities with small numbers; rate equation simplification on stat distributions
Results: 4 x #Sticks, #Robots and Arena Area
Micro-MB (10 runs)Micro-AB (100 runs)Macro (1 run)
Reducing the Macroscopic Model
Τi,Τa,Τd,Τc << Τg →Τi=Τa=Τd=Τc=0
Goal: reach mathematical tractability
Nonlinear coupling!
Reduced Macroscopic Model
Search Grip
Ns = average # robots in searching modeNg= average # robots in gripping modeN0 = # robots used in the experimentM0 = # sticks used in the experimentΓ = fraction of robots that abandon pullingTe = maximal number of iterationsk = 0,1, …Te (iteration index)
Ns(k+1) =
Ng(k+1) =
Ns(k) – pg1[M0 – Ng(k)]Ns(k)
successful
+ pg2Ng(k)Ns(k)
unsuccessful
+ pg1[M0 – Ng(k-Τg)]Γ(k;0)Ns(k-Tg)
N0 – Ns(k+1)
∏−=
−=Γk
Tkjsg
g
jNpk )](1[)0;( 2
Ns(0) = N0, Ng(0) = 0Ns(k) = Ng(k) = 0 for all k<0
Initial conditions and causality
Results Reduced Micro-AB Model
• 4 robots, 4 sticks, Ra = 40 cm • 16 robots, 16 sticks, Ra = 80 cm
• Micro-AB (100 runs) and macro models overlapped• Only qualitatively agreement with micro-MB/real robots results
Steady State Analysis (Reduced Macro Model)
• Steady-state analysis [Nn(k+1) = Nn(k)] → It can be demonstrated that :
g
optg RM
NforT+
≤∃1
2
0
0
with N0 = number of robots and M0= number of sticks, Rg approaching angle for collaboration
• Counterintuitive conclusion: an optimal Tg can exist also inscenarios with more robots than sticks if the collaboration is very difficult (i.e. Rg very small)!
∝
approaching angle for collaboration
Analysis Verification (Micro-AB and Macro Full Model)
gg RR101~ =
20 robots and 16 sticks (optimal Tg)
Example: (collaboration very difficult)
• can be computed numerically by integrating the full model ODEs or solving the full model steady-state equations
Optimal Gripping Time• Steady-state analysis → can be computed analytically in
the simplified model (numerically approximated value):
gc
g
gg
optg R
forR
NRpT
+=≤
−
+−
−=
12
21
)1(2
1ln
)2
1ln(
10
1
βββ
β
optgT
with β = N0/M0 = ratio robots-to-sticks
[Lerman et al, Alife Journal, 2001], [Martinoli et al, IJRR, 2004]
optgT
Journal PublicationsStick Pulling
• Li, Martinoli, Abu-Mostafa, Adaptive Behavior, 2004-> learning + micro-AB
• Martinoli, Easton, Agassounon, Int. J. of Robotics Res., 2004-> real + micro-MB + micro-AB + macro
• Lerman, Galstyan, Martinoli, Ijspeert, Artificial Life, 2001-> micro-MB + macro
• Ijspeert, Martinoli, Billard, and Gambardella, Auton. Robots, 2001-> real + micro-MB + micro-AB
Object Aggregation
• Agassounon, Martinoli, Easton, Autonomous Robots, 2004-> micro-MB + macro + activity regulation
• Martinoli, Ijspeert, Mondada, Robotics and Autonomous Systems-> real + micro-MB + micro-AB
Applicability of Methods Developed for Swarm
Robotics to the IS
Can we apply these Methods to the IS? • Yes:
– Assumptions are ok– Embodied units endowed with sensors, actuators, computation, several
communications forms – Diversity among units and components (manufacturing and
assembling), noise, nonlinearities, physical world laws are all there
• No:– Microscopic information to be imported is not always available, not
accurate, or extremely costly to be acquired in IS (e.g., individual cell control, physical substrate, environmental information, etc.)
– Computational efficiency for micro optimized for hundreds of units and not billions
– Calibration methods based on different experimental conditions– Role of reproduction completely different: crucial in the IS, essentially
not considered in swarm robotics (conservation laws exploited)
Can we Modify this Method for the IS? Perhaps: open question …. – Certain limitations are easy to relax (e.g., taking into account birth
and death; optimizing computational time), other are more difficult (e.g., what microscopic information from the literature can we import, what experimental data can be acquired)
– Such multi-level methods could in principle guide experimental research in the IS, if some seed results is achieved …
– Current methods are non-spatial but they can easily be expanded to spatial methods; more difficult to maintain compactness (danger: state explosion) but further methods have been developed (e.g., graph-based); no default time/space discretization such as in Cellular Automata