Distributed Intelligent Systems – W6 Machine …...Distributed Intelligent Systems – W6...
Transcript of Distributed Intelligent Systems – W6 Machine …...Distributed Intelligent Systems – W6...
Distributed Intelligent Systems – W6Machine-Learning and Multi-Machine Learning and Multi
Level Modeling Methods Applied t Di t ib t d R b ti S tto Distributed Robotic Systems
(Part I and II)( )
Outline – Part I: Machine-Learning• Challenges in multi-robot scenarios
– Credit assignment problemsg p– Single-vehicle adaptation in a world
shared by multiple vehicles
• Co-adaptation strategies• Co-shaping obstacle avoidance
– PSO vs. GA– Off-board centralized vs. on-board
distributed implementationdistributed implementation– Effect of physical constraints
• Co-shaping coordinated motionCo shaping coordinated motion
F Si l t From Single to Multi Unit Systems:Multi-Unit Systems:Co-Adaptation in a Co-Adaptation in a
Shared WorldShared World
Adaptation in Multi-Robot ScenariosAdaptation in Multi Robot Scenarios
• Collective: fitnessCollective: fitness become noisy due to partial perception, independent parallelindependent parallel actions
Credit Assignment ProblemCredit Assignment ProblemWith limited communication, no communication at all, or partial perception:
Design and Optimization of a Design and Optimization of a Sensory System for an Sensory System for an
Intelligent Vehicle
A Reasonable Traffic SimulatorA Reasonable Traffic Simulator
2001 2003 W b 2 ki i i l diff i l2001-2003: Webots 2, kinematic simulator; differential wheel vehicles; more later in the course
Evolving Sensor Configurationg g
• Number nNumber n • Type
R– Range ρ– Cone of view δ– Cost f(ρ, δ)
• Placement– Position φ– Orientation θ
An Expensive Optimization ProblemAn Expensive Optimization ProblemThe design and optimization of the sensory system is an expensive optimization problem because:
1. Large evaluation time of candidate solutions
an expensive optimization problem because:
Recipe: test candidate solutions on more abstract models
2. The performance evaluation is noisy (sensor and actuator noise; local perception in a loosely coordinated system solely based on traffic rules and road template) Recipe: noise resistant algorithms with aggregationRecipe: noise-resistant algorithms with aggregation
3. Search space high dimensionality (actually unbound)Recipe: formulate the problem as an engineering trade-offRecipe: formulate the problem as an engineering trade-off so that solution remains in a reasonable dimension space
Reducing the Evaluation TimeReducing the Evaluation Time
1. Exploit underlying computationally expensive level to generate a
b bili i di ib iprobabilistic distribution2. Randomly allocate vehicles in
detection region using this distributiondistribution
3. Evaluate a candidate solution based on a certain number of randomly placed vehiclesrandomly placed vehicles
V hi l O P b biliVehicle Occurrence Probability
• 1-Dimensional (angle) • 2-Dimensional
Probabilistic Evaluation Tests
Vehicles to be
• 1-dimensional • 2-dimensional
detected
T t hi lTest vehicle Detection region
Relative Time Cost
Evaluation Tests Time Cost Relative Cost
Static 4.6 s 11D Full Coverage 9.7 s 2.12D Full Coverage 10 3 min 1342D Full Coverage 10.3 min 1341D Probabilistic 15.6 s 3.42D Probabilistic 8 9 min 1162D Probabilistic 8.9 min 116Embodied (kinematic) 10.4 hours 8100
Example: 2500 evaluations conducted on computers with 1 5 GHz AMD CPUExample: 2500 evaluations conducted on computers with 1.5 GHz AMD CPU
Generalized Aggregation Functionsgg gAggregation function operators Ps [Scott and
Antonsson 1998]Antonsson 1998]
• μxx: fuzzy preference on factor xx• ωxx: weight on factor xx• s: degree of compensation (express the engineering trade• s: degree of compensation (express the engineering trade-
off strategy; higher values means higher willingness to find compromise among preferences)
• For design trade-offs (some compensation):• Perfect weighted sum (neutral): s = 1
l f f /d i f b• Note: Example for 2 factors/design preferences; can be generalized to arbitrary multiple factors
Properties of GeneralizedProperties of Generalized Aggregation Functions
[Scott and Antonsson 1998]
Fitness Function Formulated as E i i T d OffEngineering Trade-Off
1ss
coveragescost μωμ ⎟
⎞⎜⎛ ⋅+ μ : fuzzy preference on factor xx
1),( coveragecostsFitness
ω
ωμμ
ω ⎟⎟⎠
⎜⎜⎝ +
= μxx: fuzzy preference on factor xx(0: totally unacceptable;1: completely acceptable)
ω : weight on factor xx
Vcost
coverage
ωω
ω =ωxx: weight on factor xxs: degree of compensationV: actual number of vehicles
been in the detection region ),PDF(
1n
iiii rkCoverage α=∑
=
gduring an evaluation span
ki: 1 if the ith car at distance riand approaching angle αi
)(1
n
ii
fcost
costCost
δρ
=∑=
detected; 0 if not detectedn: number of sensors usedρi: ith sensor’s rangeδ ith ’ f i),( iii fcost δρ= δi: ith sensor’s cone of view
Preference Functions• Preference function given by the designer and/or the customer on all
relevant criteria of a candidate solution• Mapping of the actual cost in a normalized cost taking into account the pp g g
preferences• Expressed here with fuzzy sets (0: totally unacceptable; 1: completely
acceptable)
0.8
0.9
1
0.8
0.9
1
μcost
acceptab e)
μcoverage
0.5
0.6
0.7
0.8
over
age
0.5
0.6
0.7
0.8
μ cost
μcost
0.2
0.3
0.4μ cov
0.2
0.3
0.4
μ c
0 0.2 0.4 0.6 0.8 10
0.1
Coverage0 5 10 15 20 25
0
0.1
Total cost
Sample Results• Fitness evolution process and the final best design• Maximal compensation among coverage and cost• Higher importance of cost vs. coverage
s = 0, ω = 3/17
Coverage = 53%, Total_cost = 4.6
Sample Results• Fitness evolution process and the final best design• No compensation between coverage and cost
s = -∞, ω arbitrary
Coverage = 82%, Total_cost = 7.7
Sample Results• Fitness evolution process and the final best design• Maximal compensation among coverage and cost
s = 0, ω = 4• Higher importance of coverage vs. cost
Coverage = 98%, Total_cost = 11.9
Sample Results• Evolved approximate Pareto frontier for the design trade-offs
present in the case studyE h d t i t t th b t lt f ti l• Each data point represents the best result of one particular evolutionary experiment under a given combination of ω and s
[Zhang et al., Research in Engineering Design, 2008 ]
Co-Adaptation in a pCollaborative Framework
Co-Shaping Collaborative Behaviorp g
Three orthogonal axes to consider (extremities and b l d l i ibl )balanced solutions are possible):
1. Performance evaluation: individual vs. group fitness or reinforcement
l i h i2. Solution sharing: private vs. public policies
3 T di i3. Team diversity: homogeneous vs. heterogeneous learning
Search Algorithms for Multi-RobotSearch Algorithms for Multi Robot Systems
Policy Performance Sharing Diversity
i-pr-he individual private heterogeneous
i-pr-ho individual private homogeneous
i-pu-he individual public heterogeneouspi-pu-ho individual public homogeneous
g-pr-he group private heterogeneousg pr he g p p g
g-pr-ho group private homogeneous
g-pu-he group public heterogeneousg pu he group public heterogeneous
g-pu-ho group public homogeneous
Search Algorithms for MR Systems
Do not make sense (inconsistent)
Interesting (consistent)
Possible but not scalable
Policy Performance Sharing Diversity
i pr he individual private heterogeneous
g ( )
i-pr-he individual private heterogeneous
i-pr-ho individual private homogeneous
i pu he individual public heterogeneousi-pu-he individual public heterogeneous
i-pu-ho individual public homogeneous
h i t h tg-pr-he group private heterogeneous
g-pr-ho group private homogeneous
g-pu-he group public heterogeneous
g-pu-ho group public homogeneous
Population-Based Search Algorithmsfor Multi Robot Systems
Example of collaborative
for Multi-Robot Systems
Example of collaborative co-learning with binary encoding of 100 candidate solutionscandidate solutions and 2 robots
Co-Shaping Competitive BehaviorCo Shaping Competitive Behavior
fi f ≠ fi ffitness f1 ≠ fitness f2
Three Case Studies• Obstacle avoidance (today)
C di t d ti (t d )• Coordinated motion (today)• Collaborative stick pulling (next week)
Notes: 1. each case study investigates different strategies for y g g
distributing, sharing, and evaluating candidate solutions using a multi-robot system
2. each case study investigates additional peculiar methods (e.g., impact of on-board distributedness, leveraging different modeling abstractionsleveraging different modeling abstractions, measuring diversity and specialization)
Co-Shaping Obstacle Co Shaping Obstacle Avoidance
• Effects of robotic group sizeEffects of robotic group size• Effects of communication constraints
Population-Based Search Algorithms for Multi-Robot Systems
Distributed Robotic Adaptation• Standard robotic adaptation approach: evaluate
population in serialpopulation in serial• New distributed adaptation approach: robots
l t diff t ti l i ll levaluate different particles in parallel • Potentially much faster adaptation rate• Why PSO:
– appears interesting since this metaheuristic works pp gwell with small pools of candidate solutions: candidate pool size ≈ robot team size
– limited particle neighborhood sizes → scalable, on-board operation
Varying the Robotic Group SizeVarying the Robotic Group Size
Varying the Robotic Group SizeVarying the Robotic Group Size
Varying the Robotic Group SizeVarying the Robotic Group Size
Var ing the Robotic Gro p Si eVarying the Robotic Group Size
Varying the RoboticG Si GA PSO
• Same control architecture as [Floreano & Mondada, 1996] (ANN 22 i h )
Group Size – GA vs. PSO
1996] (ANN, 22 weights to tune)• Same fitness function as [Floreano & Mondada, 1996]• Similar Webots world as [Pugh et al., 2005] but 2x2 m• Robot group size: 1, 2, 5, 10, 20• GA and PSO parameters:
Varying the RoboticGroup Size – GA vs. PSO
Performance of best controllers after evolution
[Pugh and Martinoli, AAMAS 2006]
Varying the Robotic Group Size –E l i T ti E i tEvolving vs. Testing Environment
• PSO only; numberPSO only; number of particles = 10
• Gradually increase ynumber of robots on team
• Up to 10x faster learning with little performance loss
• Arena 3x3 m
[Pugh and Martinoli, Swarm Intelligence J., 2009]
Communication-Based Neighborhoods
• Default neighborhood ring topology 2• Default neighborhood - ring topology, 2 neighbors for each robot
• Problem for real robots: neighbor could be• Problem for real robots: neighbor could be very far away
• Possible solutions – use two closest robots in• Possible solutions – use two closest robots in the arena (capacity limitation), use all robots within some radius r (range limitation); w t so e ad us ( a ge tat o );reality is affected often by both
• How will this affect the performance?ow w s ec e pe o ce?
A Possible Robot DistributionA Possible Robot Distribution
Index-Based Neighborhoodg
Ring Topology - Standard
Communication-Based NeighborhoodsNeighborhoods
2-closest – Model A
Communication-Based N i hb h dNeighborhoods
Within range r – Model B
Communication-Based NeighborhoodsCommunication Based Neighborhoods• Realistic simulation results
(Webots)(Webots)• 10 Khepera III robots (one
particle per robot)• Standard neighborhood is 2• Standard neighborhood is 2
fixed particles• New particle
neighborhoods defined byneighborhoods defined by communication limitations:– Model A: particles from
closest two robots– Model B: particles from all
robots within range r = 120 cm
• No drawback to• No drawback to communication-based neighborhoods[Pugh and Martinoli, Swarm Intelligence J., 2009]
V i th C i ti RVarying the Communication RangeArena: 3x3 m, 10 robots total,
Varying the Communication Range• Realistic simulation results (Webots)• Low communication ranges → learning too slowly
High comm nication ranges fitness platea s too earl• High communication ranges → fitness plateaus too early• Limited communication range gives best performance →
scalable solution
[Pugh and Martinoli, Swarm Intelligence J., 2009]
Distributed Adaptation with Real RobotsEnabling Technology
• See Week 4 slides• On board relative positioning system
- Enabling Technology
• On-board relative positioning system• Principle:
– belt of IR emitters (LED) and receivers ( h di d )(photodiode)
– IR LED used as antennas; modulated light (carrier 10.7 MHz)RF hi b hi d d RSSI– RF chip behind, measured RSSI
– Measure range & bearing of the next robot (relative positioning) can be coupled with RF channel (e g 802 11) for headingRF channel (e.g. 802.11) for heading assessment
– Can also be used for 20 kbit/s com channel
• [Pugh et al., IEEE Trans. on Mechatronics, 2009]
Distributed Adaptation with Real Robots- Sample Results
• All operations on-board using relative positioning system• Effective learning of obstacle avoidance behavior• Effective learning of obstacle avoidance behavior• Similar results for communication-based neighborhoods• Arena 3 x 3 m
[Pugh and Martinoli, Swarm Intelligence J., 2009]
Distributed Adaptation withR l R b tReal Robots
Before adaptation (5x speed-up)
Distributed Adaptation withR l R b tReal Robots
After adaptation (5x speed-up)
Evolving Coordinated MotionEvolving Coordinated Motion• Physically connected robots• Group performance• Group performance
The SWARM-BOTS project (2001-2005)http://www.swarm-bots.org
We call “swarm-bot” an artifact composed of a number of simpler robots, called “s-bots”, capable of self assembling and self organizingcapable of self-assembling and self-organizing to adapt to its environmentS bots can connect to and disconnect from eachS-bots can connect to and disconnect from eachother to self-assemble and form structures whenneeded, and disband at will,
The coordinated motion task
• Four s-bots are connected in a swarm-bot formation
• Their chassis are randomly oriented
• The s-bots should be able to – collectively choose a direction of
motionmotion – move as far as possible
• Simple perceptrons are evolved asSimple perceptrons are evolved as controllers
Coordinated motion:h iThe traction sensor
• Connected s-bots apply turretturret
pp ypulling/pushing forces to each other when moving
• Each s-bot can measure a traction force acting on its turret/chassis
i tractiontractionconnection• The traction force indicates the
i t h b t
traction sensortraction sensor
mismatch between – the average direction of motion of the
groupgroup– the desired direction of motion of the
single s-bot
Coordinated motion:The evolutionary algorithmi d d• Binary encoded genotype
– 8 bits per real valued parameter of the neural controllers• Generational evolutionary algorithmGenerational evolutionary algorithm
– 100 individual evolved for 100 generations– 20 best individual are allowed to reproduce in each
generationgeneration– Mutation (3% per bit) is applied to the offspring
• The perceptron is cloned and downloaded to each s-boteach s-bot
• Fitness is evaluated looking at the swarm-botperformance– Each individual is evaluated with equal starting
conditions
Population-Based Search Algorithms for Multi Robot Systemsfor Multi-Robot Systems
Coordinated motion:Fit l tiFitness evaluation
• The fitness F of a genotype is given by the g yp g ydistance covered by the group:
where X(t) is the coordinate vector of the center of mass at time t, and D is the maximum ,distance that can be covered in 150 simulation cycles
• Fitness is evaluated 5 times (fixed number per• Fitness is evaluated 5 times (fixed number per candidate solution!), starting from different random initializationsTh lti i i d t th t• The resulting average is assigned to the genotype
Coordinated motion: Results Post-evaluationPost-evaluation
Replication Performance1 0.878882 0.83959
Average fitnessAverage fitness
2 0.839593 0.883384 0.715675 0 795735 0.795736 0.752097 0.834258 0.858489 0.87222
10 0.76111
Coordinated motion:Coordinated motion: Real s-bots
flexibilityflexibilitydefault (used for evolution)default (used for evolution)
Coordinated motion:Coordinated motion: Scalability
flexibility and scalabilityflexibility and scalabilityscalabilityscalability flexibility and scalabilityflexibility and scalabilityscalabilityscalability
ConclusionConclusion
Take Home Messages• Machine-learning techniques can be used not only for design and
optimization of controllers but also of further hardware features pwith the help of appropriate simulation tools
• The cost of an optimization problem is heavily influenced by the amount of noise in the evaluation function the time needed foramount of noise in the evaluation function, the time needed for evaluating a candidate solution, and the dimension of the parameter space
• Collaborative co adaptation strategies can be differentiate along• Collaborative co-adaptation strategies can be differentiate along three axes: public/private solutions; homogeneous/heterogeneous system, individual/group performance
• Multi-robot platforms can be exploited for testing in parallel multiple candidate solutions
• PSO appears to be well suited for fully distributed on-board pp yoperation and strategies based on local, geographical (com-based) neighborhoods have been successfully tested
O li P II M l i L l M d liOutline – Part II: Multi-Level Modeling
• Multi-Level Modeling Methodology– Rationale– Assumptions
• Methodological frameworkg– Overview– Some theoretical background
• An incremental bottom-up modeling recipe
Modeling Rationale and Modeling Rationale and ChoicesChoices
Motivation for Modeling
• Understanding the interplay of the various elements of the system (e g robot features robot numbersof the system (e.g., robot features, robot numbers, environment, noise level)
• Having additional tools for designing and optimizingHaving additional tools for designing and optimizing the distributed robotic system
• Delivering performance predictions for the ensembleDelivering performance predictions for the ensemble in shorter time or before doing actual experiments
• Investigating experimental conditions difficult or• Investigating experimental conditions difficult or impossible to reproduce in reality
• Formally analyzing system properties• Formally analyzing system properties
Modeling Choices• Gray-box approach: to easily incorporate a priori
information (e.g., # of agents, technological and environmental features)
• Probabilistic: to capture noisy interactions, noisy robotic components, stochastic control policies, and enable
ti h t d b t tiaggregation schemes towards abstraction
l i l l li i l diff d i• Multi-level: to represent explicitly different design choices, trade off computational speed and faithfulness to reality bridge mathematically tractable models and realityreality, bridge mathematically tractable models and reality in an incremental way
Originality and Differences with th M d l B d A hother Model-Based Approaches
• Specifically targeted to (miniature) swarm-robotic systems: exploit robust control design techniques for resource constrained individualrobust control design techniques for resource-constrained individual nodes (e.g. BB, ANN); incorporate submicroscopic constraints (e.g., technology, environment) from start, bottom-up modeling and multi-level system-centered design approachlevel system centered design approach
• Different from traditional model-based techniques in robotics: top-down approaches start from idealized node model and relaxdown approaches, start from idealized node model and relax assumptions; compensate with potential expensive/sophisticated technology and control algorithms for meeting the requirements; (multi-level) control-centered design approach( ) g pp
• Different from traditional modeling in biology (and other natural sciences): as simple as possible macroscopic models targeting asciences): as simple as possible macroscopic models targeting a given scientific question; free parameters + fitting based on macroscopic measurements since often submicroscopic/microscopic information not available/accurate
Experimental Invariant pFeatures and Modeling
A tiAssumptions
Invariant Experimental FeaturesInvariant Experimental Features• Short-range (typically 1 robot diameter), crude (noisy,
a few discrimination levels) proximity sensinga few discrimination levels) proximity sensing• Full mobility but limited navigation (no planning, no
absolute localization)absolute localization)• Limited use of long-range communication channels
available on the platforms (only as a teammate sensor)v b e o e p o s (o y s e e se so )• Reactive, behavior-based control, with a few internal
states• No overcrowded arenas• Multiple runs (typically 5+) for the same experimentalMultiple runs (typically 5 ) for the same experimental
parameters; randomized robot poses at the beginning
Modeling Assumptions g p1. State-based description for environment and multi-
robot systemrobot system2. Semi-Markovian properties: the system future state
is a function of the current state (and possibly of the ( p yamount of time spent in it)
3. Nonspatial metrics for swarm performance4. Well-mixed system: because of simple navigation,
obstacle avoidance, and convex environment the mean spatial distribution of agents over multiplemean spatial distribution of agents over multiple runs can be assumed to be homogeneous
5. Linear superposition of object/robot detection areasp p j(sparse object/robot distribution; no overcrowding)
Assumptions 1 and 2Assumptions 1 and 2• We work with statesWe work with states
S
pin poutTx
Sx
Sx: state xTx: duration of state xpin, pout: probabilities to entry and leave state x
• States can characterize both robots and the environment
Assumptions 3,4,5Assumptions 3,4,5• Trajectories and object location do not count -> 1D
Montecarlo simulationMontecarlo simulation• N objects of type i -> N x (prob. to encounter i)
2D physical space 1D probability space2D physical space 1D probability space
O1
O1O1
O2
O2
O2OFree space
O2O2
Experimental Validation of A i 3 4 5Assumption 3,4,5
PositionN b di d b lNonembodied obstacles = detection surfaces
Shape
Size Square Rect. Round All shapes Geometry
Numerical example (mean ± std dev, 3 locations, 100 h simulated time):
q p y
robot 0.31 ± 0.04 0.3 ± 0.03 0.32 ± 0.02 0.31 ± 0.03 0.31
Experimental Validation of A ti 3 4 5Assumptions 3,4,5
Symmetry DefaultSymmetryof Stick Distribution
Default
# sticks
Methodological Framework: gOverview and Theoretical
B k dBackground
Multi-Level Modeling Methodology
n
g gy
∑ ∑′ ′
′−′′=n n
nnn tNtnnWtNtnnW
dttdN )(),|()(),|()(
S S rics
trac
tionSs Sa
Ss SaS SMicroscopic : multi-agent models,
Macroscopic: rate equations, mean field approach, whole swarm
mon
met
r
Abs
t
me
Ss SaSs Sa only relevant robot features captured, 1 agent = 1 robot
Com
m
ntal
timSubmicroscopic: intra-robot (e.g.,
S&A, transceiver) and environment (e.g., physics) details reproduced f ithf ll
peri
men
Target system (physical reality):
faithfully
Exp
info on controller, S&A, communication, morphology and environmental features
Microscopic LevelMicroscopic Levelp(n,t) = probability of an agent to be in the state n at time tIf Markov properties fulfilled:
−Δ+=Δ tnpttnptnp ),(),(),(
If Markov properties fulfilled:
∑ ∑′ ′
Δ+′−′Δ+=n n
tnptnttnptnptnttnp ),(),|,(),'(),|,(
i fl tflinflow outflow
Probability the agent was in a Transition probabilitygiven state n’
tnttnp ′Δ+ )|(
Sum over all possible states n’ the agent can be in
ttnttnptnnW
t ΔΔ+
=′→Δ
),|,(lim);|(
0Transition rate
Macroscopic Level – Time-ContinuouspLeft and right side of the equation: averaging over the total number of agents, dividing by Δt, limit Δt → 0; neglect di ib i f h h i i bl ( fi ld h)
tdN )( R t E ti
distributions of the stochastic variables (mean field approach):
∑ ∑′ ′
′ ′−′=n n
nnn tNtnnWtNtnnW
dttdN )(),|()(),|()( Rate Equation
(time-continuous)
i fl tflinflow outflown, n’ = states of the agents (all possible states at each instant)N f ti ( b ) f t i t t t ti tNn = average fraction (or mean number) of agents in state n at time tW = transition rates
Macroscopic Level – Time-DiscreteRate Equation (time-discrete):
∑ ∑′ ′−′+=+ nnnn kTNkTnnTWkTNkTnnTWkTNTkN )(),|()(),|()())1((
p
inflow outflow
∑ ∑′ ′n n
k = iteration indexT = time step, sampling intervalTW = transition probability per time stepTW = transition probability per time step
Notation often simplified to:
∑ ∑′ ′
′ ′−′+=+n n
nnnn kNknnPkNknnPkNkN )(),|()(),|()()1(
T i ifi d i th t t f ll P i l l t d f T*W thT is specified in the text once of all, P is calculated from T*W or other calibration methods
Time Discretization: Th E i i R iThe Engineering Receipt
Time-discrete vs. time-continuous models:1. Assess what’s the time resolution needed for your system
performance metrics
Time discrete vs. time continuous models:
2. Consider unit-to-unit interaction essence: probabilistic/deterministic, asynchronicity role, …
3. Choose whenever possible the most computationally efficient model:3. Choose whenever possible the most computationally efficient model: time-discrete less computationally expensive than emulation of continuity (e.g. Runge-Kutta, etc.); in our systems/metrics there is no e idence of decreased prediction acc racevidence of decreased prediction accuracy
4. Advantage of time-discrete models: a single common sampling rate can be defined among different abstraction levels
Methodological Framework: gAn Incremental Bottom-Up
R iRecipe
1 Target System & Task(s)1. Target System & Task(s)Perform basic design choices for the experimental set-up:
• Hardware and software for the robotic platform• Hardware and software for the robotic platform• Environment in which robots operate• Task(s) robots must accomplishTask(s) robots must accomplish
2 Metric(s) and State Space2. Metric(s) and State Space• Define system performance metric(s)• Define state space (number of states, granularity) • Performance metric(s) and state definitions well aligned!• Exploit controller blueprint (if available) as additional source ofExploit controller blueprint (if available) as additional source of
information for defining the state space
Obstacle detected
Search AvoidanceObstacle avoided
C(k) = pg2Ns(k)Ng(k)
Grip
( ) pg2 s( ) g( )
3 Submicroscopic Model3. Submicroscopic ModelImplement faithfully your design choices in a submicroscopic model (in principle even running the samesubmicroscopic model (in principle even running the same control code; libraries and APIs are usually provided in standard commercial or open-source simulators)standard commercial or open source simulators)
4. Microscopic Model• Aggregate local interactions and reduce intra-robot details• Maintain state space’s structure as defined at step 2• Maintain individual representation (and exact discrete quantities) for• Maintain individual representation (and exact discrete quantities) for
each robotic node and environmental object of interest
Ss Sa R11 Se Sd
SiRn1
…Distributed robotic system
Ss Sa
Ss Sa
R12
R1l
…
C t
Si…
Se Sd
Si
Rnm
Caste 1
Coupling (e.g., manipulation)
Caste n
Environment…
O11 O1pOq1 Oqr
… …Sa Sb Sa Sb
5. Macroscopic Model
Distributed robotic system
• Aggregate individual nodes into one or multiple representations (castes) at
Distributed robotic system
Ss Sa
collective level• Maintain state space’s structure as
defined at step 2
Se Sd
Caste1
C
defined at step 2• Solve exactly (stochastically) or
using the ODE approximation if the system is defined as a set of ODEs
Coupling
Si Caste nsystem is defined as a set of ODEs• Exploit conservation laws (e.g. # of
robots in an enclosed arena) to
Type 1
T
Sa Sb
simplify the representation of the dynamical system
Environment
Type q
6. Parametric Calibration• Number of parameters is decreasing with the abstraction level• Calibrate a given level based on the underlying one (e.g.,
b i i i h h i l i i i h
6. Parametric Calibration
submicroscopic with physical system; microscopic with submicroscopic, macroscopic with microscopic)
• Parametric (e.g., mean only, mean and variance) or non t i ( t l di t ib ti d d t th l l l)parametric (actual distribution recorded at the lower level)
assumptions• Various methods available
Ad h i ( i i i f f i )– Ad hoc experiments (e.g., interaction time, sensor transfer functions)– System identification techniques (constrained parameter fitting)– Statistical verification techniques (e.g., trajectory analysis)
• Parameter e ample for micro and macroscopic models:• Parameter example for micro- and macroscopic models: – State durations– State-to-state transition probabilities
pin poutTstate
Conclusion
Take Home Messages• Models help understanding and generalizing properties of
real-time distributed intelligent systems• Three main levels of models: submicro, micro and macro• Macroscopic models using the mean field approach result
i ODEin ODE• Multi-level modeling allows for different approximations,
/ t ti t d ffaccuracy/computation trade-offs• A methodology with precise inter-level mapping has been
developed for a given class of experimentsdeveloped for a given class of experiments• If carefully designed, models allow also for system
optimization and closing the loop between analysis andoptimization and closing the loop between analysis and synthesis
Additional Literature – Week 6Books• T. Balch and L. E. Parker (Eds.), Robot teams: From diversity to polymorphism.
Natick, MA: A K Peters.,• Ross S. M., Introduction to Probability Models, Academic Press, San Diego,
CA, USA, 1997.
PPapers• Pugh J. and Martinoli A., “Multi-Robot Learning with Particle Swarm
Optimization”. Proc. of the Fifth ACM Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems, May 2006, Hakodate, Japan, pp. 441–448.g y y p pp
• Murciano A. and Millán J. del R., "Specialization in Multi-Agent Systems Through Learning". Biological Cybernetics, 76: 375-382, 1997.
• Dorigo M., Trianni V., Sahin E., Groß R., Labella T., Nolfi S., Baldassare G., Deneubourg J L Mondada F Floreano D and Gambardella L “EvolvingDeneubourg J.-L., Mondada F., Floreano D., and Gambardella L.. Evolving Self-organising Behaviours for a Swarm-bot”. Autonomous Robots, 17:223–245, 2004
• Mataric, M. J. “Learning in behavior-based multi-robot systems: Policies, d l d th t ” S i l I M lti di i li t di f ltimodels, and other agents”. Special Issue on Multi-disciplinary studies of multi-
agent learning, Ron Sun, editor, Cognitive Systems Research, 2(1):81-93, 2001.