Role of Cooperation in Multi-robot Systemsras-egypt.org/reading/MRS.pdf · • Applications:...
Transcript of Role of Cooperation in Multi-robot Systemsras-egypt.org/reading/MRS.pdf · • Applications:...
3/17/2014
1
Role of Cooperation in Multi-robot
http://www.iacconf.com/
p
Systems
Alaa Khamis, PhD, SMIEEE
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 1/221Tuesday March 4, 2014 © Dr. Alaa Khamis
http://www.alaakhamis.org/
http://www.ras-egypt.org/
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo
• Talk Description
• Introduction to Multirobot Systems
Outline
• Benchmark Problems of Multirobot Systems
• Challenging Problems of Multirobot Systems
• Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 2/222Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
2
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo
• Talk Description
• Introduction to Multirobot Systems
Outline
• Benchmark Problems of Multirobot Systems
• Challenging Problems of Multirobot Systems
• Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 3/223Tuesday March 4, 2014 © Dr. Alaa Khamis
Talk DescriptionThis talk provides well-grounded and informative answers to
questions like:
Wh t th i f t f lti b t t (MRS)?• What are the main features of multirobot systems (MRS)?
• What are the challenging problems of MRS?
• How to achieve different forms of cooperation in MRS?
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 4/224Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
3
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo
• Talk Description
• Introduction to Multirobot Systems
Outline
• Benchmark Problems of Multirobot Systems
• Challenging Problems of Multirobot Systems
• Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 5/225Tuesday March 4, 2014 © Dr. Alaa Khamis
Introduction to Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 6/226Tuesday March 4, 2014 © Dr. Alaa Khamis
Source: Bruno Siciliano. Robots Moving Closer to Humans. IEEE Distinguished Ambassador Seminar, IEEE RAS – Egypt Chapter. Available at: http://ras-egypt.org/activities.html
3/17/2014
4
Introduction to Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 7/227Tuesday March 4, 2014 © Dr. Alaa Khamis
Source: Lev Grossman, "2045: The Year Man Becomes Immortal," TIME Magazine, February, 2011.
Introduction to Multirobot Systems
i h l i
Kurzweil predicts personal computers with the power of the human brain by 2025
Genetics
Emerging Technologies
Nano-technology Robotics
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 8/228Tuesday March 4, 2014 © Dr. Alaa Khamis
Genetics Nano-technology Robotics
3/17/2014
5
Introduction to Multirobot SystemsShort-term goals of Robotics:
• The object-recognition capabilities of a 2-year-old child
• The language capabilities of a 4-year-old child
• The manual dexterity of a 6-year-old child
I wanna go out!
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 9/229Tuesday March 4, 2014 © Dr. Alaa Khamis
More info: http://www.springer.com/authors/author+zone?SGWID=0-168002-12-691704-0
• The social understanding of an 8-year-old child
Introduction to Multirobot Systems• Evolutionary Stages of Robot Systems
◊ Types
◊ Market size
◊ Intelligence
◊ Body/Brain Evolution
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 10/2210Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
6
Introduction to Multirobot Systems• Evolutionary Stages: Types
Service Robots forP l U
Industrial
PersonalRobotics
ServiceRobotics
Service Robots for Professional Use
Personal Use
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 11/2211Tuesday March 4, 2014 © Dr. Alaa Khamis
Robotics
Introduction to Multirobot Systems• Evolutionary Stages: Market Size
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 12/2212Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
7
Introduction to Multirobot Systems• Evolutionary Stages: Intelligence
Deliberative Cycle Cognitive Cycle
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 13/2213Tuesday March 4, 2014 © Dr. Alaa Khamis
Reactive Cycle
Brains
SI Cognitive RoboticsSwarm
Robotics
Introduction to Multirobot Systems• Evolutionary Stages: Body/Brain Evolution
100s
10s
DAI MultiagentDistributed
Robot System
Robotics
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 14/2214Tuesday March 4, 2014 © Dr. Alaa Khamis
Bodies
1AI Robotics Centralized Control
10s
Multiple Machines
1
Machine
100s
MEMS-based Multiple Machines
3/17/2014
8
• Multirobot Systems
Introduction to Multirobot Systems
Multirobot systems (MRS) are a group of robots that are
designed aiming to perform some collective behavior.
The MRS is gaining great interest because of the following
reasons:
◊ Resolving task complexity
◊ Increasing performance
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 15/2215Tuesday March 4, 2014 © Dr. Alaa Khamis
◊ Reliability
◊ Simplicity in design
• Why Multirobot Systems?: Resolving task complexity
Introduction to Multirobot Systems
Some tasks may be quite complex for a single robot to do or even
it might be impossible.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 16/2216Tuesday March 4, 2014 © Dr. Alaa Khamis
Box Pushing
Crossing a gap
3/17/2014
9
• Why Multirobot Systems?: Resolving task complexity
Introduction to Multirobot Systems
Some tasks are inherently distributed.
Heterogeneous team of an air and
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 17/2217Tuesday March 4, 2014 © Dr. Alaa Khamis
two ground
vehicles that can perform cooperative reconnaissance and
surveillance
• Why Multirobot Systems?: Resolving task complexity
Introduction to Multirobot Systems
Some tasks are diverse and
required different capabilities.
“As I look at the trends that are now starting to converge, I can envision a future in which robotic devices will become a nearly ubiquitous part of our day-to-day lives.
A robot in every home
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 18/2218Tuesday March 4, 2014 © Dr. Alaa Khamis
The challenges facing the robotics industry are similar to those we tackled in computing three decades ago.”
Bill Gates, 2007Scientific American
3/17/2014
10
• Why Multirobot Systems?: Increasing performance
Introduction to Multirobot Systems
Minimize:
Multiple robots can solve problems faster using parallelism.
Maximize:
• Area Coverage
• Object Coverage
• Task completion time
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 19/2219Tuesday March 4, 2014 © Dr. Alaa Khamis
• Radio Coverage
• Why Multirobot Systems?: Reliability
Introduction to Multirobot Systems
The introduction of multiple robots increases robustness
through redundancy.
Increasing the system reliability
because having only one robot
may work as a bottleneck for the
whole system especially in
critical times.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 20/2220Tuesday March 4, 2014 © Dr. Alaa Khamis
But when having multiple robots
doing a task and one fails, others
could still do the job.
3/17/2014
11
• Why Multirobot Systems?: Simplicity in design
Introduction to Multirobot Systems
Building several resource-bounded robots is much easier than
having a single powerful robot
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 21/2221Tuesday March 4, 2014 © Dr. Alaa Khamis
Several resource-bounded simple robots
Powerful single robot
• Why Multirobot Systems?: Simplicity in design
Introduction to Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 22/2222Tuesday March 4, 2014 © Dr. Alaa Khamis
S-bot: an autonomous, mobile robot capable of
self-assembly
ANDERS LYHNE CHRISTENSEN, REHAN O’GRADY, AND MARCO DORIGO, "Morphology Control in a Multirobot System: Distributed Growth of Specific Structures Using Directional Self-Assembly“, IEEE Robotics & Automation Magazine, December 2007.
3/17/2014
12
• Applications
Introduction to Multirobot Systems
◊ Intelligent carts
◊ UXVs◊ UXVs
◊ Cube sats
◊ Space-based construction
◊ Agricultural Foraging
◊ Killing Cancer Tumors in Human Body
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 23/2223Tuesday March 4, 2014 © Dr. Alaa Khamis
g y
◊ Search and Rescue
◊ Humanitarian demining
◊ Distributed monitoring
Unintelligent carts are commonly found in large
airports Travelers pick up carts at designated
• Applications: Intelligent Carts
Introduction to Multirobot Systems
airports. Travelers pick up carts at designated
points and leave them in arbitrary places. It is a
considerable task to re-collect them.
It is, therefore, desirable that intelligent carts (intelligent robots)
draw themselves together autonomously.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 24/2224Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
13
Unmanned Vehicles (UXVs)
• Applications: UXVs
Introduction to Multirobot Systems
Unmanned Ground Vehicles (UGV)
Unmanned Aerial Vehicles (UAV) & Micro Aerial Vehicles
(MAV)
Unmanned Underwater
Vehicles (UUV)
Unmanned Surface
Vehicles (USV)
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 25/2225Tuesday March 4, 2014 © Dr. Alaa Khamis
• Applications: UAVs
Introduction to Multirobot Systems
◊ Military Applications: Drones
Congress has mandated that by
A forward looking infrared (FLIR) camera onUAV
UAV carrying g by
2015, 1/3rd of all US military
missions should be unmanned.
There are 17,300 drones in the
US army inventory.
Th d
UAV carrying Viper Strike
Weapon System
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 26/2226Tuesday March 4, 2014 © Dr. Alaa Khamis
These drones can carry up to
3000 pounds of weapons.
Fabricated by Boeing
Source: http://www.marketresearchmedia.com/?p=509
3/17/2014
14
• Applications: Cube Sats
Introduction to Multirobot Systems
◊ Small and Pico Satellites
Small satellites are artificial
satellites of lower weights and
smaller sizes (under 500 kg).
Small satellites can be
Minisatellite, Microsatellite,
Nanosatellite, Picosatellite or
Giant Solar-powered Satellite
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 27/2227Tuesday March 4, 2014 © Dr. Alaa Khamis
Nanosatellite, Picosatellite or
Molecularsatellite.
Network of CubeSat
Picosatellite or “picosat” is an
artificial satellite with a wet
mass between 0.1 and 1 kg. More info: Klaus Schilling, IEEE Distinguished Lecture. Available at: http://ras-egypt.org/activities.html
• Applications: Search & Rescue Operations
Introduction to Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 28/2228Tuesday March 4, 2014 © Dr. Alaa Khamis
Companion slides for the book Bio-Inspired Artificial Intelligence: Theories, Methods, and Technologies by Dario Floreano and Claudio Mattiussi, MIT Press
3/17/2014
15
• Applications: Humanitarian Demining
Introduction to Multirobot Systems
GPR-based Minesweeper
…
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 29/2229Tuesday March 4, 2014 © Dr. Alaa Khamis
Alaa Khamis and Asser ElGindy "Minefield Mapping using Cooperative Multirobot Systems", Journal of Robotics, 2012.
Cooperation in Multirobot Systems
• Applications: Distributed Monitoring
Dimensions of Recent Trends
Multiplicity Multimodality Mobility
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 30/2230Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
16
Cooperation in Multirobot Systems
Domotics
• Applications: Distributed Monitoring
Health Care
Homeland Security
Logistics
Pi li d P M it i
C-SANETs
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 31/2231Tuesday March 4, 2014 © Dr. Alaa Khamis
Structural Monitoring
Network‐centric Warfare
Environment Monitoring
Pipeline and Process Monitoring
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo
• Talk Description
• Introduction to Multirobot Systems
Outline
• Benchmark Problems of Multirobot Systems
• Challenging Problems of Multirobot Systems
• Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 32/2232Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
17
• Box Pushing and Object Transportation
• Exploration and Formation Control
• Division of Labor
Benchmark Problems of MRS
Division of Labor
• Foraging
• Object/Area/Radio Coverage
• Soccer Tournaments
• Cooperative perception
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 33/2233Tuesday March 4, 2014 © Dr. Alaa Khamis
p p p
• Cooperative Target Cueing and Handoff
• Cooperative Mapping
• …
• Box Pushing and Object Transportation
Benchmark Problems of MRS
This problem’s concern is about a group of robots try to push a
box to a certain point.
Applications include transportation of heavy objects in
industrial environments or assembly of large-scale
structures, such as terrestrial buildings or planetary habitat.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 34/2234Tuesday March 4, 2014 © Dr. Alaa Khamis
Box Pushing
Rod
Object Transportation
3/17/2014
18
• Exploration and Formation Control
Benchmark Problems of MRS
In the exploration task the robots must be spread in the
environment in order to collect as much information as
possible about the surrounding area.
The formation task is focused on having the robots move in
the environment forming particular shapes.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 35/2235Tuesday March 4, 2014 © Dr. Alaa Khamis
• Division of Labor
Benchmark Problems of MRS
This cooperative behavior addresses how to dynamically assign
a set of tasks to a set of robots to maximize overall expected
performance.
RTA :
niRri ,...,2,1 ;
miTt j ,...,2,1 ;
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 36/2236Tuesday March 4, 2014 © Dr. Alaa Khamis
A set of m surveillance tasks: TA set of n mobile Sensors: R
RTA :
Alaa Khamis, Ahmed Elmogy and Fakhreddine Karray, “Complex Task Allocation in Mobile Surveillance Systems,” Journal of Intelligent and Robotic Systems, Springer, DOI: 10.1007/s10846-010-9536-2, 2011 .
3/17/2014
19
• Communication Relaying
Benchmark Problems of MRS
This cooperative behavior consists in establishing
communication through relaying in order to dramatically
increase radio coverage or expand communications links,
primarily over rugged, mountainous or urban terrains.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 37/2237Tuesday March 4, 2014 © Dr. Alaa Khamis
• Soccer
Benchmark Problems of MRS
Soccer playing is challenge problem for studying coordination
and control in multirobot systems. This domain incorporates
many challenging aspects of multirobot control, including:
◊ Collaboration,
◊ Robot control architectures,
◊ Strategy acquisition,
◊ Real-time reasoning and action,
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 38/2238Tuesday March 4, 2014 © Dr. Alaa Khamis
http://www.robocup.org/ http://www.fira.net/&
g ,
◊ Sensor fusion,
◊ Dealing with adversarial environments,
◊ Cognitive modeling, and
◊ Learning.
3/17/2014
20
• Other problems
Benchmark Problems of MRS
◊ Sorting
◊ Cooperative perception in robotics◊ Cooperative perception in robotics
◊ Cooperative Mapping
◊ Collective Robotic Search
◊ …
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 39/2239Tuesday March 4, 2014 © Dr. Alaa Khamis
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo
• Talk Description
• Introduction to Multirobot Systems
Outline
• Benchmark Problems of Multirobot Systems
• Challenging Problems of Multirobot Systems
• Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 40/2240Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
21
Challenging Problems of MRS• Algorithm Design
• Implementation and Test
• Analysis and Modelling
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 41/2241Tuesday March 4, 2014 © Dr. Alaa Khamis
• Algorithm Design
Challenging Problems of MRS
Algorithm–based Behaviour
Individual Behaviour
i-Level
Group behaviour
g-Level
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 42/2242Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
22
• Algorithm Design: i-Level Algorithm
Challenging Problems of MRS
Brain Functions
Low-level functions High-level functions
Perception, situation awareness, reasoning,
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 43/2243Tuesday March 4, 2014 © Dr. Alaa Khamis
• Fully understood
• Localized
• Partially understood
• Not fully localized
gdecision making, learning, etc.
Roboticists face the problem of designing both the physical
morphology and behaviours of the individual robots such that
• Algorithm Design: g-Level Algorithm
Challenging Problems of MRS
p gy
when those robots interact with each other and their
environment, the desired overall collective behaviours will
emerge. At present there are no principled approaches to the
design of low-level behaviours for a given desired
collective behaviour [1]
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 44/2244Tuesday March 4, 2014 © Dr. Alaa Khamis
collective behaviour [1].
“collective behavior is NOT simply the sum of each
participant’s behavior, as others emerge at the society level” [2].
[1] E. Sahin and A. Winfield, “Special issue on swarm robotics,” Swarm Intelligence, 2: 69–72, Springer Science, 2008.
[2] Pasteels et al. From Individual to Collective Behavior in Social Insects. Pages 155-175, 1987.
3/17/2014
23
Main decisional abilities:
Challenging Problems of MRS• Algorithm Design: g-Level Algorithm
- Mission planning,
- Task allocation and
- Coordinated task achievement
Management the task allocation,
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 45/2245Tuesday March 4, 2014 © Dr. Alaa Khamis
Scheduling,
Cooperation/collaboration between the entities,
Conflict avoidance, etc.
To build and rigorously test a swarm of robots in the laboratory
requires a considerable experimental infrastructure
• Implementation and Test
Challenging Problems of MRS
requires a considerable experimental infrastructure.
Real-robot experiments thus typically proceed hand-in-hand
with simulation and good tools are essential [1].
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 46/2246Tuesday March 4, 2014 © Dr. Alaa Khamis
[1] E. Sahin and A. Winfield, “Special issue on swarm robotics,” Swarm Intelligence, 2: 69–72, Springer Science, 2008.
3/17/2014
24
◊ Advanced Robotics Interface for Applications (ARIA): Robotic Sensing and Control Libraries.
Challenging Problems of MRS• Implementation and Test
◊ Open Robot Control Software (OROCOS):open-source real time control architecture for different machines.
◊ Microsoft Robotics Studio: is a Windows-based environment for robot control and simulation.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 47/2247Tuesday March 4, 2014 © Dr. Alaa Khamis
◊ Player/Stage/Gazebo: PSG is open source software that used and developed by an international community of researchers from over 30 universities/companies.
◊ Robot Operating System (ROS)
A multirobot system (specially swam systems) is typically a
t h ti li d ti ll b bl t
• Analysis and Modelling
Challenging Problems of MRS
stochastic, non-linear and partially observable system
and constructing mathematical models for both validation
and parameter optimization is challenging.
Such models would surely be an essential part of constructing a
f f l ld li i [ ]
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 48/2248Tuesday March 4, 2014 © Dr. Alaa Khamis
safety argument for real-world applications [1].
[1] E. Sahin and A. Winfield, “Special issue on swarm robotics,” Swarm Intelligence, 2: 69–72, Springer Science, 2008.
3/17/2014
25
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo
• Talk Description
• Introduction to Multirobot Systems
Outline
• Benchmark Problems of Multirobot Systems
• Challenging Problems of Multirobot Systems
• Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 49/2249Tuesday March 4, 2014 © Dr. Alaa Khamis
• Forms of Cooperation in MRS
Forms of Cooperation
Towards Cooperative Multirobot Systems
Augmentative
Cooperation
Forms of Cooperation
Integrative
Cooperation
Debative
Cooperation
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 50/2250Tuesday March 4, 2014 © Dr. Alaa Khamis
Alaa Khamis, “Cooperative Sensor and Actor Networks in Distributed Surveillance Context,” 10th International Conference on Practical Applications of Agents and Multi-Agent Systems (PAAMS'12), Salamanca, Spain, 2012.
3/17/2014
26
Agent-1
Know-howWhen agents have a similar
know-how but the agents must
• Augmentative Cooperation
Towards Cooperative Multirobot Systems
Sub‐task‐1
Percept Action
Task
Sub‐task‐2
Examples:
be multiplied to perform a task
too demanding for only one
agent. This task is then shared
into similar sub-tasks.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 51/2251Tuesday March 4, 2014 © Dr. Alaa Khamis
Agent-2
Percept Action
Know-how
p s
◊ Task Allocation
◊ Target Cueing and Handoff
◊ Optimal Deployment
◊ Group Formation ◊ Cooperative Mapping
• Augmentative Cooperation: Task Allocation
Towards Cooperative Multirobot Systems
iR 21;
miTt j ,...,2,1 ;
RTA :
niRri ,...,2,1 ; R1
R2
R3
R4
Rn
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 52/2252Tuesday March 4, 2014 © Dr. Alaa Khamis
Alaa Khamis, Ahmed Elmogy and Fakhreddine Karray, "Complex Task Allocation in Mobile Surveillance Systems," Journal of Intelligent and Robotic Systems, Springer, DOI: 10.1007/s10846-010-9536-2, 2011
A set of m surveillance tasks: TA set of n mobile Sensors: R
3/17/2014
27
• Augmentative Cooperation: Target Cueing and Handoff
non-tracked interval
Hand-off Event(Time & Location)
Towards Cooperative Multirobot Systems
Platform 1Sensing Domain
Platform 2Sensing Domain
Initiated Search
RegionTracking Sector
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 53/2253Tuesday March 4, 2014 © Dr. Alaa Khamis
Inter-platform Handoff
Platform 1Originating Platform
Platform 2Recieving Platform
A. Benaskeur, A. Khamis, H. Irandoust, “Augmentative Cooperation in Distributed Surveillance Systems for Dense Regions,” International Journal of Intelligent Defence Support Systems, 4(1): 20-49, 2011
• Augmentative Cooperation: Optimal Deployment
Towards Cooperative Multirobot Systems
Area of Interest (AOI)Mobile Sensors
Optimal strategy to deploy a set of mobile sensors in AOI:
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 54/2254Tuesday March 4, 2014 © Dr. Alaa Khamis
Yun-Qian Miao, Alaa Khamis, Mohamed S. Kamel, "Novel Mobility Model for Mobile Sensors Deployment in Surveillance Systems", International Journal of Robotics and Automation, 2012
◊ Maximize the area coverage;
◊ Maximize target detection rate;
◊ Minimize detection time.
3/17/2014
28
• Augmentative Cooperation: Group Formation
Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 55/2255Tuesday March 4, 2014 © Dr. Alaa Khamis
Ahmed Shehata and Alaa Khamis, “Adaptive Group Formation in Multi-robot Systems,” Advances in Artificial Intelligence Journal, 2013.
• Augmentative Cooperation: Group Formation
Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 56/2256Tuesday March 4, 2014 © Dr. Alaa Khamis
Ahmed Shehata and Alaa Khamis, “Adaptive Group Formation in Multi-robot Systems,” accepted to be published in Advances in Artificial Intelligence Journal, 2013.
3/17/2014
29
• Augmentative Cooperation: Cooperative Mapping
Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 57/2257Tuesday March 4, 2014 © Dr. Alaa Khamis
Alaa Khamis and Asser ElGindy "Minefield Mapping using Cooperative Multirobot Systems", Journal of Robotics
Volume 2012, http://dx.doi.org/10.1155/2012/698046.
• Augmentative Cooperation: Cooperative Mapping
Towards Cooperative Multirobot Systems
Cluster-2
Acting Agent
Detected
Cluster-1
Cluster-3
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 58/2258Tuesday March 4, 2014 © Dr. Alaa Khamis
mine
Alaa Khamis and Asser ElGindy "Minefield Mapping using Cooperative Multirobot Systems", Journal of Robotics
Volume 2012, http://dx.doi.org/10.1155/2012/698046.
3/17/2014
30
• Augmentative Cooperation: Communication Relaying
Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 59/2259Tuesday March 4, 2014 © Dr. Alaa Khamis
NetLogo simulation environment with 60 UAVs, 5 ground targets and a base station
Mohamed Wakid and Alaa Khamis. Communication Relay for Unmanned Aerial Vehicles in Autonomous Search and Rescue Mission. Research Project, RAS Research Group, GUC.
Agent-1
Action-1Know-how-1
Agent have different
complementary know-
• Integrative Cooperation
Towards Cooperative Multirobot Systems
Task
PerceptAction-1
Agents’ contributions
Examples:
◊ Multimodal Fusion
how and it is necessary to
integrate their contribution
for achieving a task.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 60/2260Tuesday March 4, 2014 © Dr. Alaa Khamis
Agent-2
Know-how-2
PerceptAction-2
◊ Multimodal Fusion
◊ Target Detection and Tracking
◊ Human-assisted Tracking
◊ Search and Rescue
3/17/2014
31
• Integrative Cooperation: Human-assisted Tracking
Towards Cooperative Multirobot Systems
LRF Visible
Human Visible
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 61/2261Tuesday March 4, 2014 © Dr. Alaa Khamis
Bahador Khaleghi, Alaa Khamis, Fakhreddine Karray, "Random Finite Set Theoretic Soft/Hard Data Fusion: Application to Target Tracking", IEEE 2010 International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2010
LRF Visible
• Integrative Cooperation: Human-assisted Tracking
Data Fusion Agent (DFA)
J
PROTÉ
ACL
XML
XML
Towards Cooperative Multirobot Systems
JADE
GÉ
Hard Data Agent (HDA)
Soft Data Agent (SDA)
Acting Agent(AA)
JavaClient
Human Interaction Agent (HIA)
Human Observer
ACL
ACL
Domain Ontology
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 62/2262Tuesday March 4, 2014 © Dr. Alaa Khamis
Action Percepts
P/S SensorsActuators
Volume of Interest (VOI)
Observations
Bahador Khaleghi, Alaa Khamis, Fakhreddine Karray, “Random Finite Set Theoretic Soft/Hard Data Fusion: Application to Target Tracking”, IEEE 2010 International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2010.
3/17/2014
32
• Integrative Cooperation: Search and Rescue
Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 63/2263Tuesday March 4, 2014 © Dr. Alaa Khamis
Mohamed Badreldin, Ahmed Hussein and Alaa Khamis, “A Comparative Study between Optimization and Market-based Multi-robot Task Allocation Approaches,” accepted to be published in Advances in Artificial Intelligence Journal, 2013.
Algorithm SA-based GA-based Market-based
Optimal Allocation Found
S ll l
• Integrative Cooperation: Search and Rescue
Towards Cooperative Multirobot Systems
Small-scale
Medium-scale
Large-scale
Capabilities matching
Time matching
Heavily constrained
Computational Cost
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 64/2264Tuesday March 4, 2014 © Dr. Alaa Khamis
Mohamed Badreldin, Ahmed Hussein and Alaa Khamis, “A Comparative Study between Optimization and Market-based Multi-robot Task Allocation Approaches,” accepted to be published in Advances in Artificial Intelligence Journal, 2013.
Computational Cost
Static Allocation [Extended Time]
Dynamic Allocation [Limited Time]
3/17/2014
33
When agent have a
similar know-how and
• Debative Cooperation
Towards Cooperative Multirobot Systems
Examples:
are faced with a unique
task, and they compare
their results for obtaining
the best solution.
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 65/2265Tuesday March 4, 2014 © Dr. Alaa Khamis
p
◊ Uncertainty Reduction
◊ Multisensor Single Target Cooperative Tracking
• Debative Cooperation: Cooperative Tracking
Towards Cooperative Multirobot Systems
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 66/2266Tuesday March 4, 2014 © Dr. Alaa Khamis
3/17/2014
34
• Debative Cooperation: Cooperative Tracking
Towards Cooperative Multirobot Systems
Case Energy saving in EKM
n=m 20 40%
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 67/2267Tuesday March 4, 2014 © Dr. Alaa Khamis
n=m 20-40%
n=(3/4)m 12-31%
n=(1/2)m 8-23%
Ahmed Elmogy, Fakhreddine Karray, and Alaa Khamis, “Auction-based Consensus Mechanism for Cooperative Tracking in Multisensor Surveillance Systems,“’ Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.14, No.1 pp. 13-20, 2010
Towards Cooperative Multirobot Systems• Debative Cooperation: Uncertainty Reduction
Extending Spatial/ Temporal Coverage
Reducing Data Imperfection
Multisensor
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 68/2268Tuesday March 4, 2014 © Dr. Alaa Khamis
Enhancing Reliability of Measurements
Multisensor
Fusion and
Integration
3/17/2014
35
Towards Cooperative Multirobot Systems• Debative Cooperation: Uncertainty Reduction
2z
1z
2
nz
fx
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 69/2269Tuesday March 4, 2014 © Dr. Alaa Khamis
Waleed Abdulhafiz and Alaa Khamis, “Handling Data Uncertainty and Inconsistency using Multisensor Data Fusion,”Advances in Artificial Intelligence Journal, 2013.
Towards Cooperative Multirobot Systems
Fusion Node
Sensor-1
Sensor-22z
1z
• Debative Cooperation: Uncertainty Reduction
Sensor-n
.
.
.
Fused estimate
nz
2z
1z
nt m
easu
rem
ents
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 70/2270Tuesday March 4, 2014 © Dr. Alaa Khamis
3z
Unc
erta
in a
nd in
cons
iste
n
nz
Waleed A. Hafiz and Alaa Khamis, “Bayesian Approach to Multisensor Data Fusion with Pre- and Post-Filtering”, 10th IEEE International Conference on Networking Sensing and Control, 2013.
3/17/2014
36
Towards Cooperative Multirobot Systems• Debative Cooperation: Uncertainty Reduction
CPU Running Time
Residual sum of squares (RSS)
Variance (P)
Coefficient of Correlation
Criterion Function (CF):
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 71/2271Tuesday March 4, 2014 © Dr. Alaa Khamis
Waleed A. Hafiz and Alaa Khamis, “Bayesian Approach to Multisensor Data Fusion with Pre- and Post-Filtering”, 10th IEEE International Conference on Networking Sensing and Control, 2013.
Towards Cooperative Multirobot Systems• Debative Cooperation: Uncertainty Reduction
MUSES_SECRET: ORF-RE Project - © PAMI Research Group – University of Waterloo 72/2272Tuesday March 4, 2014 © Dr. Alaa Khamis
Waleed A. Hafiz and Alaa Khamis, “Bayesian Approach to Multisensor Data Fusion with Pre- and Post-Filtering”, 10th IEEE International Conference on Networking Sensing and Control, 2013.
Estimated Variance using Decentralized Fusion