GRASP University of Pennsylvania Adaptive Autonomous Robot TEAMS for Situational Awareness GRASP...
-
Upload
jason-robinson -
Category
Documents
-
view
219 -
download
2
Transcript of GRASP University of Pennsylvania Adaptive Autonomous Robot TEAMS for Situational Awareness GRASP...
GRASPUniversity of Pennsylvania
Adaptive Autonomous Robot TEAMS for Situational Awareness
GRASP Laboratory University of Pennsylvania
PI: Vijay Kumar Senior Personnel:
Camillo Jose Taylor, Jim Ostrowski
Research Associates James Keller, John Spletzer, Aveek Das, Guilherme Pereira, Luiz Chaimowicz, Jong-Woo Kim, Anthony Cowley
Co-PI: Ron Arkin Senior Personnel:
Tucker Balch, Robert Burridge
Research Associates Keith O’Hara, Patrick Ulam, Alan Wagner
Co-PI: Gaurav SukhatmeSenior Personnel:
Maja Mataric, Andrew Howard, Ashley Tews
Research Associates:Srikanth Saripalli, Boyoon Jung, Brian Gerkey, Helen Yan
Co-PI: Jason RediSenior Personnel: Josh Bers,
Keith Manning
Mobile Robotics Laboratory
Georgia Institute of Technology
Robotics Research Laboratory
University of Southern CaliforniaBBN Technologies
GRASPUniversity of Pennsylvania
Future Combat SystemsThe Future Combat System (FCS) concept revolves around the creation of a network-centric force of heterogeneous platforms that is strategically responsive, lethal, survivable and sustainable
communication in active mobile nodes during network-centric warfare;
integration of multiple, heterogeneous views of the target area
GRASPUniversity of Pennsylvania
Key FCS Considerations Adapt to variations in communication
performance and strive to maximize suitably defined network-centric measures for perception, control and communication
Provide situational awareness for remotely-located war fighters in a wide range of conditions
Integrate heterogeneous air-ground assets in support of continuous operations over varying terrain
GRASPUniversity of Pennsylvania
Context
Communication Network 400 MHz (100Kbs), 2.4 GHz (10Mbs), 38 GHz (100
Mbs) Affected by foliage, buildings, terrain features,
indoor/outdoor Directionality
Small Team of Heterogeneous Robots UGVs with vision, range finders UAVs (blimp, helicopter)
GRASPUniversity of Pennsylvania
GOALS A comprehensive model and framework integrating
communications, perception, and execution
Automated acquisition of perceptual information for situational awareness
Reactive group behaviors for a team of air and ground based robots that are communications sensitive
A new framework for mobile networking in which robots use sensory information and relative position information to adapt network topology to the constraints of the task.
GRASPUniversity of Pennsylvania
GRASP Laboratory University of Pennsylvania
Mobile Robotics Laboratory
Georgia Institute of Technology
Robotics Research Laboratory
University of Southern CaliforniaBBN Technologies
Control,Vision
Behaviors, Architecture
Comms, Networking
Sensing, Mapping
MARSTEAMS
GRASPUniversity of Pennsylvania
Thrusts1. Ad Hoc Networks for Control, Perception and
Communication2. Software framework for distributed computation,
sensing, control, and human-robot interface3. Communications-sensitive operations4. Network-centric approach to situational
awareness5. Mission-specific planning and control for a team
of heterogeneous robots6. Adaptation of behaviors and networks to
changing conditions
GRASPUniversity of Pennsylvania
Thrusts and Tasks Thrust Areas Penn GT USC BBN
1 Ad Hoc Networks for Control, Perception and Communication
UP1 GT4 BBN1
2
Software framework for distributed computation, sensing, control, and human-robot interface
UP5, UP7
GT1 USC6
3 Communications-sensitive operations
UP2, UP3, UP6
GT2 GT3
USC1 BBN2, BBN3
4 Network-centric approach to situational awareness UP5
USC2, USC3, USC7
5 Mission-specific planning and control for a team of heterogeneous robots
GT1, GT2, GT5
6 Adaptation of behaviors and networks to changing conditions
UP4 GT3 USC4, USC5
GRASPUniversity of Pennsylvania
1. Ad Hoc Networks for Control, Perception and Communication
Physical Network (R, ES )
Communication Network
(R, EC )
Computational Network
(R, H )
eij={i, j, bm, bv, dm, dv}qi= {m, v }
GRASPUniversity of Pennsylvania
Models of Communication
Modeling Effect of foliage Buildings Dependence on frequency, directionality Statistical models of delays and “hot spots” from
experimental data Neighbors, path costs (delays, power) Time of last communication
QoS metrics Control/perception tasks Individual robots vs. end-to-end Move to improve reliability and network performance
Interface between network and robot software
GRASPUniversity of Pennsylvania
Self-Awareness and Cooperative Localization (Penn)
Discovery – robots can organize themselves into a team
Localization – establish relative pose information
R1
R2
R3
R4
R5
R1
R2
R3
R4
R5
GRASPUniversity of Pennsylvania
Self-Awareness and Cooperative Localization
-5 -4 -3 -2 -1 0 1 2
-2
-1
0
1
2
3
Network of UGVs and Surrogate UAV Reactive controllers that maintain, exploit
network
GRASPUniversity of Pennsylvania
Cooperative Control (Penn)
Reactive controllers that maintain and exploit network
Controllers and estimators are represented by graphs
Fundamental connection between graph structure and performance (stability, convergence)
GRASPUniversity of Pennsylvania
2. Software framework for distributed computation, sensing, control, and human-robot interface
Player/Stage (USC) Robots Sensors
Sonar IR Scanning LRF,
cameras (color blob detection)
Integration
GRASPUniversity of Pennsylvania
2. Software framework for distributed computation, sensing, control, and human-robot interface (continued)
ROCI (Penn) Discover other
processes Communicate with other
processes Monitor other processes Control other processes
ROCI Processes
Performance Monitor
GPS, Range Sensors
Motion Control
Image Acquisition
KERNEL (IPC, Security, Networking,
Distributed Database.)
GRASPUniversity of Pennsylvania
3. Communications-sensitive behaviors and operations
Networking Models (BBN) Diagnostics (BBN)
Control of Mobility Behaviors (GT) Verification and Analysis
(Penn)
Operations (Thrust 5) Mission specification (GT) Mission Planning (GT)
GRASPUniversity of Pennsylvania
4. Network-centric approach to situational awareness
Cooperative Localization Vision (Penn) Range sensors, GPS, and IMU (USC) Unreliable communication
Acquisition of 3-D information (Penn)
Cooperative behaviors (USC, Penn)
Cooperative Mapping (USC)
Semantic Markup of Maps (USC)
GRASPUniversity of Pennsylvania
5. Mission-specific planning and control for a team of heterogeneous robots
FCS scenarios (BBN, GT)
MissionLab integration (GT)
GRASPUniversity of Pennsylvania
6. Adaptation of behaviors and networks to changing conditions
Adaptation of control modes (Penn)
Reinforcement learning to adapt mode switching (sequential composition of behaviors) (USC, Penn)
Path referenced perception and selection of behaviors (USC)
Variable autonomy (USC)
Operation under stealth (USC)
GRASPUniversity of Pennsylvania
Technology Integration Air Ground
Coordination
Command and Control Vehicle
Software Mission planning Control for
communications Active perception Infrastructure for
distributed computing
GRASPUniversity of Pennsylvania
Georgia Institute of Techology
GT Personnel Faculty
Prof. Ron Arkin Prof. Tucker Balch Dr. Robert Burridge
GRAs Keith O’Hara Patrick Ulam Alan Wagner
Mobile Intelligence Inc. Dr. Doug MacKenzie
GRASPUniversity of Pennsylvania
Impact - GT Provide communication-sensitive
planning and behavioral control algorithms in support of network-centric warfare, that employ valid communications models provided by BBN
Provide an integrated mission specification system (MissionLab) spanning heterogeneous teams of UAVs and UGVs
Demonstrate warfighter-oriented tools in three contexts: simulation, laboratory robots, and in the field
GRASPUniversity of Pennsylvania
Task 1: Communication-sensitive Mission Specification
MissionLab is a usability-tested Mission-specification software developed under extensive DARPA funding (RTPC / UGV Demo II / TMR / UGCV / MARS / FCS-C programs) Adapt to incorporate air-ground
communication-sensitive command and control mechanisms
Extend to support physical and simulated experiments for objective air and ground platforms
Incorporate new communication tasks and triggers
GRASPUniversity of Pennsylvania
Task 2: Communication Sensitive Planning
Add support for terrain models and other communications relevant topographic features to MissionLab
Use plans-as-resources as a basis for multiagent robotic communication control (spatial, behavioral, formations, etc.) and integrate within MissionLab
GRASPUniversity of Pennsylvania
Task 3: Communication-Sensitive Team Behaviors
Generation and testing of a new set of reactive communications preserving and recovery behaviors
Creation of behaviors sensitive to QoS
Expansion of Behaviors in support of line-of-sight and subterranean operations
GRASPUniversity of Pennsylvania
Task 4: Communication Models and Fidelity
Work with BBN to incorporate suitable communication models into MissionLab in support of both simulation and field tests
GRASPUniversity of Pennsylvania
Task 5: Technology Integration Conduct Early-on
Demonstrations on Ground Robots at Georgia Tech
Provide our Hummer Command and Control Vehicle for Team support at Objective Demonstration
Currently being used for FCS-C Program
Fully actuated – capable of teleautonomous control
GRASPUniversity of Pennsylvania
University of Southern California
Faculty: Prof. Gaurav Sukhatme Prof. Maja Mataric
Research Associates: Dr. Andrew Howard Dr. Ashley Tews
Graduate Students: Srikanth Saripalli,
Boyoon Jung, Brian Gerkey, Helen Yan
GRASPUniversity of Pennsylvania
USC Task SummaryOutdoor simulation
Cooperative outdoor localization
Semantic representations
Stealthy behaviors
Path-referenced perception
HRI Integration
GRASPUniversity of Pennsylvania
Task 1: Stage SimulationCurrent
Multi-robot 2D simulation, models differential and omni-drive robots, sonar, IR, scanning LRF, cameras (color blob detection), pan-tilt-zoom heads, and simple 2 DOF grippers
Language independent, architecture neutral
Extensions 3D simulation for outdoor terrain. Incorporate USC helicopter and
UPenn blimp
GRASPUniversity of Pennsylvania
Task 2: Cooperative Outdoor Localization Extend existing
localization algorithms to outdoor environments.
Implement outdoor localization in the presence of partial GPS.
Validate through outdoor experiments with small teams (4 ground robots).
GRASPUniversity of Pennsylvania
Task 3: Semantic Representation and Activity Recognition
Semantic mark-up of maps with following attributes: elevation, terrain type and traversability, foliage and
coverage type, and impact on communications.
Integrate activity/motion detection algorithms to locate people in the environment.
Demonstrate semantic markup using ground robots at USC.
GRASPUniversity of Pennsylvania
Task 4: Variable Autonomy and Stealth
Develop and implement behaviors for variable autonomy incorporating operator feedback using gestures
Develop and implement a new “stealthy patrolling” behavior by integrating visibility constraints into current patrolling algorithms
Adapt and tune above behaviors using reinforcement learning to improve performance
GRASPUniversity of Pennsylvania
Task 5: Path-referenced Perception and Behaviors
Develop path-referenced perception and behaviors, which allow recall of behavioral strategy relative to priors paths taken in the mission
Integrate path-referencing which allows robots to query each other for relative locations of semantic mark-ups
GRASPUniversity of Pennsylvania
Task 6: Human Robot Interface Extend Stage to serve as a simple visual display
for war fighter. Overlay visual information with laser information in Stage.
Provide simple auditory feedback to the operator about current behavioral state of robots.
GRASPUniversity of Pennsylvania
Technology Integration Demonstrations at USC of
cooperative localization (laser based with IMU and GPS) using ground robots and USC helicopter.
Demonstration at USC of activity detection, semantic markup of terrain and stealthy traverses.
Support joint demonstration with ground robots.
GRASPUniversity of Pennsylvania
University of PennsylvaniaFaculty
Vijay Kumar Camillo Jose Taylor Jim Ostrowski
Research Associates James Keller Luiz Chaimowicz
Students John Spletzer, Aveek Das Guilherme Pereira Jong-Woo Kim Vito Sabella
GRASPUniversity of Pennsylvania
Task 1: Model of Ad Hoc Network1. Develop a comprehensive model for control, perception
and communication for situational awareness
2. Integrate models of interference, bandwidth, latency and QoS of the communication network with models of control, sensing and communication.
Performance measure
Implications for mobility
(R, H )
(R, H )
GRASPUniversity of Pennsylvania
Task 2: Control Of Mobility
1. Design controllers and behaviors in support for communications, establishing or sustaining links
2. Design controllers and behaviors in support for situational awareness
3. Formal analysis of controllers and behaviors to predict team performance
GRASPUniversity of Pennsylvania
Task 3: Adaptation
1. Performance functions for the ad hoc network and adaptation using reinforcement learning
2. Reconfiguration of network to enable integration and fusion of sensory data in support of human interaction and situational awareness
GRASPUniversity of Pennsylvania
Task 4: Human Robot Team Interface
1. Synthesis and integration for perception enabling multiple views at different spatio-temporal resolution
2. Interface for human-robot interaction ROCI Macroscope
GRASPUniversity of Pennsylvania
Task 5: Performance Metrics: Verification and Validation
1. Metrics for control, communication, and perception technologies, and performance measures for system performance. Existing measures do not incorporate the dependence
of control, communication and perception
2. Designing and conducting experiments to measure performance
GRASPUniversity of Pennsylvania
Task 6: Technology Integration
1. Coordinated motion of four UGVs and one blimp optimizing end-to-end network performance
2. Team control, realization of situational awareness using ROCI.
GRASPUniversity of Pennsylvania
Summary of TasksPenn GRASP Integrated model for control,
perception and communication for situational awareness
Synthesis and integration for perception enabling multiple views at different spatio-temporal resolution
Georgia Tech MRL Communication-sensitive
planning and behavioral control algorithms in support of network-centric warfare
Integrated mission specification system (MissionLab) spanning heterogeneous teams of UAVs and UGVs
BBN Models of QoS and metrics of
performance for network-centric warfare
Interface design between network and robot modules
Formulation of FCS needs, capabilities, and design of demonstrations
USC RRL Cooperative outdoor localization
for small teams of robots Semantic mark-up of maps with
environmental attributes and recognition of activity.
Behaviors for path-referenced perception and for clandestine operations
GRASPUniversity of Pennsylvania
MARS TEAMS Impact New paradigm and novel algorithms for
network-centric operations
Mobile nodes that reconfigure to maintain and enhance connectivity
Air-Ground coordination will directly impact FCS capabilities