April 22, 2023

Robust Distributed Task Allocation for Autonomous Multi-Agent Teams

Ph.D. Candidate: Sameera Ponda

Thesis Committee: Prof. Jonathan P. How,

Prof. Mary L. Cummings,Prof. Devavrat Shah

Motivation Modern missions involve networked heterogeneous multi-agent teams

cooperating to perform tasks Unmanned aerial vehicles (UAVs) – target tracking, surveillance Human operators – classify targets, monitor status Ground vehicles – rescue operations

Key Research Questions: How to coordinate team behavior to improve mission performance? How to hedge against uncertainty in dynamic environments? How to handle varying communication constraints?

04/22/2023 2

Problem Statement Objective: Automate task allocation to improve mission performance

Spatial and temporal coordination of team Computational efficiency for real-time implementation

Key Technical Challenges: Combinatorial decision problem (NP-hard) – computationally intractable Complex agent modeling (stochastic, nonlinear, time-varying) Constraints due to limited resources (fuel, payload, bandwidth, etc) Dynamic networks and communication requirements Robustness to uncertain and dynamic environments04/22/2023 3

25

4010

25

30

25

Problem Statement: Maximize mission score Satisfy constraints Decision variables:

Team assignments, Service times

Planning Approaches Optimal solution methods are computationally intractable for large problems

Typically use efficient approximation methods [Bertsimas ’05]

Most involve centralized planning [Bertsimas ’05] Base station plans & distributes tasks to all agents Requires full situational awareness High bandwidth, slow reaction to local changes

Motivates distributed planning [Sariel ‘05, Lemaire ‘04] Agents make plans individually & coordinate

with each other through consensus algorithms [Olfati-Saber ‘07] Faster reaction to local information Increased agent autonomy

Key questions for distributed planning: What quantities should the agents agree upon?

Information / tasks & plans / objectives / constraints How to ensure that planning is robust to inaccurate information and models?

04/22/2023 4

25

4010

25

30

25

25

4010

25

30

25

25

4010

25

30

25

Distributed Planning

Main issues: Coupling & Communication Agent score functions depend on other agents’ decisions Joint constraints between multiple agents Agent optimization is based on local information

Key challenge: How to design appropriate consensus protocols? [Johnson ‘10] Specify what information to communicate Create rules to process received information and modify plans Performance guarantees – is distributed problem good representation of centralized? Convergence guarantees – will algorithm converge to a feasible assignment?

04/22/2023 5

Centralized Problem: Maximize mission score Satisfy constraints Decision variables:

Team assignments, Service times

Distributed Problem: Maximize mission score individually Satisfy constraints Decision variables:

Agent assignments, Service times

Distributed Planning – CBBA Consensus-Based Bundle Algorithm (CBBA) [Choi, Brunet, How ‘09]

Iterations between 2 phases: Bidding & Consensus

Core features of CBBA: Sequential greedy task selection – Polynomial-time, provably good approximate solutions Guaranteed real-time convergence even with inconsistent environment knowledge

04/22/2023 6

Phase 2: Consensus

(all agents)

All agents consistent? Yes

No

Phase 1: Build Bundle & Bid on Tasks (individual agents)

3

2

N

1

Key Contributions – extensions to CBBA framework:1) Time-varying score functions (e.g. time-windows of validity for tasks)2) Guaranteeing connectivity in limited communication environments3) Robust planning for uncertain environments

CBBA with Time-Windows In realistic continuous-time missions, have time-varying task scores

04/22/2023 7

Time-window

Arrival Time

Sco

re e.g. monitor status,

security shifts Arrival Time

Sco

re

Time-criticale.g. rescue ops, target tracking

Arrival Time

Sco

re

Peak-time e.g. rendezvous, special ops

Extended CBBA to continuous-time domains [ACC 2010] Task optimization involves decisions on task

assignments and task service times Preserves convergence properties

Embedded the algorithm into dynamic planning architecture Real-time simulation framework for dynamic missions Experimental flight tests for UAV/UGV teams Demonstrates real-time feasibility

Cooperative Distributed Planning Often have fleet-wide hard constraints on assignments

Agent assignments coupled through joint team constraints

Example: Maintaining network connectivity in dynamic environments Often have limited communication radius, line-of-sight requirements As agents move around environment – dynamic networks, potential disconnects

Several issues: Some tasks rely on continuous connectivity (e.g. streaming live video) Cannot perform consensus, cannot deconflict plans How to include network connectivity constraints into distributed planner?

04/22/2023 8

Disconnected Network

25

40

10

25

30

25

Example: Baseline Scenario Motivating example – Surveillance Mission around base station

UAVs travel to tasks and stream live video back to base station Successful task execution relies on continuous connectivity Limited comm radius (RCOMM)

04/22/2023 9

10

15

30

No connectivity!

No connectivity!

0

010

Example: Network Prediction Conservative solution – predict network connectivity violations

Drop tasks if disconnects will occur Only execute tasks in local vicinity – conservative

04/22/2023 10

10

15

3010

Example: Planning with Relays Can use some agents as communication relays!

Coordinated team behavior leads to higher mission performance Goal: Develop cooperative planning algorithms to coordinate team

04/22/2023 11

10

15

3010

Relay

Relay

30

CBBA with Relays CBBA with Relays

[JSAC 2012, Globecom 2011, Infotech 2011, Globecom 2010] Generate CBBA assignments Predict network over mission duration Repair connectivity by creating relay tasks

Key features: Explicit consideration of dependency constraints Predict network topology only at select mission-

critical times – avoids discretizing time Leverages information available in CBBA

consensus phase Preserves polynomial-time and convergence

guarantees

CBBA with Relays improves performance Agents accomplish higher value tasks Guaranteed network connectivity Demonstrated real-time applicability

04/22/2023

Real-time experiment

iRobot Create

Field experiment

Pelican quad

Key questions: How to propagate uncertainty through planner to generate agent assignments? How to distribute planning given additional complexity due to uncertainty? How to ensure real-time performance and computational tractability?

Distributed Planning Under Uncertainty Uncertainty in planning process

Inaccurate models (simplified dynamics, parameter errors) Fundamentally non-deterministic processes (e.g. sensor

readings, stochastic dynamics) Dynamic local information changes Can hedge against uncertainty to improve planning

Robust planning involves several challenges Optimal solutions computationally intractable –

increased dimensionality of planning problem Non-trivial coupling of distributions – analytically

intractable Current approaches involve many limiting assumptions

04/22/2023 13

Distribution for Operator Target Identification

Figure from [D. Southern, Masters Thesis, 2010]

Target Identification Mission

Time

Task

s

Agent ScheduleLate!

Chance-Constrained CBBA – Extended CBBA to incorporate risk into planning process [ACC 2012] Model coupling using numerical approx (sampling) Preserves polynomial-time Probabilistic performance guarantees for given risk

Key features: Improved CBBA to handle non-submodular score

functions (e.g. stochastic scores) [CDC 2012] Approximate distributed agent risk given mission risk

using Central Limit Theorem assumption

Improved performance under uncertainty Higher scores within allowable risk Distributed approximation on par with centralized

Current work is exploring dynamic aspects Dynamic risk allocation Model learning using Nonparametric Bayesian techniques

[GNC 2012]04/22/2023 14

Distributed Planning Under Uncertainty

Conclusion Distributed task allocation strategies for autonomous multi-agent teams

Extended CBBA algorithm to include time-varying score functions Addressed cooperative planning in comm-limited environments using relay tasks Presented robust risk-aware distributed extensions to deterministic planning

Acknowledgments: Prof. Jonathan How for his invaluable advice and support My committee members Prof. Cummings and Prof. Shah My collaborators and colleagues at ACL, esp. Luke Johnson and Andrew Kopeikin Aero/Astro faculty and staff Graduate Aero/Astro friends!

04/22/2023 15