Adaptive Optimization of Solution Time Adaptive Optimization of Solution Time In A Distributed Multi-agent SystemIn A Distributed Multi-agent System

Amy Fedyk, Gary Kratkiewicz, Jeff Berliner, Mark Amy Fedyk, Gary Kratkiewicz, Jeff Berliner, Mark Davis, Beth DePass, Rich Lazarus, Rusty BobrowDavis, Beth DePass, Rich Lazarus, Rusty Bobrow

KIMAS, April 18, 2005

Afedyk@bbn.com

2

OutlineOutline

• Optimization goalOptimization goal

• UltraLog OverviewUltraLog Overview

• Prior ArtPrior Art

• Solution Time Optimization ChallengesSolution Time Optimization Challenges

• Techniques for Optimizing Solution TimeTechniques for Optimizing Solution Time

• ConclusionConclusion

3

Optimization GoalOptimization Goal

• Improve time to solution in a large-scale Improve time to solution in a large-scale

logistics planning applicationlogistics planning application

– Have a solution available at all times Have a solution available at all times

– Eliminate unnecessary re-workEliminate unnecessary re-work

– Minimize effects of perturbations within society Minimize effects of perturbations within society

– Continue to perform during system stresses and Continue to perform during system stresses and

communications losscommunications loss

4

UltraLog: A Large Agent SocietyUltraLog: A Large Agent Society

• UltraLog UltraLog – DARPA-funded effort to explore building logistics DARPA-funded effort to explore building logistics

systems with a distributed multi-agent architecturesystems with a distributed multi-agent architecture – The test society models The test society models demanddemand from military from military

organizations supported by a organizations supported by a logistics supply logistics supply chainchain

• Each agent models a single military organization with its Each agent models a single military organization with its physical assets, business rules, and relationships to physical assets, business rules, and relationships to other organizations other organizations

• Contains over 1000 medium weight agents distributed Contains over 1000 medium weight agents distributed across nearly 100 computers across nearly 100 computers

– Built with CougaarBuilt with Cougaar• Open source, distributed-agent architectureOpen source, distributed-agent architecture

5

Solution Time Optimization ChallengesSolution Time Optimization Challenges

• Large-scale military logistics planning application• Small changes can affect many agents within the society.Small changes can affect many agents within the society.• Supporting agents do not know when all their requests have been Supporting agents do not know when all their requests have been

received.received.

1-AD

16-CSG(1-AD)123-

MSB-POL

102-POL-SUPPLYCO

OSD

USAEUR

USEUCOM

5-CORPS

5-CORPSREAR

7-CSG(5-CORPS)

240-SSCO

3-SUPCOM-HQ

DLAHQ OSCTRANSCOM

21-TSC-HQ

110-POL-SUPPLYCO

HNS

5-CORPSARTY

FORSCOM

1-ADOrgs

16-CSGOrgs

7-CSGOrgs

5-CORPSREAROrgs

5-CORPSARTYOrgs

21-TSCOrgs

26-SSCO

900-POL-SUPPLYCO

574-SSCO

3-SUPCOMOrgs

Fuel SupplyRequests

FuelSupply Chain

6

Prior ArtPrior Art

• Adaptive systemsAdaptive systems– Gracefully degrading systemsGracefully degrading systems– Survivable systemsSurvivable systems– Self-healing systemsSelf-healing systems– Speculative computationSpeculative computation

• Effects of communication on performance Effects of communication on performance – Trade-off cost of communication and value of Trade-off cost of communication and value of

informationinformation

• Building on prior artBuilding on prior art– ““Self-pacing” systemSelf-pacing” system– Graceful degradation via speculative computationGraceful degradation via speculative computation– Improve performance by limiting information flow Improve performance by limiting information flow

in a purposeful mannerin a purposeful manner

7

Techniques For Optimizing Solution TimeTechniques For Optimizing Solution Time

1.1. Multi-Resolutional solutionsMulti-Resolutional solutions– Continuous up-to-date planContinuous up-to-date plan– Adapts to system stresses Adapts to system stresses

2.2. Control upward/downward information flowControl upward/downward information flow– Propagate change based on local consistencyPropagate change based on local consistency

3.3. Transmission of differences onlyTransmission of differences only– Each agent minimizes effects of changes by Each agent minimizes effects of changes by

transmitting only the differences between the transmitting only the differences between the previously seen and new planpreviously seen and new plan

4.4. Use predictorsUse predictors– Proxies for temporarily unavailable componentsProxies for temporarily unavailable components

8

1. Multi-Resolutional Solutions1. Multi-Resolutional Solutions

• Society generates two plans simultaneouslySociety generates two plans simultaneously

• Low-resolution solutionLow-resolution solution– Rough estimate planRough estimate plan– Produced quicklyProduced quickly– Preferred over no solutionPreferred over no solution

• High-resolution solutionHigh-resolution solution– Detailed high fidelity planDetailed high fidelity plan– Becomes available more slowlyBecomes available more slowly– Gradually replaces low-resolution solutionGradually replaces low-resolution solution– Allows the plan to evolve over timeAllows the plan to evolve over time

9

Replace Low for High-Resolution Replace Low for High-Resolution

The high-resolution solution gradually The high-resolution solution gradually replaces the low-resolution solutionreplaces the low-resolution solution

Initial SolutionLow Low Low Low Low Low Low Low Low Low

Ultimate SolutionHigh High High High High High High High High High

Near-Term Tasks Long-Term Tasks

Tim

e e

lap

sed

wh

ile p

lan

nin

g

Still Better SolutionLow Low LowHigh High High High High High Low

Better SolutionHigh Low Low Low Low Low LowHigh LowLow

10

2. Controlling Upward/Downward2. Controlling Upward/DownwardInformation FlowInformation Flow

IncomingTask

OutgoingTask

IncomingTask

Response

OutgoingTask

Response

Response toIncoming

Task

IncomingTask

LocalAgent

CustomerAgent

CustomerAgent

ProviderAgent

ProviderAgent

Response toOutgoing

Task

OutgoingTask

UpwardFlow

DownwardFlow

(1)

(2)

(3)

(4)

Information Flow in the Supply Chain

1. Local Agent receives incoming tasks from customers

2. Local Agent sends outgoing messages to providers.

3. Local Agent receives responses back from providers

4. Local Agent then sends responses back to its customers

11

2. Controlling Upward/Downward2. Controlling Upward/DownwardInformation FlowInformation Flow

IncomingTask

OutgoingTask

IncomingTask

Response

OutgoingTask

Response

Response toIncoming

Task

IncomingTask

LocalAgent

CustomerAgent

CustomerAgent

ProviderAgent

ProviderAgent

Response toOutgoing

Task

OutgoingTask

UpwardFlow

DownwardFlow

(1)

(2)

(3)

(4)

• Reduce solution time by Reduce solution time by managing re-workmanaging re-work– Local agents refrain from Local agents refrain from

sending messages if local re-sending messages if local re-work is likelywork is likely

• Incoming tasks have Incoming tasks have changedchanged

• Greatly improved stability Greatly improved stability and performance.and performance.– Test societies of 1092 agents Test societies of 1092 agents

show solution times which show solution times which are always under 12 minutes are always under 12 minutes on baseline runs.on baseline runs.

12

3. Transmit Differences Only3. Transmit Differences Only

• Minimize the affects of perturbations.Minimize the affects of perturbations.• Each agent evaluates the messages it has previously Each agent evaluates the messages it has previously

sent to its providers before sending the re-computed sent to its providers before sending the re-computed plan.plan.

• Transmission-of-differences technique reduced number Transmission-of-differences technique reduced number of unnecessary perturbations in society by an average of of unnecessary perturbations in society by an average of 26.0%.26.0%.

Transmit only the one changed task

Transmit only two changed tasks and responses

13

4. Predictors4. Predictors

• Predictors are agent proxies which provide approximations based on Predictors are agent proxies which provide approximations based on the best available data.the best available data.

• The predictors allow agents to continue planning during comms lossThe predictors allow agents to continue planning during comms loss– Customer Predictors (CP) estimate incoming customer requests.Customer Predictors (CP) estimate incoming customer requests.– Supplier Predictor (SP) estimates answers a supplier would give in Supplier Predictor (SP) estimates answers a supplier would give in

response to customer requests.response to customer requests.• Under loss of comms, agents with predictors were about 3x faster than Under loss of comms, agents with predictors were about 3x faster than

agents without predictors.agents without predictors.

CustomerAgent

SupplierAgent

CustomerAgent

CustomerAgent

SP

SP

CP

Customers estimate a supplier’s response

Supplier estimates a customer’s requests

14

ConclusionConclusion

• Multi-Resolutional solutions provide a Multi-Resolutional solutions provide a continuously available and continuously continuously available and continuously improving planimproving plan

• Controlling Upward/Downward information flow Controlling Upward/Downward information flow prevents unnecessary re-work.prevents unnecessary re-work.

• Exclusive transmission of differences minimizes Exclusive transmission of differences minimizes effects of perturbations.effects of perturbations.

• Predictors allow computation to proceed during Predictors allow computation to proceed during comms loss.comms loss.

15

For more information …For more information …

• BBN Technologies:BBN Technologies:– http://www.bbn.comhttp://www.bbn.com

• Cougaar Agent Architecture:Cougaar Agent Architecture:– http://www.cougaar.orghttp://www.cougaar.org

• Other Cougaar-related KIMAS’05 papers:Other Cougaar-related KIMAS’05 papers:– ““Watching Your Own Back: Self Managing Multi-Agent SystemsWatching Your Own Back: Self Managing Multi-Agent Systems””, M. , M.

Thome, T. Wright, et alThome, T. Wright, et al

– ““Using QoS-Adaptive Coordination Artifacts to Increase Scalability of Communication in Distributed Multi-Agent Systems”, J. Zinky, S. , J. Zinky, S. Siracuse, et alSiracuse, et al

– “A Reconfigurable Multiagent Society for Transportation Scheduling and Dynamic Rescheduling”, D. Montana, G.Vidaver, et al

– “Scalability Aspects of Agent-based Naming Services”, T. Wright and K. Kleinmann