Decentralised Coordination of Continuously Valued Control Parameters using the

Max-Sum Algorithm 

Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick JenningsSchool of Electronics and Computer Science

University of Southampton{rs06r, af2, acr, nrj}@ecs.soton.ac.uk

2

This presentation focuses on the use of Max-Sum in coordination problems with continuous parameters

From Discrete to Continuous

Max-Sum for Decentralised Coordination

Empirical Evaluation

3

Max-Sum is a powerful algorithm for solving DCOPs

Complete Algorithms

DPOPOptAPOADOPT

Communication Cost

Iterative AlgorithmsBest Response (BR)

Distributed Stochastic Algorithm (DSA)

Fictitious Play (FP)

Max-SumAlgorithm

Optimality

Max-Sum solves the social welfare maximisation problem in a decentralised way

Agents

Max-Sum solves the social welfare maximisation problem in a decentralised way

1x

2x

3x

4x

5x

6x

7x8x

Control Parameters

Max-Sum solves the social welfare maximisation problem in a decentralised way

1U

2U

3U

4U

5U

6U

7U8U

Utility Functions

Max-Sum solves the social welfare maximisation problem in a decentralised way

)( 33 xU

Localised Interaction

},,,{ 54313 xxxxx

Max-Sum solves the social welfare maximisation problem in a decentralised way

Agents

M

iiiU

1

)(maxarg xx

Social welfare:

The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph

Variable nodes Function nodes

1x

2x

3x

1U

2U

3U

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by message passing

1x

2x

3x

1U

2U

3U

Variable nodes Function nodes

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by message passing

jiadjk

iikiji xrxq\)(

)()(

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

From variable i to function j

From function j to variable i

Until now, Max-Sum was only defined for discretely valued variables

Graph Colouring)( iji xq )( iij xr

However, many problems are inherently continuous. Heading

andVelocity

Unattended Ground Sensor

ActivationTime

Autonomous Ground Robot

Thermostat

Preferred RoomTemperature

So, we extended the Max-Sum algorithm to operate in continuous action spaces

Discrete Continuous

We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)

We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)

)( iji xq )( iij xr

“Continuous” Graph Colouring

A CPLF is defined by a domain partitioning followed by value assignment

A CPLF is defined by a domain partitioning followed by value assignment

A CPLF is defined by a domain partitioning followed by value assignment

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

To make Max-Sum work on CPLFs, we need to define key two operations on them

jiadjk

iikiji xrxq\)(

)()(From variable i to function j

From function j to variable i

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

To make Max-Sum work on CPLFs, we need to define key two operations on them

jiadjk

iikiji xrxq\)(

)()(From variable i to function j

From function j to variable i1. Addition

of two CPLFs

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

To make Max-Sum work on CPLFs, we need to define key two operations on them

jiadjk

iikiji xrxq\)(

)()(From variable i to function j

From function j to variable i

2. Marginal Maximisation to a single variable

Addition of two CPLFs involves merging their domains, and then summing their values

1x

2x

1x

2x

1f 2f

Addition of two CPLFs involves merging their domains, and then summing their values

1x

2x

1x

2x

1x

2x

1. Merge domains

1f 2f

21 ff

Addition of two CPLFs involves merging their domains, and then summing their values

Addition of two CPLFs involves merging their domains, and then summing their values

2. Sum Values

Marginal maximisation is the operation of finding the maximum value of a function, if we fix all but one variable

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

From function j to variable i:

Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

)(max)(\ jjii fxgj

xx

Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

)(max)(\ jii fxgj

xx

),(max)( 2112

xxfxgx

Example: bivariate function:

Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction

),( 1 fxProject onto

axis

Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction

Project onto

axis),( 1 fx

Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction

Project onto

axis),( 1 fx

Result of projection

Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction

Extract UpperEnvelope

Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction

Extract UpperEnvelope

),(max)( 2112

xxfxgx

)( 1xg

We empirically evaluated this algorithm in a wide-area surveillance scenario

Dense deployment of sensors to detect activity within an urban environment.

Unattended GroundSensor

Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors

time

duty cycleDiscrete

time

duty cycle

time

duty cycle

Discretised time

Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors

time

duty cycleDiscrete Continuous

time

duty cycle

time

duty cycle

time

duty cycle

time

duty cycle

time

duty cycle

Continuous Max-Sum outperforms Discrete Max-Sum by up to 10%

0 5 10 15 20 25 300.75

0.80

0.85

0.90

0.95

1.00

ContinuousDiscrete

Discretisation

Solu

tion

Qua

lity

(as f

racti

on o

f opti

mal

)

Average Solution Quality over 25 Iterations

Tota

l Mes

sage

Size

Continuous Max-Sum leads to more effective use of communication resources than Discrete Max-Sum

0 5 10 15 20 25 300

5000

10000

15000

20000

ContinuousDiscrete

Discretisation

Total number of values exchanged between agents

In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum

1. No artificial discretisation

time

time

In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum

1. No artificial discretisation Continuous

Discrete

2. Better solutions

time

timeSo

lutio

n Q

ualit

y

In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum

time

time1. No artificial discretisation Continuous

Discrete

2. Better solutions ContinuousDiscrete

3. Effective communication

Solu

tion

Qua

lity

Mes

sage

Size

For future work, we wish to extend the algorithm to arbitrary continuous functions

For example, using Gaussian Processes

In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum

time

time1. No artificial discretisation Continuous

Discrete

2. Better solutions ContinuousDiscrete

3. Effective communication

Solu

tion

Qua

lity

Mes

sage

Size

Questions?