Jos e M Vidaljmvidal.cse.sc.edu/talks/negotiation-slides.pdf · 2010. 3. 3. · Zeuthen Strategy...

Negotiation

Jose M Vidal

Department of Computer Science and Engineering University of South Carolina.

March 3, 2010

Abstract

We describe automated negotiation as it applies to multiagentsystems. Chapter 6.

Negotiation

Introduction

1 Introduction

2 The Bargaining ProblemAxiomatic Solution ConceptsStrategic Solution Concepts

3 Monotonic Concession ProtocolZeuthen StrategyOne Step Protocol

4 Negotiation as Distributed Search

5 Ad-hoc Negotiation Strategies

6 Task Allocation ProblemPaymentsLying About TasksContracts

7 Complex DealsAnnealing Over Complex Deals

8 Argumentation-Based Negotiation

9 Negotiation NetworksNetwork Exchange Theory

Negotiation

Introduction

Why Negotiate?

Coordinate selfish interests.

Aggregate distributed conflicting knowledge.

Solve characteristic form games and more complex versions.

For example: NASA missions, capitol hill?

Negotiation

Introduction

Why Negotiate?

Coordinate selfish interests.

Aggregate distributed conflicting knowledge.

Solve characteristic form games and more complex versions.

For example: NASA missions, capitol hill?

Negotiation

The Bargaining Problem

1 Introduction

Negotiation

Bargaining Problem

ui : ∆→ < where ∆ is the set of deals.

δ− is the no-deal deal.

Assume that for all agents ui (δ−) = 0

Negotiation

Axiomatic Solution Concepts

1 Introduction

Negotiation

Pareto

Definition (Pareto optimal)

A deal δ is Pareto optimal if there is no other deal such thateveryone prefers it over δ. That is, there is no δ′ such that

∀iui (δ′) > ui (δ).

Negotiation

Pareto Frontier

ui (δ)

uj(δ)

Negotiation

What do we want?

We will want a Pareto deal, but which one?

Idea: Come up with some requirements first then see if asolution that meets those requirements exists.

Negotiation

What do we want?

We will want a Pareto deal, but which one?

Idea: Come up with some requirements first then see if asolution that meets those requirements exists.

Negotiation

Independence from Units

Definition (Independence of utility units)

A negotiation protocol is independent of utility units if when givenU it chooses δ and when given U ′ = {(β1u1, . . . , βI uI ) : u ∈ U} itchooses δ′ where

∀i ui (δ′) = βiui (δ).

Negotiation

Symmetry

Definition (Symmetry)

A negotiation protocol is symmetric if the solution remains thesame as long as the set of utility functions U is the same,regardless of which agent has which utility.

Negotiation

Individual Rationality

Definition (Individual rationality)

A deal δ is individually rational if

∀i ui (δ) ≥ ui (δ−).

Which means that ui (δ) ≥ 0 since we will be assuming thatui (δ

−) = 0. A deal is individually rational if all the agents prefer itover not reaching an agreement.

Negotiation

Independence from Irrelevant Alternatives

Definition (Independece of irrelavant alternatives)

A negotiation protocol is independent of irrelevant alternatives if itis true that when given ∆ it chooses δ and when given ∆′ ⊂ ∆where δ ∈ ∆′ it again chooses δ, assuming U stays constant.

That is, a protocol is independent of irrelevant alternative is thedeal it chooses does not change after we remove a deal that lost.Only removal of the winning deal changes the deal the protocolchooses.

Negotiation

Egalitarian

δ = arg maxδ′∈E

ui (δ′)

where E is the set of all deals where all agents receive the sameutility, namely

E = {δ | ∀i ,jui (δ) = uj(δ)}.

Negotiation

Egalitarian Social Welfare

Find the closest approximation:

δ = arg maxδ

ui (δ)

Negotiation

Egalitarian

ui (δ)

uj(δ)

Egalitarian deal

Egalitarian social welfare deal

Negotiation

Utilitarian Solution

δ = arg max∑

ui (δ).

Negotiation

Utilitarian Solution

ui (δ)

uj(δ)

Utilitarian deal

Negotiation

Nash Bargaining Solution

δ = arg maxδ′

∏ui (δ

Negotiation

Nash Bargaining Solution

ui (δ)

uj(δ)

Nash bargaining deal

Negotiation

Nice Nash

Nash bargaining solution is the only one that satisfies:

1 Pareto efficient

2 Independent of utility units

3 Independent of irrelevant alternatives

4 Symmetric

Negotiation

Kalai-Smorodinsky

Let u∗i be the maximum utility that i could get from the set ofall deals in the Pareto frontier.

Then, find the deal that lies in the line between the point δ−

and the point (u∗i , u∗j )

Negotiation

Kalai-Smorodinsky

ui (δ)

uj(δ)

u∗i , u∗ju∗j

Kalai-Smorodinsky deal

Negotiation

Strategic Solution Concepts

1 Introduction

Negotiation

Strategic Solutions

1 Formalize the bargaining process

2 Assume rational agents

3 Determine their equilibrium strategies for their bargainingprocess.

Negotiation

Rubinstein’s Alternating Offers

1 Two agents i and j

2 At each time step t one agent proposes a deal δ

3 The other can either accept or reject δ

4 Utilities decrease over time ui = λti ui (δ)

Negotiation

Theorem (Alternating Offers Bargaining Strategy)

The Rubinstein’s alternating offers game where the agents havecomplimentary linear utilities (ui (δ) = δ and uj(δ) = 1− ui (δ)) hasa unique subgame perfect equilibrium strategy where

agent i proposes a deal

δ∗i =1− λj

1− λiλj

and accepts the offer δj from j only if ui (δj) ≤ ui (δ∗j ),

agent j proposes a deal

δ∗j =1− λi

1− λiλj

and accepts the offer δi from i only if uj(δi ) ≤ uj(δ∗i ).

Negotiation

Alternating Offers Strategy

The theorem tells us that the best strategy for these agents ispropose a bid on the first time step which will be accepted by theother agent.

Negotiation

Monotonic Concession Protocol

1 Introduction

Negotiation

monotonic-concession

1 δi ← arg maxδ ui (δ)2 Propose δi3 Receive δj proposal4 if ui (δj) ≥ ui (δi )5 then Accept δj6 else δi ← δ′i such that uj(δ

′i ) ≥ ε+ uj(δi ) and ui (δ

′i ) ≥ ui (δ

−)7 goto 2

Negotiation

Monotonic Concession

Utility

ui (δ)

uj(δ)

ε δ2i

ε δ3i

εδ2j

εδ3j

εδ4j

Negotiation

Monotonic Concession Summary

Agents know others’ utility functions

Tricky last step: both might want other’s offer

Negotiation

Zeuthen Strategy

1 Introduction

Negotiation

Zeuthen Strategy

1 Propose my best deal.

2 Let willingness to risk conflict for i be the utility i loses byaccepting j ’s offer divided by the utility i loses by notconceding and causing conflict. That is:

riski =ui (δi )− ui (δj)

ui (δi )

3 If riski < riskj then I must concede just enough so that in thenext round I do not have to concede again.

Negotiation

Zeuthen Strategy

zeuthen-monotonic-concession

1 δi ← arg maxδ ui (δ)2 Propose δi3 Receive δj proposal4 if ui (δj) ≥ ui (δi )5 then Accept δj

6 riski ←ui (δi )−ui (δj )

ui (δi )

7 riskj ←uj (δj )−uj (δi )

uj (δj )

8 if riski < riskj

9 then δi ← δ′i such that riski (δ′i ) > riskj

10 goto 211 goto 3

Negotiation

Zeuthen Strategy

ui (δ)

uj(δ)

δi = 0 δj = 6

ui (δ) = 5− δ,uj(δ) = 2

3δδ = {0 . . . 6}δi = 0, δj = 6

riski = 5−(−1)5 = 6

5 ,riskj = 4−0

Negotiation

Zeuthen Strategy

ui (δ)

uj(δ)

δi = 0

δj = 4.9

ui (δ) = 5− δ,uj(δ) = 2

3δδ = {0 . . . 6}δi = 0, δj = 6

riski = 5−(−1)5 = 6

5 ,riskj = 4−0

4 = 1 jmust concede, morethan 1.δj < 5

Negotiation

Zeuthen Strategy

Zeuthen Characteristics

It is not guaranteed to maximize social welfare.

It is guaranteed to terminate, and any agreement it reacheswill be individually rational and Pareto optimal.

It is also in Nash equilibrium–if the other guy is using it thenyou have nothing to gain by not using it. Allows agents topublish their strategy.

But, sometimes risks are equal.

Requires agents to know eachother’s utility functions.

Negotiation

One Step Protocol

1 Introduction

Negotiation

One Step Protocol

one-step-negotiation

1 E ← {δ | ∀δ′ui (δ)uj(δ) ≥ ui (δ′)uj(δ

′)}2 δi ← arg maxδ∈E ui (δ)3 Propose δi4 Receive δj5 if ui (δj)uj(δj) < ui (δi )uj(δi )6 then Report error, j is not following strategy.7 Coordinate with j to choose randomly between δi and δj .

Negotiation

One Step Protocol

Algorithm is in Nash equilibrium.

Negotiation

Negotiation as Distributed Search

1 Introduction

Negotiation

Negotiation as Distributed Search

Hill Climbing

ui (δ)

uj(δ)

Deals that Pareto dominate δ0

Negotiation

Ad-hoc Negotiation Strategies

1 Introduction

Negotiation

Ad-hoc Negotiation Strategies

A linear discounts utility linearly.

A conceder concedes a lot initially.

An impatient demands a lot initially.

Negotiation

Task Allocation Problem

1 Introduction

Negotiation

The task allocation problem consists of:

T : tasksA: agentsci : s → < cost that i incurs in carrying out tasks s ⊆ T .δ represents allocation of tasks to agents.δ− is initial allocation

The cost function is monotonic.

The cost of doing nothing is 0.

Negotiation

δ si (δ) sj(δ) ci (δ) cj(δ) ui (δ) uj(δ)

δ1 ∅ {t1, t2, t3} 0 8 8 0δ2 {t1} {t2, t3} 1 4 7 4δ3 {t2} {t1, t3} 2 5 6 3δ4 {t3} {t1, t2} 4 7 4 1δ5 {t2, t3} {t1} 6 4 2 4δ6 {t1, t3} {t2} 5 3 3 5δ7 {t1, t2} {t3} 3 1 5 7δ8 {t1, t2, t3} ∅ 7 0 1 8

Negotiation

ui (δ)

uj(δ)

Negotiation

Payments

1 Introduction

Negotiation

Payments

1 Enable more deals by allowing payments.

2 This was the idea behind the original contract net protocol(Smith and Davis, 1981).

Negotiation

Payments

1 Enable more deals by allowing payments.

2 This was the idea behind the original contract net protocol(Smith and Davis, 1981).

Negotiation

Payments

Contract Net Protocol

manager

contractor

Negotiation

Payments

manager

contractor

task announcement

Eligibilityspecification.

Task abstraction.

Bid specification.

Expiration time.

Negotiation

Payments

manager

contractor

Negotiation

Payments

manager

contractor

Negotiation

Payments

manager

contractor

contract

Negotiation

Payments

Payments Create Deals

ui (δ)

uj(δ)

New dominant deals

Negotiation

Payments

Payments Create Deals

ui (δ)

uj(δ)

New dominant deals

Negotiation

Payments

Additive Cost Functions

More formally,

Definition

A function c(s) is an additive cost function if for all s ⊆ T it istrue that

c(s) =∑t∈s

They are easier to analyze.

Negotiation

Payments

Additive + Payments

Theorem

In a task allocation problem with an additive cost function wherewe only allow exchange of one task at a time, any protocol thatallows payments and always moves to dominant deals willeventually converge to the utilitarian solution .

Negotiation

Payments

ui (δ)

uj(δ)

Negotiation

Payments

Arbitrary Cost Functions

In general, not much we can say.

If any deal can be reached from any other deal (fullyconnected) then hill climbing will again reach the utilitariansolution.

Negotiation

Payments

Arbitrary Cost Functions

In general, not much we can say.

If any deal can be reached from any other deal (fullyconnected) then hill climbing will again reach the utilitariansolution.

Negotiation

Lying About Tasks

1 Introduction

Negotiation

Lying About Tasks

Possible Lies

Not tell others about some tasks I have.

Make up tasks and hope I end up having to do them.

Make up tasks and create them if needed.

Assume known final deal. For example, Nash bargaining solution.

Negotiation

Lying About Tasks

Task Creation Example

δ si (δ) sj(δ) ui (δ) uj(δ)

δ1 ∅ {t1} 1 3δ2 {t1} ∅ 2 1

Create phony t2.

δ1 ∅ {t1, t2} 1 5δ2 {t1} {t2} 2 3δ3 {t2} {t1} 2 3δ4 {t1, t2} ∅ 8 1

Negotiation

Lying About Tasks

δ1 ∅ {t1} 1 3δ2 {t1} ∅ 2 1

Create phony t2.

δ1 ∅ {t1, t2} 1 5δ2 {t1} {t2} 2 3δ3 {t2} {t1} 2 3δ4 {t1, t2} ∅ 8 1

Negotiation

Lying About Tasks

δ1 ∅ {t1} 1 3δ2 {t1} ∅ 2 1

Create phony t2.

δ1 ∅ {t1, t2} 1 5δ2 {t1} {t2} 2 3δ3 {t2} {t1} 2 3δ4 {t1, t2} ∅ 8 1

Negotiation

Contracts

1 Introduction

Negotiation

Contracts

Agents might want to de-commit on a contract.

Negotiation

Contracts

ui (δ)

uj(δ)

δ1 : i does task and j and pays nothing.

δ3 : i does nothing, j pays $2.

δ4 : i does task and j pays penalty of $1.

δ5 : i idle, pays $1 penalty, j pays $2.

δ2 : i does task and j and pays $2.

δ0: j does task and i is idle.

Negotiation

Contracts

ui (δ)

uj(δ)

Negotiation

Contracts

ui (δ)

uj(δ)

Negotiation

Contracts

ui (δ)

uj(δ)

Negotiation

Contracts

Contract Penalties

Penalties reduce risks.

But, if we can enforce penalties, why not just enforce originalcontracts?

Negotiation

Contracts

Contract Penalties

Penalties reduce risks.

But, if we can enforce penalties, why not just enforce originalcontracts?

Negotiation

Complex Deals

1 Introduction

Negotiation

Complex Deals

A multi-dimensional deal is composed of a set of variablesx1, x2, . . . , xn with domains D1,D2, . . .Dn.

ui (δ)

Or, ui (δ) = c1u1i (x1) + c2u2

i (x2) + · · ·+ cnuni (xn)

Yes, this is a constraint optimization problem! But nowagents do not own the variables.

Negotiation

Complex Deals

A multi-dimensional deal is composed of a set of variablesx1, x2, . . . , xn with domains D1,D2, . . .Dn.

ui (δ)

Or, ui (δ) = c1u1i (x1) + c2u2

i (x2) + · · ·+ cnuni (xn)

Yes, this is a constraint optimization problem! But nowagents do not own the variables.

Negotiation

Complex Deals

Convergence

ui (δ)uj(δ)

δ3i ,j

Pareto domi-nate δ3

Negotiation

Complex Deals

Annealing Over Complex Deals

1 Introduction

Negotiation

Complex Deals

Negotiation with Mediator

annealing-mediator

1 Generate random deal δ.2 δaccepted ← δ

3 Present δ to agents.4 if both accept5 then δaccepted ← δ

6 δ ← mutate(δ)7 goto 38 if one or more reject9 then δ ← mutate(δaccepted)

10 goto 3

Negotiation

Complex Deals

Hill Climbers and Annealers

Hill Climber Accepts a deal only if it has utility higher than itsreservation price ui (δ

−) and higher than that of thelast deal it accepted. That is, it monotonicallyincreases it reservation price as it accepts deals withhigher utility.

Annealer Use a simulated annealing algorithm. That is, theymaintain a temperature T and accept deals worsethan the last accepted deal with probability

max(1, e−∆UT ), where ∆U is the utility change

between the contracts.

Negotiation

Complex Deals

Hill-Climbers and Annealers

Ui (δ) Uj(δ)

Negotiation

Complex Deals

Ui (δ) Uj(δ)

Negotiation

Complex Deals

Ui (δ) Uj(δ)

Hill Climber

Negotiation

Complex Deals

Annealermax(1, e−

∆UT )

Ui (δ) Uj(δ)

Hill Climber

Negotiation

Complex Deals

Annealermax(1, e−

∆UT )

Ui (δ) Uj(δ)

Hill Climber

Negotiation

Complex Deals

Prisoner’s Dilemma, again!

Hill Climber Annealer

Hill Climber .73, .74 .99, .51

Annealer .51, .99 .84, .84

Negotiation

Complex Deals

Adding Tit-for-Tat

Hill Climber Annealer T4T

Hill Climber 400, 400 700, 180 500, 340

Annealer 180, 700 550, 550 550, 550

T4T 340, 500 550, 550 550, 550

Negotiation

Argumentation-Based Negotiation

1 Introduction

Negotiation

Argument-Based Negotiations

Critique

Counter-proposal

Justify

Persuade

There are also threats, rewards, and appeals.

Negotiation

Critique the proposal.

A: I propose that you provide me with service Xunder conditions P.

B: I am happy with the price of X but the deliverydate is too late.

A: I propose that I will provide you with service Yif you provide me with X .

B: I don’t want Y .

Counter-proposal

Justify

Persuade

Negotiation

Critique

Counter-proposal

A: I propose that you provide me with service X .B: I propose that I provide you with service X if

you provide me with service Z .A: I propose that I provide you with service Y if

you provide me with service X .B: I propose that I provide you with service X if

you provide me with service Z .

Justify

Persuade

Negotiation

Critique

Counter-proposal

Justify his reason for adopting a particular negotiation stance.

A: I don’t have the software for delivering service X .

Persuade

Negotiation

Critique

Counter-proposal

Justify

Persuade the other agent to change its negotiation stance.

A: Service X is much better than you think, look atthis report.

Negotiation

Critique

Counter-proposal

Justify

Persuade

These techniques help

build model of opponent’s utility function,

eliminate whole sets of deals,

change the other agent’s utility function,

change my utility function.

Negotiation

Negotiation Networks

1 Introduction

Negotiation

Definition

A negotiation network problem involves a set of agents A and setof sets of deals. Each set of deals ∆i involves only a subset ofagents ∆a

i ⊆ A and always includes the no-deal deal δ−. A solution~δ to the problem is a set of deals, one from each ∆i set, such thatall the deals that each agent is involved in are compatible witheach other. We thus define

ci (δ, δ′) =

{1 if δ and δ′ are compatible0 otherwise

Negotiation

Negotiation Network

∆1 ∆2

Negotiation

Network Exchange Theory

1 Introduction

Negotiation

−10−1

1 −1

The coercion network.

Negotiation

Equi-Resistance

Negotiation

Equi-Resistance

i ’s resistance to payment p is given by

ri =pmaxi − pi

pi − pconi

wherepmaxi = Maximum i could get, 10

andpconi = Conflict deal, 0

Negotiation

Equi-Resistance

NET tells us that exchange happens at equi-resistance:

ri =pmaxi − pi

pi − pconi

=pmaxj − pj

pj − pconj

= rj .

We can represent this graphically by simply replacing pj with10− pi in j ’s resistance equation rj and plotting the two curves riand rj . The point at which the curves cross is the point ofexchange.

Negotiation

Equi-Resistance

ri (p) rj(p)

Negotiation

Iterated Equi-Resistance

i j k10 10

Negotiation

i j k10 10

1 Apply Equi-resistance to i j10

2 Apply Equi-resistance to j k10

3 Repeat until quiescence.

Negotiation

i j k10 10

. Gives us pj = 5.

Negotiation

i j k10 10

. Gives us pj = 5.

. Let pconj = 5 and apply

equi-resistance again.

Negotiation

NET Limitations

Only tested on small networks.

Multiple equilibriums.

Might never settle down.

Still, viable descriptive solution.

Jos e M Vidaljmvidal.cse.sc.edu/talks/negotiation-slides.pdf · 2010. 3. 3. · Zeuthen Strategy...

Documents

Transcript of Jos e M Vidaljmvidal.cse.sc.edu/talks/negotiation-slides.pdf · 2010. 3. 3. · Zeuthen Strategy...

Technical Seminar Zeuthen, 6.10€¦ · Technical Seminar Zeuthen, 6.10.2009. 6. Okt. techn. Seminar 2 Outline of the talk Schematic view of the mail flow in Zeuthen What will when

Calibration of NT200+ with the external Laser Antipin Konstantin MEPhI (Moscow) Zeuthen, 19.09.2006.

March 11, 2008 Techn. Seminar Zeuthen · Instrumentation of the forward region 1 March 11, 2008 Techn. Seminar Zeuthen The CMS Beam and Radiation Monitoring Wolfgang Lohmann, DESY-monitoring

CTA Consortium meeting – DESY, Berlin/ Zeuthen ; May 2010

High Energy Neutrino Astronomy Christian Spiering DESY Zeuthen TAUP 2001.

Polarisation at Linear Colliders Achim Stahl Zeuthen 15.Oct.03.

Optimisation Studies for the BeamCal Design Lucia Bortko , DESY – Zeuthen

Zeuthen to become scientific centre of CTA · economic impact on the region. A new building will be erected on the Zeuthen campus. This building will host more than 20 new staff members

Householder Symposium XVII Book of Abstracts · Householder Symposium XVII June 1-6, 2008, Zeuthen, Germany The Householder Symposium on Numerical Linear Algebra in Zeuthen, Germany,

PROJECT : IceCube Digital Optical Module Production at DESY Zeuthen

1 Th.Naumann, DESY Zeuthen, HERMES Tracking meeting, 27.03.2003 Tracking with the Silicon Detectors Th.Naumann H1 DESY Zeuthen A short collection of experiences.

Stefan Schlenstedt DESY Zeuthen

29-Jun-09H. Henschel DESY (Zeuthen) 1 Frontend Interface Summarize.

Zeuthen, 23.04.2013 Computing Seminar Volker Gülzow, Kars Ohrenberg Networking@desy.

B . Golob, University of Ljubljana Physics in Collision 2003, Zeuthen

SILICON DETECTOR PROJECTS @ DESY Zeuthen (up to now) By Wolfgang Lange, DESY Zeuthen

Variable annuity GLBs(guaranteed living benefits) GLWBs(guaranteed lifetime withdraw benefits) GMWBs(guaranteed minimal withdraw benefits) GMIBs(guaranteed.

Pratik Majumdar DESY, Zeuthen (for the MAGIC Collaboration)

W.S. Graves DESY-Zeuthen 8/20031 Study for an xray laser at MIT Bates Laboratory William S. Graves MIT-Bates Presented at ICFA S2E workshop DESY-Zeuthen.

10.09.02 DESY Zeuthen 1 Precision drift chambers (MDT) for ATLAS muon system. G.Chelkov JINR (Dubna) DESY Zeuthen 10 September 2002.