Adauctions

Expense constrained bidder optimization in repeated auctions

Ramki Gummadi Stanford University

(Based on joint work with P. Key and A. Proutiere)

Overview

• Introduction/Motivation

• Budgeted Second Price Auctions

• A General Online Budgeting Framework

• Optimal Bids for Micro-Value Auctions

• Conclusion

Three Aspects of Sponsored Search

1. Sequential setting.

2. Micro-transactions per auction.

3. The long tail of advertisers is expense constrained.

Modeling Expense ConstraintsFixed budget over finite horizon => any balance at time is worthless.

Balance

timeT0

B

Modeling Expense ConstraintsStochastic fluctuations could cause spend rate different from target.

Balance

timeT0

B

Modeling Expense Constraints

“…the nature of what this budget limit means for the bidders themselves is somewhat of a mystery. There seems to be some risk control element to it, some purely administrative element to it, some bounded-rationality element to it, and more…”

-- “Theory research at google”, SIGACT News, 2008.

Modeling Expense ConstraintsAdd a fixed income, per unit time to the balance and relax time horizon.

Balance

time0

B

Responsibility for expense constraints Auctioneer Bidder

Bids fixed -- Auction entry throttled.

Bids adjusted dynamically.

Online bipartite matching between queries and bidders.

Online knapsack type problems.

Expense constraints = fixed budget.

Possible to model more general expense constraints.

Bid optimization

Preview

Sequential X-auction with true value v

Static X-auction with virtual value: shade* v

X can be SP, GSP, FP, etc. (any quasi linear utility)

Shade(remaining balance B) =

will be characterized explicitly.

1

1 '( )V B

Preview: Optimal Shading factors

Overview

• Introduction

• Budgeted Second Price auctions



• Conclusion

Model: Budgeted Second Price

• Discrete time, indexed • Balance: • Constant income per time slot - • I.I.D. environment sampled from– Private valuation (observable) – Competing bid (not observable)

• Decision variable is bid at time – Can depend on and , but not

Model: Budgeted Second Price

Constraint: a.s.

• Utility:

• Objective function:

The Value Function

• : max utility starting with balance

• Can use dynamic programming (“one step look ahead”) to write out a functional fixed point relation.

The Value Function

1 2,

( ) max E 1{ } 1{ }v pu b

v b u p T u p T

( )v p e v b a p

( )e v b a

But boundary conditions can not be inferred from the DP argument.

Currentauction

Loss

Win1T

2T

Future opportunity cost

Characterization of value function

“Effective price” for nominal at balance :

Theorem: Optimal bid is *:i.e: Buy all auctions with “effective price” is a functional fixed point to:

( , )*u b v

,

( ) ( ) ( , )v p

v b e v b a v p b

( , ) ( ( ) ( ))p b p e v b a v b a p

1

,( ) ( ) ( , )i i i

v pv b e v b a v p b

Value Iteration:

𝛽=0.1

Each auction has miniscule utility compared to overall utility:

Value Iteration:

𝛽=0.01

1

,( ) ( ) ( , )i i i

v pv b e v b a v p b

Numerical estimation when is small:• State space quantization errors propagate due

to lack of boundary value.• Need longer iterations over larger state space.

will be studied under scaling:

( ) ( ) ( / )V B v b v B

Limiting case: micro-value auctions

Overview

• Introduction




• Conclusion

General Online Budgeting ModelDecision Maker Environment

, i.i.d

Unobservable

Observable

Balance:

Utility:

Action

Payment:Income

𝑀𝑎𝑥𝑖𝑚𝑖𝑧𝑒∑𝑡=0

∞

𝑒−𝛽 𝑡𝔼 [𝑔(𝑢 (𝑡 ) , 𝜉 (𝑡 )) ]

Ex1: Second Price Auction

(Random environment) (Observable part) is the bid (Action) (Utility function)

(Payment function)

Ex2: GSP Auction

Random environment: Observable part: is the bidUtility function:Payment function: 1

1

( , ) 1{ } ( )L

l l l ll

g u p u p v p

11

( , ) 1{ }L

l l l ll

c u p u p p

Click events for L slots

Overview

• Introduction




• Conclusion

Limiting Regime:

( ) ( ) ( / )V B v b v B

Notation:

(( ]) [ , )E g ug u

(( ]) [ , )E c uc u

is an inverseand

is the minimum of:

Theorem

*( ), (0) ,dV

f V VdB

( )x

is the solution to:

*

*( ') , (0) ,V V V

0

( ) sup( ( ) ( ) )u

x ax g u c u x

F

* 0min ( )x x

𝑥

𝑉

𝐵

𝑉 (𝐵)

Theorem

( )x

*

) =

Application to Second Price Auctions𝐸 [𝟏𝑢>𝑝 (𝑣−𝑝 )]

p]

Second Price Auction ExampleOpponents bid p

Private Valuation

𝜙(𝑥 )

Value functions

Optimal bid

0 ( )

( )

sup( ( ) ( ) '( )) sup 1{ }( (1 '( ))

sup 1{ }1 '( )

u u v

u v

g u c u V B uE p v p V B

vu p p

VE

B

F

i.e., Static SP with shaded valuation: 1 '( )

v

V B

* at balance B solves:

Optimal Scaling factor

Optimal Bid: GSP

0

1( ) 1

sup( ( ) ( ) '( ))

sup 1{ }1 '( )

u

L

l l l lu v l

g u c u V B

vp u p p

VE

B

F

Static GSP with “virtual valuation”: 1 '( )

v

V B

Proof Overview

• Variant: Retire with payoff when .

• Value function of variant converges to ODE with initial value .

• But what is the right boundary condition ?To prove:

Because exit payoff optional Next 2 slides

Goal: Exhibit a sequence of policies parametrized by which can achieve a scaled payoff as

Lemma: For any ε > 0, there is a policy * such that ε AND

If could be played continuously, we can get arbitrarily close to ! But every now and then balance is exhausted, so we need a variant of u* that still manages to achieve nearly as much payoff

𝜂∗≤ lim inf 𝑉 𝛽(0)

time

B(t)

B

Play U*

𝜂∗≤ lim inf 𝑉 𝛽(0)

Show that fraction of time spent in green phase by the random walk gets arbitrarily close to 1 as ->0

Overview

• Introduction

• MDP for budgeted SP auctions



• Conclusion

Conclusion

• A two parameter model for expense constraints in online budgeting problems.

• Optimal bid can be mapped to static auction with a shaded virtual valuation.

• Paper has more contents: MFE analysis and a finite horizon model.

Adauctions

Education

Transcript of Adauctions