Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University...
-
Upload
quinn-turney -
Category
Documents
-
view
218 -
download
3
Transcript of Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University...
![Page 1: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/1.jpg)
Expense constrained bidder optimization in repeated auctions
Ramki Gummadi Stanford University
(Based on joint work with P. Key and A. Proutiere)
![Page 2: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/2.jpg)
Overview
• Introduction/Motivation
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
![Page 3: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/3.jpg)
Three Aspects of Sponsored Search
1. Sequential setting.
2. Micro-transactions per auction.
3. The long tail of advertisers is expense constrained.
![Page 4: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/4.jpg)
Modeling Expense ConstraintsFixed budget over finite horizon => any balance at time is worthless.
Balance
timeT0
B
![Page 5: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/5.jpg)
Modeling Expense ConstraintsStochastic fluctuations could cause spend rate different from target.
Balance
timeT0
B
![Page 6: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/6.jpg)
Modeling Expense Constraints
“…the nature of what this budget limit means for the bidders themselves is somewhat of a mystery. There seems to be some risk control element to it, some purely administrative element to it, some bounded-rationality element to it, and more…”
-- “Theory research at google”, SIGACT News, 2008.
![Page 7: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/7.jpg)
Modeling Expense ConstraintsAdd a fixed income, per unit time to the balance and relax time horizon.
Balance
time0
B
![Page 8: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/8.jpg)
Responsibility for expense constraints Auctioneer Bidder
Bids fixed -- Auction entry throttled.
Bids adjusted dynamically.
Online bipartite matching between queries and bidders.
Online knapsack type problems.
Expense constraints = fixed budget.
Possible to model more general expense constraints.
![Page 9: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/9.jpg)
Bid optimization
![Page 10: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/10.jpg)
Preview
Sequential X-auction with true value v
Static X-auction with virtual value: shade* v
X can be SP, GSP, FP, etc. (any quasi linear utility)
Shade(remaining balance B) =
will be characterized explicitly.
1
1 '( )V B
![Page 11: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/11.jpg)
Preview: Optimal Shading factors
![Page 12: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/12.jpg)
Overview
• Introduction
• Budgeted Second Price auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
![Page 13: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/13.jpg)
Model: Budgeted Second Price
• Discrete time, indexed • Balance: • Constant income per time slot - • I.I.D. environment sampled from– Private valuation (observable) – Competing bid (not observable)
• Decision variable is bid at time – Can depend on and , but not
![Page 14: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/14.jpg)
Model: Budgeted Second Price
Constraint: a.s.
• Utility:
• Objective function:
![Page 15: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/15.jpg)
The Value Function
• : max utility starting with balance
• Can use dynamic programming (“one step look ahead”) to write out a functional fixed point relation.
![Page 16: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/16.jpg)
The Value Function
1 2,
( ) max E 1{ } 1{ }v pu b
v b u p T u p T
( )v p e v b a p
( )e v b a
But boundary conditions can not be inferred from the DP argument.
Currentauction
Loss
Win1T
2T
![Page 17: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/17.jpg)
Future opportunity cost
Characterization of value function
“Effective price” for nominal at balance :
Theorem: Optimal bid is *:i.e: Buy all auctions with “effective price” is a functional fixed point to:
( , )*u b v
,
( ) ( ) ( , )v p
v b e v b a v p b
( , ) ( ( ) ( ))p b p e v b a v b a p
![Page 18: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/18.jpg)
1
,( ) ( ) ( , )i i i
v pv b e v b a v p b
Value Iteration:
𝛽=0.1
Each auction has miniscule utility compared to overall utility:
![Page 19: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/19.jpg)
Value Iteration:
𝛽=0.01
1
,( ) ( ) ( , )i i i
v pv b e v b a v p b
![Page 20: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/20.jpg)
Numerical estimation when is small:• State space quantization errors propagate due
to lack of boundary value.• Need longer iterations over larger state space.
will be studied under scaling:
( ) ( ) ( / )V B v b v B
Limiting case: micro-value auctions
![Page 21: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/21.jpg)
Overview
• Introduction
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
![Page 22: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/22.jpg)
General Online Budgeting ModelDecision Maker Environment
, i.i.d
Unobservable
Observable
Balance:
Utility:
Action
Payment:Income
𝑀𝑎𝑥𝑖𝑚𝑖𝑧𝑒∑𝑡=0
∞
𝑒−𝛽 𝑡𝔼 [𝑔(𝑢 (𝑡 ) , 𝜉 (𝑡 )) ]
![Page 23: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/23.jpg)
Ex1: Second Price Auction
(Random environment) (Observable part) is the bid (Action) (Utility function)
(Payment function)
![Page 24: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/24.jpg)
Ex2: GSP Auction
Random environment: Observable part: is the bidUtility function:Payment function: 1
1
( , ) 1{ } ( )L
l l l ll
g u p u p v p
11
( , ) 1{ }L
l l l ll
c u p u p p
Click events for L slots
![Page 25: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/25.jpg)
Overview
• Introduction
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
![Page 26: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/26.jpg)
Limiting Regime:
( ) ( ) ( / )V B v b v B
Notation:
(( ]) [ , )E g ug u
(( ]) [ , )E c uc u
![Page 27: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/27.jpg)
is an inverseand
is the minimum of:
Theorem
*( ), (0) ,dV
f V VdB
( )x
is the solution to:
*
![Page 28: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/28.jpg)
*( ') , (0) ,V V V
0
( ) sup( ( ) ( ) )u
x ax g u c u x
F
* 0min ( )x x
𝑥
𝑉
𝐵
𝑉 (𝐵)
Theorem
( )x
*
![Page 29: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/29.jpg)
) =
Application to Second Price Auctions𝐸 [𝟏𝑢>𝑝 (𝑣−𝑝 )]
p]
![Page 30: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/30.jpg)
Second Price Auction ExampleOpponents bid p
Private Valuation
𝜙(𝑥 )
Value functions
![Page 31: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/31.jpg)
Optimal bid
0 ( )
( )
sup( ( ) ( ) '( )) sup 1{ }( (1 '( ))
sup 1{ }1 '( )
u u v
u v
g u c u V B uE p v p V B
vu p p
VE
B
F
i.e., Static SP with shaded valuation: 1 '( )
v
V B
* at balance B solves:
![Page 32: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/32.jpg)
Optimal Scaling factor
![Page 33: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/33.jpg)
Optimal Bid: GSP
0
1( ) 1
sup( ( ) ( ) '( ))
sup 1{ }1 '( )
u
L
l l l lu v l
g u c u V B
vp u p p
VE
B
F
Static GSP with “virtual valuation”: 1 '( )
v
V B
![Page 34: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/34.jpg)
Proof Overview
• Variant: Retire with payoff when .
• Value function of variant converges to ODE with initial value .
• But what is the right boundary condition ?To prove:
Because exit payoff optional Next 2 slides
![Page 35: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/35.jpg)
Goal: Exhibit a sequence of policies parametrized by which can achieve a scaled payoff as
Lemma: For any ε > 0, there is a policy * such that ε AND
If could be played continuously, we can get arbitrarily close to ! But every now and then balance is exhausted, so we need a variant of u* that still manages to achieve nearly as much payoff
𝜂∗≤ lim inf 𝑉 𝛽(0)
![Page 36: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/36.jpg)
time
B(t)
B
Play U*
𝜂∗≤ lim inf 𝑉 𝛽(0)
Show that fraction of time spent in green phase by the random walk gets arbitrarily close to 1 as ->0
![Page 37: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/37.jpg)
Overview
• Introduction
• MDP for budgeted SP auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
![Page 38: Expense constrained bidder optimization in repeated auctions Ramki Gummadi Stanford University (Based on joint work with P. Key and A. Proutiere)](https://reader035.fdocuments.us/reader035/viewer/2022062712/56649c8f5503460f949493b8/html5/thumbnails/38.jpg)
Conclusion
• A two parameter model for expense constraints in online budgeting problems.
• Optimal bid can be mapped to static auction with a shaded virtual valuation.
• Paper has more contents: MFE analysis and a finite horizon model.