Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu...
-
Upload
beverly-hoover -
Category
Documents
-
view
214 -
download
0
Transcript of Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu...
![Page 1: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/1.jpg)
Searching for Extremes Among Distributed Data Sources with Optimal ProbingZhenyu (Victor) Liu
Computer Science Department, UCLA
![Page 2: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/2.jpg)
Why Extremes?Central Server
Sensor 1 Sensor 2 Sensor n
query: highest raindrop
Sensor i (the highest one), plus its value
Identifying severe weatherconditions (flood / drought)
Central Server
link 1 link 2 link n
query: slowest link
link i (the slowest one), plus its transferring speed
a network path from L.A. to N.Y.
Identifying the networkbottleneck
Central Server
Amazon Barns & Noble CampusI.com
query: best Web site for “Computer Algorithms”
Website i (the best one), plus the matching Web pages
Identifying the best Web database for a user’s query
![Page 3: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/3.jpg)
What Is the Challenge?
Constant communication between sensors and the central server is too expensive
Can the central server contact only a few sensors (i.e. use probing) to find out the maximum?
Central Server
Sensor 1 Sensor 2 Sensor n
query: highest raindrop
Sensor i (the highest one), plus its value
![Page 4: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/4.jpg)
A Motivating Example
Central Server
Sensor 1
Sensor 2
Sensor n
expensivecommunicationcost
Sensor 2
the possible value range of Sensor 1
actual value ofSensor 1 (unknown)
( )Sensor n
Sensor 1 ( )
( )
a) The central server without the latest sensor updates
Central ServerSensor 1
Sensor 2
Sensor n
Sensor 2
( )Sensor n
Sensor 1
( )
1000
probe
1000
b) Probing sensors’ reading to reduce uncertainty
![Page 5: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/5.jpg)
Data Model
The reading of each source as a random variable, X1, …, Xn
[li, ui] as Xi’s value range Bounded model: li, ui as real numbers Unbounded model: [-, ui], [li, +], [-, +]
Given Xi’s probability distribution in [li, ui] fi(x), Fi(x)
X1, …, Xn independent Probing Xi results in xi, costs ci
uniform-cost model, c1=c2= … = 1 non-uniform-cost model
![Page 6: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/6.jpg)
U(<X2, 880>) = 0.12, cost: probing 1U(<X2, 880>) = 0, cost: probing 2
Uncertainty in The Answer
Two variables X1 and X2, uniform distribution
0
f1(x)
880
0.12
1000 X1
X2
f2(x)
600
f1(x)
900800
f2(x)
![Page 7: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/7.jpg)
Uncertainty / Probing Cost Tradeoff
Uncertainty in the answer
0
Less probing,high uncertainty
More probinglow uncertainty
Probing cost
Tradeoff point
The user-specified uncertainty threshold
![Page 8: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/8.jpg)
The Problem
Given the uncertainty data model, design a probing policy
P: X1PX2
P…XnP
that incurs the least probing cost finds the maximum variable with an uncertainty
lower than Brute force searching takes n!
![Page 9: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/9.jpg)
Optimal Probing under Zero-Uncertainty = 0, i.e. return an absolutely correct answer Two policies
P1: X1X2
P2: X2X1
0
f1(x)X1
X2900800
f2(x)
1000
f1(x)
f2(x)
![Page 10: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/10.jpg)
Optimal Probing under Zero-Uncertainty Theorem 1: X1, …, Xn are ranked in a
descending order of their upper bounds, i.e., u1 > … > un,
P: X1X2…Xn
is optimal in the zero-uncertainty case The upper bound ui as a “representative
point” for Xi
![Page 11: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/11.jpg)
Optimal Probing under Non-Zero-Uncertainty = 0.15 Two policies
P1: X1X2, saves the 2nd probing if X1>885
P2: X2X1, saves the 2nd probing if X2>850
0
f1(x)X1
X2900800
f2(x)
1000
885
850
![Page 12: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/12.jpg)
Critical Point
Critical point,i [li, ui] s.t.P(Xi > i ) =
Lemma 1: With two variables X1 and X2, the optimal policy always probes the one with the larger critical point
0
f1(x)X1
X2900800
f2(x)
1000
885
850
x8501
8852
F1(x)
F2(x)
1
0.85(1-)
![Page 13: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/13.jpg)
Deriving The Optimal Policy from The Critical Points? Theorem 2: The optimal policy should always
place Xi before Xj if:Cond1: i > j
Cond2: x >j, Fi(x) < Fj(x)
x
1-1
Fi(x)Fj(x)
j i
![Page 14: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/14.jpg)
Applying Theorem 2 to Derive The Optimal Policy
x
1-1
2 1n
F1(x)F2(x)Fn(x)
Case 1:
Optimal policy:P: X1X2…Xn
![Page 15: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/15.jpg)
Applying Theorem 2 to Derive The Optimal Policy Case 2: Possible candidate
policies {X1,X2,X3} {X4,X5}
and X1 must be before X2
X1X2X3X4X5
X1X2X3X5X4X1
X3X2X4X5
X1X3X2X5X4
X3X1X2X4X5
X3X1X2X5X4
x
1-
1
F3(x)
F4(x)
F5(x)
F1(x)F2(x)
![Page 16: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/16.jpg)
Experimental Set-up
166 rainfall sensors across Washington State
Recording the rainfall at each sensor location, on every day over the past 46 years
![Page 17: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/17.jpg)
Probability Distribution
From the historical data, generate one distribution per sensor per day
Distinguish two kinds of historical data: Yesterday was dry Yesterday was rainy
![Page 18: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/18.jpg)
Preliminary Results
Complexity of optimal-policy searching
![Page 19: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/19.jpg)
Future Experimental Study
The behavior of the optimal policy on the rainfall sensor data Uncertainty threshold vs. number of sensor probing
The behavior of the optimal policy on synthetic datasets Reduction in the search space vs. number of sensor probing
![Page 20: Searching for Extremes Among Distributed Data Sources with Optimal Probing Zhenyu (Victor) Liu vicliu@cs.ucla.edu Computer Science Department, UCLA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649ebc5503460f94bc59ab/html5/thumbnails/20.jpg)
Summary
Under the proposed data model, find the maximum variable with uncertainty less than
Optimal probing policy = 0, sort variables according to their upper
bounds > 0, derive probing preferences (Xi before Xj)
and reduce the search space