Randomized Algorithms CS648
description
Transcript of Randomized Algorithms CS648
![Page 1: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/1.jpg)
Randomized AlgorithmsCS648
Lecture 17Miscellaneous applications of Backward analysis
1
![Page 2: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/2.jpg)
MINIMUM SPANNING TREE
2
![Page 3: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/3.jpg)
Minimum spanning tree
3
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6
![Page 4: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/4.jpg)
Minimum spanning tree
Algorithms: • Prim’s algorithm• Kruskal’s algorithm• Boruvka’s algorithm
4
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6
Less known but it is the first algorithm for MST
![Page 5: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/5.jpg)
Minimum spanning tree
: undirected graph with weights on edges, .
Deterministic algorithms:Prim’s algorithm 1. O(( + ) log ) using Binary heap2. O( + log ) using Fibonacci heapBest deterministic algorithm: O( + ) bound
– Too complicated to design and analyze– Fails to beat Prim’s algorithm using Binary heap
5
![Page 6: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/6.jpg)
Minimum spanning tree
When finding an efficient solution of a problem appears hard, one should strive to design an efficient verification algorithm.
MST verification algorithm: [King, 1990]Given a graph and a spanning tree , it takes O( + ) time todetermine if is MST of .
Interestingly, no deterministic algorithm for MST could use this algorithm to achieve O( + ) time.
6
![Page 7: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/7.jpg)
Minimum spanning tree
: undirected graph with weights on edges, .
Randomized algorithm:Karger-Klein-Tarjan’s algorithm [1995]1. Las Vegas algorithm2. O( + ) expected timeThis algorithm uses • Random sampling• MST verification algorithm• Boruvka’s algorithm• Elementary data structure
7
![Page 8: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/8.jpg)
Minimum spanning tree
: undirected graph with weights on edges, .
Randomized algorithm:Karger-Klein-Tarjan’s algorithm [1994]1. Las Vegas algorithm2. O( + ) expected time
• Random sampling :How close is MST of a random sample of edges to MST of original graph ?
The notion of closeness is formalized in the following slide.
8
![Page 9: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/9.jpg)
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?Answer: ??
9
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
¿𝒏𝒌 (𝒎−𝒌)
![Page 10: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/10.jpg)
USING BACKWARD ANALYSIS FORMISCELLANEOUS APPLICATIONS
10
![Page 11: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/11.jpg)
PROBLEM 1SMALLEST ENCLOSING CIRCLE
11
![Page 12: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/12.jpg)
Smallest Enclosing Circle
12
![Page 13: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/13.jpg)
Smallest Enclosing Circle
Question: Suppose we sample points randomly uniformly from a set of points, what is the expected number of points that remain outside the smallest circle enclosing the sample?
13
For = , the answer is
![Page 14: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/14.jpg)
PROBLEM 2SMALLEST LENGTH INTERVAL
14
![Page 15: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/15.jpg)
0 1
Sampling points from a unit interval
Question: Suppose we select points from interval [0,1], what is expected length of the smallest sub-interval ?• for , it is ?? • for , it is ??
• General solution : ??
This bound can be derived using two methods.• One method is based on establishing a relationship between uniform
distribution and exponential distribution.• Second method (for nearly same asymptotic bound) using Backward
analysis.
𝟏
(𝒌+𝟏 )𝟐
![Page 16: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/16.jpg)
PROBLEM 3MINIMUM SPANNING TREE
16
![Page 17: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/17.jpg)
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?
17
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
![Page 18: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/18.jpg)
USING BACKWARD ANALYSIS FORTHE 3 PROBLEMS :
A GENERAL FRAMEWORK
18
![Page 19: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/19.jpg)
A General framework
Let be the desired random variable in any of these problems/random experiment.
• Step 1: Define an event related to the random variable .
• Step 2: Calculate probability of event using standard method based on definition. (This establishes a relationship between )
• Step 3: Express the underlying random experiment as a Randomized incremental construction and calculate the probability of the event using Backward analysis.
• Step 4: Equate the expressions from Steps 1 and 2 to calculate E[].
19
![Page 20: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/20.jpg)
PROBLEM 3MINIMUM SPANNING TREE
20
![Page 21: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/21.jpg)
A BETTER UNDERSTANDING OF LIGHT EDGES
21
![Page 22: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/22.jpg)
Minimum spanning tree
22
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
13
4 2
19
6
x
b
ac
d
h
y
u
v
181
19
22
10 15
11
5
16
4
31
31
Random sampling
(𝑽 ,𝑬 )
(𝑽 ,𝑹)
![Page 23: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/23.jpg)
Minimum spanning tree
23
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
13
4 2
19
6
x
b
ac
d
h
y
u
v
1
19
10 15
11
5
16
4
31
18
2231
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)
![Page 24: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/24.jpg)
Minimum spanning tree
24
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
1
19
10 15
11
5
16
4
7
342
3
17
23
2
19
6
31
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)𝑬 ¿
![Page 25: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/25.jpg)
Minimum spanning tree
25
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
7
1
19
10
342
3
15
11
5
16
17
23
4 2
19
6
31
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)𝑬 ¿
Light
![Page 26: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/26.jpg)
First useful insight
Lemma1: An edge is light with respect to if and only if belongs to MST().
26
![Page 27: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/27.jpg)
Minimum spanning tree
27
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
7
1
19
10
342
3
15
11
5
16
17
23
4 2
19
6
31
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)𝑬 ¿
Light heavy
![Page 28: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/28.jpg)
Minimum spanning tree
28
b
ac
d
h x
y
u
v
7
1
3 3
54 2
6
b
ac
d
h x
y
u
v
7
1
19
10
3 3
15
11
5
16
17
4 2
42
23
19
6
MST()
(𝑽 ,𝑬 )
(𝑽 ,𝑹)𝑬 ¿
Light heavy
MST()
Is there any relationship among MST(), MST()
and Light edges from ?
![Page 29: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/29.jpg)
Second useful insight
Lemma2: Let • and • be the set of all edges from that are light with respect to . Then
MST() = MST()
This lemma is used in the design of randomized algorithm for MST as follows (just a sketch):• Compute MST of a sample of edges (recursively). Let it be T’.• There will be expected edges light edges among all unsampled edges.• Recursively compute MST of T’ edges which are less than on expectation.
29
![Page 30: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/30.jpg)
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?
30
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
We shall answer the above question using the Generic framework. But before that, we need to get a better understanding of the
corresponding random variable.
![Page 31: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/31.jpg)
31
𝒌
𝑹
𝑬 ¿MST()
![Page 32: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/32.jpg)
32
𝒌
𝑹
𝑬 ¿MST()
Light
![Page 33: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/33.jpg)
33
𝒌
𝑹
𝑬 ¿Light heavy
MST()
![Page 34: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/34.jpg)
: random variable for the number of light edges in when is a random sample of edges.
: set of all subsets of of size . : number of light edges in when . = ??
34
𝟏¿𝑺∨¿ ∑
𝒂∈𝑺𝒇 (𝒂 ) ¿
Can you express in terms of and only ?
![Page 35: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/35.jpg)
Step 1
Question: Let be a uniformly random sample of edges from .What is the prob. that an edge selected randomly from is a light edge ?
35
Two methods to find
![Page 36: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/36.jpg)
Step 2Calculating using definition
36
![Page 37: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/37.jpg)
Step 2 Calculating using definition
37
𝒌
𝒂
𝑬 ¿MST()
Light heavy
Light edges=
![Page 38: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/38.jpg)
Step 2Calculating using definition
: set of all subsets of of size .The probability is equal to
38
𝒇 (𝒂)𝒎−𝒌
𝟏¿𝑺∨¿¿
![Page 39: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/39.jpg)
Step 3
Expressing the entire experiment as Randomized Incremental Construction
A slight difficulty in this process is the following:• The underlying experiment talks about random sample from a set.• But RIC involves analyzing a random permutation of a set of elements. Question: What is relation between random sample from a set and a random permutation of the set ?
Spend some time on this question before proceeding further.
39
![Page 40: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/40.jpg)
random sample and random permutation
Observation: is indeed a uniformly random sample of
40
Random permutation of
𝒓 𝒎−𝒓𝑨 𝑬 ¿
![Page 41: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/41.jpg)
Step 3
The underlying random experiment as Randomized Incremental Construction: • Permute the edges randomly uniformly.• Find the probability that th edge is light relative to the first edges.
Question: Can you now calculate probability ?
Spend some time on this question before proceeding further.
41
![Page 42: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/42.jpg)
Step 3
42
Random permutation of
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊−𝟏
![Page 43: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/43.jpg)
Step 3
: a random variable taking value 1 if is a light edge with respect to MST().
43
Random permutation of
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊−𝟏 𝑬 {𝑬 ¿𝒊−𝟏
![Page 44: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/44.jpg)
Step 3
: a random variable taking value 1 if is a light edge with respect to MST().
Question: What is relation between and ’s?Answer: ??
44
Random permutation of
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊−𝟏 𝑬 {𝑬 ¿𝒊−𝟏
𝒑=𝐏¿¿
![Page 45: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/45.jpg)
Calculating ).
: set of all subsets of of size . ) =
depends upon
45
Forward analysis
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊−𝟏
MST()
Random permutation of
![Page 46: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/46.jpg)
Calculating ).
: set of all subsets of of size . )=
46
Backward analysis
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊
Random permutation of
![Page 47: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/47.jpg)
𝐏¿ ¿
= ??
??
47
Backward analysis
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊
MST()
¿MST (𝒂)∨¿𝒊 ¿
Random permutation of
Use Lemma 2.
𝐏 ( 𝒊 thedge𝐛𝐞𝐥𝐨𝐧𝐠𝐬 ¿MST (𝒂))
![Page 48: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/48.jpg)
Calculating )
: set of all subsets of of size . )=
48
Backward analysis
𝒆𝒊𝒆𝟏𝒆𝟐 …
𝑬 𝒊
Random permutation of
![Page 49: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/49.jpg)
Combining the two methods for calculating
Using method 1:
Using method 2:
)
Hence:
49
![Page 50: Randomized Algorithms CS648](https://reader031.fdocuments.us/reader031/viewer/2022013012/568160c6550346895dcff492/html5/thumbnails/50.jpg)
Theorem: If we sample edges uniformly randomly from an undirected graph on vertices and edges, the number of light edges among the unsampled edges will be less than on expectation.
50