Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π...
Transcript of Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π...
![Page 1: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/1.jpg)
Randomized Algorithms CS648
Lecture 2 β’ Three interesting problems
β’ Balls into bins β’ Balls out of bin β’ Randomized Quick Sort
β’ Random Variable and Expected value
1
![Page 2: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/2.jpg)
Recap from last lecture
β’ Probability space (π,P)
β’ Events
β’ Probability of union of events
β’ Partition theorem :
If events π1,β¦, ππpartition the sample space π then
β’ Conditional probability
2
P[A|B] = π[πβ©π]
π[π]
P(AUB) = P(A) + P(B) β P(Aβ©B)
P(B) = P(ππβ©Bπ )
![Page 3: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/3.jpg)
BALLS INTO BINS
3
![Page 4: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/4.jpg)
Balls into Bins
Ball-bin Experiment: There are π balls and π bins.
Each ball selects its bin randomly uniformly and independent of other balls
and falls into it.
Applications: β’ Hashing
β’ Load balancing in distributed environment 4
1 2 3 β¦ π β¦ π
1 2 3 4 5 β¦ π β 1 π
![Page 5: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/5.jpg)
Balls into Bins
Question : What is the probability that there is at least one empty bin ?
5
1 2 3 β¦ π β¦ π
1 2 3 4 5 β¦ π β 1 π
![Page 6: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/6.jpg)
Balls into Bins
What is the probability space (Ξ©,P) ?
β’ | Ξ© | = ππ
β’ P(Ο) = 1/ ππ for each ΟΟ΅ Ξ©
6
1 2 3 β¦ π β¦ π
1 2 3 4 5 β¦ π β 1 π
![Page 7: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/7.jpg)
Balls into Bins
Let us define event π¬ππ
β’ Events π¬ππ and π¬π
π are ??
β’ Events π¬ππ and π¬π
π are ??
β’ Events π¬ππ and π¬π
π are ??
7
1 2 3 β¦ π β¦ π
1 2 3 4 5 β¦ π β¦ π β 1 π
disjoint
Independent
Independent
: πth ball falls into πth bin.
![Page 8: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/8.jpg)
Balls into Bins
π¬ππ : πth ball enters πth bin.
β’ Pr[π¬ππ] = ??
β’ Pr[π¬ππ] = ??
β’ Pr[πth bin is empty] = ??
=(1 β 1
π )π
8
1π
1 β1
π
Pr[π¬1π β©π¬2
π β¦β©π¬ππ] = Pr[π¬1
π ] β¨― β¦Pr[π¬ππ]
1 2 3 4 5 β¦ π β¦ π β 1 π
1 2 3 β¦ π β¦ π
![Page 9: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/9.jpg)
Balls into Bins
β’ Pr[πth bin is empty] = (1 β 1
π )π
β’ Pr[πth and πth bin are empty] = ??
β’ Pr[a specified set of π bins are empty] = ??
9
(1 β 2π
)π
(1 β ππ )π
1 2 3 β¦ π β¦ π
1 2 3 4 5 β¦ π β¦ π β 1 π
![Page 10: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/10.jpg)
Balls into Bins
Question: What is the probability that there is at least one empty bin ?
Attempt 1: Explore the sample space associated with the βballs into binsβ.
Attempt 2: ??
Define ππ : βπth bin is emptyβ
Event βthere is at least one empty binβ = ππππ=1
10
Express the event as union of some events β¦
![Page 11: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/11.jpg)
Balls into Bins
Theorem: For events π1,β¦, ππdefined over a probability space (π,P), then
P( ππππ=1 ) = P(ππ)π
β P(ππ ππ )π<π
+ P(ππ ππ ππ )π<π<π
β β¦
(β1)π+1 P(π1 π2 β¦ ππ)
--------------------------------------------------------------------
= (π1)(1 β 1
π )π
β (π2)(1 β 2
π )π
+ (π3)(1 β 3
π )π
β β¦
+ β1 π+1(ππ) (1 β π
π )π
β¦ 11
π(π,π)
![Page 12: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/12.jpg)
Balls into Bins
Homework Exercise:
What is the probability that there are exactly π empty bins ?
Hint: You will need to use π(π,π) with suitable values of π.
12
![Page 13: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/13.jpg)
BALLS OUT OF BIN
13
![Page 14: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/14.jpg)
Balls Out of Bin
There is a bin containing π red balls and π blue balls.
Balls are taken out from the bag
Question: What is the probability that appears ahead of all blue balls. ?
14
uniformly randomly and without replacement.
What if π = 1 ?
![Page 15: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/15.jpg)
Balls Out of Bin
There is a bin containing 1 red ball and π blue balls.
Balls are taken out from the bag
Question: What is the probability that appears ahead of all blue balls ?
Question: What is the probability that appears at πth place ? 15
uniformly randomly and without replacement.
1
π + 1
1
π + 1
![Page 16: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/16.jpg)
Balls Out of Bin
There is a bin containing π red balls and π blue balls.
Balls are taken out from the bag
Question: What is the probability that appears ahead of all blue balls ?
Homework: Give formal arguments in support of your answer. 16
uniformly randomly and without replacement.
1
π + 1
![Page 17: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/17.jpg)
RANDOMIZED QUICK SORT
17
![Page 18: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/18.jpg)
18
How to solve a problem ?
![Page 19: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/19.jpg)
Perspective
19
does matter
![Page 20: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/20.jpg)
Randomized Quick Sort
Input: π¨[0..π β 1]
RandomizedQuickSort(π¨,π, π) //For the first call, π =0, π=π β 1
{ If (π < π)
π an element selected randomly uniformly from π¨[π..π];
π Partition(π¨,π,π,x);
RandomizedQuickSort(π¨,π, π β π);
RandomizedQuickSort(π¨,π + π, π)
}
Assumption : All elements are distinct (if not, break the ties arbitrarily)
Notation : ππ : πth smallest element of array π¨.
Question: What is the probability that ππ is compared with ππ ?
20
Recall that the execution of RandomizedQuickSort is totally immune to the permutation of π¨.
![Page 21: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/21.jpg)
Randomized Quick Sort
Input: π¨[0..π β 1]
RandomizedQuickSort(π¨,π, π) //For the first call, π =0, π=π β 1
{ If (π < π)
π an element selected randomly uniformly from π¨[π..π];
π Partition(π¨,π,π,x);
RandomizedQuickSort(π¨,π, π β π);
RandomizedQuickSort(π¨,π + π, π)
}
The sample space : all recursion trees (rooted binary trees on π nodes).
21
![Page 22: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/22.jpg)
Randomized Quick Sort
Input: π¨[π..π β π]
RandomizedQuickSort(π¨,π, π) //For the first call, π =0, π=π β 1
{ If (π < π)
π an element selected randomly uniformly from π¨[π..π];
π Partition(π¨,π,π,x);
RandomizedQuickSort(π¨,π, π β π);
RandomizedQuickSort(π¨,π + π, π)
}
22
π4
π2
π1 π3
π6
π5 π7
π = 7 1
7β 1
3β 1
3
![Page 23: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/23.jpg)
Randomized Quick Sort
Input: π¨[π..π β π]
RandomizedQuickSort(π¨,π, π) //For the first call, π =0, π=π β 1
{ If (π < π)
π an element selected randomly uniformly from π¨[π..π];
π Partition(π¨,π,π,x);
RandomizedQuickSort(π¨,π, π β π);
RandomizedQuickSort(π¨,π + π, π)
}
23
π = 7 π5
π6
π7
π2
π1 π4
π3
1
7β 1
4β 1
2β 1
2
![Page 24: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/24.jpg)
Randomized Quick Sort
Question: What is the probability that ππ is compared with ππ ?
Attempt 1: Explore the sample space associated with Randomized Quick Sort.
add the probability of each recursion tree in which ππ is compared with ππ.
Attempt 2: ??
24
View the execution of RandomizedQuickSort from perspective of ππ and ππ
Not a feasible way to calculate the probability
![Page 25: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/25.jpg)
Randomized Quick Sort from perspective of ππ and ππ
In order to analyze the Randomized Quick Sort algorithm from the perspective of ππ and ππ, we do the following:
β’ We visualize elements π1, β¦ , ππ arranged from left to right in increasing order of values.
β’ This visualization ensures that the two subarrays which we sort recursively lie to left and right of the pivot element. In this way we can focus on the subarray containing ππ and ππeasily.
β’ Note that this visualization is just for the sake of analysis. It will be grossly wrong if you interpret it as if we are sorting an already sorted array.
25
Go through the next few slides slowly and patiently, pondering at each step. Never accept anything until and unless you can see the underlying truth yourself.
![Page 26: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/26.jpg)
Randomized Quick Sort from perspective of ππ and ππ
26
Elements of A arranged in Increasing order of values
ππ ππ
![Page 27: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/27.jpg)
Randomized Quick Sort from perspective of ππ and ππ
Observation:
ππ and ππ get compared during an instance of Randomized Quick Sort iff
the first pivot element from πΊππ is either ππ or ππ.
Let us define two events.
π΅π : first pivot element selected from πΊππ during Randomized Quick Sort is ππ.
π΅π : first pivot element selected from πΊππ during Randomized Quick Sort is ππ.
Pr[ππ and ππ get compared] = ??
27
ππ ππ
πΊππ
Pr[π΅πUπ΅π]
![Page 28: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/28.jpg)
Randomized Quick Sort from perspective of ππ and ππ
Pr[ππ and ππ get compared] = Pr[π΅πUπ΅π]
= Pr[π΅π] + Pr[π΅π] - Pr[π΅π β©π΅π]
= Pr[π΅π] + Pr[π΅π]
= 1
πβπ+1 + 1
πβπ+1
= 2
πβπ+1
28
ππ ππ
πΊππ
What relation exists between π΅π and π΅π ?
π΅π and π΅π are disjoint events.
What is Pr[π΅π] ? Pr[π΅π] =
1
|πΊππ|.
![Page 29: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/29.jpg)
Randomized Quick Sort from perspective of ππ and ππ
Theorem: During Randomized-Quick-Sort on π elements,
the probability ππ and ππ are compared with probability 2
πβπ+1 .
Inferences: β’ Probability depends upon the rank separation π β π + 1
β’ Probability ----------?----------- the size of the array.
β’ Probability ππ and ππ+1 are compared = ? .
β’ Probability of comparison of π0 and ππβ1 = ? .
29
is independent of
1 2
π
![Page 30: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/30.jpg)
PROBABILITY THEORY
β’ Random variable
and
β’ expected value
30
![Page 31: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/31.jpg)
Random variable
31
Randomized-Quick-Sort
on array of size n
Number of HEADS in 5 tosses
Sum of numbers in 4 throws
Number of comparisons
![Page 32: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/32.jpg)
Random variable
Definition: A random variable defined over a probability space (Ξ©,P)
is a mapping Ξ© R.
β’ πΏ(Ο) : the value of πΏ on elementary event Ο β π .
Notations for random variables :
β’ πΏ, π, πΌ, β¦(capital letters)
32
Ξ©
R
Set of real numbers
πΏ
![Page 33: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/33.jpg)
Many Random Variables for the same Probability space
Random Experiment: Throwing a dice two times
β’ X : the largest number seen
β’ Y : sum of the two numbers seen
33
X(Ο) = 6 Ο
Y(Ο) = 9
![Page 34: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/34.jpg)
Expected Value of a random variable (average value)
Definition: Expected value of a random variable X defined over a probability space (Ξ©,P) is
E[X] = X(Ο) β¨― P(Ο)ΟΟ΅ Ξ©
34
Ξ©
X= a X= b
X= c
E[X] = a⨯ P(X = a)aϡ X
![Page 35: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/35.jpg)
Examples
Random experiment 1: A fair coin is tossed n times
Random Variable X: The number of HEADS
E[X] = π β¨― P(X =π)π
= π β¨―πππ (1 2 )
π (1 2 )πβπ
= π 2
Random Experiment 2: 4 balls into 3 bins
Random Variable X: The number of empty bins
E[X] = π β¨― P(X =π)π
35
![Page 36: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/36.jpg)
Can we solve these problems ? Random Experiment 1 : There are π balls and π bins.
Each ball falls into a bin selected randomly uniformly and independently.
Random Variable X: The number of empty bins
E[X]= ??
Random Experiment 2 : A bin contains π red balls and π blue balls in a bin.
The balls are taken out randomly uniformly without replacement.
Random Variable X: The number of red balls preceding all blue balls.
E[X]= ??
Random Experiment 3 Randomized Quick sort on π elements Random Variable X: The number of comparisons
E[X]= ??
36
![Page 37: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/37.jpg)
β’ Spend at least 2 hours today trying to solve the 3 problems given in the previous slide.
β’ These 2 hours will be very valuable for you in the long run.
I really mean it.
37
![Page 38: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/38.jpg)
Fun with probability
38
![Page 39: Randomized Algorithms - ict.iitk.ac.inΒ Β· Randomized Quick Sort from perspective of π and π In order to analyze the Randomized Quick Sort algorithm from the perspective of π](https://reader033.fdocuments.us/reader033/viewer/2022041501/5e2219411111a5654e426e0d/html5/thumbnails/39.jpg)
Stick problem
We break the stick at π points selected randomly uniformly.
39
What is the probability that the π + π pieces can be joined to form a polygon ?