Lecture 4-cs648
-
Upload
rajiv-omar -
Category
Technology
-
view
35 -
download
4
Transcript of Lecture 4-cs648
![Page 1: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/1.jpg)
Randomized AlgorithmsCS648
Lecture 4• Linearity of Expectation with applications(Most important tool for analyzing randomized algorithms)
1
![Page 2: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/2.jpg)
RECAP FROM THE LAST LECTURE
2
![Page 3: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/3.jpg)
Random variable
Definition: A random variable defined over a probability space (Ω,P) is a mapping Ω R.
Examples:o The number of HEADS when a coin is tossed 5 times.o The sum of numbers seen when a dice is thrown 3 times.o The number of comparisons during Randomized Quick Sort on an array of
size n.
Notations for random variables : • X, Y, U, …(capital letters)• X() denotes the value of X on elementary event .
3
![Page 4: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/4.jpg)
Expected Value of a random variable(average value)
Definition: Expected value of a random variable X defined over a probability space (Ω,P) is
E[X] =
4
Ω
X= a X= b
X= c
E[X] =
![Page 5: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/5.jpg)
Examples
Random experiment 1: A fair coin is tossed n times Random Variable X: The number of HEADS E[X] = = =
Random Experiment 2: balls into bins Random Variable X: The number of empty bins E[X] =
5
![Page 6: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/6.jpg)
Can we solve these problems ?
Random Experiment 1 balls into bins Random Variable X: The number of empty bins
E[X]= ??
Random Experiment 2 Randomized Quick sort on elements Random Variable Y: The number of comparisons
E[Y]= ??
6
![Page 7: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/7.jpg)
Balls into Bins(number of empty bins)
Question : X is random variable denoting the number of empty bins. E[X]= ??
Attempt 1: (based on definition of expectation) E[X] = = =
7
1 2 3 … … n
1 2 3 4 5 … m-1 m
A subset of bins
This is a right but useless answer !
![Page 8: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/8.jpg)
Randomized Quick Sort(number of comparisons)
Question : Y is random variable denoting the number of comparisons. E[Y]= ??
Attempt 1: (based on definition of expectation) E[Y] =
8
We can not proceed from this point …
A recursion tree associated with Randomized Quick Sort
![Page 9: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/9.jpg)
9
1 2 3 4 5 … m-1 m
1 2 3 … … n
Balls into Bins(number of empty bins)
Randomized Quick Sort(number of comparisons)
![Page 10: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/10.jpg)
Balls into Bins(number of empty bins)
Question: Let be a random variable defined as follows. =
What is E[] ?Answer : E[] = + = =
10
1 2 3 … … n
1 2 3 4 5 … m-1 m
![Page 11: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/11.jpg)
Balls into Bins(any relation between and ’s ?)
Consider any elementary event.
11
1 2 3 4 5
1 2 3 4 5 6
0 1 0 1 0
An elementary event
![Page 12: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/12.jpg)
Sum of Random Variables
Definition: Let be random variables defined over a probability space (Ω,P) such that
for each ϵ Ω Then is said to be the sum of random variables and .A compact notation :
Definition: Let and be random variables defined over a probability space (Ω,P) such that
for each ϵ Ω Then is said to be the sum of random variables .A compact notation :
12
𝑼=𝑽 +𝑾
𝑼=𝑽 𝟏+𝑽 𝟐+…+𝑽 𝒏
![Page 13: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/13.jpg)
Randomized Quick Sort(number of comparisons)
Question : Let , for any , be a random variable defined as follows. =
What is E[] ?Answer : E[] = + = =
13
Elements of A arranged in Increasing order of values
𝑒𝑖
![Page 14: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/14.jpg)
Randomized Quick Sort(any relation between and ’s ?)
Consider any elementary event.
Question: What is relation between and and ?Answer:
14
1 0 … 0 1 1 … 0
… …
Hence
Any elementary event
![Page 15: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/15.jpg)
What have we learnt till now?
Balls into Bin experimentX: random variable denoting the number of empty bins
Aim: E[X]= ??
E[] =
Randomized Quick SortY: random variable for the number of comparisons
Aim: E[Y]= ??
E[] =
15
Hence
E E
![Page 16: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/16.jpg)
The main question ?
Let be random variables defined over a probability space (Ω,P) such that ,
16
![Page 17: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/17.jpg)
Balls into Bins(number of empty bins)
: random variable denoting the number of empty bins. Using Linearity of Expectation [] for
17
1 2 3 … … n
1 2 3 4 5 … m-1 m
![Page 18: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/18.jpg)
Randomized Quick Sort(number of comparisons): r. v. for the no. of comparisons during Randomized Quick Sort on elements.
Using Linearity of expectation:
18
𝑯 𝑛≤ l𝑜𝑔𝑒𝑛+0.58
![Page 19: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/19.jpg)
Linearity of Expectation
Theorem: • (For sum of 2 random variables)If are random variables defined over a probability space (Ω,P) such that , then
• (For sum of more than 2 random variables) If , then
19
![Page 20: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/20.jpg)
Where to use Linearity of expectation ?
Whenever we need to find E[U] but none of the following work• E[] = • E[] =
In such a situation,
Try to express as , such that it is “easy” to calculate . Then calculate using
20
![Page 21: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/21.jpg)
Think over the following questions?
• Let be random variables defined over a probability space (Ω,P) such that , for some real no. , then
Answer: yes (prove it as homework)
• Why does linearity of expectation holds always ? (even when and are not independent)
Answer: (If you have internalized the proof of linearity of expectation, this question should appear meaningless.)
21
![Page 22: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/22.jpg)
Think over the following questions?
Definition: (Product of random variables)Let be random variables defined over a probability space (Ω,P) such that
for each ϵ Ω Then is said to be the product of random variables and .A compact notation is
• If , then
Answer: No (give a counterexample to establish it.)• If and both and are independent then
Answer: Yes (prove it rigorously and find out the step which requires independence)
22
![Page 23: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/23.jpg)
Independent random variables
In the previous slides, we used the notion of independence of random variable. This notion is identical to the notion of independence of events:
Two random variables are said to be independent if knowing the value of one random variable does not influence the probability distribution of the other.In other words,
for all ϵ and ϵ .
23
![Page 24: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/24.jpg)
Some Practice problemsas homework
• Balls into bin problem:• What is the expected number of bins having exactly 2 balls ?
• We toss a coin n times, what is the expected number of times pattern HHT appear ?
• A stick has n joints. The stick is dropped on floor and in this process each joint may break with probability p independent of others. As a result the stick will be break into many substicks.– What is the expected number of substicks of length 3 ?– What is the expected number of all the substicks ?
24
![Page 25: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/25.jpg)
PROBLEMS OF THE NEXT LECTURE
25
![Page 26: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/26.jpg)
Fingerprinting Techniques
Problem 1:Given three ⨯ matrices , , and , determine if .
Best deterministic algorithm: • ;• Verify if ? Time complexity:
Randomized Monte Carlo algorithm: Time complexity: Error probability: for any .
26
![Page 27: Lecture 4-cs648](https://reader036.fdocuments.us/reader036/viewer/2022062708/558c786dd8b42a716c8b476c/html5/thumbnails/27.jpg)
Fingerprinting Techniques
Problem 2:Given two large files A and B of bits located at two computers which are connected by a network. We want to determine if A is identical to B. The aim is to transmit least no. of bits to achieve it.
Randomized Monte Carlo algorithm: Bits transmitted : Error probability: for any .
27