Randomness and Computation - inf.ed.ac.uk · lnln ( n ) with probability at least 1 - 1 n. Proof...

Randomness and Computationor, “Randomized Algorithms”

Mary Cryan

School of InformaticsUniversity of Edinburgh

RC (2017/18) – Lectures 9 and 10 – slide 1

warm-up: Birthday Paradox

30 people in a room. What is the probability they share a birthday?

I Assume everyone is equally likely to be born any day (uniform atrandom). Exclude Feb 29 for neatness.

I Generate birthdays one-at-a-time from the pool of 365 (principleof deferred decisions).

Probability p30diff that all birthdays are different is

p30diff =

365 − (i − 1)365

(1 −

(i − 1)365

(1 −

Recall that 1+x < ex for all x ∈ R, hence (1− j365 ) < e−j/365 for any j .

30 people in a room. What is the probability they share a birthday?

I Assume everyone is equally likely to be born any day (uniform atrandom). Exclude Feb 29 for neatness.

I Generate birthdays one-at-a-time from the pool of 365 (principleof deferred decisions).

Probability p30diff that all birthdays are different is

p30diff =

365 − (i − 1)365

(1 −

(i − 1)365

(1 −

Recall that 1+x < ex for all x ∈ R, hence (1− j365 ) < e−j/365 for any j .

p30diff <

e−j/365 =

e−∑29

j=1 j) 1

365=(e−435) 1

last step using∑n

j=1 j = n(n+1)2 . And

(e−435

) 1365 ∼ e−1.19 ∼ 0.3. So

with probability of at least 0.7, two people at the party share abirthday.

More general framework:

n birthday options, m party guests

p30diff <

e−j/365 =

e−∑29

j=1 j) 1

365=(e−435) 1

last step using∑n

j=1 j = n(n+1)2 . And

(e−435

) 1365 ∼ e−1.19 ∼ 0.3. So

p30diff <

e−j/365 =

e−∑29

j=1 j) 1

365=(e−435) 1

last step using∑n

j=1 j = n(n+1)2 . And

(e−435

) 1365 ∼ e−1.19 ∼ 0.3. So

warm-up: General Birthday Paradox

Probability pall−m−diff that all are different is

pall−m−diff =

(1 −

(j − 1)n

m−1∏

(1 −

Continuing,

pall−m−diff ≤m−1∏

e−j/n =

m−1∏

e−∑m−1

j=1 j) 1

n= e−

(m−1)m2n ,

approximately e−m2/2n.Suppose we set m = b√nc, then e−m2/2n becomes ∼ e−0.5 ∼ 0.6.

pall−m−diff =

(1 −

(j − 1)n

m−1∏

(1 −

Continuing,

e−j/n =

m−1∏

e−∑m−1

j=1 j) 1

n= e−

(m−1)m2n ,

pall−m−diff =

(1 −

(j − 1)n

m−1∏

(1 −

Continuing,

e−j/n =

m−1∏

e−∑m−1

j=1 j) 1

n= e−

(m−1)m2n ,

Balls in Bins

I m balls, n bins, and balls thrown uniformly at random into bins(usually one at a time).

I Magic bins with no upper limit on capacity.

I Common model of random allocations and their affect on overallload and load balance, typical distribution in the system.

I (by the birthdays analysis) we know that for m = Ω(√

n), thenthere is some constant probability c > 0 of a birthday clash(visualiser).

I “Classic" question - what does the distribution look like form = n? Max load? (with high probability results are what wewant).

Balls in Bins maximum load

Lemma (5.1)Let n balls be thrown independently and uniformly at random into nbins. Then for sufficiently large n, the maximum load is boundedabove by 3 ln(n)

ln ln(n) with probability at least 1 − 1n .

Proof The probability that bin i receives ≥ M balls is at most(

)nn−M

Expanding( n

), this is

n . . . (n − M + 1)M!

1nM ≤ 1

To bound (M!)−1 note that for any k , we have kk

k! ≤∑∞

i=0k i

i! = ek ,hence 1

k! ≤ ( ek )

k . Or use Stirling . . .

)nn−M

Expanding( n

), this is

n . . . (n − M + 1)M!

1nM ≤ 1

k! ≤∑∞

i=0k i

i! = ek ,hence 1

k! ≤ ( ek )

)nn−M

Expanding( n

), this is

n . . . (n − M + 1)M!

1nM ≤ 1

k! ≤∑∞

i=0k i

i! = ek ,hence 1

k! ≤ ( ek )

)nn−M

Expanding( n

), this is

n . . . (n − M + 1)M!

1nM ≤ 1

k! ≤∑∞

i=0k i

i! = ek ,hence 1

k! ≤ ( ek )

Proof of Lemma 5.1 cont’d.So bin i gets ≥ M bins with probability at most

Set M =def3 ln(n)ln ln(n) . Then the probability that any bin gets ≥ M balls is

(using the Union bound) at most

e · ln ln(n)3 ln(n)

) 3 ln(n)ln ln(n)

≤ n ·(

ln ln(n)ln(n)

) 3 ln(n)ln ln(n)

= eln(n)(

ln ln(n)ln(n)

) 3 ln(n)ln ln(n)

Again using properties of ln, this expands as

eln(n)(

eln ln ln(n)−ln ln(n)) 3 ln(n)

ln ln(n)= eln(n)

(e−3 ln(n)+3 ln(n) ln ln ln(n)

ln ln(n)

) 3 ln(n)ln ln(n)

≤ n ·(

ln ln(n)ln(n)

) 3 ln(n)ln ln(n)

= eln(n)(

ln ln(n)ln(n)

) 3 ln(n)ln ln(n)

eln(n)(

ln ln(n)= eln(n)

ln ln(n)

) 3 ln(n)ln ln(n)

≤ n ·(

ln ln(n)ln(n)

) 3 ln(n)ln ln(n)

= eln(n)(

ln ln(n)ln(n)

) 3 ln(n)ln ln(n)

eln(n)(

ln ln(n)= eln(n)

ln ln(n)

Proof of Lemma 5.1 cont’d.Grouping the ln(n)s in the exponents, and evaluating, we have

e−2 ln(n) · e3 ln(n) ln ln ln(n)ln ln(n) =

1n2 n3 ln ln ln(n)

ln ln(n) .

If we take n “sufficiently large" (n ≥ eee4

will do it), then ln ln ln(n)ln ln(n) ≤ 1/3,

hence the probability of some bin having ≥ M balls is at most

Can derive a matching proof to show that “with high probability" therewill be a bin with Ω( ln(n)

ln ln(n) ) balls in it.

Proof of Lemma 5.1 cont’d.Grouping the ln(n)s in the exponents, and evaluating, we have

e−2 ln(n) · e3 ln(n) ln ln ln(n)ln ln(n) =

1n2 n3 ln ln ln(n)

ln ln(n) .

If we take n “sufficiently large" (n ≥ eee4

will do it), then ln ln ln(n)ln ln(n) ≤ 1/3,

hence the probability of some bin having ≥ M balls is at most

Can derive a matching proof to show that “with high probability" therewill be a bin with Ω( ln(n)

ln ln(n) ) balls in it.

Ω(·) bound on the maximum load (chat)

I We implicitly used the Union Bound in our proof of Lemma 5.1,when we multiplied by n on slide 7. However, in reality, bin i hasa lower chance of being “high" (say Ω( ln(n)

ln ln(n) )) if other bins arealready “high" (the “high-bin" events are negatively correlated).

I This means that we can’t use the same approach as inTheorem 5.1 to prove a partner result of Ω( ln(n)

ln ln(n) ).

I Solution is to use the fact that for the binomial distributionB(m, 1

n ) for an individual bin, that as n → ∞,

Pr[X = k ] =

)k (1 −

)m−k

→ e−m/n(m/n)k

(ie, close to the probabilities for the Poisson distribution withparameter µ = m/n)

I The Poisson’s aren’t independent but the dependance can belimited to an extra factor of e

√m (Section 5.4).

References and Exercises

I Sections 5.1, 5.2 of “Probability and Computing". And if you areinterested in the Ω bound for the Θ( ln(n)

ln ln(n) ) result, readSections 5.3 and 5.4 also.

Exercises

I Exercise 5.3 (balls in bins when m = c · √n).

I Exercise 5.10 (sequences of empty bins; this is a bit more tricky)

Randomness and Computation - inf.ed.ac.uk · lnln ( n ) with probability at least 1 - 1 n. Proof...

Documents

Transcript of Randomness and Computation - inf.ed.ac.uk · lnln ( n ) with probability at least 1 - 1 n. Proof...