Limit Theorems - STATISTICS -- Lecture no. 6k101.unob.cz/~neubauer/pdf/stat_lecture6.pdf ·...

Limit Theorems

Limit TheoremsSTATISTICS – Lecture no. 6

Jirı Neubauer

Department of Econometrics FEM UO Brnooffice 69a, tel. 973 442029email:[email protected]

3. 11. 2009

Jirı Neubauer Limit Theorems

Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem

The Law of Large Numbers

If we repeat some experiment independently we can createusing given observed values distribution of relative frequenciesand calculate some measures (mean, median, variance . . . ).

This distribution (measures) we call sample distribution(sample measures).

Under particular conditions we can expect that the sampledistribution (measures) will converge toward a theoreticaldistribution (measures). The more repetitions of theexperiment the better convergence.



The Law of Large Numbers

Notice that the convergence of the sample values towardtheoretical ones is not the convergence in the sense ofmathematical convergence, but the probability convergence.

The probability convergence – if the number of experimentsincreases, the probability of deviation between sample valuesand theoretical values decreases.



Convergence in probability

Definition

If the sequence of random variables X1,X2, . . . ,Xn, . . . fulfils

limn→∞

P(|Xn − c | < ε) = 1, ε > 0,

it is said that the sequence {Xn} converges in probability to theconstant c , we write

XnP→ c .



Chebyshev’s Inequality

Theorem

For any random variable X with the mean E (X ), the finitevariance D(X ) and for every ε > 0 we have

P(|X − E (X )| < ε) ≥ 1− D(X )

ε2.

Chebyshev’s inequality is useful fist of all in the theoretical field. Itallow us to estimate some probabilities of random variables withunknown distribution.



Bernoulli’s Theorem

Theorem

If the random variable X denotes the number of occurrence of theevent in the sequence of n independent experiments, where π isthe probability of occurrence of the event in one experiment, thenfor every ε > 0 is

limn→∞

P

(∣∣∣∣Xn − π

∣∣∣∣ < ε

)= 1.



de Moivre-Laplace Theorem

Theorem

Let X be a random variable with binomial distributionX ∼ B(n, π) a For the standardized random variable

U =X − nπ√nπ(1− π)

we havelim

n→∞P(U ≤ u) = Φ(u),

where Φ(u) is the distribution function of the standard normaldistribution N(0, 1).

aX = X1, X2, . . . , Xn, where Xi , i = 1 . . . , n, are independent Bernoullirandom variables E(Xi ) = π, D(Xi ) = π(1− π), which means E(X ) = nπ andD(X ) = nπ(1− π).



de Moivre-Laplace Theorem

The de Moivre-Laplace theorem says that for n →∞ the binomialdistribution converges to the normal distribution.Given approximation is acceptable if

nπ(1− π) > 9 and1

n + 1< π <

n

n + 1.



de Moivre-Laplace Theorem for proportion

Theorem

Let X be a random variable with binomial distribution X ∼ B(n, π)The random variable X

n has the mean E(

Xn

)= π and the variance

D(

Xn

)= π(1−π)

n . For the standardized random variable

U =Xn − π√π(1− π)

√n

we havelim

n→∞P(U ≤ u) = Φ(u),

where Φ(u) is the distribution function of the standard normaldistribution N(0, 1).



Levy-Lindeberg’s Theorem

Theorem

Let the random variable be X = X1 + X2 + · · ·+ Xn, whereXi , i = 1, . . . , n are independent random variables with the samedistribution with the mean E (Xi ) = µ and the finite varianceD(Xi ) = σ2, a For the standardized random variable

U =X − nµ√

nσ2

we havelim

n→∞P(U ≤ u) = Φ(u),

where Φ(u) is the distribution function of the standard normal

distribution N(0, 1).

aE(X ) = nµ and D(X ) = nσ2



Levy-Lindeberg’s Theorem for the Mean

Theorem

Let the random variable X be the mean of n independent randomvariables X1,X2, . . . ,Xn, with the same distribution and the meanE (Xi ) = µ and the finite variance D(Xi ) = σ2, i = 1, . . . , n, then

E (X ) = µ and D(X ) =σ2

n

and for the standardized random variable

U =X − µ

σ

√n

we havelim

n→∞P(U ≤ u) = Φ(u),

where Φ(u) is the distribution function of the standard normal

distribution N(0, 1).Jirı Neubauer Limit Theorems



For M = X1 + · · ·+ Xn is:

M =n∑

i=1Xi ∼ as.N(nµ, nσ2),E (M) = nµ,D(M) = nσ2

U = M−E(M)√D(M)

= M−nµ√nσ2

∼ as.N(0, 1)

P(M ≤ m) = F (m) ≈ Φ(

m−nµ√nσ2

)P(−u1−α/2 < m−nµ√

nσ2< u1−α/2

)= 1− α




For M = X1 + · · ·+ Xn is:

M =n∑


U = M−E(M)√D(M)

= M−nµ√nσ2

∼ as.N(0, 1)

P(M ≤ m) = F (m) ≈ Φ(

m−nµ√nσ2

)

P(−u1−α/2 < m−nµ√

nσ2< u1−α/2

)= 1− α




For M = X1 + · · ·+ Xn is:

M =n∑


U = M−E(M)√D(M)

= M−nµ√nσ2

∼ as.N(0, 1)

P(M ≤ m) = F (m) ≈ Φ(

m−nµ√nσ2

)P(−u1−α/2 < m−nµ√

nσ2< u1−α/2

)= 1− α




For the sample mean X is:

X = 1n

n∑i=1

Xi ∼ as.N(µ, σ2

n ),E (M) = µ,D(M) = σ2

n

U = X−E(X )√D(X )

= X−µσ

√n ∼ as.N(0, 1)

P(X ≤ x) = F (x) ≈ Φ(

x−µσ

√n)

P(−u1−α/2 < x−µ

σ

√n < u1−α/2

)= 1− α



Continuity Correction

In the case of using the normal distribution as an approximation ofa distribution of a discrete random variable, it is recommended toapply so called continuity correction which improves thisapproximation.If we calculate P(X ≤ x) or P(X ≥ x) by normal approximation,we get undervalued results. On the contrary if we calculateP(X < x) or P(X > x) by normal approximation, we getovervalued results.



Continuity Correction

Some examples of continuity correction:

before correction x < 3 x ≤ 3 x = 5 x ≥ 7 x > 7after correction x < 2.5 x < 3.5 4.5 < x < 5.5 x > 6.5 x > 7.5



Example 1

The probability that you hit the target is 0.8. What is theprobability that the difference between the number of hits in thesequence of 200 shots and the mean of the this number will not belarge than 10?



Example 1

The binomial distribution:

E (X ) = nπ = 200 · 0.8 = 160

D(X ) = nπ(1− π) = 200 · 0.8 · (1− 0.8) = 32

P(150 ≤ X ≤ 170) = p(150) + p(151) + · · ·+ p(170) =

=(200150

)0.8150 · 0.250 +

(200151

)0.8151 · 0.249 + · · ·+

+(200170

)0.8170 · 0.230 = 0.937



Example 1

de Moivre-Laplace theorem:

F (x) ≈ Φ

(x − nπ√nπ(1− π)

)

P(150 ≤ X ≤ 170) = F (170)− F (149) ≈ Φ(

170−160√32

)−

−Φ(

149−160√32

)= 0.936



Example 1

de Moivre-Laplace theorem (with continuity correction):

F (x) ≈ Φ

(x − nπ√nπ(1− π)

)

P(150 ≤ X ≤ 170)≈P(149.5 < X < 170.5) = F (170.5)− F (149.5) =

= Φ(

170.5−160√32

)− Φ

(149.5−160√

32

)= 0.937



Example 1

Chebyshev’s inequality:

P(|X − E (X )| < ε) ≥ 1− D(X )

ε2

E (X ) = nπ = 200 · 0.8 = 160

D(X ) = nπ(1− π) = 200 · 0.8 · (1− 0.8) = 32

P(|X − 160)| < 10) ≥ 1− 32

102= 0.68

P(|X − 160)| < 11) ≥ 1− 32

112= 0.736



Example 2

In some elections the coalition obtained 52 % of votes. What is theprobability that in the public opinion research of the size 2600respondents the opposition won?



Example 2

X . . . the number of respondents who voted the opposition

X ∼ B(2600; 0.48)

E (X ) = nπ = 2600 · 0.48 = 1248

D(X ) = nπ(1− π) = 2600 · 0.48 · (1− 0.48) = 648.96



Example 2

P(X > 1300) = 1− P(X ≤ 1300) = 1− [p(0) + · · ·+ p(1300)] =

= 1−[(2600

0

)· 0.480 · 0.522600 + · · ·+

+(26001300

)· 0.481300 · 0.521300

]= 1− 0.98031 = 0.01969



Example 2

de Moivre-Laplace theorem:

F (x) ≈ Φ

(x − nπ√nπ(1− π)

)

P(X > 1300) = 1− P(X ≤ 1300) = 1− F (1300) ≈≈ 1− Φ

(1300−1248√

648.96

)= 1− 0.97939 = 0.02061



Example 2

de Moivre-Laplace theorem (with continuity correction):

F (x) ≈ Φ

(x − nπ√nπ(1− π)

)

P(X > 1300) = 1− P(X ≤ 1300) ≈ 1− P(X < 1300.5) =

= 1− Φ(

1300.5−1248√648.96

)=

= 1− 0.98034 = 0.01966


Limit Theorems - STATISTICS -- Lecture no. 6k101.unob.cz/~neubauer/pdf/stat_lecture6.pdf ·...

Documents

Transcript of Limit Theorems - STATISTICS -- Lecture no. 6k101.unob.cz/~neubauer/pdf/stat_lecture6.pdf ·...