Special Continuous Probability Distributions Normal Distributions Lognormal Distributions
Parametric continuous distributions
Transcript of Parametric continuous distributions
![Page 1: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/1.jpg)
Parametric continuous distributions
Li Li
Department of Mathematics and Statistics
1
![Page 2: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/2.jpg)
Continuous uniform distributionA continuous uniform random variable X has probability densityfunction (PDF)
f (x) =1
b � a, a x b,
and 0 elsewhere. Here a and b are two real numbers. We referthe distribution as uniform (a, b).• For example, a continuous uniform distribution over [0, 1]
(often referred as uniform (0, 1)) has density f (x) = 1 for0 < x < 1 and 0 elsewhere.
• Interesting fact: given any continuous random variable Yand its cumulative distribution function F (y), a CDFtransformation of Y , i.e F (Y ) has a uniform (0, 1)distribution.
• For example, if Y has a density f (y) = �e��y for y � 0,then its CDF is F (y) = 1 � e��y for y � 0. We have
1 � e�Y ⇠ Uniform(0, 1).
2
I-7 Sample-1 space
9yob=/
-
to xcfloi)
![Page 3: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/3.jpg)
Pfc - X ad )'
is:*.=DIb - a
Pla - x ax .) =b - a
-X r Uniform ( o , t)FC . ) : CDF function(Y :X) Y has distribution Ftl
![Page 4: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/4.jpg)
Where is continuous uniform distribution used?
• Simulations are used to model complicated processes,estimate distributions of estimators (using methods suchas bootstrap), and have dramatically increased the use ofan entire field of statistics.
• In simulations, we often generate random numbers from adesired distribution.
• Uniform (0, 1) is the where random number generationstart.
• To generate a random variable that has CDFF (y) = 1 � e��y for y � 0, we can use the following steps(a) generate a random number u from Uniform (0, 1).(b) transform the random number u by the inverse of the CDF
for the density we desire, i.e. �log(1 � u)/�.
3
=
=
![Page 5: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/5.jpg)
CDFCDF of Uniform (a, b)
4
O xsa
Fix)= ( x a <xeb
l l X>b
b-aFl b) = Fa = I
![Page 6: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/6.jpg)
Mean and variance of uniform (a, b)Mean and variance of uniform (a, b):
5
µ= aztbb'- a' (b-a)(b'+bata't g.= (b
-al'
T2
µ -- fab-x.b.tadx-M-f.jx.EE,dx)- BT> fix)
gz, Ely'
) - µ"
= ba =fab¥adx -HII= b÷÷,=bII=¥a, Iba -4th
= ¥ata2- ¥4'
![Page 7: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/7.jpg)
fix, = ft"" " "o
fax) O E X < I
fix
x
Ek) =/!! x fix, d× =/!x. findX+f! x. facedX
![Page 8: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/8.jpg)
Normal distribution
1) Characteristics
4 Pla - X ab)
s) percentiles )4) Normal approximationsto Binomial distribution
and Poisson distribution
![Page 9: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/9.jpg)
Normal distributionsGiven parameters µ and �, PDF of the Normal distribution is
f (x) =1p
2⇡�2e� (x�µ)2
2�2 ,�1 < x < +1.
Figure: Normal distribution density functions. Density function isdenoted as �µ,�2(x).
µ– location parameter or mean of the distribution and �–scaleparameter or standard deviation of the distribution.
6
--
E- ( x ) = µVar(X) = 62
÷.
I-
![Page 10: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/10.jpg)
• A Normal distribution with µ = 0 and � = 1 is referred as“standard Normal distribution”.
• Normal distributions are also called ”Gaussiandistributions”.
• Normal distribution is sometimes informally called the “bellcurve”.
• A random variable with a Gaussian distribution is said tobe normally distributed, denoted by X ⇠ N(µ,�2). Forexample, X ⇠ N(3, 22).
7
- -
= e.
-
![Page 11: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/11.jpg)
Important facts about Normal distributions
• All Normal curves have the same overall shape:symmetric, single-peaked, bell-shaped.
• Any specific Normal curve is completely described bygiving its mean µ and its standard deviation �.
• The mean is located at the center of the symmetric curveand is the same as the median. Changing µ withoutchanging � moves the Normal curve along the horizontalaxis without changing its spread.
• The standard deviation � controls the spread of a Normalcurve. Curves with larger standard deviations are morespread out or wider.
8
![Page 12: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/12.jpg)
Important facts about Normal distributions
• The average of many independent processes (such asmeasurement errors) often have distributions that arenearly normal.
• If X1,X2, . . . ,Xn are independent Bernoulli randomvariables with the same success rate p, X̄ (average ofX1, . . . ,Xn) follows a Normal distribution N(p, p(1�p)
n )approximately.
• If X1,X2, . . . ,Xn are independent Poisson random variableswith the same rate of event � and time interval T , X̄ followsa Normal distribution N(�T , �T
n ) approximately.
9
- - e- I.' m - t
Si-
![Page 13: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/13.jpg)
For a Normal distribution with mean µ and standard deviation �:
• Approximately 68% of the observations fall within � of themean µ.
• Approximately 95% of the observations fall within 2� of themean µ.
• Approximately 99.7% of the observations fall within 3� ofthe mean µ.
10
![Page 14: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/14.jpg)
The empirical rule
Figure: The 68–95–99.7 rule.
11
PIX > pets) p( µ-36C Xc µ-130) 99.75%=
O - 00272-
pl X > put so or¥ Oi 00135
'
-2
×e )= I - 99.73%
'III:'s:* D '
oars
- I 0. 0027t
= 0.5 - ° pfµ. o - X spits ) = 68.27%
pl X > pete . -50 ) Pl M - Zo s X C la +20) 95.45%
![Page 15: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/15.jpg)
How do we find P(a < X < b)
For a Normal distribution with mean µ and variance �2 andX ⇠ N(µ,�2), probability
P(a < X < b) =Z b
a
1p2⇡�2
e� (x�µ)2
2�2 dx
can be calculated by pnorm((b � µ)/�)� pnorm((a � µ)/�) inR.
12
O
=
IT -
pnormicex) : returns F- ( x ) where FC . ) is the CDF of
stand art Normal distribution .
Plas x ab) - E. ( BI ) - E. l )
![Page 16: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/16.jpg)
How do we find the p-th percentile?
The p-th percentile c where
P(X < c) =Z c
1
1p2⇡�2
e� (x�µ)2
2�2 dx = p/100
can be calculated by qnorm(p/100) ⇤ � + µ.
13
P-th percentile of a continuous distribution with density
th) : solve fkff.
j
flydx =÷o EPE 100
⇐
![Page 17: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/17.jpg)
CDFThe cumulative distribution function (CDF) of a Normaldistribution function is denoted as
�µ,�2(x) =Z x
�1
1p2⇡�2
e� (u�µ)2
2�2 du.
Figure: Normal distribution CDF functions. CDF functions aredenoted as �µ,�2(x).
14
TQ , , zz( x )
1) Monotone increasingIoz , I ' ( x ) within the sample
space of the
distribution
2) o E IDFCx) E l
![Page 18: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/18.jpg)
Standard Normal distribution and z-score
Standard Normal distribution:
• If a variable X has any Normal distribution N(µ,�2), thenthe standardized variable
Z =X � µ
�
has the standard Normal distribution N(0, 1). For example,if X follows a Normal distribution with mean 3 and variance4, i.e. X ⇠ N(3, 22), then Z = X�µ
� ⇠ N(0, 1).• For a real number a, the standardized value of a is
z =a � µ
�
is called the z-score of a.
15
ppl a ex ab)
-.
-
•
![Page 19: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/19.jpg)
Etz CH = P ( Z e z )= PHI e z )=p ( x e ot ta)
=L!" .. e-'
dx
shows, Got = ¥1
= f.to ¥, e-E
dy-
= Teo, ,lZ)
![Page 20: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/20.jpg)
Use of z-score
• When the z-score of an observation has an absolute valuegreater than 3, this observation can be viewed roughly asan outlier or unusual.
• z-score can be used to compare two observations fromtwo populations that have different Normal distributions.
16
![Page 21: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/21.jpg)
Example
Consider for two high school senior students,
• student A scored 670 on the Mathematics part of the SAT.4The distribution of SAT Math scores in 2010 was Normalwith mean 516 and standard deviation 116.
• student B took the ACT and scored 46 on the Mathematicsportion. ACT Math scores for 2010 were Normallydistributed with mean 21.0 and standard deviation of 5.3.
(a) Find the z- scores for both students.(b) Assuming that both tests measure the same kind of ability,
who had a higher score? Are any of these two test scoresoutlying?
17
6=116-
- -µ-- 516
-
-
µ=21,7=5.3
![Page 22: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/22.jpg)
18
670 - 516A) ZA = 11-6=1.3276
46-21ZB = 7.3 = 4.7170
![Page 23: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/23.jpg)
Normal table
• Only used in test situation these days.• It is a one to one mapping of z to �0,1(z) (Standard Normal
CDF) for z goes from �3.99 to 3.99.• Given z, use the table we can find �0,1(z). Given p such
that �0,1(z) = p, we can also find z. This gives us thep ⇤ 100-th percentile of the Standard Normal distribution.
19
Goal : X - N ( µ. o') ; P (a cX ab)from the Normal-
table.
- -
- - -
![Page 24: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/24.jpg)
Normal table
Figure: One to one mapping of z to �0,1(z) for z from �3.99 to 0.
20
-3.91 -3.92 oil 't ) -- 0.00013ez⇐⇐"→÷÷
÷-
![Page 25: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/25.jpg)
Normal table
Figure: One to one mapping of z to �0,1(z) for z from 0 to 3.99.
21
-T
so:÷%-
÷÷÷÷:.¥l¥g¥# - - - *
![Page 26: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/26.jpg)
What is the probability for a standard Normal variable Z takevalues less than 1.47?
• Locate 1.4 in the left-hand column of the Normal table• Then locate the remaining digit seven as .07 in the top row.• The entry opposite 1.4 and under .07 is 0.9292. This is the
cumulative proportion we seek.
What is the 92.9-th percentile of a Standard Normaldistribution?—It is 1.47.
22
![Page 27: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/27.jpg)
For a Normal random variable follows a distribution that has amean µ and variance �2, how to find the probability for therandom variable to fall within an interval [a, b]?
• Note that P(a < X < b) = P(a�µ� < X�µ
� < b�µ� ) and X�µ
�follows a standard Normal distribution.
• Hence P(a < X < b) = �0,1(b�µ� )� �0,1(
a�µ� ).
• Calculate b�µ� and a�µ
� . Then use the Normal table to findthe corresponding probabilities �0,1(
b�µ� ) and �0,1(
a�µ� )
respectively.
23
X- N ( µ , 04
-00 do
a- -
-
O U
![Page 28: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/28.jpg)
Suppose X ⇠ N(2, 22), show thatP(1 < X < 3) = �(0.5)� �(�0.5) = 0.383.
24
in.×÷?'S
" " "
'Ic -2 c
'II
= p f - o is a Z c o . 5 ) Z - N(oil)
= -I - a5) =
pl a c Z c b) = PLZ ab) - pl Zsa) ex,
e¥¥¥n⇐¥#¥
![Page 29: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/29.jpg)
Finding percentiles using the Normal table
For a Normal random variable follows a distribution that has amean µ and standard deviation �, how to find the the p-thpercentile of the distribution?(a) Note that our goal is to find c such that P(X < c) = p/100.
Since P(X < c) = �0,1(c�µ� ), we are solving c from
�0,1(c�µ� ) = p/100.
(b) Use the table to find the p-th percentile of the standardNormal distribution. Denoted it by zp.
(c) Then c = zp ⇤ � + µ.
25
O. -
![Page 30: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/30.jpg)
Suppose X ⇠ N(2, 22), show that the 92.9�th percentile of thedistribution is 1.47 ⇤ 2 + 2 = 4.94.
26
--
µ}
"
..
-
Ebo!1.477=92.9Goal : find p -th percentile of distribution N (µ,
62)
i. e find a such that
PC X E c ) = Plioo where X-Ngaio's
→ -
= Iof ) -- I011 I DO
C-µ
g- = Zp
C = 2-p - stfu
![Page 31: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/31.jpg)
Normal probability plot
How to tell whether observations from a population follows aNormal distribution? (Chapter 7)
• Normal probability plot or QQ plot.• Shapiro-Wilk test.
27
![Page 32: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/32.jpg)
Normal approximation to Binomial distributions
Normal approximation to the Binomial distribution: If X s abinomial random variable with parameters n and p,
Z =X � nppnp(1 � p)
follows a standard Normal distribution approximately. Theapproximation is close if np > 5 and n(1 � p) > 5.https://newonlinecourses.science.psu.edu/stat414/node/179/
28
![Page 33: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/33.jpg)
Normal approximation to Binomial distributions
• To approximate a binomial probability with a normaldistribution, a continuity correction is applied as follows:
P(X x) = P(X x + 0.5) ⇡ P
Z x + 0.5 � npp
np(1 � p)
!
and
P(X � x) = P(X � x � 0.5) ⇡ P
Z � x � 0.5 � npp
np(1 � p)
!
• For example, if n = 20 and p = 0.3, thenP(X 7) ⇡ �0,1(
7+0.5�6p4.2
) = 0.758,P(X � 7) ⇡ 1 � �0,1(
7�0.5�6p4.2
) = 1 � 0.596 = 0.404.
• Note that P(X < x) = P(X x � 1) andP(X > x) = P(X � x + 1).
29
![Page 34: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/34.jpg)
Example
Assume that in a digital communication channel, the number ofbits received in error can be modeled by a Binomial randomvariable, and assume that the probability that a bit is received inerror is 1 ⇥ 10�5. If 16 million bits are transmitted, what is theprobability that 150 or fewer errors occur?
30
![Page 35: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/35.jpg)
Normal approximation to Poisson distribution
Normal approximation to the Poisson distribution: If X is aPoisson random variable with E(X ) = �T and V (X ) = �T ,
P(X x) = P(X x + 0.5) = P✓
Z x + 0.5 � �Tp�T
◆
and
P(X � x) = P(X � x � 0.5) = P✓
Z � x � 0.5 � �Tp�T
◆
The approximation is generally good for �T > 5.
31
![Page 36: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/36.jpg)
Chi-square distributions
If Z follows a standard Normal distribution then:
V = Z 2 ⇠ �21,
where �21 is called a chi-square distribution with 1 degree of
freedom which has density
f (v) =1
�(1/2)20.5 v0.5�1e�v/2, v � 0.
32
![Page 37: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/37.jpg)
Chi-square distributions
More generally, if Z1,Z2, . . . ,Zk are independent (one does notaffect the distribution of another), standard Normal randomvariables, then
V =kX
i=1
Z 2i ⇠ �2
k
which denotes a chi-squared distribution with k degrees offreedom. For example, V = Z 2
1 + Z 22 ⇠ �2
2.The density of thedistribution is
f (v) =1
�(k/2)2k/2 vk/2�1e�v/2, v � 0.
Chi-squared distributions are used primarily in hypothesistesting.
33
![Page 38: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/38.jpg)
Distributions of random sample mean andrandom sample variance
Suppose X1, . . . ,Xn are independent Normal random variables,which all follow distribution N(µ,�2), which means they areidentical.
• Denote X̄ = 1nPn
i=1 Xi as the random sample mean.• Denote S2 = 1
n�1Pn
i=1(Xi � X̄ )2 as the random samplevariance.
Then the results are true regardless what values of µ and �2
are:• X̄ ⇠ N(µ,�2/n)• (n � 1)S2/�2 ⇠ �2
n�1
34
![Page 39: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/39.jpg)
Example
Suppose X1, . . . ,X20 are independent Normal randomvariables, which all follow distribution N(2, 32), which meansthey are identical.
• Denote X̄ = 120P20
i=1 Xi as the random sample mean.
• Denote S2 = 119P20
i=1(Xi � X̄ )2 as the random samplevariance.
• X̄ ⇠ N(2, 9/20)• 19S2/9 ⇠ �2
19
35
![Page 40: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/40.jpg)
Simple random sample
If X1, . . . ,Xn is called a simple random sample if• X1,X2, . . . ,Xn are independent random variables.• X1,X2, . . . ,Xn follow the same distribution, i.e. they are
identical.
36
![Page 41: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/41.jpg)
Central limit theorem (CLT)
If X1, . . . ,Xn is a random sample of size n taken from apopulation or a distribution (not necessarily Normal distribution)with mean µ and variance �2 and if X̄ is the sample mean, then
X̄ ⇠ N(µ,�2/n)
for large n. For example,• If X1,X2, . . . ,X10 are independent random variables
following an uniform distribution (0, 1), then X̄ follows aNormal distribution N(0.5, 1/12/10), i.e. N(0.5, 0.0083).
• If X1,X2, . . . ,Xn are independent Bernoulli randomvariables with the same success rate 0.4, X̄ (average ofX1, . . . ,Xn) follows a Normal distribution N(0.4, 0.24
n )approximately.
37
![Page 42: Parametric continuous distributions](https://reader033.fdocuments.us/reader033/viewer/2022050611/627389bccc2d4c2c2421558f/html5/thumbnails/42.jpg)
Animation of CLT
https://www.youtube.com/watch?v=Pujol1yC1_A
38