Mescaline:The Cberrris'try and Ph.arrnacology ofits Analogs'
Normal Distribution, Continuous Analogs, Derived...
Transcript of Normal Distribution, Continuous Analogs, Derived...
![Page 1: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/1.jpg)
Normal Distribution, Continuous Analogs,Derived Distributions
EECS 126 Spring 2020
February 6, 2020
![Page 2: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/2.jpg)
Agenda
Announcements
Review
Normal (Gaussian) Distribution
Analogs to Discrete Probability / RVs
Derived Distributions
![Page 3: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/3.jpg)
Announcements
I Lab 1 and Lab 0 Self-Grades are due Friday (2/7).
I HW3 will be released today and is due next Wednesday(2/12).
I These slides will be posted later.
![Page 4: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/4.jpg)
Questions
Let R be equal to the distance from the origin of a point randomlysampled on a unit ball. What is the
I CDF of R?
I PDF of R?
I Expectation of R?
![Page 5: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/5.jpg)
Answers
Let R be the distance from the origin of a point randomly sampledon a unit ball. What is the
I CDF of R?FR(r) = P(R ≤ r) = 3
4π ·43πr
3 = r3.
I PDF of R?fR(r) = d
dr r3 = 3r2.
I Expectation of R?∫ 10 r · 3r2 = 3
4 .
![Page 6: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/6.jpg)
Normal Distribution
The Normal distribution is seen abundantly in nature (e.g. examscores). This can be explained by the Central Limit Theorem(CLT), which we will go over later in the course.
The Normal distribution is parameterized by its mean µ andvariance σ2 (denoted N (µ, σ2)). Its PDF and CDF are
fX (x) =1√
2πσ2e−(x−µ)
2/2σ2
FX (x) = Φ(x), (cannot be expressed in elementary functions)
![Page 7: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/7.jpg)
Proof that it Integrates to 1
We will show this when µ = 0 and σ2 = 1. Idea is to show(∫∞−∞ fX (x) dx
)2= 1.
(∫ ∞−∞
fX (x) dx)2
=(∫ ∞−∞
fX (x) dx)(∫ ∞
−∞fX (y) dy
)=
∫ ∞−∞
∫ ∞−∞
fX (x)fX (y) dy dx
=
∫ ∞−∞
∫ ∞−∞
( 1√2π
e−x2
2
)( 1√2π
e−y2
2
)dy dx
=
∫ ∞−∞
∫ ∞−∞
1
2πe−
x2+y2
2 dy dx
![Page 8: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/8.jpg)
Proof that it Integrates to 1 (cont.)
Key is now to use polar integration: x2 + y2 = r2 anddy dx = r dr dθ.∫ ∞
−∞
∫ ∞−∞
1
2πe−
x2+y2
2 dy dx =
∫ 2π
0
∫ ∞0
1
2πe−
r2
2 r dr dθ
=
∫ ∞0
e−r2
2 r dr
Now letting u = r2
2 so that du = r dr∫ ∞0
e−r2
2 r dr =
∫ ∞0
e−u du =[−e−u
]u=∞u=0
=[−e−∞ − (−e−0)
]= 0− (−1) = 1
![Page 9: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/9.jpg)
Properties of the Normal DistributionI The sum of two independent Normals is Normal. If
X ∼ N(µ1, σ21), Y ∼ N(µ2, σ
22), and Z = X + Y , then
Z ∼ N(µ1 + µ2, σ21 + σ22)
I The sum of two dependent Normals isn’t always Normal.Consider the following example.
X = N(0, 1)
Y =
{X w.p. 1
2
−X w.p. 12
They are both Normal but X + Y is not Normal.
I A Normal multiplied by a constant is Normal. IfX ∼ N(µ, σ2) and Y = aX , then
Y ∼ N(a · µ, a2 · σ2)
![Page 10: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/10.jpg)
Scaling to the Standard Normal
I The properties on the previous slide allow us to convert anyNormal into the standard Normal.
I If X ∼ N(µ, σ2), then
Z =X − µσ
is distributed with Z ∼ N(0, 1).
I Intuition: I got SDs on midterm 1.
![Page 11: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/11.jpg)
Question
Suppose male height is distributed as N (70, 5) and female heightis distributed as N (64, 4). What’s the probability that a randomlychosen male is taller than a randomly chosen female? You mayexpress your answer in terms of Φ, the standard normal CDF.
![Page 12: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/12.jpg)
Answer
Express X ∼ N (70, 5) and Y ∼ N (64, 4). We want to findP(X > Y ).
P(X > Y ) = P(X + (−Y ) > 0)
= P(N (70, 5) + (−N (64, 4)) > 0)
= P(N (70, 5) +N (−1 · 64, (−1)2 · 4)) > 0)
= P(N (6, 9) > 0)
= P(N (0, 9) > −6)
= P(N (0, 1) >−6
3)
= P(N (0, 1) < 2)
So the answer is Φ(2) ≈ 0.977.
![Page 13: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/13.jpg)
Joint PDFs
We know that multiple discrete RVs have a joint PMF. Similarly,multiple continuous RVs have a joint PDF.
I Discrete PMF: pX ,Y (x , y)
P((X ,Y ) ∈ A) =∑
(x ,y)∈A
pX ,Y (x , y)
I Continuous PDF: fX ,Y (x , y)
P((X ,Y ) ∈ A) =
∫AfX ,Y (x , y)dxdy
I Still needs to be non-negative.
I Still needs to integrate to 1.
![Page 14: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/14.jpg)
Joint CDFs
I Both discrete and continuous RVs have the same definitionfor CDFs and joint CDFs:
FX (x) = P(X ≤ x)
FX ,Y (x , y) = P(X ≤ x ,Y ≤ y)
I Continuous RVs:
fX (x) =d
dxFX (x)
fX ,Y (x , y) =∂2
∂x∂yFX ,Y (x , y)
![Page 15: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/15.jpg)
Marginal Probability Density
I Discrete
pX (x) =∑y∈Y
pX ,Y (x , y)
I Continuous
fX (x) =
∫ ∞−∞
fX ,Y (x , y) dy
I fX (x) is still a density, not a probability.
![Page 16: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/16.jpg)
Conditional Probability Density
I Discrete
pX |Y (x | y) =pX ,Y (x , y)
pY (y)
I Continuous
fX |Y (x | y) =fX ,Y (x , y)
fY (y)
I By definition, Multiplication Rule still holds.
![Page 17: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/17.jpg)
Independence
Similar to discrete, 3 equivalent definitions.
I For all x and y ,
fX ,Y (x , y) = fX (x)fY (y)
I For all x and y ,
fX |Y (x | y) = fX (x)
I For all x and y ,
fY |X (y | x) = fY (y)
![Page 18: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/18.jpg)
Bayes RuleI Discrete (simple form)
pX |Y (x | y) =pY |X (y | x)pX (x)
pY (y)
I Discrete (extended form)
pX |Y (x | y) =pY |X (y | x)pX (x)∑
x ′∈X pY |X (y | x ′)pX (x ′)
I Continuous (simple form)
fX |Y (x | y) =fY |X (y | x)fX (x)
fY (y)
I Continuous (extended form)
fX |Y (x | y) =fY |X (y | x)fX (x)∫∞
−∞ fY |X (y | t)fX (t) dt
![Page 19: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/19.jpg)
Conditional Expectation
I Discrete
E [Y | X = x ] =∑y∈Y
y · pY |X (y | x)
I Continuous
E [Y | X = x ] =
∫ ∞−∞
y · fY |X (y | x) dy
![Page 20: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/20.jpg)
Combining Discrete and Continuous RVs
I You can also have discrete and continuous RVs defined jointly.
I Ex. let X be the outcome of a dice roll and Y be Exp(X ).
∀x ∈ {1, · · · , 6}, pX (x) =1
6fY |X (y | x) = xe−xy
![Page 21: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/21.jpg)
Change of Variables / Derived Distributions
I Let X be a discrete RV with P(X = 0) = 12 , P(X = 1) = 1
2 ,and Y = 2X . Write down the distribution of Y .
I Change of variables in PMF:
P(Y = y) = P(2X = y) = P(X =y
2)
I In general, if X is a discrete RV, Y = f (X ),
P(Y = y) = P(f (X ) = y) = P(X ∈ f −1(y))
![Page 22: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/22.jpg)
Change of Variables / Derived DistributionsI Let X ∼ U[0, 1], and Y = 2X . Then is it true that
fY (y) = P(Y = y) = P(2X = y) = P(X =y
2) = fX (
y
2)
I No, this won’t integrate to 1.
∀x ∈ [0, 1], fX (x) = 1∫ 2
0fY (y)dy =
∫ 2
0fX (
y
2)dy = 2
I You have to use the CDF.
FY (y) = P(Y ≤ y) = P(2X ≤ y) = P(X ≤ y
2) = FX (
y
2)
I Taking the derivative on both sides,
fY (y) =d
dyFX (
y
2) =
1
2· fX (
y
2).
![Page 23: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/23.jpg)
Change of Variables / Derived Distributions
I Let X ∼ U[0, 1], and Y = 2X . Then is it true that
fY (y) = P(Y = y) = P(2X = y) = P(X =y
2) = fX (
y
2)
I Note: PDF does not satisfy fY (y) = P(Y = y).
fY (y)dy = P(y ≤ Y ≤ y + dy)
= P(y ≤ 2X ≤ y + dy)
= P(y
2≤ X ≤ y
2+
dy
2)
= fX (y
2) · dy
2
This implies fY (y) = 12 · fX ( y2 ).
![Page 24: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/24.jpg)
Change of Variables / Derived Distributions
I In general, if X is continuous RV, Y = g(X ). Then we have
FY (y) = P(Y ≤ y) = P(g(X ) ≤ y).
I When g is monotone increasing,
P(g(X ) ≤ y) = P(X ≤ g−1(y)) = FX (g−1(y)) (1)
I When g is monotone decreasing,
P(g(X ) ≤ y) = P(X ≥ g−1(y)) = 1− FX (g−1(y)) (2)
![Page 25: Normal Distribution, Continuous Analogs, Derived Distributionsee126/sp20/slides/EECS_126_Spring_… · Normal Distribution, Continuous Analogs, Derived Distributions EECS 126 Spring](https://reader033.fdocuments.us/reader033/viewer/2022053101/605f5d196b5093218867ddce/html5/thumbnails/25.jpg)
References
Introduction to probability. DP Bertsekas, JN Tsitsiklis - 2002