Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable...
Transcript of Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable...
![Page 1: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/1.jpg)
Chapter 2. Continuous random variables
![Page 2: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/2.jpg)
Outline
Review of probability: events and probability
Random variable
Probability and Cumulative distribution function
Review of discrete random variable
Introduction to continuous random variable
Expected values
Variance and standard deviation
Variance for 2-dim random variable
Quantiles and Cumulative distribution function
Standard continuous univariate distributionsUniformExponentialNormal
![Page 3: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/3.jpg)
Review of Probability
Probability considers an experiment before it is performed.Probability is a measure of the chance that an event may occur inthe experiment. Tossing a coin or conducting an election survey isan example of an experiment.An event is a subset of the sample space, the set of all possibleoutcomes. Seeing tail in coin or a positive response from thesurvey is an event.The legitimate questions are then: What is the probability ofseeing tail in the experiment of tossing a coin, or what is theprobability of getting a positive response in a survey?
![Page 4: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/4.jpg)
The Axioms of Probability
Mathematically, probability is a function P which assigns to eachevent A in the sample space Ω a number P(A) in [0, 1] such that
Axiom 1: P(A) ≥ 0 for all A ⊆ Ω;
Axiom 2: P(Ω) = 1;
Axiom 3: P(A ∪ B) = P(A) + P(B) if A ∩ B = ∅ for anyA,B ⊆ Ω.
When an event has an associated probability to occur, this givesrise to uncertainty.Uncertainty principle is the fundamental of statistical inference.
![Page 5: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/5.jpg)
An example of random variable
A random variable X is a function from sample space Ω to realnumbers R.
Exercise 2.2.1
(a) Experiment 1: In a presential election with two candidates M
and O, the possible outcomes are Ω = Candidate M wins,Candidate O wins. If we define a random variable X thatmaps from Ω to 0, 1. Then the probability of the eventCandidate O wins is equivalent to P(X = 1).
(b) Experiment 2: A national air quality monitoring systemautomatically collects measurements of ozone level atdesignated sites. The possible outcomes are Ω = x : x ≥ 0.If we define a random variable X to be the numericalmeasurements, the probability that ozone level falls below acertain level c is given by P(X ≤ c).
![Page 6: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/6.jpg)
Random variable
A random variable X is a function that associates a uniquenumber with each possible outcome of an experiment.
Associated with each random variable X is a probabilitydistribution function that describes the chance of all possibleoutcomes of X .
Often in scientific investigation, X represents the variable ofmain interest that can be measured or observed.
All observable events are expressed in terms of a randomvariable.
![Page 7: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/7.jpg)
Distribution function
In order to describe all possible outcomes of an experiment, wefocus on an event of the basic form
X ≤ x
for fixed x , where x can take any value. How how can one expressa general event a < X ≤ b using the basic form with setoperations?
a < X ≤ b = X ≤ b ∩ X ≤ ac .
If we have a rule of assigning probability to an event of the basicform, then the probability of any event can be determined.
![Page 8: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/8.jpg)
Cumulative Distribution function
For any univariate randomvariable X , cumulative
distribution function, c.d.f.,FX : R → [0, 1], is defined by
FX (x) ≡ F (x) = P(X ≤ x) .
Moreover, F (−∞) = 0 andF (∞) = 1 and F is anon-decreasing function. Anexample is drawn in Figure 1.Note that for a ≤ b,
P(a < X ≤ b) = F (b) − F (a) .
Why is it so?
0
1
F (a)
F (b)
a bx
Cumulative distribution function
Figure: Example of cumulative distributionfunction.
![Page 9: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/9.jpg)
Discrete random variable
Discrete random variables are probability models describing theoutcomes of experiments with countable sample space, oftenassigning only the integers or the non-negative integers.Examples include
college membership, exam grades(A,B,C,D,E), number ofgoals in a match, number of children in a family.
![Page 10: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/10.jpg)
Discrete random variable
Discrete random variables are probability models describing theoutcomes of experiments with countable sample space, oftenassigning only the integers or the non-negative integers.Examples include
college membership, exam grades(A,B,C,D,E), number ofgoals in a match, number of children in a family.
The probability distribution function FX (x) = P(X ≤ x) can beobtained by
P(X ≤ x) = P(X ≤ int(x)) =
int(x)∑
k=−∞
P(X = x) ,
where int(x) denotes the largest integer smaller than or equal to x ,e.g. int(5.2) = 5, int(3) = 3, int(-2.1) = -2.
![Page 11: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/11.jpg)
Probability mass function
The probability distribution of a discrete random variable X ischaracterised by probability mass function, p.m.f, p(x), where
p(x) = P(X = x) .
The probability mass function p(x) satisfies
0 ≤ p(x) ≤ 1 for all x ;
∑
∞
x=−∞p(x) = 1
For any event A, P(X ∈ A) =∑
x∈A p(x).For example,
P(a < X ≤ b) = P(X = a + 1) + P(X = a + 2) + · · · + P(X = b)
= p(a + 1) + p(a + 2) + · · · + p(b)
![Page 12: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/12.jpg)
00.
10.
20.
3
Probability mass function
P(a < X ≤ b)
a b
x
Figure: Example of probability mass function. Shaded area representsP(a < X ≤ b).
![Page 13: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/13.jpg)
Example of discrete random variable
Exercise 2.4.1For a random variable X that takes values 0,1 with probabilitiesθ, 1 − θ, obtain P(X ≤ x) for all x ≥ 0 and plot the cumulativedistribution function.
![Page 14: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/14.jpg)
P(X ≤ x) =
0 if x < 0θ if 0 ≤ x < 11 if x ≥ 1
![Page 15: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/15.jpg)
Continuous random variableWhen the outcome of an experiment is a measurement on acontinuous scale, such as ozone level measurements in the earlierexample, the random variable is called continuous random
variable. Examples include
![Page 16: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/16.jpg)
Continuous random variableWhen the outcome of an experiment is a measurement on acontinuous scale, such as ozone level measurements in the earlierexample, the random variable is called continuous random
variable. Examples include
height, weight, direction, waiting times in the hospital, priceof stock
![Page 17: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/17.jpg)
Continuous random variableWhen the outcome of an experiment is a measurement on acontinuous scale, such as ozone level measurements in the earlierexample, the random variable is called continuous random
variable. Examples include
height, weight, direction, waiting times in the hospital, priceof stock
Again, the cumulative distribution function is defined by
F (x) = FX (x) = P(X ≤ x) .
However, if X is continuous random variable
P(X = x) = 0 for all x .
and hence
P(a < X < b) = P(a ≤ X < b) = P(a < X ≤ b) = P(a ≤ X ≤ b) .
![Page 18: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/18.jpg)
Probability density function
The probability density function, p.d.f. f (x) a continuousrandom variable X is defined by
f (x) =d
dxFX (x)
so that it satisfies
FX (x) =
∫ x
−∞
f (u) du .
The probability density function f (x) satisfies
f (x) ≥ 0 for all x ;
∫
∞
−∞f (x) dx = 1.
For any event A, P(X ∈ A) =∫
x∈Af (x) dx .
![Page 19: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/19.jpg)
Interpretation of probability density function
Due to results from calculus,
P(a < X ≤ b) = FX (b) − FX (a)
=
∫ b
−∞
f (x) dx −∫ a
−∞
f (x) dx
=
∫ b
a
f (x) dx
![Page 20: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/20.jpg)
Probability density function
P(a < X ≤ b)
a bx
f(x
)
Figure: Example of probability density function. P(a < X ≤ b) is thearea under the curve between x = a and x = b.
![Page 21: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/21.jpg)
Exercise 2.5.1For a random variable X with cumulative distribution function
FX (x) =
x if 0 ≤ x ≤ 10 otherwise
(a) Find P(0.3 < X ≤ 0.5).
(b) Find the p.d.f of X .
![Page 22: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/22.jpg)
(a) P(0.3 < X ≤ 0.5) = F (0.5) − F (0.3) = 0.5 − 0.3 = 0.2
(b) f (x) =
1 if 0 ≤ x ≤ 10 if x < 0 or x > 1
![Page 23: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/23.jpg)
Expectation
If X is a discrete random variable with probability massfunction p(x) on 0, 1, · · · , then the expected value
of X is
µX = E[X ] =∞∑
x=0
xp(x) .
If X is a continuous random variable with probabilitydensity function f (x) on (−∞,∞), then the expected
value of X is
µX = E[X ] =
∫
∞
−∞
xf (x) dx .
![Page 24: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/24.jpg)
Expectations of functions of random variables
Suppose Y = g(X ) where g is a fixed function.
If X is a discrete random variable with probability massfunction p(x) on 0, 1, · · · , then the expected value
of Y is
µY = E[Y ] =
∞∑
x=0
g(x)p(x) .
If X is a continuous random variable with probabilitydensity function f (x) on (−∞,∞), then the expected
value of Y is
µY = E[Y ] =
∫
∞
−∞
g(x)f (x) dx .
![Page 25: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/25.jpg)
Exercise 2.6.1Let f (x) = exp(−x) for all x ≥ 0. Find (i)E[X ] (= µX ) (ii)E[X 2]and (iii)E[(X − µX )2]
![Page 26: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/26.jpg)
(i) µX = E[X ] =
∫
∞
0x exp(−x) dx
= [−x exp(−x)]∞0 +
∫
∞
0exp(−x) dx
= 0 + [− exp(−x)]∞0 = 0 − (−1) = 1
(ii) E[X 2] =
∫
∞
0x2 exp(−x) dx
= [−x2 exp(−x)]∞0 +
∫
∞
02x exp(−x) dx
= 0 + 2 × 1 = 2
(iii) E[(X − µX )2] =
∫
∞
0(x − 1)2 exp(−x) dx
=
∫
∞
0(x2 − 2x + 1) exp(−x) dx
= 2 − 2 × 1 + 1 = 1
![Page 27: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/27.jpg)
Properties of Expected values
TheoremIf X has expectation E[X ] and Y is a linear function of X as
Y = aX + b then Y has expectation
E[Y ] = aE[X ] + b .
![Page 28: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/28.jpg)
Properties of Expected values
TheoremIf X has expectation E[X ] and Y is a linear function of X as
Y = aX + b then Y has expectation
E[Y ] = aE[X ] + b .
More generally, the following properties hold:
E[g(X ) + h(X )] = E[g(X )] + E[h(X )] (1)
E[cg(X )] = cE[g(X )] (2)
E[aX + b] = aE[X ] + b (3)
Note that we proved them in MATH 104 for discrete randomvariables.
![Page 29: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/29.jpg)
Using linear properties of expectation, we may computeE[(X − a)2] by
E [(X − a)2] = E [X 2 − 2aX + a2]
= E [X 2] − E [2aX ] + E [a2]
= E [X 2] − 2aE [X ] + a2
![Page 30: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/30.jpg)
Variance and Standard Deviation
If X is a random variable with expected valueµX = E[X ], the variance of X is
σ2X = Var[X ] = E[(X − µX )2]
=
∑
∞
x=0(x − µX )2p(x) discrete r.v.∫
∞
−∞(x − µX )2f (x) dx continous r.v.
The variance of X can be calculated as
σ2X = E[X 2] − µ2
X .
The standard deviation of X is
σX =√
Var[X ] .
![Page 31: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/31.jpg)
Variance and Standard Deviation: example
Exercise 2.7.1For f (x) = exp(−x) for all x ≥ 0, Find σX .
![Page 32: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/32.jpg)
From the previous example,σ2X = 1 so σX = 1.
![Page 33: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/33.jpg)
Properties of Variance and Standard Deviation
TheoremIf Var[X ] exists and Y = a + bX , then Var[Y ] = b2Var[X ]. Hence,
the standard deviation of Y is σY = |b|σX .
![Page 34: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/34.jpg)
Properties of Variance and Standard Deviation
TheoremIf Var[X ] exists and Y = a + bX , then Var[Y ] = b2Var[X ]. Hence,
the standard deviation of Y is σY = |b|σX .
Why do you need to take the absolute value in the aboveexpression?
![Page 35: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/35.jpg)
0 1 2 3 4 5
0.0
0.1
0.2
0.3
0.4
y1
0 1 2 3 4 5
0.1
0.3
0 1 2 3 4 5
0.0
0.1
0.2
0.3
0.4
y1
0.1
0.5
1
Probability mass functionProbability mass function
DensityDensity
µ = 2.5
µ = 2.5
σ = 1.1
σ = 1.1
µ = 0.83
µ = 0.83
σ = 0.83
σ = 0.83
x
x
x
x
Figure: Expectations and Standard deviations for discrete and continuousrandom variables.
![Page 36: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/36.jpg)
Correlation
With multivariate data, we need to characterise dependence:The correlation between X and Y , denoted Corr(X ,Y ), anddefined as:
The correlation between two random variables X andY is
Corr(X ,Y ) =E [X − E[X ]Y − E[Y ]]
√
Var(X )Var(Y )
=E[(X − µX )(Y − µY )]
σXσY
![Page 37: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/37.jpg)
..
.
.
....
....
. .....
....
....
.
..
.. . . . .
.. . .
...
.. .
...
..
... . . . ..
..
.
.... .
.
...
. ...
. . .
.. .
........
..
...
.
..
....
...
. .....
......
....
. ...
.. . . . .
.. . .
.
. ..
.
...
..
.. . . ..
..
.
....
.. ...
..
.. . .
.
..
.. .
.
.
...
.
..
.. .
Correlation near 1
..
.
.
....
....
. .....
......
....
...
...
.. . . . .
.. . .
.
. ..
.. .
...
..
... . . . ..
..
.
.... .
.
...
. ...
...
.. . .
... .
........
..
.....
..
..
.
.
.
.
..
..
.cent
re h
ere
spre
ad
centre here
spread
Correlation near 0
loose clustering tight clustering
Figure: Mean and Variance are measures of location and scale;Correlation measures linear association between variables.
Measure of clustering around a straight line with a slope ∈ [−1, 1]Correlation is a scale free measure of linear dependence betweentwo variables.
![Page 38: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/38.jpg)
TheoremIf X and Y are independent, then Corr(X ,Y ) = 0.
![Page 39: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/39.jpg)
Covariance
Recall that the variance of a random variable X is given byVar(X ) = E
[
(X − µX )2]
. The covariance between two randomvariables X and Y is defined in a similar way:
The covariance between random variables X and Y
isCov(X ,Y ) = E [(X − µX )(Y − µY )] , .
so that Var(X ) = Cov(X ,X ) and Cov(X ,Y ) is the expectedproduct of the deviations of each variable from its expected value.
![Page 40: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/40.jpg)
Covariance cont.
Exercise 2.8.1Show that covariance is equivalent to
Cov(X ,Y ) = E[XY ] − µXµY .
![Page 41: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/41.jpg)
Cov(X ,Y ) = E[(X − µX )(Y − µY )]
= E[XY − µXY − µY X + µXµY ]
= E[XY ] − µXE[Y ] − µY E[X ] + µXµY
= E[XY ] − µXµY − µY µX + µXµY
= E[XY ] − µXµY
![Page 42: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/42.jpg)
Cov(X ,Y ) = E[(X − µX )(Y − µY )]
= E[XY − µXY − µY X + µXµY ]
= E[XY ] − µXE[Y ] − µY E[X ] + µXµY
= E[XY ] − µXµY − µY µX + µXµY
= E[XY ] − µXµY
We have the following interpretation:
Cov(X ,Y ) = ρσXσY ,
where Corr(X ,Y ) = ρ, and σX and σY are the square roots of thevariances of X and Y respectively.
![Page 43: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/43.jpg)
Quantiles
Often interest is in the values of a continuous random variablewhich are not exceeded with a given probability, e.g. income oflower 10% income tax payer or score of top 5% students.
Let X be a random variable and p any value such that0 ≤ p ≤ 1. The the pth quantile of the distribution ofX is the value xp that satisfies:
P(X ≤ xp) = p.
When p = 0.5, the quantitle x0.5 is called median.
![Page 44: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/44.jpg)
pth QuantileSee Figure 6 for visualisation.
0
p
1
xpx
F(x
)Cumulative distribution function
Figure: xp is the pth quantile obtained from c.d.f.
![Page 45: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/45.jpg)
QuartilesThe quartiles of a distribution are the values at which we can cutthe distribution into four equally likely slices: (x0.25, x0.5, x0.75).Figure 7 shows quartiles in c.d.f and p.d.f.
x(.25) x(.75)
0.25
0.5
0.75
1
x(.5) x(.75)
0
0.5
Cumulative distribution function Density
f(x
)
F(x
)
xx
Figure: Quartiles (x0.25, x0.5, x0.75) shown on c.d.f. and p.d.f. respectively.
![Page 46: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/46.jpg)
Uniform distribution
This distribution is used to model variables that can take any valueon a fixed interval, when the probability of occurrence does notvary over the interval.
The p.d.f. of a Uniform random variable X , dis-tributed on the interval (a, b) is given by:
f (x ;θ) =
1
b − aif a < x < b;
0 otherwise,
where θ = (a, b) and Θ is the set of (a, b) suchthat −∞ < a < b < ∞. This is written asX ∼ Uniform(a, b).
![Page 47: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/47.jpg)
a x0 b
01/
(b−
a)
P(a < X ≤ x0)
x
Figure: P.d.f. for Uniform(a, b) random variable. Shaded area representsP(a < X ≤ x0).
![Page 48: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/48.jpg)
Cumulative distribution function and quantiles
Exercise 2.10.1For X ∼ Uniform(a, b),
(i) Find the c.d.f. and sketch its graph.
(ii) Find the mean µX and variance σ2X .
(iii) Find the median and compare to the mean
![Page 49: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/49.jpg)
(i) F (x) =∫ x
a1
b−adu = x−a
b−afor a ≤ x ≤ b and 1 for x ≥ b.
![Page 50: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/50.jpg)
(i) F (x) =∫ x
a1
b−adu = x−a
b−afor a ≤ x ≤ b and 1 for x ≥ b.
(ii) µX =∫ b
ax 1
b−adx = a+b
2 and
E[X 2] =∫ b
ax2 1
b−adx = a2+ab+b2
3
So σ2X = E[X 2] − µ2
X = (a−b)2
12 .
![Page 51: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/51.jpg)
(i) F (x) =∫ x
a1
b−adu = x−a
b−afor a ≤ x ≤ b and 1 for x ≥ b.
(ii) µX =∫ b
ax 1
b−adx = a+b
2 and
E[X 2] =∫ b
ax2 1
b−adx = a2+ab+b2
3
So σ2X = E[X 2] − µ2
X = (a−b)2
12 .
(iii) F (x) = x−ab−a
= 0.5 so x0.5 = a + 0.5(b − a) = (a + b)/2,same as the mean.
![Page 52: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/52.jpg)
Exercise 2.10.2Numerically evaluate the p.d.f., c.d.f. and the quantile function ofa Uniform distribution.
dunif(0.5, -2, 2) # p.d.f. Uniform(-2,2) at x=0.5, f(0.5)=0.25
punif(0.3, 0, 1) # c.d.f. of Uniform(0,1) at x=0.3, P(X<3)=0.3
x = seq(-1, 1, length=101)
fx = dunif(x, -1,1)
plot(x, fx)
lines(x, fx)
![Page 53: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/53.jpg)
Exponential distribution
This distribution is often used to model variables that are the timesuntil specific events happen when the events occur at random at agiven rate over time.
The p.d.f. of an Exponential random variable X is
f (x ;θ) =
θ exp(−θx) for x > 0,0 otherwise,
where 0 < x and 0 < θ. This is written asX ∼ Exponential(θ) and θ ∈ Θ = (0,∞).
![Page 54: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/54.jpg)
0 1 2 3 4 5
0.0
0.2
0.4
0.6
0.8
1.0
x
f(x)
P(X>1)
Figure: P.d.f.s for Exponential(θ) random variables for θ = 1.
![Page 55: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/55.jpg)
Shape of Exponential density function
Exercise 2.10.3Old Q: How is the shape of the function related to the parameter
θ? Which one has lower tail probability P(X > 10)?Find P(X > 1) when θ = 1.
![Page 56: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/56.jpg)
P(X > 1) =
∫
∞
1exp(−x) dx
= [− exp(−x)]∞1 = 0 − (− exp(−1))
= exp(−1) = 0.3678
![Page 57: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/57.jpg)
Cumulative distribution function and quantiles
The c.d.f. of the Exponential(θ) distribution is
F (x) =
0 if x ≤ 0 ,1 − exp(−θx) if x > 0 .
Exercise 2.10.4For X ∼ Exponential(θ),
(i) Derive the cumulative distribution from the p.d.f.
(ii) Find the median.
(iii) The mean µX of the Exponential(θ) is 1θ. Compare themedian and the mean. Which one is larger? Why does themedian differ from the mean?
![Page 58: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/58.jpg)
(i)
F (x) =
∫ x
0f (u) du =
∫ x
0θ exp(−θu) du = 1−exp(−θx)
(ii) Solving F (xp) = p gives xp = θ−1 log(1 − p)−1. Formedian, p = 0.5 so the median is x0.5 = 1
θlog 2.
(iii) µX = 1/θ > (1/θ) log 2 = x0.5. As the distribution isnot symmetric, the mean and the median are not thesame. The median is always smaller because there ishigher concentration of probability on smaller valuesbut with thin probability values are stretching to the farright.
![Page 59: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/59.jpg)
Exercise 2.10.5Numerically evaluate the p.d.f., c.d.f. and the quantile function ofan Exponential distribution.
dexp(3, rate=2) # p.d.f. of Exponential(2) at x=3, f(3)=0.004957504
pexp(3, 5) # c.d.f. of Exponential(5) at x=3, P(X<3)=0.9999997
x = seq(0, 4, length=100)
fx = dexp(x, rate=2)
plot(x, fx)
lines(x, fx)
![Page 60: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/60.jpg)
Exercise 2.10.6Suppose that a goal is scored at random in a fixed time of the cupfinal and the time until the event can be modelled by anExponential distribution with rate parameter θ = 2/3 hours. If thefirst goal has been scored just now, what is the probability ofwaiting time until the next goal is
(i) more than 30 minutes
(ii) between 30 and 50 minutes
![Page 61: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/61.jpg)
Let X be the random variable of the waiting time. ThenX ∼ Exponential(2/3) and F (x) = 1 − exp(−(2/3)x).
(i) P(X > 1/2) = 1 − F (1/2) = exp(−2/3 · 1/2) = 0.7165
(ii)
P(1/2 < X < 5/6) = F (5/6) − F (1/2)
= exp(−2/3 · 1/2) − exp(−2/3 · 5/6)= 0.1428
![Page 62: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/62.jpg)
Normal distribution: background
quoted from gqview weblib/Gauss.html
The normal distribution was introduced by the French
mathematician Abraham De Moivre in 1733. De Moivre used this
distribution to approximate probabilities of winning in various
games of chance involving coin tossing.
It was later used by the German mathematician Karl Gauss to
prredict the location of astronomical bodies and became known as
the Gaussian distribution.
In the late nineteenth century statisticians started to believe that
most data sets would have histograms with the Gaussian
bell-shaped form and that all normal data sets would follow this
form and so the curve came to be known as the normal curve.
![Page 63: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/63.jpg)
Normal distribution
The p.d.f. of a Normal random variable X is
f (x ;θ) =1√2πσ
exp
−1
2
(
x − µ
σ
)2
,
where θ = (µ, σ), −∞ < µ < ∞, 0 < σ and −∞ <x < ∞. This is written as X ∼ N(µ, σ2) and θ ∈ Θ =(−∞,∞) × (0,∞).
E[X ] = µ , Var(X ) = σ2
![Page 64: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/64.jpg)
Shape of Normal density function
Exercise 2.10.7How is the shape of the function related to the parameter θ?Which one has higher probability of P(|X | > 3)?
−3 0 3
00.
20.
40.
60.
8
sigma=0.5sigma=1sigma=1.5
x
Figure: P.d.f.s for Normal(µ, σ2) random variables where µ = 0 andσ = 0.5, 1, 1.5.
![Page 65: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/65.jpg)
The larger σ, the more spread. So σ = 1.5 has the largestprobability of P(|X | > 3) and σ = 0.5 has the smallest.
![Page 66: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/66.jpg)
Cumulative distribution function and quantiles
The normal c.d.f. is
F (x) =
∫ x
−∞
f (u) du =
∫ x
−∞
1√2πσ2
exp
− 1
2
(u − µ
σ
)2
, du .
This does not have a closed form expression so numericalevaluation is required, if we want to obtain probabilities of theform P(X ≤ x) or quantiles.
![Page 67: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/67.jpg)
Exercise 2.10.8Numerically evaluate the p.d.f., c.d.f. and the quantile function ofa Normal distribution.
pnorm(0, mean=2, sd=sqrt(5)) # P(X<0) when X ~ N(2,5)
pnorm(0, 2, sqrt(5)) # P(X<0) when X ~ N(2,5) 0.1855467
1-pnorm(-2, 0, 2) # P(X >-2) when X ~ N(0,4) 0.8413447
qnorm(0.975, 0, 1) # u such that P(X<u)=0.975, 1.959964
Note that the R functions for the Normal distribution use thestandard deviation σ, not the variance σ2.
![Page 68: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/68.jpg)
Exercise 2.10.9A normal distribution is proposed to model the variation in heightof women with parameters µ = 160 and σ2 = 25 measured in cm.Find the proportion of tall women, defined as over 175cm tall.
![Page 69: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/69.jpg)
Let X be the random variable of women’ height then X ∼Normal(160, 52). So
P(X > 175) =
∫
∞
175
1√2π · 5
exp
− 1
2
(x − 160
5
)2
dx
![Page 70: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/70.jpg)
Let X be the random variable of women’ height then X ∼Normal(160, 52). So
P(X > 175) =
∫
∞
175
1√2π · 5
exp
− 1
2
(x − 160
5
)2
dx
In the above example we have expressed the proportion in terms ofan integral and as the number of deviations from the mean.The integral is impossible to calculate analytically so numericalevaluation is required to obtain probabilities or quantiles.
![Page 71: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/71.jpg)
Standardardization of the random variable
It is useful to express such probabilities in terms of a standardizedrandom variable, with µ = 0 and σ = 1.
If X ∼ N(µ, σ2) then
Z =X − µ
σ∼ N(0, 1),
and conversely if Z ∼ N(0, 1), then
X = µ + σZ ∼ N(µ, σ2) .
![Page 72: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/72.jpg)
Standardardization of the random variable
It is useful to express such probabilities in terms of a standardizedrandom variable, with µ = 0 and σ = 1.
If X ∼ N(µ, σ2) then
Z =X − µ
σ∼ N(0, 1),
and conversely if Z ∼ N(0, 1), then
X = µ + σZ ∼ N(µ, σ2) .
The formal proof will be given in M230 and here it is sufficient tonote that
E[Z ] = 0 Var[Z ] = 1 .
![Page 73: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/73.jpg)
Standard normal distribution
A random variable Z is said to have a standard normal
distribution with mean 0 and standard deviation 1 if its p.d.f. isgiven by
f (z) =1√2π
exp(−z2/2) ,
where −∞ < z < ∞ and is denoted by Z ∼ N(0,1).The cumulative distribution function, i.e. the area under the curve,of the standard normal variable Z is given by
Φ(z) = P(Z ≤ z) =
∫ z
−∞
1√2π
exp(−x2/2) dx .
Values of Φ(z) are obtained from a table of standard normalprobabilities or from computer software such as R: pnorm(z)
z -3.00 -2.33 -1.67 -1.00 -0.33 0.33 1.00
Φ(z) 0.0013 0.0098 0.0478 0.1587 0.3694 0.6306 0.8413
![Page 74: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/74.jpg)
Standard normal distribution: example
Exercise 2.10.10We repeat the previous example to illustrate the standardizationprocedure:
P(X > 175) = P(X − 160
5>
175 − 160
5)
= P(Z > 3) = 1 − P(Z ≤ 3)
= 1 − Φ(3)
= 1 − 0.9987 = 0.0013
![Page 75: Chapter 2. Continuous random variablesparkj1/math105/mainslide_ch2.pdf · Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction](https://reader030.fdocuments.us/reader030/viewer/2022040409/5ec6adc938387536650b8389/html5/thumbnails/75.jpg)
Probabilities of Normal distribution
µµ − σ µ + σµ − 2σ µ + 2σµ − 3σ µ + 3σP(µ − σ < X < µ + σ) = 0.683
P(µ − 2σ < X < µ + 2σ) = 0.954
P(µ − 3σ < X < µ + 3σ) = 0.997
Figure: Illustration of coverage probability of Normal(µ, σ2) distribution.