Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Prbability Theory and Mathematical StatisticsLecture 05: Probability Distributions and
Probability Densities IIMultivariate Probabilities
Chih-Yuan Hung
School of Economics and ManagementDongguan University of Technology
March 27, 2019
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Introduction
In the one r.v. case, we know that many different randomvariables can be defined over one and the same sample space.
Rolling two dice, we can consider many different r.v.s
Consider the following example
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example
Two caplets are selected at random from a bottle containing 3aspirin, 2 sedative, and 4 laxative caplets. If X and Y are,respectively, the numbers of aspirin and sedative caplets includedamong the 2 caplets drawn from the bottle, find the probabilitiesassociated with all possible pairs of values of X and Y .
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
It is preferable to express the probabilities by means of a functionwith the values f (x , y) = P(X = x ,Y = y) for any pair of valuesof (x , y) within the range of X and Y .The example above, we can have
f (x .y) =(3x)(
2y)(
42−x−y)
(92)for x = 0, 1, 2; y = 0, 1, 2; 0 ≤ x+ y ≤ 2
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Joint Probability Distribution
Definition
If X and Y are discrete random variables, the function given byf (x , y) = P(X = x ,Y = y) for each pair of values (x , y) withinthe range of X and Y is called the joint probability distributionof X and Y .
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Joint Probability Distribution
Theorem (7)
A bivariate function can serve as the joint probability distributionof a pair of discrete random variables X and Y if and only if itsvalues, f (x , y), satisfy the conditions
1 f (x , y) ≥ 0 for each pair of values (x , y) within its domain;
2 ∑x
∑yf (x , y) = 1, where the double summation extends over all
possible pairs (x , y) within its domain.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example 13
Determine the value of k for which the function given by
f (x , y) = kxy , for x = 1, 2, 3; y = 1, 2, 3
can serve as a joint probability distribution.Solution: Substituting the various values of x and y , we get
yf (x , y) 1 2 3
x1 k 2k 3k2 2k 4k 6k3 3k 6k 9k
To satisfy the first condition of Theorem 7, the constant k must benonnegativeTo satisfy the second condition,
k + 2× 2k + 2× 3k + 4k + 2× 6k + 9k = 1,
which is k = 136 .
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
If we want to know that the probability that the values of tworandom variables are less than or equal to some real numbers xand y .
Definition (Joint Distribution Function)
If X and Y are discrete random variables, the function given by
F (x , y) = P(X ≤ x ,Y ≤ y) = ∑s≤x
∑t≤y
f (s, t)
for −∞ < x < ∞;−∞ < y < ∞
where f (s, t) is the value of the joint probability distribution of Xand Y at (s, t), is called the joint distribution function, or thejoint cumulative distribution of X and Y .
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Exercise 48 (c)
Theorem
If F (x , y) is the value of the joint distribution function of twodiscrete random variables X and Y at (x, y), show that If a < band c < d , then F (a, c) ≤ F (b, d).
Proof.
By definition,
F (a, c) = ∑x≤a
∑y≤c
f (x , y)
F (b, d) = ∑x≤b
∑y≤d
f (x , y)
When a < b and c < d , we have
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Proof.
F (b, d) = ∑x≤b
∑y≤d
f (x , y)
= ∑x≤b
(∑y≤c
f (x , y) + ∑c<y≤d
f (x , y)
)= ∑
x≤b∑y≤c
f (x , y) + ∑x≤b
∑c<y≤d
f (x , y)
= ∑x≤a
∑y≤d
f (x , y) + ∑a<x≤b
∑y≤d
f (x , y) + ∑x≤b
∑c<y≤d
f (x , y)
=F (a, c) + some summation of f (x , y)
Since f (x , y) ≥ 0 for all x in the range of X and y in the range ofY , we have
F (a, c) ≤ F (b, d)
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Multivariate Distributions
With reference to Example 12,
F (1, 1) =P(x ≤ 1, y ≤ 1)
=f (0, 0) + f (0, 1) + f (1, 0) + f (1, 1)
=1
6+
2
9+
1
3+
1
6
=8
9
Note that, the joint distribution is defined for all real numbers.
Using the same example,
F (−2, 1) = P(X ≤ −2,Y ≤ 1) = 0
andF (3.7, 4.5) = P(X ≤ 3.7,Y ≤ 4.5) = 1.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Continuous Cases
Definition
A bivariate function with values f (x , y) defined over the xy -planeis called a joint probability density function of the continuousrandom variables X and Y if and only if
P(X ,Y ) ∈ A =∫∫A
f (x , y)dxdy
for any given A in the xy -plane
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Theorem (8)
A bivariate function can serve as a joint probability densityfunction of a pair of continuous random variables X and Y if itsvalues, f (x , y), satisfy the conditions
1. f (x , y) ≥ 0 for −∞ < x < ∞ −∞ < y < ∞
2.∫ ∞
−∞
∫ ∞
−∞f (x , y)dxdy = 1
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (15)
Given the joint probability density function
f (x , y) =
{35x(y + x) for 0 < x < 1, 0 < y < 2
0 elsewhere
of two random variables X and Y , find P [(X ,Y ) ∈ A], where A isthe region {(x , y)|0 < x < 1
2 , 1 < y < 2}
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
P [(X ,Y ) ∈ A] =P
(0 < x <
1
2, 1 < y < 2
)=∫ 2
1
∫ 12
0
3
5x(y + x)dxdy
=3
5
∫ 2
1
(x2y
2+
x3
3
)∣∣∣∣ 120
dy
=3
5
∫ 2
1
(y
8+
1
24
)dy
=3
5
(y2
16+
y
24
)∣∣∣∣21
=11
80
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Joint Distribution Function
Definition (9)
If X and Y are continuous random variables, the function given by
F (x , y) = P(X ≤ x ,Y ≤ y) =∫ y
−∞
∫ x
−∞f (s, t)dxdy
for −∞ < x < ∞; −∞ < y < ∞where f (s, t) is the jointprobability density of X and Y at (s, t), is called the jointdistribution function of X and Y .
Note that partial differentiation in Definition 9 leads to
f (x , y) =∂2
∂x∂yF (x , y)
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (16)
If the joint probability density of X and Y is given by
f (x , y) =
{x + y for 0 < x < 1, 0 < y < 1
0 elsewhere
find the joint distribution function of these two random variables.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
The region of A ∈ {(x , y)|0 < x < 1, 0 < y < 1} can be dividedby four parts
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
For 0 < x < 1 and 0 < y < 1, we get
F (x , y) =∫ y
0
∫ x
0s + tdsdt =
1
2xy(x + y)
for x > 1 and 0 < y < 1, we get
F (x , y) =∫ y
0
∫ 1
0s + tdsdt =
1
2y(y + 1)
for 0 < x < 1 and y > 1, we get
F (x , y) =∫ 1
0
∫ x
0s + tdsdt =
1
2x(x + 1)
and for x > 1 and y > 1, we get
F (x , y) = 1
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
Summarize these,
F (x , y) =
0 for x ≤ 0ory ≤ 012xy(x + y) for 0 < x < 1, 0 < y < 112y(1 + y) for x ≥ 1, 0 < y < 112x(1 + x) for 0 < x < 1, y ≥ 1
1 for x ≥ 1, y ≥ 1
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (17)
Find the joint probability density of the two random variables Xand Y whose joint distribution function is given by
F (x , y) =
{(1− e−x )(1− e−y ) for x > 0, y > 0
0 elsewhere
Also use the joint probability density to determineP(1 < X < 3, 1 < Y < 2).
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
Since partial differentiation yields
∂2
∂x∂yF (x , y) = e−(x+y )
for x > 0 and y > 0 and 0 elsewhere, we find the joint probabilitydensity of X an Y is given by
f (x , y) =
{e−(x+y ) for x > 0 and y > 0
0 elsewhere
Thus, the integration yields∫ 3
1
∫ 2
1e−(x+y )dxdy =(e−1 − e−3)(e−1 − e−2)
=e−2 − e−3 − e−4 + e−5
=0.074
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Geometric Presentation of pdf and CDF
We are finding the volume under the surface of probability density
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Generalization of Multivariate Probability
The values of the joint probability distribution of n discrete randomvariables X1,X2, ..., and Xn are given by
f (x1, x2, ..., xn) = P(X1 = x1,X2 = x2, ...,Xn = xn)
And the distribution function is given by
F (x1, x2, ..., xn) = P(X1 ≤ x1,X2 ≤ x2, ...,Xn ≤ xn)
for −∞ < x1 < ∞,−∞ < x2 < ∞, ...,−∞ < xn < ∞
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (18)
If the joint probability distribution of three discrete randomvariables X, Y, and Z is given by
f (x , y , z) =(x + y)z
63for x = 1, 2; y = 1, 2, 3; z = 1, 2
find P(X = 2,Y + Z ≤ 3)
Solution
P(X = 2,Y + Z ≤ 3) =f (2, 1, 1) + f (2, 1, 2) + f (2, 2, 1)
=3
63+
6
63+
4
63
=13
63
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Generalization of Multivariate Density
The joint distribution function is given by
F (x1, x2, ..., xn) =∫ xn
−∞...∫ x2
−∞
∫ x1
−∞f (t1, t2, ..., tn)dt1dt2...dtn
for −∞ < x1 < ∞,−∞ < x2 < ∞, ...,−∞ < xn < ∞. Also thedensity function is given by
f (x1, x2, ..., xn) =∂n
∂x1∂x2...∂xnF (x1, x2, ..., xn)
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example
If the trivariate probability density of X1,X2, and X3 is given by
f (x1, x2, x3) =
{(x1 + x2)e−x3 for 0 < x1 < 1, 0 < x2 < 1, x3 > 0
0 elsewhere
find P [(x1, x2, x3) ∈ A], where A is the regin
{(x1, x2, x3)|0 < x1 <1
2,
1
2< x2 < 1, x3 < 1}
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
P [(x1, x2, x3) ∈ A] =P
(0 < x1 <
1
2,
1
2< x2 < 1, x3 < 1
)=∫ 1
0
∫ 1
12
∫ 12
0(x1 + x2)e
−x3dx1dx2dx3
=∫ 1
0
∫ 1
12
(1
8+
x22)e−x3dx2dx3
=∫ 1
0
1
4e−x3dx3
=1
4(1− e−1)
=0.158
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Marginal Distributions
Consider the following example:
Example (20)
In Example 12 we derived the joint probability distribution of tworandom variables X and Y , the number of aspirin caplets and thenumber of sedative caplets included among two caplets drawn atrandom from a bottle containing three aspirin, two sedative, andfour laxative caplets.Find the probability distribution of X alone and that of Y alone.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
The results of Example 12 are shown in the following table,together with the marginal totals, that is, the totals of therespective rows and columns:
xf (x , y) 0 1 2
y
0 16
13
112
712
1 29
16
728
2 136
136
512
12
112
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Marginal Distribution
Definition
If X and Y are discrete random variables and f (x , y) is the valueof their joint probability distribution at (x , y), the function given by
g(x) = ∑y
f (x , y)
for each x within the range of X is called the marginaldistribution of X. Correspondingly, the function given by
h(y) = ∑x
f (x , y)
for each y within the range of Y is called the marginaldistribution of Y.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Marginal Density
Definition
If X and Y are continuous random variables and f (x , y) is thevalue of their joint probability density at (x , y), the function givenby
g(x) =∫ ∞
−∞f (x , y)dy
is called the marginal density of X. Correspondingly, the functiongiven by
h(y) =∫ ∞
−∞f (x , y)dx
is called the marginal density of Y.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (21)
Given the joint probability density
f (x , y) =
{23 (x + 2y) for 0 < x < 1, 0 < y < 1
0 elsewhere
find the marginal densities of X and Y .
Solution
g(x) =∫ ∞
−∞f (x , y)dy =
∫ 1
0
2
3(x + 2y)dy =
2
3(x + 1)
for 0 < x < 1 and g(x) = 0 elsewhere. Likewise,
h(y) =∫ ∞
−∞f (x , y)dx =
∫ 1
0
2
3(x + 2y)dx =
1
3(1 + 4y)
for 0 < x < 1 and g(x) = 0 elsewhere.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Joint Marginal Distribution
When we are dealing with more than two random variables, we canspeak of the joint marginal distributions of several of therandom variables.
Example (22)
Considering again the trivariate probability density of Example 19,
f (x1, x2, x3) =
(x1 + x2)e−x3 for 0 < x1 < 1, 0 < x2 < 1,
x3 > 0
0 elsewhere
find the joint marginal density of X1 and X3 and the marginaldensity of X1 alone.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
Performing the necessary integration, we find that the jointmarginal density of X1 and X3 is given by
m(x1, x3) =∫ 1
0(x1 + x2)e
−x3dx2 =(x1 +
1
2
)e−x3
for 0 < x1 < 1 and x3 > 0 and m(x1, x3) = 0 elsewhere. Usingthis result, we find that the marginal density of X1 alone is given by
g(x1) =∫ ∞
0
∫ 1
0f (x1, x2, x3)dx2dx3
=∫ ∞
0m(x1, x3)dx3
=∫ ∞
0
(x1 +
1
2
)e−x3dx3 = x1 +
1
2
for 0 < x1 < 1 and g(x1) = 0 elsewhere.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Conditional Distributions
Review of conditional probability. For Event A and B in samplespace S ,
P(A|B) = P(A∩ B)
P(B)
Suppose A and B are X = x and Y = y ,
P(X = x |Y = y) =P(X = x ,Y = y)
P(Y = y)=
f (x , y)
h(y)
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Definition
Conditional Distribution If f (x , y) is the value of the jointprobability distribution of the discrete random variables X and Yat (x , y) and h(y) is the value of the marginal distribution of Y aty , the function given by
f (x |y) = f (x , y)
h(y)h(y) 6= 0
for each x within the range of X is called the conditionaldistribution of X given Y = y. Correspondingly, if g(x) is thevalue of the marginal distribution of X at x , the function given by
w(y |x) = f (x , y)
g(x)g(x) 6= 0
for each y within the range of Y is called the conditionaldistribution of Y given X = x.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (23)
With reference to Examples 12 and 20, find the conditionaldistribution of X given Y = 1.
Solution
Substituting the appropriate values from the table in Example 20,we get
f (0|1) =29718
=4
7
f (1|1) =16718
=3
7
f (2|1) = 0718
= 0
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Definition
Conditional Density If f (x , y) is the value of the joint density ofthe continuous random variables X and Y at (x , y) and h(y) isthe value of the marginal density of Y at y , the function given by
f (x |y) = f (x , y)
h(y)h(y) 6= 0
for −∞ < x < ∞, is called the conditional density of X given Y= y. Correspondingly, if g(x) is the value of the marginal densityof X at x , the function given by
w(y |x) = f (x , y)
g(x)g(x) 6= 0
for −∞ < y < ∞, is called the conditional density of Y given X= x.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (24)
With reference to Example 21, find the conditional density of Xgiven Y = y , and use it to evaluate P(X ≤ 1
2 |Y = 12).
Solution
Using the results obtained on the previous page, we have
f (x |y) = f (x , y)
h(y)=
23 (x + 2y)13 (1 + 4y)
=2x + 4y
1 + 4y
for 0 < x < 1 and f (x |y) = 0 elsewhere.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
Now,
f
(x
∣∣∣∣12)=
2x + 4 · 121 + 4 · 12
=2x + 2
3
Thus,
P
(X ≤ 1
2
∣∣∣∣Y =1
2
)=∫ 1
2
0
2x + 2
3dx
=5
12
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Geometric Presentation
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (25)
Given the joint probability density
f (x , y) =
{4xy for 0 < x < 1, 0 < y < 1,
0 elsewhere
find the marginal densities of X and Y and the conditional densityof X given Y = y .
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
Performing the necessary integrations, we ge
g(x) =∫ ∞
−∞f (x , y)dy =
∫ 1
04xydy
=2xy2∣∣∣∣1y=0
= 2x
for 0 < x < 1, and g(x) = 0 elsewhere; also
h(y) =∫ ∞
−∞f (x , y)dx =
∫ 1
04xydx
=2x2y
∣∣∣∣1x=0
= 2y
for 0 < y < 1, and h(y) = 0 elsewhere.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
Then, substituting into the formula for a conditional density, we get
f (x |y) = f (x , y)
h(y)=
4xy
2y= 2x
for 0 < x < 1, and f (x |y) = 0 elsewhere.
When we are dealing with two or more random variables, questionsof independence are usually of great importance.Here, f (x |y) = 2x does not depend on the given value of Y = y ,but this is clearly not the case in Example 24
f (x |y) = 2x + 4y
1 + 4y
Thus we have the analogy definition of independent randomvariables and its probability distributions.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
INDEPENDENCE OF DISCRETE RANDOM VARIABLES
Definition
If f (x1, x2, ..., xn) is the value of the joint probability distributionof the discrete random variables X1,X2, ...,Xn at (x1, x2, ..., xn)and fi (xi ) is the value of the marginal distribution of Xi at xi fori = 1, 2, ..., n, then the n random variables are independent if andonly if
f (x1, x2, ..., xn) = f1(x1) · f2(x2) · ... · fn(xn)
for all (x1, x2, ..., xn) within their range.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (26)
Considering n independent flips of a balanced coin, let Xi be thenumber of heads (0 or 1) obtained in the ith flip for i = 1, 2, ..., n.Find the joint probability distribution of these n random variables.
Solution
fi (xi ) =1
2
n random variable are independent, the joint probabilitydistribution is given by
f (x1, x2, ..., xn) =f1(x1) · f2(x2) · ... · fn(xn)
=1
2
1
2...
1
2=
(1
2
)n
where xi = 0 or 1 for i = 1, 2, ..., n.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Example (27)
Given the independent random variables X1, X2, and X3 with theprobability densities
f1(x1) =
{e−x1 for x1 > 0
0 elsewhere
f2(x2) =
{2e−2x2 for x2 > 0
0 elsewhere
f3(x3) =
{3e−3x3 for x3 > 0
0 elsewhere
find their joint probability density, and use it to evaluate theprobability P(X1 + X2 ≤ 1,X3 > 1).
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Solution
The joint probability density:
f (x1, x2, x3) =f1(x1) · f2(x2) · f3(x3)=e−x1 · 2e−2x2 · 3e−3x3
=6e−x1−2x2−3x3
for x1 > 0, x2 > 0, x3 > 0, and f (x1, x2, x3) = 0 elsewhere. Thus,
P(X1 + X2 ≤ 1,X3 > 1) =∫ ∞
1
∫ 1
0
∫ 1−x2
06e−x1−2x2−3x3dx1dx2dx3
=(1− 2e−1 + e−2)e−3
=0.02
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Homework
Work with your partner (in group)
hand in the homework to the editor group on duty 3 before17:00, Saturday.
Group editor on duty shall organize the final answers and sendthe file of final answer to [email protected] before nextTuesday
HW Chapter 3: 69, 97, 99, 101, 103, 105, 107, 109.
Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending
Questions??
Top Related