Functions of Random Variables - National Tsing Hua University

Functions ofRandom Variables

Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs

MomentGeneratingFunctions

7.1

Lecture #7Functions of Random Variables

BMIR Lecture Series on Probability and StatisticsFall 2015

Ching-Han Hsu, Ph.D.Department of Biomedical Engineering

and Environmental SciencesNational Tsing Hua University


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.2

Function of Single Variable

Theorem

Suppose that X is a discrete random variable withprobability distribution fX(x). Let Y = h(X) define aone-to-one transformation between the values of X and Yso that the equation y = h(x) can be solved uniquely for xin terms of y. Let the solution be x = u(y). Then theprobability distribution of the random variable y is

P(Y = y) = fY(y) = fX(u(y)) = P(X = u(y)) (1)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.3

Function of Single Variable: Example

Example

Let X be a geometric random variable with probabilitydistribution

fX(x) = p(1− p)x−1, x = 1, 2, . . .

Find the probability distribution of Y = X2.

• Since X > 0, Y = X2 (or X =√

Y) is one-to-onetransformation.

• Therefore, the distribution of the random variable Y is

fY(y) = fX(√

y) = p(1− p)√

y−1, y = 1, 4, 9, . . .


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.4

Function of Two Random Variables

Theorem

Suppose that X1 and X2 are two discrete randomvariables with joint probability distribution fX1X2(x1, x2) (orf (x1, x2)). Let Y1 = h1(X1,X2) and Y2 = h2(X1,X2) defineone-to-one transformations between the points (x1, x2)and (y1, y2) so that equations y1 = h1(x1, x2) andy2 = h2(x1, x2) can be solved uniquely for x1 and x2 interms of y1 and y2, i.e., x1 = u1(y1, y2) and x2 = u2(y1, y2).Then the joint probability distribution of Y1 and Y2 is

fY1,Y2(y1, y2) = fX1X2 [u1(y1, y2), u2(y1, y2)] (2)

Note that the distribution of Y1 can be found by summingover the y2, i.e., the marginal probability distribution of Y1,

fY1(y1) =∑

y2

fY1,Y2(y1, y2)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.5

MGF of Sum Two Independent Poisson RVs I

Example

Suppose that X1 and X2 are two independent Poissonrandom variables with parameters λ1 and λ2. Find theprobability distribution of Y = X1 + X2.

• Let Y = Y1 = X1 + X2 and Y2 = X2. Hence,X1 = Y1 − Y2 and X2 = Y2.

• Since X1 and X2 are independent, the jointdistribution of X1 and X2 is

fX1X2(x1, x2) =λx1

1 e−λ1

x1!·λx2

2 e−λ2

x2!x1 = 0, 1, 2, . . . , x2 = 0, 1, 2, . . .


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.6

MGF of Sum Two Independent Poisson RVs II

• The joint distribution of Y1 and Y2 is

fY1,Y2(y1, y2) = fX1X2 [u1(y1, y2), u2(y1, y2)]

= fX1X2(y1 − y2, y2)

=λy1−y2

1 e−λ1

(y1 − y2)!·λy2

2 e−λ2

y2!

= e−(λ1+λ2) λy1−y21 λy2

2(y1 − y2)!y2!

y1 = 0, 1, 2, . . . , y2 = 0, 1, 2, . . . , y1

Note that x1 ≥ 0 so that y1 − y2 ≥ 0.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.7

MGF of Sum Two Independent Poisson RVs III

• The marginal distribution of Y1 is

fY1(y1) =

y1∑y2=0

fY1,Y2(y1, y2)

= e−(λ1+λ2)y1∑

y2=0

λy1−y21 λy2

2(y1 − y2)!y2!

=e−(λ1+λ2)

y1!

y1∑y2=0

y1!

(y1 − y2)!y2!λy1−y2

1 λy22

=e−(λ1+λ2)

y1!(λ1 + λ2)y1

• The random variable Y = Y1 = X1 + X2 has Poissondistribution with parameter λ1 + λ2.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.8

Function of Single Variable I

Theorem

Suppose that X is a continuous random variable withprobability distribution fX(x). Let Y = h(X) define aone-to-one transformation between the values of X and Yso that the equation y = h(x) can be solved uniquely for xin terms of y. Let the solution be x = u(y). Then theprobability distribution of the random variable y is

fY(y) = fX(u(y))|J| (3)

where J = ddy u(y) = u′(y) = dx

dy is called the Jacobian ofthe transformation and the absolute value |J| of J is used.

• Suppose that the function y = h(x) is an increasingfunction of x.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.9

Function of Single Variable II

• now, we have

P(Y ≤ a) = P[X ≤ u(a)] =

∫ u(a)

−∞fX(x)dx

• Since x = u(y), we obtain dx = u′(y)dy and

P(Y ≤ a) =

∫ a

−∞fX(u(y))u′(y)dy

• The density function of Y is

fY(y) = fX(u(y))u′(y) = fX(u(y))J.

• If y = h(x) is a decreasing function of x, a similarargument holds.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.10

Function of Single Variable: Example 1 I

Example

Suppose that X has pdf

f (x) =

{2x, 0 < x < 10, elsewhere

(4)

Let h(X) = 3X + 1. How to find the pdf of Y = h(X).

• Since y = h(x) = 3x + 1 is an increasing function of x,we obtain x = u(y) = (y− 1)/3, J = u′(y) = 1/3,

fY(y) = fX(u(y))u′(y) = 2(y− 1)/3 · 1/3 =29

(y− 1).


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.11

Function of Single Variable: Example 1 II

• Intuitive approach

FY(y) = P(Y ≤ y)

= P[(3X + 1) ≤ y] = P[X ≤ (y− 1)/3]

=

∫ (y−1)/3

02xdx = [(y− 1)/3]2.

fY(y) =ddy

FY(y)

=

{29(y− 1), 1 < y < 40, elsewhere


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.12

Function of Single Variable: Example 2 I

Example

Suppose that X has pdf

f (x) =

{2x, 0 < x < 10, elsewhere

(5)

Let h(X) = e−X. How to find the pdf of Y = h(X).

• Since y = h(x) = e−x is a decreasing function of x, weobtain x = u(y) = − ln y, J = u′(y) = −1/y,

fY(y) = fX(u(y))|J|

= −2 ln y · 1/y =−2y

ln y,

1/e < y < 1.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.13

Function of Single Variable: Example 2 II

• Intuitive approach

FY(y) = P(Y ≤ y)

= P(e−X ≤ y) = P(X ≥ − ln y)

=

∫ 1

− ln y2xdx = 1− (− ln y)2.

fY(y) =ddy

FY(y) =−2y

ln y,

1/e < y < 1.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.14

Function of Single Variable: Non-monotonicTransformation I

Example

Suppose that X has pdf f (x). Let h(X) = X2. How to findthe pdf of Y = h(X).

• Since y = h(x) = x2 is neither an increasing nor adecreasing function of x, we can not directly use theEq. (3).

• We can obtain the pdf of Y = X2 as follows:

FY(y) = P(Y ≤ y) = P(X2 ≤ y)

= P(−√y ≤ X ≤ √y)

= FX(√

y)− FX(−√y)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.15

Function of Single Variable: Non-monotonicTransformation II

fY(y) =ddy

FY(y) =ddy

[FX(√

y)− FX(−√y)]

=dxdy

ddx

[FX(√

y)− FX(−√y)]

=fX(√

y)

2√

y−

fX(−√y)

−2√

y

=1

2√

y[fX(√

y) + fX(−√y)]


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.16

Non-monotonic Transformation: Normal vs Chi-Square I

Example

Suppose that X has pdf N(0, 1) Let h(X) = X2. How to findthe pdf of Y = h(X).

• X is standard normal distribution with pdf

f (x) =1√2π

e−x2/2

• We can obtain the pdf of Y = X2 as follows:

fY(y) =1

2√

y[fX(√

y) + fX(−√y)]

=1√

y1√2π

e−y/2

=1

21/2√πy1/2−1e−y/2, y > 0.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.17

Non-monotonic Transformation: Normal vs Chi-SquareII• The gamma distribution has the pdf

f (x; γ, λ) =λγxγ−1e−λx

Γ(γ)=

λ

Γ(γ)(λx)γ−1e−λx, x > 0.

When λ = 1/2 and γ = 1/2, 1, 3/2, 2, . . ., it becomeschi-square distribution:

f (x; γ, 1/2) =1

Γ(γ)

(12

)γxγ−1e−x/2, x > 0.

• Since√π = Γ(1

2), we can rewrite fY(y) as

fY(y) =1

Γ(12)

(12

)1/2

y1/2−1e−y/2

Y has a chi-square distribution.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.18

General Transformation

Theorem

Suppose that X is a continuous random variable withprobability distribution fX(x), and Y = h(X) is atransformation that is not one-to-one. If the interval overwhich X is defined can be partitioned into m mutuallyexclusive disjoint sets such that each of the inversefunctions x1 = u1(y), x2 = u2(y), . . . , xm = um(y) of y = h(x)is one-to-one, the probability distribution of Y is

fY(y) =

m∑i=0

fX[ui(y)]|Ji| (6)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.19

Function of Two Random Variables

Theorem

Suppose that X1 and X2 are two continuous randomvariables with joint probability distribution fX1X2(x1, x2) (orf (x1, x2)). Let Y1 = h1(X1,X2) and Y2 = h2(X1,X2) defineone-to-one transformations between the points (x1, x2)and (y1, y2) so that equations y1 = h1(x1, x2) andy2 = h2(x1, x2) can be solved uniquely for x1 and x2 interms of y1 and y2, i.e., x1 = u1(y1, y2) and x2 = u2(y1, y2).Then the joint probability distribution of Y1 and Y2 is

fY1,Y2(y1, y2) = fX1X2 [u1(y1, y2), u2(y1, y2)] · |J| (7)

where J is the Jacobian and is given by the followingdeterminant:

J =

∣∣∣∣∂x1/∂y1 ∂x1/∂y2∂x2/∂y1 ∂x2/∂y2

∣∣∣∣ (8)

and the absolute value of the determinant is used.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.20

Function of Two Random Variables I

Example

Suppose that X1 and X2 are two independent exponentialrandom variables with fX1(x1) = 2e−2x1 andfX2(x2) = 2e−2x2 . Find the probability distribution ofY = X1/X2.

• The joint probability distribution of X1 and X2 is

fX1X2(x1, x2) = 4e−2(x1+x2)

• Let Y = Y1 = h1(X1,X2) = X1/X2 andY2 = h2(X1,X2) = X1 + X2.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.21

Function of Two Random Variables II

• The inverse solutions of y1 = x1/x2 and y2 = x1 + x2are x1 = y1y2/(1 + y1) and x2 = y2/(1 + y1), and itfollows that

J =

∣∣∣∣∣y2

(1+y1)2y1

(1+y1)−y2

(1+y1)21

(1+y1)

∣∣∣∣∣ =y2

(1 + y1)2

• Then the joint probability distribution of Y1 and Y2 is

fY1,Y2(y1, y2) = fX1X2 [u1(y1, y2), u2(y1, y2)] · |J|

= 4e−2(y1y2/(1+y1)+y2/(1+y1)) · y2

(1 + y1)2

= 4e−2y2y2

(1 + y1)2

y1 > 0, y2 > 0.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.22

Function of Two Random Variables III

• We need to find the distribution of Y = Y1 = X1/X2.The marginal distribution of Y1 is

fY1(y1) =

∫ ∞0

fY1,Y2(y1, y2)dy2

=

∫ ∞0

4e−2y2y2

(1 + y1)2 dy2

=1

(1 + y1)2 , y > 0

= fY(y)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.23

Example: Function of Two N(0, 1) I

Example

Let X1 and X2 be two independent normal randomvariables and both have a N(0, 1) distribution. LetY1 = X1+X2

2 and Y2 = X2−X12 . Show that Y1 and Y2 are

independent.

• Since X1 and X2 are independent, the joint pdf is

fX1,X2(x1, x2) = fX1(x1) · fX2(x2) =1

2πexp

−12 (x2

1+x22) .

• X1 = u1(Y1,Y2) = Y1 − Y2 andX2 = u2(Y1,Y2) = Y1 + Y2.

• The Jacobian

J =

∣∣∣∣∂x1/∂y1 ∂x1/∂y2∂x2/∂y1 ∂x2/∂y2

∣∣∣∣ =

∣∣∣∣1 −11 1

∣∣∣∣ = 2


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.24

Example: Function of Two N(0, 1) II

• The joint pdf of Y1 and Y2 is

fY1,Y2(y1, y2) = fX1X2 [u1(y1, y2), u2(y1, y2)] · |J|

=2

2πexp{

−12 [(y1−y2)2+(y1+y2)2]}

=1(√

2π)2 (

1√2

)2 exp

−12

y21(

1√2

)2 +y22(

1√2

)2

=1

√2π ·

(1√2

) exp

−12

y21(

1√2

)2

· 1√

2π ·(

1√2

) exp

−12

y22(

1√2

)2

.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.25

Example: Function of Two N(0, 1) III

• Therefore, fY1,Y2(y1, y2) = fY1(y1) · fY2(y2).

• Y1 and Y2 both are normal distribution with N(0, 12).


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.26

Example: Function of Multiple Random Variables I

Example

Let X1,X2, . . . ,Xn be independent and identicallydistributed random variables from a uniform distributionon [0, 1]. Let Y = max{X1,X2, . . . ,Xn} andZ = min{X1,X2, . . . ,Xn}. What are the pdf’s of Y and Z?

• The pdf of Y:

FY(y) = P(Y ≤ y) = P(max{X1,X2, . . . ,Xn} ≤ y)

= P(X1 ≤ y, . . . ,Xn ≤ y)

= P(X1 ≤ y)× · · · × P(Xn ≤ y)

= yn.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.27

Example: Function of Multiple Random Variables II

Hence fy(y) = dFY(y)dy = nyn−1, 0 ≤ y ≤ 1.

FZ(z) = P(Z ≤ z) = P(min{X1,X2, . . . ,Xn} ≤ z)

= 1− P(X1 > z, . . . ,Xn > z)

= 1− P(X1 > z)× · · · × P(Xn > z)

= 1− (1− z)n.

Hence fZ(z) = dFZ(z)dz = n(1− z)n−1, 0 ≤ z ≤ 1.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.28

Moment Generating Function (mgf)

Definition (Discrete Random Variable)

Let X be a discrete random variable with probability massfunction P(X = xi) = f (xi), i = 1, 2, . . .. The function MX iscalled the moment generating function of X is defined by

MX(t) = E(etX) =

∞∑i=1

etxi f (xi) (9)

Definition (Continuous Random Variable)

If X be a continuous random variable with probabilitydensity function f (x), we define the moment generatingfunction of X by

MX(t) = E(etX) =

∫ ∞−∞

etxf (x)dx (10)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.29

MGF of Poisson Distribution

Example

If X is a Poisson random variable, f (x) = e−λλx

x! , withparameter λ > 0, find its moment generating function.

Solution

MX(t) =

∞∑x=0

etx e−λλx

x!

= e−λ∞∑

x=0

(λet)x

x!

= e−λeλet= eλ(et−1) (11)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.30

MGF of Exponential Distribution

Example

If X is an exponential random variable, f (x) = λe−λx, withparameter λ > 0, find its moment generating function.

Solution

MX(t) =

∫ ∞0

etxλe−λxdx

= λ

∫ ∞0

e−x(λ−t)dx

=λ

λ− t, t < λ (12)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.31

MGF of Normal Distribution(I)

Example

If X ∼ N(0, 1), find its moment generating function.

Solution

MX(t) = E{etX} =1√2π

∫ ∞−∞

etxe−x2/2dx

=1√2π

et2/2∫ ∞−∞

e−t2/2etxe−x2/2dx

= et2/2[

1√2π

∫ ∞−∞

e−(x−t)2/2dx]

= et2/2 (13)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.32

MGF of Gamma Distribution I

Example

If X is a Gamma distribution with pdf

f (x; γ, λ) =

{λγ

Γ(γ)xγ−1e−λx, 0 < x <∞0, elsewhere

,

find its moment generating function.

MX(t) = E{etX} =

∫ ∞0

etx λγ

Γ(γ)xγ−1e−λxdx

=

∫ ∞0

λγ

Γ(γ)xγ−1e−(λ−t)xdx


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.33

MGF of Gamma Distribution II

Let y = (λ− t)x, then x = y/(λ− t) and dx = dy/(λ− t).We have

MX(t) =

∫ ∞0

λγ

Γ(γ)xγ−1e−(λ−t)xdx

=

∫ ∞0

λγ

Γ(γ)

(y

(λ− t)

)γ−1

e−y 1(λ− t)

dy

=

(λ

λ− t

)γ ∫ ∞0

1Γ(γ)

yγ−1e−ydy

=

(λ

λ− t

)γ=

1(1− t/λ)γ


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.34

Properties of MGF

• Recall the Taylor series of the function ex:

ex = 1 + x +x2

2!+ · · ·+ xn

n!+ · · ·

Thus

etx = 1 + tx +(tx)2

2!+ · · ·+ (tx)n

n!+ · · ·

• The moment generating of the random variable X is

MX(t) = E(etX)

= E(

1 + tX +(tX)2

2!+ · · ·+ (tX)n

n!+ · · ·

)= 1 + tE(X) +

t2E(X2)

2!+ · · ·+ tnE(Xn)

n!+ · · ·


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.35

Properties of MGF

• Consider the first derivative of MX(t) with respect to tand obtain

M′X(t) = E(X) + tE(X2) + · · ·+ ntn−1 E(Xn)

n!+ · · ·

Taking t = 0, we have

M′X(t)|t=0 = M′X(0) = E{X} = µX

• Consider the second derivative of MX(t) and obtain

M′′X(t) = E(X2) + tE(X3) · · ·+ n(n− 1)tn−2 E(Xn)

n!+ · · ·

The second moment of X is M′′X(0) = E{X2}. Then,the variance of X is given by

V(X) = E{X2} − E2{X} = M′′X(0)− (M′X(0))2


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.36

Properties of MGF

• Differentiating k times and then setting t = 0 yields

M(k)X (t)|t=0 = E{Xk} (14)

• If X is a random variable and α is a constant, then

MX+α(t) = eαtMX(t) (15)

MαX(t) = MX(αt) (16)

• Let X and Y be two random variables with momentgenerating functions MX(t) and MY(t), respectively. IfMX(t) = MY(t) for all values of t, then X and Y havethe same probability distribution.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.37

MGF of Normal Distribution(II)

Example

Use Eq. (13) to find the first four moments of the normalrv X.

Solution

• M′X(t) = ddt e

t2/2 = et2/2 · t = t ·MX(t).E{X1} = M′X(0) = 0.

• M′′X(t) = MX(t) + t ·M′X(t) = MX(t) + t2 ·MX(t).E{X2} = M′′X(0) = 1.

• M(3)X (t) = (2t) ·MX(t)+(1+ t2) ·M′X(t) = (3t+ t3) ·MX(t).

E{X3} = M(3)X (0) = 0.

• M(4)X (t) = (3 + 3t2) ·MX(t) + (3t + t3) ·M′X(t).

E{X4} = M(4)X (0) = 3.


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.38

MGF of Normal Distribution(III)

Example

Use Eqs. (14) and (15) to find the mgf of a normaldistribution with mean µ and variance σ2.

• Assume X be a normal distribution with mean µ andvariance σ2.

• If Z = X−µσ , Z is a standard normal random variable.

• The mgf of X is

MX(t) = MσZ+µ(t) = eµtMσZ(t)

= eµtMZ(σt)

= eµte(σt)2/2

= eµt+σ2t2/2 (17)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.39

Sum of Independent RVs: MGF I

Theorem

If X1,X2, . . . ,Xn are independent random variables withmoment generating functions MX1(t),MX2(t), . . . ,MXn(t),respectively, and if Y = X1 + X2 + · · ·+ Xn then themoment generating function of Y is

MY(t) = MX1(t) ·MX2(t) · · · · ·MXn(t) (18)

• For the continuous case,

MY(t) = E(etY) = E(et(X1+X2+···+Xn))

=

∫ ∞−∞· · ·∫ ∞−∞

et(x1+x2+···+xn)f (x1, x2, . . . , xn)

dx1dx2 · · · dxn


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.40

Sum of Independent RVs: MGF II

• Since the variables are independent, we have

f (x1, x2, . . . , xn) = f (x1)f (x2) · · · f (xn)

• Therefore,

MY(t) =

∫ ∞−∞

etx1 f (x1)dx1 ·∫ ∞−∞

etx2 f (x2)dx2

· · · ·∫ ∞−∞

etxn f (xn)dxn

= MX1(t) ·MX2(t) · · · · ·MXn(t)


Ching-Han Hsu,Ph.D.

Discrete RVs

Continuous RVs


7.41

MGF of Sum Two Independent Poisson RVs

Example

Suppose that X1 and X2 are two independent Poissonrandom variables with parameters λ1 and λ2. Find theprobability distribution of Y = X1 + X2.

• The mgf’s of X1 and X2 are MX1(t) = eλ1(et−1) andMX2(t) = eλ2(et−1), respectively.

• The mgf of Y is

MY(t) = MX1(t) ·MX2(t) = eλ1(et−1) · eλ2(et−1)

= e(λ1+λ2)(et−1) (19)

• Therefore, Y is also a Poisson random variable withparameters λ1 + λ2.

Functions of Random Variables - National Tsing Hua University

Documents

Transcript of Functions of Random Variables - National Tsing Hua University