STA 611: Introduction to Mathematical Statistics Lecture 5 ...

STA 611: Introduction to Mathematical StatisticsLecture 5: Random Variables and Distributions

Instructor: Meimei Liu

STA 611 (Lecture 05) Random Variables and Distributions 1 / 14

Chapter 3 - continued

Chapter 3 sections

3.1 Random Variables and Discrete Distributions3.2 Continuous Distributions3.3 The Cumulative Distribution Function3.4 Bivariate Distributions3.5 Marginal Distributions3.6 Conditional DistributionsJust skim: 3.7 Multivariate Distributions (generalization ofbivariate)3.8 Functions of a Random Variable3.9 Functions of Two or More Random Variables

SKIP: pages 180 -186

SKIP: 3.10 Markov Chains


Chapter 3 - continued 3.8 Functions of a Random Variable

Transformations of Random Variables

If X is a random variable then any function of X , g(X ), is also arandom variableSometimes we are interested in Y = g(X ) and need thedistribution of Y

Example: Say we have the distribution of the service rate X , thenwhat is the distribution of the average waiting time Y = 1/X?

We can use the distribution of X to get the distribution of Y :

P(Y ∈ A) = P(g(X ) ∈ A)

Depending on the function g we can sometimes obtain a tractableexpression for the probability of Y

Example: P(Y ≤ y) = P(1/X ≤ y) = P(X ≥ 1/y)



Transformations of Random Variables

Inverse mapping

Let g(x) : X → Y. The inverse mapping is defined as

g−1(A) = {x ∈ X : g(x) ∈ A}

For a set of one point we write

g−1({y}) = g−1(y) = {x ∈ X : g(x) = y}

We can therefore write:

P(Y ∈ A) = P(g(X ) ∈ A) = P({x ∈ X : g(x) ∈ A})= P(X ∈ g−1(A))



Transformation of a discrete random variable

If Y = g(X ) where X is a discrete r.v. with support X then Y isalso a discrete r.v. and

fY (y) = P(Y = y) = P(X ∈ g−1(y))

=∑

x∈g−1(y)

P(X = x) =∑

x∈g−1(y)

fX (x)

for all y ∈ Y = {y : y = g(x), x ∈ X}

Example:Let X ∼ Binomial(n,p), i.e.

fX (x) = P(X = x) =(

nx

)px(1− p)n−x , x = 0,1,2, . . . ,n

What is the pf of Y = n − X (i.e. the number of failures)?



Example 1: Exponential distribution (Continuous r.v.)

Let X be a random variable withpdf

fX (x) = λ exp(−λx), x > 0

where λ is a positive constant

We say that X is exponentially distributed with parameter (rate) λ orX ∼ Exp(λ).

1 What is the cdf of X?2 What is the distribution of Y = αX where α is a positive constant?3 What is the distribution of W = X 2 ?



Example 2: Double exponential distributionAlso called Laplace distribution

Let X be a random variable withpdf

fX (x) =λ

2exp(−λ|x |), x ∈ R

where λ is a positive constant

What is the cdf of X?What is the cdf of W = X 2 ?



Example 3: cdf transformation

Again we consider the exponential distribution.

Let X ∼ Exp(λ), then X has the pdf

fX (x) = λe−λx , x > 0

where λ is a positive constant.(a) Let FX (x) be the cdf found in Ex. 1. Find the distribution of

Y = FX (X ).

(b) Find the inverse cdf F−1X and the distribution of F−1

X (U) whereU ∼ Uniform(0,1).



Example 3: cdf transformation

Again we consider the exponential distribution.

Let X ∼ Exp(λ), then X has the pdf

fX (x) = λe−λx , x > 0

where λ is a positive constant.(a) Let FX (x) be the cdf found in Ex. 1. Find the distribution of

Y = FX (X ).(b) Find the inverse cdf F−1

X and the distribution of F−1X (U) where

U ∼ Uniform(0,1).



Probability integral transformation

Theorem1 Let X have a continuous cdf F and let Y = F (X ). Then

F (X ) ∼ Uniform(0,1).2 Let Y ∼ Uniform(0,1) and let F be a continuous cdf with quantile

function F−1. Then X = F−1(Y ) has cdf F .

This theorem is useful when we want to generate randomnumbers from some distribution.If F−1 is available in closed form we can simply generate uniformrandom numbers and then transform them using F−1.Therefore, much of the effort concerning generating (pseudo)random numbers has been concentrated on generating uniformrandom numbers.







This theorem is useful when we want to generate randomnumbers from some distribution.

If F−1 is available in closed form we can simply generate uniformrandom numbers and then transform them using F−1.Therefore, much of the effort concerning generating (pseudo)random numbers has been concentrated on generating uniformrandom numbers.







This theorem is useful when we want to generate randomnumbers from some distribution.If F−1 is available in closed form we can simply generate uniformrandom numbers and then transform them using F−1.

Therefore, much of the effort concerning generating (pseudo)random numbers has been concentrated on generating uniformrandom numbers.







This theorem is useful when we want to generate randomnumbers from some distribution.If F−1 is available in closed form we can simply generate uniformrandom numbers and then transform them using F−1.Therefore, much of the effort concerning generating (pseudo)random numbers has been concentrated on generating uniformrandom numbers.



Monotone transformations of continuous r.v.’s

TheoremLet X be a random variable with pdf fX (x) and support X and letY = g(X ) where g is a monotone function.

Suppose fX (x) is continuous on X and that g−1(y) has a continuousderivative on Y = {y : y = g(x), x ∈ X}.

Then the pdf of Y is

fY (y) =

{fX (g−1(y))

∣∣∣ ddy g−1(y)

∣∣∣ if y ∈ Y0 otherwise

Refer to Theorem 3.9.5 for the multivariate case.



Example 4: Transformation of the Gamma distribution

Consider the Gamma distribution with parameters n and β

Let X ∼ Gamma(n, β). Then X has the pdf

f (x) =1

(n − 1)!βn xn−1e−x/β

What is the pdf of Y = 1/X?

The distribution of Y is called the Inverse gamma distribution.



Linear functions

A straightforward corollary:

Linear functionLet X be a random variable with pdf fX (x) and let Y = aX + b, a 6= 0.Then

fY (y) =1|a|

f(

y − ba

)

Example: Let X have the pdf

fX (x) =1

σ√

2πexp

(−(x − µ)2

2σ2

)This is the pdf of the Normal distribution with parameters µ (mean) andσ2 (variance). Notation: X ∼ N(µ, σ2).

Find the pdf of Y = X−µσ .



Linear functions

A straightforward corollary:

Linear functionLet X be a random variable with pdf fX (x) and let Y = aX + b, a 6= 0.Then

fY (y) =1|a|

f(

y − ba

)Example: Let X have the pdf

fX (x) =1

σ√

2πexp

(−(x − µ)2

2σ2

)This is the pdf of the Normal distribution with parameters µ (mean) andσ2 (variance). Notation: X ∼ N(µ, σ2).

Find the pdf of Y = X−µσ .


Chapter 3 - continued 3.9 Functions of Two or More Random Variables

Sum of two random variables – Convolution

What is the distribution of Z = X + Y?

If X and Y discrete random variables we get

P(Z = z) =∑

i

P(X = i ,Y = z − i)

P(Z = z) =∑

i

P(X = i)P(Y = z − i) if X and Y are independent

X and Y continuous random variables:

fZ (z) =∫ ∞−∞

fX ,Y (t , z − t)dt

fZ (z) =∫ ∞−∞

fX (t)fY (z − t)dt if X and Y are independent

this is called the convolution formula. Example: What is the distribution

of X + Y for independent standard normals X and Y ?




What is the distribution of Z = X + Y?If X and Y discrete random variables we get

P(Z = z) =∑

i

P(X = i ,Y = z − i)

P(Z = z) =∑

i



fZ (z) =∫ ∞−∞


fZ (z) =∫ ∞−∞








P(Z = z) =∑

i

P(X = i ,Y = z − i)

P(Z = z) =∑

i



fZ (z) =∫ ∞−∞


fZ (z) =∫ ∞−∞


this is called the convolution formula.

Example: What is the distribution






P(Z = z) =∑

i

P(X = i ,Y = z − i)

P(Z = z) =∑

i



fZ (z) =∫ ∞−∞


fZ (z) =∫ ∞−∞





Chapter 3 - continued

END OF CHAPTER 3


STA 611: Introduction to Mathematical Statistics Lecture 5 ...

Documents

Transcript of STA 611: Introduction to Mathematical Statistics Lecture 5 ...