Chapter 4Continuous Random Variables and Probability Distributions

4.1 - Probability Density Functions 4.2 - Cumulative Distribution Functions and

Expected Values

4.3 - The Normal Distribution

4.4 - The Exponential and Gamma Distributions 4.5 - Other Continuous Distributions

4.6 - Probability Plots

X = # “clicks” on a Geiger counter in normal background radiation.

0 T

Poisson Distribution (discrete)

For x = 0, 1, 2, …, this calculates P(x Events) in a random sample of n trials

coming from a population with rare P(Event) = .

But it may also be used to calculate P(x Events) within a random interval of

time units, for a “Poisson process” having a known “Poisson rate” α.

Recall…

X = # “clicks” on a Geiger counter in normal background radiation.

0 T

X = time between “clicks” on a Geiger counter in normal

background radiation.

Time between events is often modeled by the Exponential Distribution (continuous).

failures, deaths, births, etc.

• “Time-to-Event Analysis”• “Time-to-Failure

Analysis”• “Reliability Analysis”• “Survival Analysis”

Poisson Distribution (discrete)

For x = 0, 1, 2, …, this calculates P(x Events) in a random sample of n trials

coming from a population with rare P(Event) = .

But it may also be used to calculate P(x Events) within a random interval of

time units, for a “Poisson process” having a known “Poisson rate” α.

0

ye 0

lim cy

ce

( ) 1?f x dx

0

1( )x

xf x dx e dx

Time between events is often modeled by the Exponential Distribution (continuous).

parameter > 0

X = Time between events

0

1

pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp()

( )

Check pdf?

( ) 0 is clearf x

0

y

ye dy

Let ;

then

xy

dxdy

lim 1 c

ce

1

[ ] ( )E X x f x dx

0

x

x

x e dx

Time between events is often modeled by the Exponential Distribution (continuous).

parameter > 0

X = Time between events

0

1

pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp()

( )

Calculate the expected time between events

u dv uv v du Integration by Parts

xxu dv e dx

xdxdu v e

00

x x

xx e e dx

0lim 0

c x

xcc e e dx

0

Recall1 1

x

xe dx

0

[ ] ( )E X x f x dx

Time between events is often modeled by the Exponential Distribution (continuous).

parameter > 0

X = Time between events

0

1

pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp()

( )

Calculate the expected time between events

Mean

Similarly for the variance…

2 2 2( ) ( ) ( )E X x f x dx

2 2 2 2 2( )E X x f x dx

2 2

0

1 x

xx e dx

u dv uv v du Integration by Parts

etc... = 2

[ ] ( )E X x f x dx

Time between events is often modeled by the Exponential Distribution (continuous).

parameter > 0

X = Time between events

0

1

X ~ Exp()

( )

Calculate the expected time between events

Mean

2 2Variance

Determine the cdf

( ) ( ) ( )x

F x P X x f t dt

00

1( )xt t

xF x e dt e

( ) 1 , 0x

F x e x

pdf1 , 0( )0, 0

x

e xf xx

[ ] ( )E X x f x dx

Time between events is often modeled by the Exponential Distribution (continuous).

parameter > 0

X = Time between events

0

1 pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp() Calculate the expected time between events

Mean

2 2Variance

Determine the cdf

( ) ( ) ( )x

F x P X x f t dt

00

1( )xt t

xF x e dt e

( ) 1 , 0x

F x e x

cdf

1 , 0( )

0, 0

x

e xF x

x

Note:

“Reliability Function” R(t)

( ) 1 ( )x

P X x F x e

“Survival Function” S(t)

Time between events is often modeled by the Exponential Distribution (continuous).

parameter > 0

X = Time between events

0

1 pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp() Example: Suppose mean time between events is known to be…

cdf

1 , 0( )

0, 0

x

e xF x

x

Mean = 2 years

Then for x 0,

2( ) ( ) 1 .x

F x P X x e

Calculate ( 3 years).P X 32(3) ( 3) 1

0.77687F P X e

Calculate the “Poisson rate” .

Poisson Distribution (discrete)

For x = 0, 1, 2, …, this calculates P(x Events) in a random sample of n trials

coming from a population with rare P(Event) = .

But it may also be used to calculate P(x Events) within a random interval of

time units, for a “Poisson process” having a known “Poisson rate” α.

Ex: Suppose the mean number of instantaneous clicks/sec is = 10, then the mean time between any two successive clicks is = 1/10 sec.

Ex: Suppose the mean number of instantaneous clicks/sec is = 10, then the mean time between any two successive clicks is = 1/10 sec.

0 T

X = Time between events is often modeled by the Exponential Distribution (continuous).

The mean number of events during this time interval (0, T) is .T Therefore, the mean number of events in one unit of time is .T

However, the mean time between events was just shown to be = .

Connection?

1 second

1 ( ) .

Time between events is often modeled by the Exponential Distribution (continuous).

parameter > 0

X = Time between events

0

1 pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp() Example: Suppose mean time between events is known to be…

cdf

1 , 0( )

0, 0

x

e xF x

x

Mean = 2 years

Then for x 0,

2( ) ( ) 1 .x

F x P X x e

Calculate ( 3 years).P X 32(3) ( 3) 1

0.77687F P X e

Calculate the “Poisson rate” .

1 1 event 0.5 events/yr2 years

0 T

Another property …(Event = “Failure,” etc.)

|t

|t tNo Failure

What is the probability of “No Failure” up to t + t, given “No Failure” up to t?

( | )P X t t X t ( )( )

P X t t X tP X t

1 ( )

1 ( )F t tF t

( ) ( ) 1x

F x P X x e

t t

t

e

e

t

e

independent of time t;

only depends on t

“Memory-less” property of the Exponential distribution

The conditional property of “no failure” from ANY time t to a future time t + t of fixed duration t, remains constant.

Models many systems in the “prime of their lives,” e.g., a random 30-yr old individual in the USA.

( 1) ( )n n n ( 1) ! n n

1

0( ) xx e dx

More general models exist…, e.g., The Gamma Distribution

In order to understand this, it is first necessary to understand the ”Gamma Function”

Def: For any > 0,

• Discovered by Swiss mathematician Leonhard Euler (1707-1783) in a different form.

• “Special Functions of Mathematical Physics” includes Gamma, Beta, Bessel, classical orthogonal polynomials (Jacobi, Chebyshev, Legendre, Hermite,…), etc.

• Generalization of “factorials” to all complex values of (except 0, -1, -2, -3, …).• The Exponential distribution is a special case of the Gamma distribution!Basic Properties:

(1) 1 Proof:0

(1) xe dx 0

lim cy

ce

lim 1 c

ce

1

( 1) ( ) Proof:0

( 1) xx e dx

u dv uv v du Integration by Parts

1

x

xu x dv e dxdu x dx v e

1

0 0

x xx e x e dx

1

00 xx e dx

( )

Let = n = 1, 2, 3, …

12

1

0( ) xx e dx

(1) 0! 1 (2) 1! 1 (3) 2! 2

(4) 3! 6

(5) 4! 24

The Gamma Function

~ Gamma( , )X

( ) 1?f x dx

1

0

1( )

xx e dx

1

0

1( )

xx e dx

1 ( ) ( )

1

General Gamma Distribution

Gamma Function

11( ) for 0( )

xf x x e x

Note that if = 1, then pdf is

1

pdf parameters , 01 , 0( ) ( )

0, 0

x

x e xf xx

Standard Gamma Distribution

Note that if = 1, then pdf is

1( ) , 0x

f x e x

Exponential Distribution

2 2

= “shape parameter”

= “scale parameter”

1

0( ) xx e dx

~ Gamma( , )X 1

0( ) xx e dx

General Gamma Distribution

Gamma Function

Standard Gamma Distribution

0.5

1: ~ Exp(1)X

2 3

~ Gamma( ,1) X

1

pdf1( ) for 0( )

xf x x e x

= “shape parameter”

= “scale parameter”

= 1

WLOG…

1

0

1

0

cdf ( ) ( )

( )1( )

1( )

x

x y

x y

F x P X x

f y dy

y e dy

y e dy

1

0( ) xx e dx

Gamma

Function~ Gamma( ,1) X

Standard Gamma Distribution

1

pdf1( ) for 0( )

xf x x e x

= “shape parameter”

= “scale parameter”

1

0( ) xx e dx

Gamma

Function

1

0

1

0

cdf ( ) ( )

( )1( )

1( )

x

x y

x y

F x P X x

f y dy

y e dy

y e dy

“Incomplete

Gamma Function”

1

0

x yy e dy

1

pdf1( ) for 0( )

xf x x e x

~ Gamma( ,1) X

Standard Gamma Distribution

(No general closed form expression, but still continuous and monotonic from 0 to 1.)

= “shape parameter”

= “scale parameter”

~ Gamma( , )X 1

0( ) xx e dx

General Gamma Distribution

Gamma Function

1

pdf parameters , 01 , 0( ) ( )

0, 0

x

x e xf xx

Note that if = 1, then pdf is

1( ) , 0x

f x e x

Exponential Distribution

2 2

Return to…

= “shape parameter”

= “scale parameter”

( ) , 0xf x e x = “Poisson rate”

(= 1/ = )

Theorem: Suppose r.v.’s 1 2 3, , , , are , ~ Exp( ).nX X X X independent

“independent, identically distributed” (i.i.d.)

Then their sum 1 2 3 ~ Gamma( , ).nX X X X n e.g., failure time in machine components

~ Gamma( , )X 1

0( ) xx e dx

General Gamma Distribution

Gamma Function

1

pdf parameters , 01 , 0( ) ( )

0, 0

x

x e xf xx

2 2

= “shape parameter”

= “scale parameter”

Example: Suppose X = time between failures is known to be modeled by a Gamma distribution, with mean = 8 years, and standard deviation = 4 years. Calculate the probability of failure before 5 years.

2 28

4

42

4 1 24

1( )2 (4)

x

f x x e

3 21

(16) 3!

x

x e

3 21 , 096

x

x e x

( ) ( )F x P X x 3 20

196

txt e dt

(5) ( 5)F P X

5 3 20

196

t

t e dt

2 28

4

~ Gamma( , )X 1

0( ) xx e dx

General Gamma Distribution

Gamma Function

1

pdf parameters , 01 , 0( ) ( )

0, 0

x

x e xf xx

2 2

= “shape parameter”

= “scale parameter”

Example: Suppose X = time between failures is known to be modeled by a Gamma distribution, with mean = 8 years, and standard deviation = 4 years. Calculate the probability of failure before 5 years.

42

4 1 24

1( )2 (4)

x

f x x e

(5) ( 5)F P X

5.6

3.51.6

5.6 3

3

3.5 1 1.63.5

1( )(1.6) (3.5)

x

f x x e

Recall... ( 1) ( ) for any 0.

7 5 52 2 2

5 3 32 2 2

5 3 1 12 2 2 2

158

22

= 1

= 2

= 3

= 4 = 5 = 6

= 7

Chi-Squared Distributionwith = n 1 degrees of freedom df = 1, 2, 3,…

, 22

12 2

2

1 , 0( ) 2 ( 2)0, 0

x

x e xf xx

Special case of the Gamma distribution:

“Chi-squared Test” used in statistical analysis of categorical data.

23

F-distributionwith degrees of freedom 1 and

2 .

“F-Test” used when comparing means of two or more groups (ANOVA).

24

T-distributionwith (n – 1) degrees of freedom df = 1, 2,

3, …

“T-Test” used when analyzing means of one or two groups.

df = 1df = 2df = 5df = 10

25

T-distributionwith 1 degree of freedom

df = 1

2

1 1( ) ,1

f xx

x

“Cauchy distribution”

26

T-distributionwith 1 degree of freedom

2

1 1( ) ,1

f xx

x

“Cauchy distribution”

pdf: ( )f x dx

2

1 11

dxx

improper integral at both endpoints

0

2 20

1 1 11 1

dx dxx x

0

2 20

1 1 1lim lim1 1

b

aa bdx dx

x x

01 1

0

1 lim (tan ) lim (tan )b

aa bx x

1 11 lim ( tan ) lim (tan )a b

a b

12 2

1 1 12 2

12

12

0, 0a b

01 1

0

1 lim (tan ) lim (tan )b

aa bx x

1 11 lim ( tan ) lim (tan )a b

a b

12 2

0

2 20

1 1 11 1

dx dxx x

27

T-distributionwith 1 degree of freedom

2

1 1( ) ,1

f xx

x

“Cauchy distribution”

pdf: ( )f x dx

2

1 11

dxx

improper integral at both endpoints

0

2 20

1 1 1lim lim1 1

b

aa bdx dx

x x

( )x f x dx

2

11x dxx

0

2 20

11 1x xdx dxx x

0

2 20

1 lim lim1 1

b

aa b

x xdx dxx x

21

xyx

0

1 1 12 2

12

12

0, 0a b

0

2 20

1 lim lim1 1

b

aa b

x xdx dxx x

0

2 20

11 1x xdx dxx x

( )x f x dx

2

11x dxx

02 21 12 2 0

1 lim ln(1 ) lim ln(1 )b

aa bx x

28

T-distributionwith 1 degree of freedom

2

1 1( ) ,1

f xx

x

“Cauchy distribution”improper integral at both endpoints

21xyx

0 2 21 12 2

1 lim ln(1 ) lim ln(1 )a b

a b

“indeterminate form”

|a

|b

12

12

0, 0a b

0

2 20

1 lim lim1 1

b

aa b

x xdx dxx x

0

2 20

11 1x xdx dxx x

( )x f x dx

2

11x dxx

02 21 12 2 0

1 lim ln(1 ) lim ln(1 )b

aa bx x

29

T-distributionwith 1 degree of freedom

2

1 1( ) ,1

f xx

x

“Cauchy distribution”improper integral at both endpoints

21xyx

0 2 21 12 2

1 lim ln(1 ) lim ln(1 )a b

a b

“indeterminate form”

12

12

does not exist!

0, 0a b

30

Classical Continuous Probability Distributions

● Normal distribution

● Log-Normal ~ X is not normally distributed (e.g., skewed), but Y = “logarithm of X” is normally distributed

● Student’s t-distribution ~ Similar to normal distr, more flexible

● F-distribution ~ Used when comparing multiple group means

● Chi-squared distribution ~ Used extensively in categorical data analysis

● Others for specialized applications ~ Gamma, Beta, Weibull…