Inferential Statistics: Probability Distributions, Hypothesis Testing, and Statistical Inference
1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of...
-
Upload
randell-hopkins -
Category
Documents
-
view
221 -
download
0
Transcript of 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of...
![Page 1: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/1.jpg)
1
Probability and Statistical Inference (9th Edition)
Chapter 5 (Part 2/2)
Distributions of Functions of Random Variables
November 25, 2015
![Page 2: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/2.jpg)
2
Outline
5.5 Random Functions Associated with Normal Distributions
5.6 The Central Limit Theorem
5.7 Approximations for Discrete Distributions
5.8 Chebyshev’s Inequality and Convergence in Probability
![Page 3: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/3.jpg)
3
Random Functions Associated with Normal Distributions
Theorem: Assume that X1, X2,…, Xn are independent random variables with distributions N(μ1,σ1
2), N(μ2,σ22),…, N(μn,σn
2), respectively. Then,
n
i
n
iiiii
n
iii ccNXcY
1 1
22
1
, is
![Page 4: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/4.jpg)
4
Random Functions Associated with Normal Distributions
Proof: Recall the mgf of N(,2) is
),( is Y,
2exp
2exp
1
22
1
1
222
1
1
222
1
n
iii
n
iii
n
iii
n
iii
n
i
iiii
n
iixY
ccNTherefore
ct
ct
tctctcMtM
i
![Page 5: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/5.jpg)
5
Example 1: If X1 and X2 are independent normal random variables N(µ1,σ1
2) and N(µ2,σ22), respec
tively, then
X1 + X2 is N(µ1+µ2, σ12+σ2
2), and
X1 - X2 is N(µ1-µ2, σ12+σ2
2)
Random Functions Associated with Normal Distributions
![Page 6: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/6.jpg)
6
Example 2: If X1,X2,…,Xn correspond to random samples from a normal distribution N(μ,σ2), then the sample mean
is N(μ,σ2/n)
Proof:
1
1
n
iiXn
X
n,N is X Therefore,
2 since ,
2
22
2
22
2
t
n
tn
tExp
n
tMM
tExptM
t
nn
x
Random Functions Associated with Normal Distributions
![Page 7: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/7.jpg)
7
Random Functions Associated with Normal Distributions
One important implication of the distribution of is that it has a greater probability of falling in an interval containing μ than does a single sample Xk
The larger the sample size n, the smaller the variance of the sample mean
“Mean” is a constant, but “sample mean” is a random variable
X
![Page 8: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/8.jpg)
8
Random Functions Associated with Normal Distributions
For example, assume that X1, X2,…, Xn are random samples from N(50,16) distribution. Then, is N(50,16/n). The following figure shows the pdf of with different values of n
XX
![Page 9: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/9.jpg)
9
Random Functions Associated with Normal Distributions
![Page 10: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/10.jpg)
10
Random Functions Associated with Normal Distributions
Recall:Let Z1, Z2, …, Zn be i.i.d. N(0,1). Then, w =
Z12+ Z2
2+ …+ Zn2 is χ2(n)
Let X1, X2,…, Xn be independent chi-square random variables with k1, k2,…, kn degrees of freedom, i.e., χ2(k1), χ2(k2),…, χ2(kn), respectively. Then, Y=X1+X2+…+Xn is χ2(k1
+k2+…+kn)
![Page 11: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/11.jpg)
11
Random Functions Associated with Normal Distributions
Theorem: Let X1, X2,…, Xn be random samples from the N(μ,σ2) distribution. The sample mean and sample variance are given by
Then,
(a) and are independent
(b) is χ2(n-1)
![Page 12: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/12.jpg)
12
Random Functions Associated with Normal Distributions We will accept (a) without proving it Proof of (b):
![Page 13: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/13.jpg)
13
Random Functions Associated with Normal Distributions
![Page 14: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/14.jpg)
14
Random Functions Associated with Normal Distributions
![Page 15: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/15.jpg)
15
Random Functions Associated with Normal Distributions
It is interesting to observe that
That is, when the actual mean is replaced by the sample mean, one degree of freedom is lost
1 W
and
2
12
2
2
12
2
nisXX
nisX
U
n
i
i
n
i
i
![Page 16: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/16.jpg)
16
Central Limit Theorem
It is useful to first review some related theorems Theorem (Sample Mean): Let X1, X2, …, Xn be a sequenc
e of i.i.d. random variables with mean and variance 2. Then, the sample mean
is a random variable with mean and variance 2/n
![Page 17: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/17.jpg)
17
Central Limit Theorem
Theorem (Strong Law of Large Numbers): Let X1, X2, …, Xn be a sequence of i.i.d. random variables with mean Then, with probability 1,
That is,
(The sample mean converges almost surely, or converges with probability 1, to the expected value)
![Page 18: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/18.jpg)
18
Central Limit Theorem
Theorem (Strong Law of Large Numbers)(Cont.): This theorem holds for any distribution of the Xi’s This is one of the most well-known results in probability t
heory
![Page 19: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/19.jpg)
19
Central Limit Theorem
Theorem (Central Limit Theorem): Let X1, X2, …, Xn be a sequence of i.i.d. random variables with mean and variance 2. Then the distribution of
is N(0,1) as
That is,
(convergence in distribution)
![Page 20: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/20.jpg)
20
Central Limit Theorem
Theorem (Central Limit Theorem)(Cont.): While tends to “degenerate” to zero (Strong La
w of Large Numbers), the factor in
“spreads out” the probability enough to prevent this degeneration
![Page 21: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/21.jpg)
21
Central Limit Theorem
Theorem (Central Limit Theorem)(Cont.): One observation that helps make sense of this result is t
hat, in the case of normal distribution (i.e., X1, X2, …, Xn are i.i.d. normal), is N(,2/n)
Hence, is (exactly) N(0,1) for each positive value of n
Thus, in the limit, the distribution must also be N(0,1)
![Page 22: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/22.jpg)
22
Central Limit Theorem
Theorem (Central Limit Theorem)(Cont.): The powerful fact is that this theorem holds for any distri
bution of the Xi’s It explains the remarkable fact that the empirical frequen
cies of so many natural “populations” exhibit a bell-shaped (i.e., normal) curve
The term “central limit theorem” traces back to George Polya who first used the term in 1920 in the title of a paper. Polya referred to the theorem as “central” due to its importance in probability theory
![Page 23: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/23.jpg)
23
Central Limit Theorem
The Central Limit Theorem and the Strong Law of Large Numbers are the two fundamental theorems of probability
![Page 24: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/24.jpg)
24
Central Limit Theorem Example 1 (Normal Approximation to the Uniform Sum Di
stribution (a.k.a. the Irwin-Hall Distribution)): Let Xi, i=1,2,… be i.i.d. U(0,1). Compare the graph of the pdf of Y=X1+X2+…+Xn, with the graph of the N(n(1/2), n(1/12)) pdf
n=2
![Page 25: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/25.jpg)
25
Central Limit Theorem
n=4
![Page 26: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/26.jpg)
26
Central Limit Theorem Example 2 (Normal Approximation to the Uniform Sum Di
stribution (a.k.a. the Irwin-Hall Distribution)): Let Xi, i=1,2,…,10 be i.i.d. U(0,1). Estimate P(X1+X2+…+X10 > 7)
Solution: With
and by the central limit theorem,
![Page 27: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/27.jpg)
27
Central Limit Theorem
Example 3 (Normal Approximation to the Chi-Square Distribution): Let X1,X2,…,Xn be i.i.d. N(0,1). Then,
is chi-square with n degrees of freedom, with E(Y)=n and Var(Y)=2n
Recall the pdf of Y is
Let
![Page 28: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/28.jpg)
28
Central Limit Theorem
The pdf of W is given by
Compare the pdf of W and the pdf of N(0,1):
n=20 n=100
![Page 29: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/29.jpg)
29
Approximations for Discrete Distributions
The beauty of the central limit theorem is that it holds regardless of the underlying distribution (even discrete)
![Page 30: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/30.jpg)
30
Approximations for Discrete Distributions
Example 4 (Normal Approximation to the Binomial Distribution): X1,X2,… Xn are random samples from a Bernoulli distribution with μ=p and σ2 = p(1-p). Then, Y=X1+X2+…+Xn is binomial b(n,p). The central limit theorem states that
is N(0,1) as n approaches infinity
![Page 31: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/31.jpg)
31
Approximations for Discrete Distributions
Thus, if n is sufficiently large, the distribution of Y is approximately N(np,np(1-p)), and the probabilities for the binomial distribution b(n,p) can be approximated with this normal distribution, i.e.,
for sufficiently large n
![Page 32: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/32.jpg)
32
Approximations for Discrete Distributions
Consider n=10, p=1/2, i.e., Y~b(10,1/2). Then, by CLT, Y can be approximated by the normal distribution with mean 10(1/2)=5 and variance 10(1/2)(1/2)=5/2. Compare the pmf of Y and the pdf of N(5,5/2):
![Page 33: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/33.jpg)
33
Approximations for Discrete Distributions
Example 5 (Normal Approximation to the Poisson Distribution):
Recall the Poisson pmf
where parameter is both the mean and variance of the distribution
Poisson random variable counts the number of discrete occurrences (sometimes called “events” or “arrivals”) that take place during a time-interval of given length
![Page 34: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/34.jpg)
34
Approximations for Discrete Distributions
A random variable having a Poisson distribution with mean 20 can be thought of as the sum Y of the observations of a random sample of size 20 from a Poisson distribution with mean 1. Thus,
has a distribution that is approximately N(0,1), and the distribution of Y is approximately N(20,20)
![Page 35: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/35.jpg)
35
Approximations for Discrete Distributions
Compare the pmf of Y and the pdf of N(20,20):
![Page 36: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/36.jpg)
36
Markov’s Inequality
Theorem (Markov’s Inequality): If X is a continuous random variable that takes only nonnegative values, then for any a>0,
The inequality is valid for all distributions (discrete or continuous)
![Page 37: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/37.jpg)
37
Markov’s Inequality
Proof:
![Page 38: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/38.jpg)
38
Markov’s Inequality
Intuition behind Markov’s Inequality, using a fair dice (discrete) example:
Then,
…
![Page 39: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/39.jpg)
39
Chebyshev’s Inequality
Theorem (Chebyshev’s Inequality): If X is a continuous random variable with mean and variance 2, then for any k>0,
The inequality is valid for all distributions (discrete or continuous) for which the standard deviation exists
![Page 40: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/40.jpg)
40
Chebyshev’s Inequality
Proof: Since (X-)2 is a nonnegative random variable, we can apply Markov’s inequality (with a=k2) to obtain
Thus,
![Page 41: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/41.jpg)
41
Chebyshev’s Inequality (Another Form)
Chebyshev’s Inequality (another form):
Chebyshev’s inequality states that the probability that X differs from its mean by at least k standard deviations is less than or equal to 1/k2
It follows that the probability that X differs from its mean by less than k standard deviations is at least 1-1/k2
![Page 42: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/42.jpg)
42
Chebyshev’s Inequality
The importance of Markov’s and Chebyshev’s inequalities is that they enable us to derive (sometimes loose but still useful) bounds on probabilities when only the mean, or both the mean and the variance, of the probability distribution are known
![Page 43: 1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.](https://reader036.fdocuments.us/reader036/viewer/2022062305/5697c0131a28abf838cccc10/html5/thumbnails/43.jpg)
43
Chebyshev’s Inequality
Example 1: If it is known that X has a mean of 25 and a variance of 16, then, a lower bound for P(17<X<33) is given by
and an upper bound for P(|X-25|>=12) is
The results hold for any distribution with mean 25 and variance 16