Lecture 15 Parameter Estimation Using Sample Mean

Last Time Sums of R. V.s

Moment Generating Functions MGF of the Sum of Indep. R.Vs Sample Mean (7.1) Deviation of R. V. from the Expected Value (7.2) Law of Large Numbers (part of 7.3)

Probability & Stochastic ProcessesYates & Goodman (2nd Edition) NTUEE SCC_06_200915 - 1

Lecture 15: Parameter Estimation Using Sample Mean

Today Law of Large Numbers (Cont.) Central Limit Theorem (CLT) Application of CLT The Chernoff Bound Point Estimates of Model Parameters Confidence Intervals

Reading Assignment: 6.6 – 6.8, 7.3-7.4

Probability & Stochastic ProcessesYates & Goodman (2nd Edition) NTUEE SCC_06_200915 - 2

Lecture 15: Parameter Estimation Using Sample Mean

Next Time: Final Exam

Scope: Chapters 4 – 7 Time: 15:30 -17:30

Reading Assignment: 6.6 – 6.8, 7.3-7.4

Probability & Stochastic ProcessesYates & Goodman (2nd Edition) NTUEE SCC_06_200915 - 3

Law of Large Numbers: Strong and Weak Jakob Bernoulli, Swiss Mathematician, 1654-1705 [Ars Conjectandi, Basileae, Impensis

Thurnisiorum, Fratrum, 1713The Art of Conjecturing; Part Four showing The Use and Application of the Previous Doctrine to Civil, Moral and Economic Affairs Translated into English by Oscar Sheynin, Berlin 2005]

Bernoulli and Law of Large Number.pdf

S&WLLN.doc

Visualization of Law of Large Numbers

15 - 5

Who are they?

For Sum of iid Uniform RVs

For Sum of iid Binomial RVs

15 - 13

Table 1: Normal Distribution Table (from Ulberg, 1987)

Probability and Stochastic ProcessesA Friendly Introduction for Electrical and Computer EngineersSECOND EDITION

Roy D. Yates David J. Goodman

Chapter 7

Parameter Estimation Using the Sample Mean

12 - 43

Theorem 7.7

If X has finite variance, then the sample mean MN(X) is a sequence of consistent estimates of E[X].

Strong Law of Large Numbers Please refer to the supplementary material

12 - 50

Theorem 7.10

E[VN(X)] = (n-1)/n Var[X]

Application to Histogram Construction Application of P(A) estimation to historgram construction

Approximation of CDF

1. Discretization of RV Values

Lecture 15: Parameter Estimation Using Sample Mean

Next Time: Final Exam

Scope: Chapters 4 – 7 Time: 15:30 -17:30

Reading Assignment: 6.6 – 6.8, 7.3-7.4

Probability & Stochastic ProcessesYates & Goodman (2nd Edition) NTUEE SCC_06_200915 - 69