6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302...
Transcript of 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302...
![Page 1: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/1.jpg)
6.ParameterEs,ma,on
ECE302Fall2009TR3‐4:15pmPurdueUniversity,SchoolofECE
Prof.IlyaPollak
![Page 2: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/2.jpg)
ParameterEs,ma,on:Framework
Measurement Inference
Unobserved parameter x, a nonrandom but unknown constant
Observation Y, a random variable
Given an observation Y=y, estimate x
![Page 3: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/3.jpg)
ParameterEs,ma,on:Framework
Measurement Inference
Unobserved parameter x, a nonrandom but unknown constant
E.g., • the President’s approval rating
Observation Y, a random variable
Given an observation Y=y, estimate x
![Page 4: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/4.jpg)
ParameterEs,ma,on:Framework
Measurement Inference
Unobserved parameter x, a nonrandom but unknown constant
E.g., • the President’s approval rating • the number of tanks in enemy’s army
Observation Y, a random variable
Given an observation Y=y, estimate x
![Page 5: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/5.jpg)
ParameterEs,ma,on:Framework
Measurement Inference
Unobserved parameter x, a nonrandom but unknown constant
E.g., • the President’s approval rating • the number of tanks in enemy’s army • the bias of a coin
Observation Y, a random variable
Given an observation Y=y, estimate x
![Page 6: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/6.jpg)
ParameterEs,ma,on:Framework
Measurement Inference
Unobserved parameter x, a nonrandom but unknown constant
E.g., • the President’s approval rating • the number of tanks in enemy’s army • the bias of a coin
Observation Y, a random variable
E.g., • people’s responses to poll quesions • serial numbers of captured enemy tanks • results of several flips of the coin
Given an observation Y=y, estimate x
![Page 7: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/7.jpg)
ParameterEs,ma,on:Framework
Measurement Inference
Unobserved parameter x, a nonrandom but unknown constant
E.g., • the President’s approval rating • the number of tanks in enemy’s army • the bias of a coin
Observation Y, a random variable
E.g., • people’s responses to poll quesions • serial numbers of captured enemy tanks • results of several flips of the coin
Given an observation Y=y, estimate x
In order to have any hope of reasonably estimating x from observing Y, the distribution of Y must depend on x.
![Page 8: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/8.jpg)
ParameterEs,ma,on:Framework
Measurement Inference
Unobserved parameter x, a nonrandom but unknown constant
E.g., • the President’s approval rating • the number of tanks in enemy’s army • the bias of a coin
Observation Y, a random variable
E.g., • people’s responses to poll quesions • serial numbers of captured enemy tanks • results of several flips of the coin
Given an observation Y=y, estimate x
In order to have any hope of reasonably estimating x from observing Y, the distribution of Y must depend on x. To emphasize this dependence, we will write: fY(y;x) or pY(y;x) Viewed as a function of x, fY(y;x) or pY(y;x) is called a likelihood function.
![Page 9: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/9.jpg)
ParameterEs,ma,on:Framework
Measurement Inference
Unobserved parameter x, a nonrandom but unknown constant
E.g., • the President’s approval rating • the number of tanks in enemy’s army • the bias of a coin
Observation Y, a random variable
E.g., • people’s responses to poll quesions • serial numbers of captured enemy tanks • results of several flips of the coin
Given an observation Y=y, estimate x
In order to have any hope of reasonably estimating x from observing Y, the distribution of Y must depend on x. To emphasize this dependence, we will write: fY(y;x) or pY(y;x) Viewed as a function of x, fY(y;x) or pY(y;x) is called a likelihood function.
E.g., • Find the value x* of x, which maximizes the probability of the observation Y=y, i.e., the value x* that maximizes the likelihood function fY(y;x) or pY(y;x). This is called the maximum likelihood (ML) estimate.
![Page 10: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/10.jpg)
Parameteres,ma,onfrommul,pleobserva,ons
• Typically,wewilles,mateaparameterxfromseveralobserva,onsY=(Y1,…,Yn).
![Page 11: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/11.jpg)
Parameteres,ma,onfrommul,pleobserva,ons
• Typically,wewilles,mateaparameterxfromseveralobserva,onsY=(Y1,…,Yn).– ORen,itisunrealis,ctoproduceausefules,matebasedonasingleobserva,on.
– E.g.,youcannotreallyes,mateP(heads)foracoinbasedonasinglecointoss.
![Page 12: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/12.jpg)
Parameteres,ma,onfrommul,pleobserva,ons
• Typically,wewilles,mateaparameterxfromseveralobserva,onsY=(Y1,…,Yn).– ORen,itisunrealis,ctoproduceausefules,matebasedonasingleobserva,on.
– E.g.,youcannotreallyes,mateP(heads)foracoinbasedonasinglecointoss.
Given observations Y = Y1,…,Yn( ), an estimator of x is a random variable
of the form X̂ = g(Y) for some function g.
![Page 13: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/13.jpg)
Parameteres,ma,onfrommul,pleobserva,ons
• Typically,wewilles,mateaparameterxfromseveralobserva,onsY=(Y1,…,Yn).– ORen,itisunrealis,ctoproduceausefules,matebasedonasingleobserva,on.
– E.g.,youcannotreallyes,mateP(heads)foracoinbasedonasinglecointoss.
Given observations Y = Y1,…,Yn( ), an estimator of x is a random variable
of the form X̂ = g(Y) for some function g.
Note: the distribution of Y depends on x.Hence, the distribution of X̂ also depends on x.
![Page 14: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/14.jpg)
Parameteres,ma,onfrommul,pleobserva,ons
• Typically,wewilles,mateaparameterxfromseveralobserva,onsY=(Y1,…,Yn).– ORen,itisunrealis,ctoproduceausefules,matebasedonasingleobserva,on.
– E.g.,youcannotreallyes,mateP(heads)foracoinbasedonasinglecointoss.
Given observations Y = Y1,…,Yn( ), an estimator of x is a random variable
of the form X̂ = g(Y) for some function g.
Note: the distribution of Y depends on x.Hence, the distribution of X̂ also depends on x.Sometimes, we will denote the estimator by X̂n , to emphasizethe role of the number of observations.
![Page 15: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/15.jpg)
Parameteres,ma,onfrommul,pleobserva,ons:es,ma,onerrorandbias
Suppose we have an estimator X̂n of x based on the observations of Y = Y1,…,Yn( ).The mean Ex X̂n⎡⎣ ⎤⎦ and variance varx X̂n( ) both depend on x.
![Page 16: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/16.jpg)
Parameteres,ma,onfrommul,pleobserva,ons:es,ma,onerrorandbias
Suppose we have an estimator X̂n of x based on the observations of Y = Y1,…,Yn( ).The mean Ex X̂n⎡⎣ ⎤⎦ and variance varx X̂n( ) both depend on x.
The estimation error, denoted Xn , is defined by Xn = X̂n − x
![Page 17: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/17.jpg)
Parameteres,ma,onfrommul,pleobserva,ons:es,ma,onerrorandbias
Suppose we have an estimator X̂n of x based on the observations of Y = Y1,…,Yn( ).The mean Ex X̂n⎡⎣ ⎤⎦ and variance varx X̂n( ) both depend on x.
The estimation error, denoted Xn , is defined by Xn = X̂n − x
The bias of the estimator, denoted bx X̂n( ), is the expected value
of the estimation error: bx X̂n( ) = ExXn⎡⎣ ⎤⎦ = Ex X̂n⎡⎣ ⎤⎦ − x
![Page 18: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/18.jpg)
Parameteres,ma,onfrommul,pleobserva,ons:es,ma,onerrorandbias
Suppose we have an estimator X̂n of x based on the observations of Y = Y1,…,Yn( ).The mean Ex X̂n⎡⎣ ⎤⎦ and variance varx X̂n( ) both depend on x.
The estimation error, denoted Xn , is defined by Xn = X̂n − x
The bias of the estimator, denoted bx X̂n( ), is the expected value
of the estimation error: bx X̂n( ) = ExXn⎡⎣ ⎤⎦ = Ex X̂n⎡⎣ ⎤⎦ − x
X̂n is unbiased if Ex X̂n⎡⎣ ⎤⎦ = x, for every possible value of x.
![Page 19: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/19.jpg)
Parameteres,ma,onfrommul,pleobserva,ons:es,ma,onerrorandbias
Suppose we have an estimator X̂n of x based on the observations of Y = Y1,…,Yn( ).The mean Ex X̂n⎡⎣ ⎤⎦ and variance varx X̂n( ) both depend on x.
The estimation error, denoted Xn , is defined by Xn = X̂n − x
The bias of the estimator, denoted bx X̂n( ), is the expected value
of the estimation error: bx X̂n( ) = ExXn⎡⎣ ⎤⎦ = Ex X̂n⎡⎣ ⎤⎦ − x
X̂n is unbiased if Ex X̂n⎡⎣ ⎤⎦ = x, for every possible value of x.
X̂n is asymptotically unbiased if limn→∞
Ex X̂n⎡⎣ ⎤⎦ = x, for every possible value of x.
![Page 20: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/20.jpg)
Consistentes,mators
X̂n is consistent if the sequence X̂n converges in probability to x,for every possible x
![Page 21: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/21.jpg)
Consistentes,mators
X̂n is consistent if the sequence X̂n converges in probability to x,for every possible x :
limn→∞P X̂n − x ≥ ε( ) = 0,
for every ε > 0 and every possible x.
![Page 22: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/22.jpg)
Meansquaredes,ma,onerror
ExXn2⎡⎣ ⎤⎦ = Ex
Xn⎡⎣ ⎤⎦( )2 + varx Xn( )
![Page 23: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/23.jpg)
Meansquaredes,ma,onerror
ExXn
2⎡⎣ ⎤⎦ = ExXn⎡⎣ ⎤⎦( )2
+ varx Xn( ) = b
x
2 X̂n( ) + varx X̂n − x( )
![Page 24: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/24.jpg)
Meansquaredes,ma,onerror
ExXn
2⎡⎣ ⎤⎦ = ExXn⎡⎣ ⎤⎦( )2
+ varx Xn( ) = b
x
2 X̂n( ) + varx X̂n − x( ) = b
x
2 X̂n( ) + varx X̂n( )
![Page 25: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/25.jpg)
Maximumlikelihoodes,ma,on• Unknownparameterx
• Observa,onsY=(Y1,…,Yn)whosejointdistribu,ondependsonx:– jointPMFpY(y;x)ifYisdiscrete;– jointPDFfY(y;x)ifYiscon,nuous.
![Page 26: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/26.jpg)
Maximumlikelihoodes,ma,on• Unknownparameterx
• Observa,onsY=(Y1,…,Yn)whosejointdistribu,ondependsonx:– jointPMFpY(y;x)ifYisdiscrete;– jointPDFfY(y;x)ifYiscon,nuous.
• Whenviewedasfunc,onsofx,pY(y;x)andfY(y;x)arecalledlikelihoodfunc1ons.
![Page 27: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/27.jpg)
Maximumlikelihoodes,ma,on• Unknownparameterx
• Observa,onsY=(Y1,…,Yn)whosejointdistribu,ondependsonx:– jointPMFpY(y;x)ifYisdiscrete;– jointPDFfY(y;x)ifYiscon,nuous.
• Whenviewedasfunc,onsofx,pY(y;x)andfY(y;x)arecalledlikelihoodfunc1ons.
• Themaximumlikelihoodes,mateofxbasedonanobserva,onyofY:
x̂n =argmax
xpY (y; x) if Y is discrete
argmaxx
fY (y; x) if Y is continuous
⎧⎨⎪
⎩⎪
![Page 28: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/28.jpg)
Ex.9.2:Es,ma,ngthemeanofaBernoullir.v.
• BiasedcoinwithP(heads)=x.• Y1,…,Yn=nindependenttossesofthecoin(Yi=1forheads,Yi=0fortails).
• FindtheMLes,matorforx.
![Page 29: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/29.jpg)
Es,ma,ngthemeanofaBernoullir.v.• BiasedcoinwithP(heads)=x.• Y1,…,Yn=nindependenttossesofthecoin(Yi=1forheads,Yi=0fortails).
• FindtheMLes,matorforx.
• Letk=#headsinntosses.
![Page 30: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/30.jpg)
Es,ma,ngthemeanofaBernoullir.v.• BiasedcoinwithP(heads)=x.• Y1,…,Yn=nindependenttossesofthecoin(Yi=1forheads,Yi=0fortails).
• FindtheMLes,matorforx.
• Letk=#headsinntosses.Then
pY (y; x) = pYi (yi; x)i=1
n
∏
![Page 31: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/31.jpg)
Es,ma,ngthemeanofaBernoullir.v.• BiasedcoinwithP(heads)=x.• Y1,…,Yn=nindependenttossesofthecoin(Yi=1forheads,Yi=0fortails).
• FindtheMLes,matorforx.
• Letk=#headsinntosses.Then
pY (y; x) = pYi (yi; x)i=1
n
∏ = xi:yi =1∏ (1− x)
i:yi =0∏
![Page 32: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/32.jpg)
Es,ma,ngthemeanofaBernoullir.v.• BiasedcoinwithP(heads)=x.• Y1,…,Yn=nindependenttossesofthecoin(Yi=1forheads,Yi=0fortails).
• FindtheMLes,matorforx.
• Letk=#headsinntosses.Then
pY (y; x) = pYi (yi; x)i=1
n
∏ = xi:yi =1∏ (1− x)
i:yi =0∏ = xk (1− x)n− k
![Page 33: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/33.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
![Page 34: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/34.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
It’seasiertomaximizethelog‐likelihood:
ln pY (y; x) = k ln x + (n − k)ln(1− x)
(Takinglogsisokunlessx=0orx=1.)
![Page 35: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/35.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
It’seasiertomaximizethelog‐likelihood:
ln pY (y; x) = k ln x + (n − k)ln(1− x)
(Takinglogsisokunlessx=0orx=1.)Tofindthemax,differen,ate:
∂∂xln pY (y; x) =
kx−n − k1− x
=k − kx − nx + kx
x(1− x)=k − nxx(1− x)
![Page 36: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/36.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
It’seasiertomaximizethelog‐likelihood:
ln pY (y; x) = k ln x + (n − k)ln(1− x)
(Takinglogsisokunlessx=0orx=1.)Tofindthemax,differen,ate:
∂∂xln pY (y; x) =
kx−n − k1− x
=k − kx − nx + kx
x(1− x)=k − nxx(1− x)
x̂n =kn
, x̂n ≠ 0, x̂n ≠ 1
![Page 37: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/37.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
It’seasiertomaximizethelog‐likelihood:
ln pY (y; x) = k ln x + (n − k)ln(1− x)
(Takinglogsisokunlessx=0orx=1.)Tofindthemax,differen,ate:
∂∂xln pY (y; x) =
kx−n − k1− x
=k − kx − nx + kx
x(1− x)=k − nxx(1− x)
x̂n =kn
, x̂n ≠ 0, x̂n ≠ 1
I.e., this derivation will not work for k = 0 and k = n.
![Page 38: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/38.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
It’seasiertomaximizethelog‐likelihood:
ln pY (y; x) = k ln x + (n − k)ln(1− x)
(Takinglogsisokunlessx=0orx=1.)Tofindthemax,differen,ate:
∂∂xln pY (y; x) =
kx−n − k1− x
=k − kx − nx + kx
x(1− x)=k − nxx(1− x)
x̂n =kn
, x̂n ≠ 0, x̂n ≠ 1
I.e., this derivation will not work for k = 0 and k = n.
Checkthatthisisamaximum:∂2
∂x2ln pY (y; x) = −
kx2
−n − k(1− x)2
< 0
![Page 39: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/39.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
x̂n =kn
, unless k = 0 or k = n.
![Page 40: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/40.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
x̂n =kn
, unless k = 0 or k = n.
If k = 0 then pY (y; x) = (1− x)n
![Page 41: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/41.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
x̂n =kn
, unless k = 0 or k = n.
If k = 0 then pY (y; x) = (1− x)n which is maximized by x̂n = 0 =kn
![Page 42: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/42.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
x̂n =kn
, unless k = 0 or k = n.
If k = 0 then pY (y; x) = (1− x)n which is maximized by x̂n = 0 =kn
If k = n then pY (y; x) = xn
![Page 43: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/43.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
x̂n =kn
, unless k = 0 or k = n.
If k = 0 then pY (y; x) = (1− x)n which is maximized by x̂n = 0 =kn
If k = n then pY (y; x) = xn which is maximized by x̂n = 1 = kn
![Page 44: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/44.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
x̂n =kn
, unless k = 0 or k = n.
If k = 0 then pY (y; x) = (1− x)n which is maximized by x̂n = 0 =kn
If k = n then pY (y; x) = xn which is maximized by x̂n = 1 = kn
Hence, the formula x̂n =kn
actually works for any k.
![Page 45: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/45.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
x̂n =kn
, unless k = 0 or k = n.
If k = 0 then pY (y; x) = (1− x)n which is maximized by x̂n = 0 =kn
If k = n then pY (y; x) = xn which is maximized by x̂n = 1 = kn
Hence, the formula x̂n =kn
actually works for any k.
ML estimate: x̂n =kn=y1 +…+ yn
n
![Page 46: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/46.jpg)
Es,ma,ngthemeanofaBernoullir.v.pY (y; x) = x
k (1− x)n− k
x̂n =kn
, unless k = 0 or k = n.
If k = 0 then pY (y; x) = (1− x)n which is maximized by x̂n = 0 =kn
If k = n then pY (y; x) = xn which is maximized by x̂n = 1 = kn
Hence, the formula x̂n =kn
actually works for any k.
ML estimate: x̂n =kn=y1 +…+ yn
n
ML estimator: X̂n =Y1 +…+Yn
n
![Page 47: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/47.jpg)
Es,ma,ngthemeanofaBernoullir.v.
ML estimator: X̂n =Y1 +…+Yn
n
![Page 48: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/48.jpg)
Es,ma,ngthemeanofaBernoullir.v.
ML estimator: X̂n =Y1 +…+Yn
n
E X̂n⎡⎣ ⎤⎦ =E Y1[ ] +…+ E Yn[ ]
n= x, hence, this estimator is unbiased.
![Page 49: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/49.jpg)
Es,ma,ngthemeanofaBernoullir.v.
ML estimator: X̂n =Y1 +…+Yn
n
E X̂n⎡⎣ ⎤⎦ =E Y1[ ] +…+ E Yn[ ]
n= x, hence, this estimator is unbiased.
By WLLN, X̂n → x in probability. Hence, this estimator is consistent.
![Page 50: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/50.jpg)
Germantankproblem• TotalnGermantanks,numbered1,2,…,n.• ThenumbernisunknowntotheAllies.
![Page 51: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/51.jpg)
Germantankproblem• TotalnGermantanks,numbered1,2,…,n.• ThenumbernisunknowntotheAllies.
• Tankswithserialnumbersy1,…,ykhavebeencapturedorotherwiseobservedbytheAllies.
![Page 52: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/52.jpg)
Germantankproblem• TotalnGermantanks,numbered1,2,…,n.• ThenumbernisunknowntotheAllies.
• Tankswithserialnumbersy1,…,ykhavebeencapturedorotherwiseobservedbytheAllies.
• Objec,ve:basedonobservingy1,…,yk,es,maten.
![Page 53: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/53.jpg)
Germantankproblem• TotalnGermantanks,numbered1,2,…,n.• ThenumbernisunknowntotheAllies.
• Tankswithserialnumbersy1,…,ykhavebeencapturedorotherwiseobservedbytheAllies.
• Objec,ve:basedonobservingy1,…,yk,es,maten.
• Model:y1,…,ykareobserva,onsofY1,…,Ykwhicharearandomcombina,onofknumbersfrom{1,2,…,n}.
![Page 54: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/54.jpg)
Germantankproblem• TotalnGermantanks,numbered1,2,…,n.• ThenumbernisunknowntotheAllies.
• Tankswithserialnumbersy1,…,ykhavebeencapturedorotherwiseobservedbytheAllies.
• Objec,ve:basedonobservingy1,…,yk,es,maten.
• Model:y1,…,ykareobserva,onsofY1,…,Ykwhicharearandomcombina,onofknumbersfrom{1,2,…,n}.
• Approach1:MLes,ma,on.
![Page 55: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/55.jpg)
Germantanks:MLes,ma,on
There are nk
⎛⎝⎜
⎞⎠⎟
sets of k distinct numbers which are subsets of {1,2,…,n}.
![Page 56: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/56.jpg)
Germantanks:MLes,ma,on
There are nk
⎛⎝⎜
⎞⎠⎟
sets of k distinct numbers which are subsets of {1,2,…,n}.
Assuming each is equally likely, pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
![Page 57: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/57.jpg)
Germantanks:MLes,ma,on
There are nk
⎛⎝⎜
⎞⎠⎟
sets of k distinct numbers which are subsets of {1,2,…,n}.
Assuming each is equally likely, pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
Thetotalnumberoftanksmustbegreaterthanorequaltothelargestobservedserialnumber,max(y1,…,yk).Let’scallitmk.
![Page 58: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/58.jpg)
Germantanks:MLes,ma,on
There are nk
⎛⎝⎜
⎞⎠⎟
sets of k distinct numbers which are subsets of {1,2,…,n}.
Assuming each is equally likely, pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
Thetotalnumberoftanksmustbegreaterthanorequaltothelargestobservedserialnumber,max(y1,…,yk).Let’scallitmk.
As a function of n, pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if n ≥ mk
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
![Page 59: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/59.jpg)
Germantanks:MLes,ma,on
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if n ≥ mk = max(y1,…yk )
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
![Page 60: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/60.jpg)
Germantanks:MLes,ma,on
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if n ≥ mk = max(y1,…yk )
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
n
pY (y;n)
mk
![Page 61: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/61.jpg)
Germantanks:MLes,ma,on
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if n ≥ mk = max(y1,…yk )
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
n
pY (y;n)
mk
Hence, the ML estimate is n̂k = mk = max(y1,…, yk )
![Page 62: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/62.jpg)
IstheMLes,matorofnunbiased?
![Page 63: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/63.jpg)
IstheMLes,matorofnunbiased?ML estimator is Mk = max(Y1,…,Yk )
![Page 64: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/64.jpg)
IstheMLes,matorofnunbiased?ML estimator is Mk = max(Y1,…,Yk )
Tocomputethebias,wewill:• ComputetheCDFofMk
![Page 65: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/65.jpg)
IstheMLes,matorofnunbiased?ML estimator is Mk = max(Y1,…,Yk )
Tocomputethebias,wewill:• ComputetheCDFofMk• ComputethePMFofMk
![Page 66: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/66.jpg)
IstheMLes,matorofnunbiased?ML estimator is Mk = max(Y1,…,Yk )
Tocomputethebias,wewill:• ComputetheCDFofMk• ComputethePMFofMk• Computethebias,E[Mk]−n
![Page 67: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/67.jpg)
CDFoftheMLes,matorofnML estimator is Mk = max(Y1,…,Yk )
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
![Page 68: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/68.jpg)
CDFoftheMLes,matorofnML estimator is Mk = max(Y1,…,Yk )
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
FMk(r) = P Mk ≤ r( ) = P Y1 ≤ r,…,Yk ≤ r( )
![Page 69: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/69.jpg)
CDFoftheMLes,matorofnML estimator is Mk = max(Y1,…,Yk )
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
FMk(r) = P Mk ≤ r( ) = P Y1 ≤ r,…,Yk ≤ r( )
=1nk
⎛⎝⎜
⎞⎠⎟
⋅ # subsets of {1,2,...,r} of size k[ ]
![Page 70: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/70.jpg)
CDFoftheMLes,matorofnML estimator is Mk = max(Y1,…,Yk )
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
FMk(r) = P Mk ≤ r( ) = P Y1 ≤ r,…,Yk ≤ r( )
=1nk
⎛⎝⎜
⎞⎠⎟
⋅ # subsets of {1,2,...,r} of size k[ ] =0, if r ≤ k −1 ⎧
⎨⎪
⎩⎪
![Page 71: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/71.jpg)
CDFoftheMLes,matorofnML estimator is Mk = max(Y1,…,Yk )
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
FMk(r) = P Mk ≤ r( ) = P Y1 ≤ r,…,Yk ≤ r( )
=1nk
⎛⎝⎜
⎞⎠⎟
⋅ # subsets of {1,2,...,r} of size k[ ] =
0, if r ≤ k −1
rk
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪⎪
⎩
⎪⎪⎪⎪
![Page 72: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/72.jpg)
CDFoftheMLes,matorofnML estimator is Mk = max(Y1,…,Yk )
pY (y;n) =
1nk
⎛⎝⎜
⎞⎠⎟
, if {y1,…, yk} ⊂ {1,2,…,n}
0, otherwise
⎧
⎨
⎪⎪
⎩
⎪⎪
FMk(r) = P Mk ≤ r( ) = P Y1 ≤ r,…,Yk ≤ r( )
=1nk
⎛⎝⎜
⎞⎠⎟
⋅ # subsets of {1,2,...,r} of size k[ ] =
0, if r ≤ k −1
rk
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
1, if r ≥ n
⎧
⎨
⎪⎪⎪⎪
⎩
⎪⎪⎪⎪
![Page 73: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/73.jpg)
PMFoftheMLes,matorofn
FMk(r) = P Mk ≤ r( ) =
0, if r ≤ k −1
rk
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
1, if r ≥ n
⎧
⎨
⎪⎪⎪⎪
⎩
⎪⎪⎪⎪
fMk(r) = P Mk = r( ) = P Mk ≤ r( ) − P Mk ≤ r −1( ) = FMk
(r) − FMk(r −1)
![Page 74: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/74.jpg)
PMFoftheMLes,matorofn
FMk(r) = P Mk ≤ r( ) =
0, if r ≤ k −1
rk
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
1, if r ≥ n
⎧
⎨
⎪⎪⎪⎪
⎩
⎪⎪⎪⎪
fMk(r) = P Mk = r( ) = P Mk ≤ r( ) − P Mk ≤ r −1( ) = FMk
(r) − FMk(r −1)
=0, if r ≤ k −1 and r ≥ n +1?, if r = k,k +1,…,n
⎧⎨⎪
⎩⎪
![Page 75: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/75.jpg)
PMFoftheMLes,matorofn
FMk(r) = P Mk ≤ r( ) =
0, if r ≤ k −1
rk
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
1, if r ≥ n
⎧
⎨
⎪⎪⎪⎪
⎩
⎪⎪⎪⎪
fMk(r) = P Mk = r( ) = P Mk ≤ r( ) − P Mk ≤ r −1( ) = FMk
(r) − FMk(r −1)
=
0, if r ≤ k −1 and r ≥ n +1
rk
⎛⎝⎜
⎞⎠⎟− r −1
k⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
![Page 76: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/76.jpg)
PMFoftheMLes,matorofn
FMk(r) = P Mk ≤ r( ) =
0, if r ≤ k −1
rk
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
1, if r ≥ n
⎧
⎨
⎪⎪⎪⎪
⎩
⎪⎪⎪⎪
fMk(r) = P Mk = r( ) = P Mk ≤ r( ) − P Mk ≤ r −1( ) = FMk
(r) − FMk(r −1)
=
0, if r ≤ k −1 and r ≥ n +1
rk
⎛⎝⎜
⎞⎠⎟− r −1
k⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
=
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
![Page 77: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/77.jpg)
PMFoftheMLes,matorofn
fMk(r) =
0, if r ≤ k −1 and r ≥ n +1
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
![Page 78: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/78.jpg)
Ausefulcombinatorialiden,ty
fMk(r) =
0, if r ≤ k −1 and r ≥ n +1
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
1 = fMk(r)
r= k
n
∑
because fMk is a PMF
![Page 79: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/79.jpg)
Ausefulcombinatorialiden,ty
fMk(r) =
0, if r ≤ k −1 and r ≥ n +1
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
1 = fMk(r)
r= k
n
∑ =
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑ =1nk
⎛⎝⎜
⎞⎠⎟
r −1k −1
⎛⎝⎜
⎞⎠⎟r= k
n
∑
because fMk is a PMF
![Page 80: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/80.jpg)
Ausefulcombinatorialiden,ty
fMk(r) =
0, if r ≤ k −1 and r ≥ n +1
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
1 = fMk(r)
r= k
n
∑ =
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑ =1nk
⎛⎝⎜
⎞⎠⎟
r −1k −1
⎛⎝⎜
⎞⎠⎟r= k
n
∑
Therefore, r −1k −1
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = nk
⎛⎝⎜
⎞⎠⎟
because fMk is a PMF
![Page 81: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/81.jpg)
Anotherusefulcombinatorialiden,ty
Let r ' = r + d +1 and k ' = k + d +1. Then
r + dk + d
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = r '−1k '−1
⎛⎝⎜
⎞⎠⎟r '= k '
n+d+1
∑
![Page 82: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/82.jpg)
Anotherusefulcombinatorialiden,ty
r −1k −1
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = nk
⎛⎝⎜
⎞⎠⎟
Let r ' = r + d +1 and k ' = k + d +1. Then
r + dk + d
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = r '−1k '−1
⎛⎝⎜
⎞⎠⎟r '= k '
n+d+1
∑ = n + d +1k '
⎛⎝⎜
⎞⎠⎟
![Page 83: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/83.jpg)
Anotherusefulcombinatorialiden,ty
r −1k −1
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = nk
⎛⎝⎜
⎞⎠⎟
Let r ' = r + d +1 and k ' = k + d +1. Then
r + dk + d
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = r '−1k '−1
⎛⎝⎜
⎞⎠⎟r '= k '
n+d+1
∑ = n + d +1k '
⎛⎝⎜
⎞⎠⎟= n + d +1
k + d +1⎛⎝⎜
⎞⎠⎟
![Page 84: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/84.jpg)
Anotherusefulcombinatorialiden,ty
r −1k −1
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = nk
⎛⎝⎜
⎞⎠⎟
Let r ' = r + d +1 and k ' = k + d +1. Then
r + dk + d
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = r '−1k '−1
⎛⎝⎜
⎞⎠⎟r '= k '
n+d+1
∑ = n + d +1k '
⎛⎝⎜
⎞⎠⎟= n + d +1
k + d +1⎛⎝⎜
⎞⎠⎟
r + dk + d
⎛⎝⎜
⎞⎠⎟r= k
n
∑ = n + d +1k + d +1
⎛⎝⎜
⎞⎠⎟
![Page 85: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/85.jpg)
MeanoftheMLes,matorofn
fMk(r) =
0, if r ≤ k −1 and r ≥ n +1
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
E Mk[ ] = r
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑
![Page 86: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/86.jpg)
MeanoftheMLes,matorofn
fMk(r) =
0, if r ≤ k −1 and r ≥ n +1
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
E Mk[ ] = r
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑ =
r!(k −1)!(r − k)!
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑
![Page 87: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/87.jpg)
MeanoftheMLes,matorofn
fMk(r) =
0, if r ≤ k −1 and r ≥ n +1
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
E Mk[ ] = r
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑ =
r!(k −1)!(r − k)!
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑
=
k ⋅ r!k!(r − k)!
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑
![Page 88: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/88.jpg)
MeanoftheMLes,matorofn
fMk(r) =
0, if r ≤ k −1 and r ≥ n +1
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
, if r = k,k +1,…,n
⎧
⎨
⎪⎪⎪
⎩
⎪⎪⎪
E Mk[ ] = r
r −1k −1
⎛⎝⎜
⎞⎠⎟
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑ =
r!(k −1)!(r − k)!
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑
=
k ⋅ r!k!(r − k)!
nk
⎛⎝⎜
⎞⎠⎟
r= k
n
∑ =knk
⎛⎝⎜
⎞⎠⎟
rk
⎛⎝⎜
⎞⎠⎟r= k
n
∑
![Page 89: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/89.jpg)
Simplifyingalijlefurther…
E Mk[ ] = knk
⎛⎝⎜
⎞⎠⎟
n +1k +1
⎛⎝⎜
⎞⎠⎟=k ⋅ (n +1)!(k +1)!(n − k)!
n!k!(n − k)!
=k(n +1)!k!(k +1)!n!
=k(n +1)k +1
![Page 90: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/90.jpg)
ThebiasoftheMLes,matorofnb(Mk ) = E Mk[ ]− n = k(n +1)
k +1− n =
k − nk +1
![Page 91: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/91.jpg)
ThebiasoftheMLes,matorofnb(Mk ) = E Mk[ ]− n = k(n +1)
k +1− n =
k − nk +1
Therefore,theMLes,matorisbiased!
![Page 92: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/92.jpg)
ThebiasoftheMLes,matorofnb(Mk ) = E Mk[ ]− n = k(n +1)
k +1− n =
k − nk +1
Therefore,theMLes,matorisbiased!What’stheintui,onhere?
![Page 93: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/93.jpg)
ThebiasoftheMLes,matorofnb(Mk ) = E Mk[ ]− n = k(n +1)
k +1− n =
k − nk +1
Therefore,theMLes,matorisbiased!What’stheintui,onhere?UnlessweobserveALLserialnumbers,weareneverguaranteedtoseethelargestone.Infact,typically,wewillNOTseethelargestone,especiallyifthenumberofobserva,onsisalotsmallerthanthetotalnumberoftanks.
![Page 94: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/94.jpg)
ThebiasoftheMLes,matorofnb(Mk ) = E Mk[ ]− n = k(n +1)
k +1− n =
k − nk +1
Therefore,theMLes,matorisbiased!What’stheintui,onhere?UnlessweobserveALLserialnumbers,weareneverguaranteedtoseethelargestone.Infact,typically,wewillNOTseethelargestone,especiallyifthenumberofobserva,onsisalotsmallerthanthetotalnumberoftanks.So,ifwees,matethenumberoftanksasthelargestserialnumberobserved,wewillsystema,callyunderes,mate.
![Page 95: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/95.jpg)
Anunbiasedes,matorofn
TocorrectforthebiasoftheMLes,mator,let’suseadifferentes,mator:
N̂k =k +1k
Mk −1
E Mk[ ] = k(n +1)k +1
b(Mk ) =k − nk +1
![Page 96: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/96.jpg)
Anunbiasedes,matorofn
TocorrectforthebiasoftheMLes,mator,let’suseadifferentes,mator:
N̂k =k +1k
Mk −1
E Mk[ ] = k(n +1)k +1
b(Mk ) =k − nk +1
E N̂k⎡⎣ ⎤⎦ =k +1k
E Mk[ ]−1 = k +1k
⋅k(n +1)k +1
−1 = n
![Page 97: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/97.jpg)
Anunbiasedes,matorofn
TocorrectforthebiasoftheMLes,mator,let’suseadifferentes,mator:
N̂k =k +1k
Mk −1
E Mk[ ] = k(n +1)k +1
b(Mk ) =k − nk +1
E N̂k⎡⎣ ⎤⎦ =k +1k
E Mk[ ]−1 = k +1k
⋅k(n +1)k +1
−1 = n
The estimator N̂k is therefore unbiased.
![Page 98: 6. Parameter Esmaon - Purdue Engineeringipollak/ece302/FALL09/notes/...6. Parameter Esmaon ECE 302 Fall ... • the number of tanks in enemy’s army • the bias of a coin Observation](https://reader035.fdocuments.us/reader035/viewer/2022070610/5af4bcaa7f8b9a4d4d8e1229/html5/thumbnails/98.jpg)
References• R.BugglesandH.Brodie.AnempiricalapproachtoeconomicintelligenceinWorldWarII.JournaloftheAmericanSta1s1calAssocia1on,42(237):72—91,March,1947.
• Howasta,s,calformulawonthewar,TheGuardian,July20,2006,www.guardian.co.uk/world/2006/jul/20/secondworldwar.tvandradio
• Sametechniquehasrecentlybeenappliedtoes,mateiPhoneproduc,onnumbers,see– WhyiPhonesarejustlikeGermantanks,TheGuardian,Oct.2008,
www.guardian.co.uk/technology/blog/2008/oct/08/iphone.apple