Electron Spin Resonance Study on the Magnetic Properties ...
Properties of Estimators New Update Spin
Transcript of Properties of Estimators New Update Spin
-
8/2/2019 Properties of Estimators New Update Spin
1/43
PROPERTIES OF ESTIMATORS
-
8/2/2019 Properties of Estimators New Update Spin
2/43
1. Unbiased: E (W) = (centered around truevalue)
2. Consistent: W for large number ofobservations3. Efficient: W1 is better than W2 if
4. Minimum Variance: efficient ANDachieve CRLB
5. Sufficient statistic for
22 )2()1( WEWE
-
8/2/2019 Properties of Estimators New Update Spin
3/43
Let be an estimator for , we saythat is unbiased for if E( )= .If E( )= +b() with b()0, then
is a biased estimator for withbias b().
If one writes of estimation, then anaccurate estimator would be the one resulting insmall estimation errors, so that estimated values willbe near the true value.
error
-
8/2/2019 Properties of Estimators New Update Spin
4/43
If an estimator tn estimates then differencebetween them (tn- ) is called the estimationerror. Bias of the estimator is defined as theexpectation value of this difference:
B=E(tn-)=E(tn)- If the bias is equal to zero then the estimation
is called unbiased. For example sample meanis an unbiased estimator:
0)(1
)(1
)1
()(111
n
i
n
i
i
n
i
i xEn
xEn
xn
ExE
-
8/2/2019 Properties of Estimators New Update Spin
5/43
Here we used the fact that expectation andsummation can change order (Remember thatexpectation is integration for continuous randomvariables and summation for discrete random
variables.) and the expectation of each samplepoint is equal to the population mean.
Knowledge of population distribution was not
necessary for derivation of unbiasedness of thesample mean. This fact is true for the samplestaken from population with any distribution forwhich the first moment exists..
-
8/2/2019 Properties of Estimators New Update Spin
6/43
A biased estimator can be an unbiasedestimator when n is large.
If , then the estimator is
asymptotically unbiased.
lim ( ) 0n
b
-
8/2/2019 Properties of Estimators New Update Spin
7/43
Suppose that X1, X2,, Xn constitute a randomsample from a distribution with mean 1 andstandard deviation 1, and that Y1, Y2,, Yn(independent of Xis) constitute a random sample
from a distribution with mean 2and standarddeviation 2.
Use the rules of expected value to show that X Yis an unbiased estimator of1 -2.
Solution:21
][][][
YEXEYXE
-
8/2/2019 Properties of Estimators New Update Spin
8/43
We have shown that when sampling from
, one finds that the maximum likelihood estimators of andare
Recalling that the distribution of
Thus, we know that the distribution of
Hence,
2
1 2( , )N
2
22
1 2
( 1)
andn S
Xn
2is ( , ), we see that ( ) ;X N E X
n
is unbiased estimator of .X
2 2 2 2
12
1
( 1) 1is where ( )
1
n
n i
i
nS S X X
n
2 2 22 2
2
( 1)( ) ( 1)
1 1
n SE S E n
n n
-
8/2/2019 Properties of Estimators New Update Spin
9/43
Therefore, S2is an unbiased estimator of2
Consequently, since
Therefore,
2 2 2
2
( 1) ( 1) ( 1)( ) [ ] [ ]
n n nE E S E S
n n n
22 2
is a biased estimator of
-
8/2/2019 Properties of Estimators New Update Spin
10/43
Expectation value of the square of the differencesbetween estimator and the expectation of theestimator is called its variance:
Exercise: What is the variance of the sample mean.
As we noted if estimator for is tn then difference
between them is error of the estimation. Expectationvalue of this error is bias. Expectation value of squareof this error is called mean square error (m.s.e.):
2)( ntEM
2))(( nn tEtEV
-
8/2/2019 Properties of Estimators New Update Spin
11/43
It can be expressed by the bias and thevariance of the estimator:
M.S.E is equal to square of the estimators bias plusvariance of the estimator. If the bias is 0 then MSE isequal to the variance.
In estimation it is usually trade of betweenunbiasedness and minimum variance. In ideal worldwe would like to have minimum variance unbiasedestimator. It is not always possible.
)()(
))(())(())()(()()(
2
2222
nn
nnnnnnnn
tBtV
tEtEtEtEtEtEtEtM
-
8/2/2019 Properties of Estimators New Update Spin
12/43
-
8/2/2019 Properties of Estimators New Update Spin
13/43
UNBIASED!!!!
-
8/2/2019 Properties of Estimators New Update Spin
14/43
Let S2 be a sample variance for a random sample from and . Is this
estimator biased or unbiased estimator? Note: meanof chi-square is it df.Solution:
since E(S2)2; therefore S2 is biased estimator and the
biased value is - 2
/n.
1 2, ,..., nX X X
2( , )N
2
2( )
ix x
sn
222 2
1 12
2
2 22
22
2
2
22
( )( 1) ~ ; ~
( )( ) [ ]
( )[ ]
( 1)
i
n n
i
i
x xsn
n
x xE s E
n
x xE
n
nn
n
-
8/2/2019 Properties of Estimators New Update Spin
15/43
If an estimate is such that its biasedconverged to zero as n, then we say that itis asymptotically unbiased.
S2 is asymptotically unbiased when n
22 2
2 2
( )
lim ( ) .n
E s n
E s
-
8/2/2019 Properties of Estimators New Update Spin
16/43
Solution:
2
2
( )If , does this estimator is unbiased
1
estimator of ?
ix x
wn
2
2
2
22
2
2
2
( )
( ) [ ]( 1)
( )[ ]
1
( 1)1
i
i
x x
E w E n
x xE
n
nn
w is unbiased estimator for 2
-
8/2/2019 Properties of Estimators New Update Spin
17/43
THEOREM Let be an unbiased estimator for
based on a random sample of size n,from function , assume we can differentiate
inside the integral whenever needed, then CRLB forthe Var(y) is given by
1 2( , ,..., )ny U x x x
1 2, ,..., nx x x
( ; )f x
2
2
22
2
1
ln ( , )
1
ln ( , )
y
f xnE
f xn E
Cramer RaoLower bound
-
8/2/2019 Properties of Estimators New Update Spin
18/43
Corrollary Suppose y is nor unbiased; say E(y)=+b(),
then CRLB for y is:
2
2
22
2
1 '( )
ln ( , )y
b
f xn E
-
8/2/2019 Properties of Estimators New Update Spin
19/43
Get the CRLB and find the variance for thisestimator
1,0
( ; ) ; ~ ( 1, )
0 ,
x
e xf x y gamma
elsewhere
-
8/2/2019 Properties of Estimators New Update Spin
20/43
Solution:
2
2
2 2 3
2
2 2 3
2 2
2
ln ( ; ) ln
ln ( ; ) 1
ln ( ; ) 1 2
ln ( ; ) 1 2 ( )[ ]
1 2
1
xf x
f x x
f x x
f x E xE E
2
2
2
2
1
1
CRLB:
y
y
n
n
2 2 2
1
( ) 1.
( ) 1.
; ( )
E X
V X
xM x E X
n
x
-
8/2/2019 Properties of Estimators New Update Spin
21/43
Therefore, variance for equal to CRLB.
If an unbiased estimator achive the CRLB;
we say that is efficient for estimating andis called uniformly minimum- variance (orbest) unbiased estimator
2
2
22 2
2 2
( ) ( ) ( )
1
( )
1( )
1 1
i
i
xV V x V
n
V xn
V xn
nn n n
-
8/2/2019 Properties of Estimators New Update Spin
22/43
If we consider all possible unbiased estimator ofsome parameter , the one with the smallestvariance is called the most efficient estimator of .
1
2
3
Sampling distribution of different estimators of
Unbiasedestimators
is the bestestimator of
1
-
8/2/2019 Properties of Estimators New Update Spin
23/43
Let be an estimator that is unbiased for .The relative efficiency for is defined to be:
( ) 1( )
CRLBeffV
-
8/2/2019 Properties of Estimators New Update Spin
24/43
If and are biased estimators of parameter ofa given population, the measure of efficiency ofrelative to is denoted by the ratio
where
1 2
2 2
( ) [( ) ] ( ) ( )MSE E Var b
1
2
2
1 2
1
( ) ( , )( )
MSEeffMSE
Mean square error
CAREFUL HERE
-
8/2/2019 Properties of Estimators New Update Spin
25/43
What it means Conclusion21 2
1
( ) ( , )
( )
MSEeff
MSE
1 2 ( , ) 1eff
1 2 ( , ) 1eff
1 2 ( , ) 1eff
1 2 ( ) ( )MSE MSE
1 2 ( ) ( )MSE MSE
1 2 ( ) ( )MSE MSE
1 2
1 2
1 2
as efficient as
more efficientthan
less efficientthan
-
8/2/2019 Properties of Estimators New Update Spin
26/43
If and are unbiased estimators of parameter of a given population, and , then we saythat is relatively more efficient than .
The measure of efficiency of relative to isdenoted by the ratio
1 2
1 2 ( ) ( )Var Var
1
2
1
2
2
1 21
( ) ( , )
( )
Vareff
Var
CAREFUL HERE
-
8/2/2019 Properties of Estimators New Update Spin
27/43
What it means Conclusion21 2
1
( ) ( , )
( )
Vareff
Var
1 2 ( , ) 1eff
1 2 ( , ) 1eff
1 2 ( , ) 1eff
1 2 ( ) ( )Var Var
1 2 ( ) ( )Var Var
1 2 ( ) ( )Var Var
1 2
1 2
1 2
as efficient as
more efficientthan
less efficientthan
-
8/2/2019 Properties of Estimators New Update Spin
28/43
-
8/2/2019 Properties of Estimators New Update Spin
29/43
Suppose that X1 and X2 are random samplesfrom a normal distribution with mean andstandard deviation . Two estimators formean are as below:
Check whether the two estimators andare unbiased.
Determine the measure of efficiency ofrelative to .
1 21
2
3
X X
1 22
2
X X
1
2
1
2
-
8/2/2019 Properties of Estimators New Update Spin
30/43
Check whether the two estimators and areunbiased.
1 21
1 2
2( )
3
1 2( ) ( )
3 3
1 2
3 3
X XE E
E X E X
1 22
1 2
( )2
1 1( ) ( )
2 2
1 1
2 2
X XE E
E X E X
1
2
Since both and ,then the twoestimators and are unbiased.
1( )E 2( )E
1
2
-
8/2/2019 Properties of Estimators New Update Spin
31/43
Determine the measure of efficiency of relative to
1 21
1 2
2 2
2
2( )
3
1 4( ) ( )9 9
1 4
9 9
59
X XVar Var
Var X Var X
1 2
2
1 2
2 2
2
( )2
1 1( ) ( )4 4
1 1
4 4
12
X XVar Var
Var X Var X
1
2
Since both and ,then the twoestimators and are unbiased.
1( )E 2( )E
1
2
-
8/2/2019 Properties of Estimators New Update Spin
32/43
So, the measure of efficiency of relative to
2
1 2
1
2
2
( , )
2
59
90.9 110
Vareff
Var
1 2
Since that is ,then theestimator is 0.9 times more efficient than .
12
1 2 ( , ) 1eff 1 2
( ) ( )Var Var
-
8/2/2019 Properties of Estimators New Update Spin
33/43
Suppose that X1 , X2 and X3 arerandom samples from a normaldistribution with mean and standard
deviation . Determine the efficiencymeasure of relative to1 2 324
X X X
1 2 3
3
X X X
-
8/2/2019 Properties of Estimators New Update Spin
34/43
-
8/2/2019 Properties of Estimators New Update Spin
35/43
-
8/2/2019 Properties of Estimators New Update Spin
36/43
An unbiased estimator is said to beconsistent if the difference between theestimator and the parameter grows smaller
as the sample size grows larger. E.g. is a consistent estimator of
because: V(X) is
That is, as ngrows larger, the variance of
grows smaller.
X
X
-
8/2/2019 Properties of Estimators New Update Spin
37/43
-
8/2/2019 Properties of Estimators New Update Spin
38/43
Let be a random sample fromfunction f(x;), . Let y1=U( ) be astatistic with pdf g(y1,). We say that y1 is asufficient statistic for the family if and only if
Where for every fix value of thefunc H( ) does not depend on , thusthe conditional pdf of given
is free of
1 2, ,..., nx x x
1 2, ,..., nx x x
( , ),iF f x
1 2 1 21 1
1 2
, ( , )... ( , ) joint pdf for , ,...,
( , ) ( , )
( , ,..., )
n n
n
f x f x f x x x x
g y g y
H x x x
y1=U( )1 2, ,..., nx x x
1 2, ,..., nx x x
1 2, ,..., nx x x 1 1Y y
-
8/2/2019 Properties of Estimators New Update Spin
39/43
-
8/2/2019 Properties of Estimators New Update Spin
40/43
i. find joint pdf for
ii.
Conc: the given func. Is a sufficient statistic.
1 2, ,..., nx x x
1 2
1 1
( , ) ( , )... ( , ) ( , )!
ixn n
n i
i i
ef x f x f x f x
x
1! ( )!!
!
( )! ( )!
ii
i i
xnxn
i ii
x xn ni
i i
eex xx
n xn e n e
x x
Free of
-
8/2/2019 Properties of Estimators New Update Spin
41/43
Note: There is another way to find sufficient statistic by
Neymans Factorization Theorem.
Theorem: Let be a random sample from function
f(x;), . Let y1=U( ) be a statistic of
g(y1,) that y1 is a sufficient statistic for family if and only if there is exist a
function k1(.) if k2(.)where for each fix value of y1=U( ) thefunction k
2( ) does not depend on in
form or in domain.
1 2, ,..., nx x x
1 2, ,..., nx x x
( , ),iF f x 1 1 1 2 2 1 2
1
( ; ) ( ( , ,..., )) ( , ,..., )n
n n
i
f x k U x x x k x x x
1 2, ,..., nx x x
1 2, ,..., nx x x
Is just a statistic
-
8/2/2019 Properties of Estimators New Update Spin
42/43
Let is a r.s from
Solution:
1 2, ,..., nx x x
1
2
(2 )[ (1 )] ,0 1
[ ( )]( ; )
0 ,
Find sufficient statistic for
x x xf x
elsewhere
-
8/2/2019 Properties of Estimators New Update Spin
43/43