1 UNBIASEDNESS AND EFFICIENCY Much of the analysis in this course will be concerned with three...

1

UNBIASEDNESS AND EFFICIENCY

Much of the analysis in this course will be concerned with three properties of estimators: unbiasedness, efficiency, and consistency. The first two, treated here, relate to finite sample analysis: analysis where the sample has a finite number of observations.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

2


Consistency, a property that relates to analysis when the sample size tends to infinity, is treated in a later slideshow.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

3


Suppose that you wish to estimate the population mean X of a random variable X given a sample of observations. We will demonstrate that the sample mean is an unbiased estimator, but not the only one.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

4


We will start with the proof in the previous sequence. We use the second expected value rule to take the 1/n factor out of the expectation expression.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

5


Next we use the first expected value rule to break up the expression into the sum of the expectations of the observations.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

6


Thinking about the sample values {X1, …, Xn} at the planning stage, each expectation is equal to X, and hence the expected value of the sample mean, before we actually generate the sample, is X.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

7


However, the sample mean is not the only unbiased estimator of the population mean. We will demonstrate this supposing that we have a sample of two observations (to keep it simple).

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

Generalized estimator 2211 XXZ

8


We will define a generalized estimator Z which is the weighted sum of the two observations, 1 and 2 being the weights.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X


9


We will analyze the expected value of Z and determine the condition that must be satisfied by the weights for Z to be an unbiased estimator.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

X

XXXEXE

XEXEXXEZE

212211

22112211


10


We begin by decomposing the expectation using the first expected value rule.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

X

XXXEXE

XEXEXXEZE

212211

22112211


11


Now we use the second expected value rule to bring 1 and2 out of the expected value expressions.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

X

XXXEXE

XEXEXXEZE

212211

22112211


12


The expected value of X in each observation, before we generate the sample, is X.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

X

XXXEXE

XEXEXXEZE

212211

22112211


Thus Z is an unbiased estimator of X if the sum of the weights is equal to one. An infinite number of combinations of 1 and 2 satisfy this condition, not just the sample mean.

XXn

nn

nn

XEXEn

XXEn

XXn

EXE

1...

1

...1

...1

1

11

Unbiasedness of X

X

XXXEXE

XEXEXXEZE

212211

22112211

121 if

13



How do we choose among them? The answer is to use the most efficient estimator, the one with the smallest population variance, because it will tend to be the most accurate.

14


0

0

pro

bab

ilit

y d

ensi

ty

estimator B

estimator A

X

Efficiency

In the diagram, A and B are both unbiased estimators but B is superior because it is more efficient.

15


0

0

pro

bab

ilit

y d

ensi

ty

estimator B

estimator A

X

Efficiency

16


We will analyze the variance of the generalized estimator and find out what condition the weights must satisfy in order to minimize it.


Efficiency of X

2

121

221

21

222

221

222

221

22112211

22112

122

1

,cov2varvar

var

21

X

X

XX

XX

Z

XXXX

XX

17


The first variance rule is used to decompose the variance.


2

121

221

21

222

221

222

221

22112211

22112

122

1

,cov2varvar

var

21

X

X

XX

XX

Z

XXXX

XX

Efficiency of X

18


Note that we are assuming that X1 and X2 are independent observations and so their covariance is zero. The second variance rule is used to bring 1 and 2 out of the variance expressions.


2

121

221

21

222

221

222

221

22112211

22112

122

1

,cov2varvar

var

21

X

X

XX

XX

Z

XXXX

XX

Efficiency of X

19


The variance of X1, at the planning stage, is X2. The same goes for the variance of X2.


2

121

221

21

222

221

222

221

22112211

22112

122

1

,cov2varvar

var

21

X

X

XX

XX

Z

XXXX

XX

Efficiency of X

20


Now we take account of the condition for unbiasedness and re-write the variance of Z, substituting for 2.

121 if


2

121

221

21

222

221

222

221

22112211

22112

122

1

,cov2varvar

var

21

X

X

XX

XX

Z

XXXX

XX

Efficiency of X

2

121

221

21

222

221

222

221

22112211

22112

122

1

,cov2varvar

var

21

X

X

XX

XX

Z

XXXX

XX

121 if

21



The quadratic is expanded.

Efficiency of X

22


To minimize the variance of Z, we must choose 1 so as to minimize the final expression.

21

21

2 122 XZ


Efficiency of X

23


We differentiate with respect to 1 to obtain the first-order condition.

5.00240dd

2111

2

Z

21

21

2 122 XZ


Efficiency of X

5.00240dd

2111

2

Z

The expression is minimized for 1 = 0.5. It follows that 2 = 0.5 as well. So we have demonstrated that the sample mean is the most efficient unbiased estimator, at least in this example. (Note that the second differential is positive, confirming that we have a minimum.) 24


21

21

2 122 XZ


04d

d21

22

Z

Efficiency of X

Alternatively, we could find the minimum graphically. Here is a graph of the expression as a function of 1.

25


0

0.2

0.4

0.6

0.8

1

1.2

0 0.2 0.4 0.6 0.8 1

1f

1

Efficiency of X

Again we see that the variance is minimized for 1 = 0.5 and so the sample mean is the most efficient unbiased estimator.

0

0.2

0.4

0.6

0.8

1

1.2

0 0.2 0.4 0.6 0.8 1

1f

1

26


Efficiency of X

Copyright Christopher Dougherty 2012.

These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be

used as a resource for teaching an econometrics course. There is no need to

refer to the author.

The content of this slideshow comes from Section R.6 of C. Dougherty,

Introduction to Econometrics, fourth edition 2011, Oxford University Press.

Additional (free) resources for both students and instructors may be

downloaded from the OUP Online Resource Centre

http://www.oup.com/uk/orc/bin/9780199567089/.

Individuals studying econometrics on their own who feel that they might benefit

from participation in a formal course should consider the London School of

Economics summer school course

EC212 Introduction to Econometrics

http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx

or the University of London International Programmes distance learning course

EC2020 Elements of Econometrics

www.londoninternational.ac.uk/lse.

2012.10.31

1 UNBIASEDNESS AND EFFICIENCY Much of the analysis in this course will be concerned with three...

Documents

Transcript of 1 UNBIASEDNESS AND EFFICIENCY Much of the analysis in this course will be concerned with three...