Econometrics I 18

7/25/2019 Econometrics I 18

1/49

Part 18: Maximum Likelihood Estimation8-1/49

Econometrics I

Professor William Greene

Stern School of Business

Deartment of Economics


2/49


3/49


Econometrics I

Part 18 Maximum

Likelihood Estimation


4/49


Maximum Likelihood Estimation

!his defines a class of estimators "ased on the articulardistri"ution assumed to ha#e $enerated the o"ser#edrandom #aria"le%

&ot estimatin$ a mean ' least s(uares is not a#aila"le

Estimatin$ a mean )ossi"l*+, "ut also usin$ information

a"out the distri"ution


5/49


-d#anta$e of !he MLE

The main adanta!eof ML estimators is thatamon$ all Consistent Asymptotically NormalEstimators, MLEs ha#e o"timal as#m"toti$

"ro"erties%The main disadanta!eis that the* are not

ne$essaril# ro&ust to failures of thedistri"utional assumtions% !he* are #er*

deendent on the articular assumtions%The o't $ited disadanta!eof their mediocre

small samle roerties is o#erstated in #ie. ofthe usual aucit* of #ia"le alternati#es%


6/49

Part 18: Maximum Likelihood Estimation8-(/49

Proerties of the MLE

)onsistent: &ot necessaril* un"iased, ho.e#er *s#m"toti$all# normall# distri&uted: Proof

"ased on central limit theorems

*s#m"toti$all# e''i$ient: -mon$ the ossi"leestimators that are consistent and as*mtoticall*

normall* distri"uted ' counterart to Gauss/

Marko# for linear re$ression +nariant: !he MLE of $)+ is $)the MLE of +


7/49Part 18: Maximum Likelihood Estimation8-,/49

Settin$ 0 the MLE

The distri&ution o' the o&sered randomaria&le is ritten as a 'un$tion o' the"arameters to &e estimated

P)*idata,.+ 2 Pro"a"ilit* densit* arameters%

The likelihood 'un$tion is $onstru$ted 'rom thedensit#

3onstruction: 4oint ro"a"ilit* densit* functionof the o"ser#ed samle of data ' $enerall* theroduct .hen the data are a randomsamle%


8/49Part 18: Maximum Likelihood Estimation8-8/49

)Lo$+ Likelihood 5unction

f)*i,xi+ 2 ro"a"ilit* densit* of o"ser#ed *i

$i#en arameter)s+ and ossi"l* data, xi%

6"ser#ations are indeendent 4oint densit* 2 if)*i,xi+ 2 L)#0+ f)*i,xi+ is the contri"ution of o"ser#ation i to

the likelihood% !he MLE of maximi7es L)#0+ In ractice it is usuall* easier to maximi7e

lo$L)#0+ 2 ilo$f)*i,xi+



!he MLE

!he lo$/likelihood function: lo$L)data+

!he likelihood e(uation)s+:

5irst deri#ati#es of lo$L e(ual 7ero at the MLE%

)1n+9ilo$f)*ixi+MLE %

am"le statisti$% )!he 1n is irrele#ant%+

;5irst order conditions< for maximi7ation

- moment condition / its counterart is thefundamental theoretical result E=lo$L> 2 %



-#era$e !ime 0ntil 5ailure

Estimatin$ the a#era$e time until failure, , of li$ht "ul"s%*i2 o"ser#ed life until failure%

f)*i+ 2 )1+ex)/*i+

L)+ 2 ? if)*i+2 -nex)/9*i+

lo$L)+ 2 /nlo$ )+ / 9*iLikelihood e(uation: lo$L)+2 /n@ 9*iA2

Solution: MLE 2 9*i n% &ote: E=*i>2 &ote, lo$f)*i+2 /1@ *iA

Since E=*i>2 , E=lo$f)+>2%)6ne of the Ce$ularit* conditions discussed "elo.+



!he Linear )&ormal+ Model

Definition of the likelihood function / oint densit* of theo"ser#ed data, .ritten as a function of the arameters.e .ish to estimate%

Definition of the maximum likelihood estimator as that

function of the o"ser#ed data that maximi7es thelikelihood function, or its lo$arithm%

5or the model: *i 2 xi @ i, .here i &=,A>,the maximum likelihood estimators of and A are&2 )00+/10# and sA2 een%!hat is, least s(uares is ML for the sloes, "ut the#ariance estimator makes no de$rees of freedomcorrection, so the MLE is "iased%



&ormal Linear Model

!he lo$/likelihood function

2 ilo$ f)*i+

2 sum of lo$s of densities%

5or the linear re$ression model .ith normall* distri"uted

distur"ances

lo$L 2 i= /Flo$ A / Flo$ A / F)*i' xi+AA>%

2 /nA=lo$A@ lo$A @ sAA>sA2 een



Likelihood E(uations

!he estimator is defined "* the function of the data that e(uateslo$/Lto % )Likelihood e(uation+

!he deri#ati#e #ector of the lo$/likelihood function is the score function% 5or there$ression model,

! 2 =lo$L, lo$LA>2 lo$L 2 i=)1A+xi)*i/ xi+> 0/2%

lo$LA 2i=-1)AA+ @ )*i/ xi+A)AH+> 2 /&AA=1 ' sAA>

5or the linear re$ression model, the first deri#ati#e #ector of lo$L is

)1A+0)# - 0+ and )1AA+ i=)*i/ xi +AA / 1> )1+ )11+

&ote that .e could comute these functions at anyand A% If .e comutethem at &and e

en, the functions .ill "e identicall* 7ero%



Information Matrix

!he ne$ati#e of the second deri#ati#es matrix of the lo$/

likelihood,

/6 2

forms the "asis for estimatin$ the #ariance of the MLE%

It is usuall* a random matrix%

2

log'

ii

f



Jessian for the Linear Model

7ote that the o'' dia!onal elements hae ex"e$tation ero%

A A

AA

A A

A A A

iAi i

AA

i iA Hi i

lo$L lo$L

lo$L 2 /

lo$L lo$L

1)* +

1

2 1 1)* + )* +

A

i i i i

i i i

x x x x

x x x


16/49Part 18: Maximum Likelihood Estimation8-1(/49

Information Matrix

).hich should look familiar+% !he off dia$onal terms $o to

7ero )one of the assumtions of the linear model+%

!his can "e comuted at an* #ector and scalar A% Koucan take exected #alues of the arts of the matrix to $et

A i

H

1

/E= >2n

A

i ix x 2

6

2


17/49



)om"utin! the *s#m"toti$ arian$e

We .ant to estimate /E=6>N/1 !hree .a*s:

)1+ 4ust comute the ne$ati#e of the actual second deri#ati#es matrixand in#ert it%

)A+ Insert the maximum likelihood estimates into the kno.n exected

#alues of the second deri#ati#es matrix% Sometimes )1+ and )A+ $i#ethe same ans.er )for examle, in the linear re$ression model+%

)O+ Since E=6> is the #ariance of the first deri#ati#es, estimate this .iththe samle #ariance )i%e%, mean s(uare+ of the first deri#ati#es, thenin#ert the result% !his .ill almost al.a*s "e different from )1+ and)A+%

Since the* are estimatin$ the same thin$, in lar$e samles, all three.ill $i#e the same ans.er% 3urrent ractice in econometrics oftenfa#ors )O+% Stata rarel* uses )O+% 6thers do%


19/49



-lication: Doctor isits

German Indi#idual Jealth 3are data: n2A,AOQ

Model for num"er of #isits to the doctor: Poisson re$ression )fit "* maximum likelihood+

Income, Education, Gender



Poisson Model

[ ]'

ensit of 3bserve*

exp( )Prob! " # + $ "

Log Li0eli1oo*

logL( + % ) " log log

Li0eli1oo* E.uations " erivatives of log li0eli1oo*

exp( ) exp( ) "

j

i ii i

n

i i i ii

i i ii

j

y y=

+

= =

x

y X

x xx

[ ]

[ ]

'

' '

log "

" "

i i

n

i i i ii

n n

i i i i ii i

L y

y

=

= =

+ =

x

x x 0

x x


22/49


-s*mtotic ariance of the MLE

4ariance of t1e first *erivative vector:

3bservations are in*epen*ent 5irst *erivative vector

is t1e sum of n in*epen*ent terms 61e variance is t1e

sum of t1e variances 61e variance of eac1 term is

4

ar! ( )$ " 4ar! $ "

-umming terms

log4ar

i i i i i i i i i i

ni i ii

y y

L=

= =

x x x x x

x x XX


23/49


Estimators o' the *s#m"toti$ )oarian$e

*s#m"toti$ )oarian$e Matrix

( )

Conventional Estimator / 7nverse of t1e information matrix

89sual8

8ern*t% ;all% ;all% ;ausman8 (;;;)

!( ) $!( ) $

n

i i ii

n n

i i i i i i i ii i

y y

=

= =

=

=

x x XX

g g x x

2

( )

n

i i i iiy

= = x x


24/49


:o&ust Estimation

andi$h Estimator

6

-1

;; 6

-1

+s this a""ro"riate< =h# do e do this


33/49


!estin$ the Model(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%(

) Poisson Aeression )

) Maximum >iBelihood 5stimates )

) ependent varia*le @C42+ )

) Num*er of o*servations 3. )

) 2terations completed )) >o liBelihood function %10.1 ) >o liBelihood

) Num*er of parameters - )

) Aestricted lo liBelihood %10/..#1 ) >o >iBelihood with onl a

) Mc'adden Pseudo A%sDuared #0&13 ) constant term#

) Chi sDuared -/3#/3 ) 7lo> E lo>F0G

) erees of freedom 3 )

) Pro*7Chi+Dd 9 value = #0000000 )

(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%(

Likelihood ratio test that all three sloes are 7ero%


34/49


Wald

!est--> M*T:+0 ? List ? &1 &2@4 ? 11 ar&2@42@4 ? A1BC11>A1DMatrix H1 Matrix 411

has 3 rows and 1 columns# has 3 rows and 3 columns

1 1 3

(%%%%%%%%%%%%%% (%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

1) #31&- 1) #-/&.&%0- %#-&&.0.%0. #1.&%0&

) %#&-. ) %#-&&.0.%0. #000-/ %#1.0&&/%0&3) %#0-/ 3) #1.&%0& %#1.0&&/%0& #//-.&%0&

Matrix Aesult has 1 rows and 1 columns#

1

(%%%%%%%%%%%%%%

1) -./#3/ >A statistic was -/3#/3


35/49


3ho. St*le !est for Structural 3han$e

oes t1e same mo*el appl to 2 (e use* t1e 8C1o>8 (5) test

5or mo*els fit b maximum li0eli1oo*% >e use a

test base* on t1e li0eli1oo* function 61e same

mo*el is fit t

( )'

o t1e poole* sample an* to eac1 group

C1i s.uare* " 2 log log

egrees of free*om " (


36/49

Part 18: Maximum Likelihood Estimation8-3(/49

Poisson Ce$ressions%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Poisson Aeression

ependent varia*le @C42+

>o liBelihood function %0//#01&3 FPooled, N = 3.G

>o liBelihood function %-3/.#-01 FMale, N = 1--3G

>o liBelihood function %-.&/#00 F'emale, N = 130/3G

%%%%%%%%(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

4aria*le) Coefficient +tandard 5rror *6+t#5r# P7)8)9: Mean of hs = ocvis ; Ahs = < $

Calc ; >male = lol $

Poisson ; 'or 7female = 1 ; >hs = ocvis ; Ahs = < $Calc ; >female = lol $

Calc ; K = ColFist

; ChisD = F>male ( >female % >poolG

; Ct*F#&,BG $

(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%(

) >isted Calculator Aesults )

(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%(C"2+L = 00#01.01

Aesult= 1#&1&/

Jhe hpothesis that the same model applies to men and

women is reected#


38/49


Pro"erties o' the

Maximum Likelihood EstimatorWe .ill sketch formal roofs of these results:

!he lo$/likelihood function, a$ain!he likelihood e(uation and the information matrix%

- linear !a*lor series aroximation to the first order conditions:

!)ML+ 2 !)+ @ J)+ )ML - +

)under re$ularit*, hi$her order terms .ill #anish in lar$e samles%+6ur usual aroach% Lar$e samle "eha#ior of the left and ri$ht hand sides is the same%* Proo' o' $onsisten$#% )Proert* 1+!he limitin$ #ariance of n)ML - +% We are usin$ the central limit theorem here%Leads to as#m"toti$ normalit# )Proert* A+% We .ill deri#e the as*mtotic #ariance of

the MLE%Estimatin! the arian$eof the maximum likelihood estimator%E''i$ien$#).e ha#e not de#eloed the tools to ro#e this%+ !he 3ramer/Cao lo.er

"ound for efficient estimation )an as*mtotic #ersion of Gauss/Marko#+%+narian$e% )- VERYhand* result%+ 3ouled .ith the Slutsk* theorem and the delta

method, the in#ariance roert* makes estimation of nonlinear functions ofarameters #er* eas*%


39/49


Ce$ularit* 3onditions

Deri#in$ the theor* for the MLE relies on certain ;re$ularit*1ere an*

MLE

n n

MLE i ii i

i i

i i

f f

= =

= =

H g

H g

g H


41/49


)onsisten$#

( ) ( ) ( )

( ) ( ) ( )

'

' '

'

' '

@

ivi*e bot1 sums b t1e sample sie

' ' '@ " o

61e approximation is no> exact because of t1e 1ig1er or*er termAs n

n n

MLE i ii i

n n

MLE i ii in n n

= =

= =

+

H g

H g

( ) ( ){ }

( ) ( )

''

'

'

% t1e t1ir* term vanis1es 61e matrices in brac0ets are sample

means t1at converge to t1eir expectations

'% a positive *efinite matrix

' % one of

n

i ii

ni ii

En

En

=

=

=

H H

g g 0

( )

t1e regularit con*itions

61erefore% collecting terms%

@ @ or plim "MLE MLE 0


42/49


43/49


*s#m"toti$ arian$e


44/49



45/49



46/49

Part 18: Maximum Likelihood Estimation8-4(/49

*s#m"toti$ istri&ution


47/49

Part 18: Maximum Likelihood Estimation8-4,/49

E''i$ien$#@ arian$e Aound


48/49


In#ariance

!he maximum likelihood estimator of a function of

, sa* h)

+ is h)MLE+% !his is not al.a*s true ofother kinds of estimators% !o $et the #ariance of

this function, .e .ould use the delta method%

E%$%, the MLE of F2).T+ is &)een+


49/49

Estimatin$ the !o"it Model

+ = >

i i

x x n ii ii=1

i i i

Log likelihood for the tobit model for estimation of and :

y1logL= (1-d)log d log

d 1 if y 0, 0 if y = 0. Derivatives are very comlicated,

!essian is

( ) ( )( )

( ) ( )

+ + + + +

i ii i

= -

x x

x x

n

i i ii=1

"

i i i

nightmarish. #onsider the $lsen transformation%:

=1& , =- & . ($ne to one' =1& , &

logL= log (1-d)log d log y

log (1-d) log d(log (1 & ")log" (1& ") y )

( )

( )

=

=

=

=

i

i

i

xx

x

n

i=1

n

i i ii 1

n

i i ii 1

logL(1-d) de

logL 1d ey

%ote on the ni*+eness of the L in the obit odel,/ conometrica, 12.

Econometrics I 18

Documents

Transcript of Econometrics I 18