Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards...
Transcript of Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards...
![Page 1: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/1.jpg)
Outline1 Log likelihood and score
2 Fisher information
3 Cramer Rao Lower Bound
4 Hammersley Chapman Robbins IneqReview
Jesinsine quality f EX E Efcx iff is convex if strictly unless HE
Rao Blacku.ee Thm If 210 D convex in dTX sufficient then JCT Eff IT is better
Bias VarianceTradeoff i MSEIO.ro BiasoCS5tVaroCoIf TCX is complete sufficient andglo is U estimable then there existsa unique unbiased estimator of theform JCT and it is UMVU
2 ways to find it1 Find any unbiased SCT2 Rao Blackwellize any unbiased RX
![Page 2: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/2.jpg)
LoglikelihoodyscoreAssume P has densities fo wit m EIRD
Common support x pdx O same to
Recall l 0 x log foldThought of as random function of O
Defy The is TICO x plays a keyrole in many areas of statistics espasymptotes
Can think of as local sufficient statistic i
fq zx ellootz x
Z MCG x
poolx for 2 0
Differentialidentities assuming enough regularity
I faxed dugdF O f eco D ell dµC
E Teco D Ox tonly true if these are theSame value of 0
![Page 3: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/3.jpg)
E o Sc E.io Eo eednEoLEfoo3 iEolEf 8I
VarololcoxB EoL Teco X
JIO X X x 17Same O same 0
Called Fisherinfor mation
It is possible to extend this definition to certain
cases where he is not even differentiable
e g Laplace location family but for our
purposes we can just assume sufficient
regularity
Try with another statistic JCX let
g O Eo Sad unbiased estimator
glo f seederagco fore elder Eoloradrecoix
Codex Teco x s.IE Ere o
![Page 4: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/4.jpg)
Combining these results with Cauchy Schwarz
gives us the CramertowerBoundor Information Lower Bound
1paray Vards varocl.comDZCovoCS Ico xD2
Varo 501 70
OEIRD gCG4R Var d I Tgco Tco bgco
Inter If go is estimand no unbiased estimator
can have smaller variance than Fgco JcojygCo
EI i i d sample
X X dpdC Oe
X pdx Ipo CxiLet L co x logoff xi
LCOix El.co xii
TCO Varocolco xVarocqecoixiNJ 10 where J.CO is Fisher info
in single observationLower bound scales like n t SDI n Ye for regular famili
![Page 5: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/5.jpg)
Efficiency
CR LB is not nec attainable
We define the efficiency of an unbiased estimator as
effs grafts f YI.gg ifgcos oe1R
effoco E 1
We say SCX is efficient if effoco L FO
Depends on Corro Stx Peco X
Corio Kx Ico x
effoldVargo VargCIco
Conf Edo
El
MX is efficient Carrots Eco I FO
Rarely achieved in finite samples but we can
approach it asymptotically as n co
![Page 6: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/6.jpg)
EI Exponential Families
fz x ez Taa Aq Ux
lly x z'Tx Acn log Hx
Fl z x Tix AttyTEX Ez TX
Vargo Varzltan T'AczTH z x PAG
Eal T lez D TAG
So any unbiased est of z has Varzlor Z TAG
![Page 7: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/7.jpg)
ltanmesley.ch RoobinsIneq
CR LB requiresdifferentiation under integral
Can make more general statement if we
replace TKO D with finite difference
Potecx
Fl elcoteix ko
l
I E olio x small E
Eo lot D 1 poder I 1 0
assuming common support or pagespg
Covolo toff D fo eo 1 poder
Eoff Edoglote glo
Varo s z gcote g o2
FEEDCRIB follows from E o but sgp gives begboty
![Page 8: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/8.jpg)
furvedfanity pdx end thx O c IR
ICO x z olTx BCO t log h6
TICO X 7210 Tcx VB co
Tyco TG T A zoo
Oz O TX Eotc x
i meant
n
![Page 9: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/9.jpg)
Doubts about unbiasedness
The 4M VUE might be very inefficientor inadmissible or just dumb in cases where
another approach makes much more sense
Ex X Bin 1000,0Estimate glo IPO Xz soo
UMV4E is 19 25003 why11 500 Conclude gco 100
X 499 Conclude gCO 0
This is not epistemically reasonableCould do much better with e.g ML E or a
Bayes estimator
In fact our theorem should make us
suspicions of UMVU E's everyidiotic function of T is a UMVUE
of its own expectation
![Page 10: Outline - University of California, Berkeleywfithian/courses/stat210a/lecture7.pdfparay Vards varocl.comDZCovoCS Ico xD2 Varo 501 70 OEIRD gCG4R Var d I Tgco Tco bgco Inter If g o](https://reader031.fdocuments.us/reader031/viewer/2022040516/5e74cd219c555b2aa912204e/html5/thumbnails/10.jpg)
Gets worse Ex 4.7 in Keener
X n Truncated Poisson O
foG0 10
X G eo 4 1,3
O 0
EstimategO e
0mass lost 10
truncationKeener shows UMVUE is S C I
Idiotic but we cannot improve using
any unbiased estimator
Sometimes insisting on unbiased ness leadsus to absurd results
Funny example Basu's elephant story on
course website
Unbiasedness has bad reputation but othermethods have their problems too