co eff - IMAGINEimagine.enpc.fr/~dalalyan/Download/Grenoble2015_1.pdf · L, ~••). ~ €"+ ~ >,....
Transcript of co eff - IMAGINEimagine.enpc.fr/~dalalyan/Download/Grenoble2015_1.pdf · L, ~••). ~ €"+ ~ >,....
THE.ORETI CAL GuARANTËE.S FOR
THE LASSO
CA. T)ALALYAN)CRE S T - EN SA E" JUNE 1.0 -11
GRENOBLt.
TL.. LASSO CtVl.d relo...+ed -te.ck4t..<..t/) bCèh.e.d ~ c.o~e'X
r-d~xo....t;~ beeGLII\o\ e. '((!'X+~M~ ••(..(.ec~'!>.p bec.Cl..U.4.e ~
· ik, cow\'p \.-l. tJ;~l'\.oJ. eff ici.~'1• s~~ -.tk.o,e.h c.a...! ~-k CM +Wr ~+a.;hs.hc.oJ
"-C.c..lAf"'-~ .T~ l'Y\.(.~ - cctA..C"le w;:1I. to~ ~ ~s s~e..oV\J ~~ec..+.
A (..C.l"c.J~ Jev» ~ F(t>ble~ vJ ;Vl-te("~i ~V'I ~~af;s+ich /ty1a.~n.e..
~~ ~ be ("ùïiH~ M fo~s. \Je. observe +k ~o..btL,Y-1 1"') YO\ QV\ Gl. 5 CI.NV\~ ~ -rt i..r)cLi...vidu.~ (randorn~) dr~1"I
tC"0>'11\ Gt ~oe fopdaliQ-V1, ECÀ.cl eJ ~~ Lnck"iJ~ l~ rl.O'I\.-
exR.~.J;v~ described b~ a. vec+o-r -9J fe.a..t~ (J~o+e.J ~7Xi E rRP r -l:k tVl~"iolt.{.J nurnbe.. l ') .
1~i.!. G\M1M"t'le.d -+h_~ 4-k4+ }cx- ~ peu-o.,W\e'+r; c to..t"t\i,~ [r,.1e>J ~+n6(.J-;~ ~ ~~ve· - ..for st,;)~e r~E; (RP -
• {~eo.,c!. i..) of.' ('t 1 Xi')::: Pxl ~*
•
-1.-
-----_-------- ---------------------
MC\A.Y'\ e.xCc..'W\r~ (..V\ c.I~c\~
G ~QvII\ ~V\..e a.A" m()J-e!.
P"l, ::;: vf (1. )G'"l.) ) wh~ ~~)0 lS a.. vu..c.L~o..Ylœ pa.t"'-l'Mete.r
y~ :: ~ ~* + ~i ) ~Z ~ vi (0 )f5")
.coot-~.l;,- r-e'à\es~
P"t :: Bern()u.~ ( 1:é"rrt )
1
- Po1.~~O'V\Ï"ec~re~~~
p~ -:: Po.-\.ss 0-". ( e' ') 1 1. 6 IR
REMf\RK T~ a.~rem~~ed MOJ~ ~. themb~ oj ~ fa.~~ ~~e.vtet~~e.d ~~ar m.od~ (GLM'). It +k di'me.vvnQ-V\ p QJ +~c.
. .fea.+~ vect&-1' ï.S çmol! compa-red -h,. Tl,) the..rt these moc1e.Js C(.('e pa..ra-
. rne.tric. ((.Md) +kefx-c J a..re. u.,nt4bt. to de~C('i,~e comrfk..x r~lMe",ct.
T~ s+~da.rd. a.tfroa.cl lVl ~uc.lC\. ~Q../.;(w\ L!) +0 eMbed ~kori~~ 1eo..-+~ "ec~ Îvrt-o (À. sf(À.u ~ fY\ucÂ.. EQ.J'~er.dim€.4'lslo-Y'.
. . . . r·· ?MO'rt.. . prec,i,~ ) '-Cft'! c.hoose 0.. f\<)V\- t.VltèC:U- m~p;V\.d ~: R ~ lR )t.ei,.I{. r MIA..ci Q~rd €A +~ p} OI.V\J ref~a.~ +k te()..t~ ve,-- .
· t<X" x~ 1" x::: <f (~i) E IRfi.
FroM r\.Q~. EM) c.J~ Q))S~~ -fk+ f ÎS . ~~e. ) p~~ MUc-t.
~ru-de.r ~ n.. 1V\ lSu.ok ~.. ~~-J;m~~i~~ ~+~Ovt;~J L~j)
· "Lmr~~ +0 Je.v~r ~~~~+S+Gl.k~{1c.a.1· ·l.V\t~~œ wittu,ut· 1u-rHvu- ~·~f-h~·· QI'l ~*. ...... ..
E.><A(.\ SPARSliY ASSUMP11<JN: OVl~ 0.. tec...Y (OOrck~oJe.s t;)J r}f....a,re. ~- ~~. s;::. --H= ~ ~: ~~ ~ 0) ::= Il~~Uo «Y\. •
... -'-- -
ln 0.. ~ ro-rdd SCe.t'lGU'"IO w-Jh a. k~ - clt,mEWJ~<;;w\cJ. pCU'"a.wae-te.rJ
th..re-e m~ rr-obe.e..ms a.re ~d in..tUv>-t,'
• VW~~ee.. seL..c.t;o",: ~iV€A'\ {(Xi) Yi.J}i:21}""I'" provi.de 0vI"I
€l:>tiN"lO.'+Oi 3' c. f i}...)p ~ o} +~ SPC\.rS~td pa..#ern. J~ =::: f J ~ {-1'-"}r ~ :; ~~ ~ o} .
• eA+jrna..~iO'V"' : Con.S+nAC+ ~ b",sed O'\A {(X(.I y;)t .•-i""/1.<
S~ 4.0.,4. for ~ ~t,V~ n..or~ II· Il) -tle. ~ ~e.ôhmwhc;;M. Hf - ~~\\ l~ (),)) ~W'I~ G.4 r~b~.
• rç-e,.ctc.e-+t6"Yl Cot\':.+ru.c-t t s~ 4h~+ :i[~ ts &o..se
.to xI r* .f.o; e~e..('d i. 6 f 1\ r") v'l ~ er 1 Y't'l-OH. 3EU\er~~ ~
tk Ji~+ribJiCW\ ® P ï t.s Jo~e +0 ® P.T * .(. -c '\ Xi, ~ -V" .., :t.-1. ~
ReYY\~~ 1\'"l ~e.o.ero1/ prek-k9V\. 'lS 4k t~ r-e4~rid +k k~~U.Mf{;~ ) ~ke~ VtM""ia.1k ~~d;01,;\ re~u..(..reI) +k S~('Q1n~e...\t-
(.U!.~ff;~, T~ QhS.~ftlO'V\s <.cil 1 he. ~l'YIr~ed~ +kd€/>1..~Vl rn~ +r 1 x
Remru-~ I~ 4k caAe ;t a.. G~~ ~cu- mcJJ. ) fk.e.~ V\
ai5-t-GtNlCQ. be.-tLO"eeVl ~ Ji~A(ibt.c,..tï~ _® PxZll> ~",J ~ P4"(J~l.ZI r 1"z:1 r
M~~re..J. by ~ k~bo..c! - Le(,b~r JLve..r~enc.R. ~s ~eAl by
kLŒp.{~,~p..q~~;:~. !.(:x7~-i[f)'\~JX(~-n!I~.- 3-
IT THE. LASSO AND THE. 'DANi2IG 5ELECTOR
Frol'V'\ f\.O\.J OV'\) we Je~ w-~~ +k. G~ ~e.GU'" ModQ..1.We obse.rv~ (.xL) y~') J l~;fr-"V\ Su.ei tha.+
•)
T~ c~ be. (.Or'I+t~ IV) 0.. ved-o-r tOïYYl CV:>
y= X~~ + ~ 1 ~ rv ~V\.(o)G7I~J. (0)
We o.J.>~l.V'Y\e :f<-OWI V\O\..J 19-Vl -ik\- X iS de+e.rmlV\À~~C_'
DEF. FGr 0.. +~~ pcu-arYleter ~ '> 0, ~ aJ[ +-ka. ~S'so
0... SOU~ e~ +k op+iVY\is C\..tlC,,", pro1}"e.wI:
rLo.M "- =-d""'" [t lIY-x~ll~ + )..1l~1ld.} (:L)~f1R? l ' 1 J
~a.'+i.;iG~CM) SF-'~+~rroW\O-t;<1Ooi- ~~Qi~od ?~~~
REl"\ARK. Tk +e..rWl LASSO ~ foe~ FP~e.cl 6y T,bsLro..ni Il-'\
~9<3 b . Tk t1. - pe.n~ -tern"\ ~ be Seev"\ a..-') a.. Col'\.ye_)(
f"'ee.~')(~O'V\ 9'~ +k m~ l'\.a...+ur~ ~o- p~~ .
. Prof 0 s-i.-h~ 1...For ~ À ') 0 ) ~ SotvdiOVl. ~~ (1.) SO.:h~tL~
~ Il Xl ( Y - X rlo.s~o ') Il co ~ À.
Pro\:>t. We - ~~+. L(~ ') ;: 1: 1\y- X~ Il: ~d P(~'): Àll~ll~.If. 1.5 ~I~G..\tka.-+ L(~')~P(r) ts cot'\.~ex J L(~) 1-50 Cc>o
~\~ P(~ ')- ~~ V\.QV\~-chtteA'"e-,1-i~ble.. Adcor~ +0 +ktil'~+- Q-{"der Co Vlet. t;- ~ .fo..r 0.. CoV\vex· ftu'lc--l-i P'Vt) fo lS ~ -
rn;V\;~;Ôer ~ L(~)-+ P(~) i.~t+k- ~~ "ec.J~r b~vt'j~
- 4-.
~bo ~ s~ - Jiffere.-,-I-;J °t Le?) + P(~).
o ~L(~) -t- P(~)) ~) 0 f: VL(~) -i- dP(~)(:;:::;)-y L(~) E ~P(~)
(:?) ~ XT (y- )(~') E- ~ P (r ).
o~ tk Jk R.~d)~ s~-chffe<~-I-ioJ. O} tk Ma..vr~d r t-4 ll~111
~s CO>'V\f~ &t cJ.Q_ vecton V E rRP s~vl +hQ.,~
YJ = 1. L~ ~~ '/ 0Yi :. - 1\ ·I.f ~~ <: 0
y~é: [- "J-" J 'l.~ t -.:0Tke~ J Y E:d I\~\\1- l~~e~ RQ-t- nvlll)O ~ 1..~d h~c.Q..
~ XI (y-x ~) E ê> P(~) = ~~ 1\ ~ IL~ ? ~ IlxT ('( - x~')\l~~À. ~
DEF For CÀ. ~">O.) <..:Je. dl ~ J)~+Ôid Sd~c-t-~ (:os)
~ S~Ov\ ~ -tka.. Of4-;~;sa...t-I~ rrohle.1IV'
f1>S E- CU'~ yY"Ii~ {Il~ll~~ (fL)~E: :B>..
(..3~ $~= [~GlR?" ~ IIXT(y-x~)IL>o~À~.
Remcu-R -os h~ Ioe~ ~rt>p~d b7 CC<N\J~ &. Tc...o (UJo1-).
(:2.) ~ be CCV)+ i:Vl-+o Gl.. l~GU"" p~~ra...W\ <4'k~ (4) ca..-, be~'!lo+ ~~ ~ se~VlJ- o,der Cone pro8r~vn (SDC.?).
Rem~t It X ts or+ko~~a!) t,e.. ~~ ~ XiX ;;::.IF J ~
-bk L~o ~J ~ J)S cu-e- e.ttl.t.a! ~ CO-l.\Ila.de. l.Vi.~ ~
SQtt- - Tkr~tch,d e.-,-kW\~tQ.rr ST ::: Si~-A (~~) (1~ l - )\')+ ) J --1) ... } F ~ ~ ~ Xïy.d
- 5-
Pror~~QY\!2. 4+ MOde.\ (.1:)') be trt.<Jl.. and ]*- {~:~:fo}.~J S <:: (0,-1) be. a.... tole.c-e-r"tC4. le.ve.L Q.N\J J~~ili\e. tke. e..,,~+
ë :: [ Il xT~ t'oO ~ O""}J. va..to~ (p 18) '. If tk c.olU-W\fI- ~ of
X Mt.. ~~~ +h~ * Il X,; II~ = i the.~;) lP (~) ~ 1- SIi) 4 >.. ~ o-..J :L 1D~(p/d) 1 -H•....., E>V\ é <.Je I..••.ve_ [CT '0'1]
Il Cfl>S_ f~)(J")' 111. ~ '\(r1>S_ ()3~1'1-iii) ït >.. ~ 1'3,j ,H.o"21E) '. th••.••0" E (Je I,.._ve. [BRT'O~)
1I(~a._ ~*)(J*)C IL, ( 31((~1>S_ f)J* IL,REMARK RolA.~hL.a ~r~d'V\d 1 t~~ teS~+ te.lIs t.c)) ~ ihe.
a.cc.c..c..r~ ef ~+~wu~..+;~ ik k8e. vec+er ~* lS Of ik SQ.~e..
orde.r (;V) fu ~~ 9~ ~fIMQ.+;h~ -lk M("('~ c,;~JJer 5~b-
vec"+9<'" g*r)"* .PROOF W-e ~~t ~~= (XJ)T ~ ) J--f) ...)t'\.. SiVlGe. xj
-IS J~+erMlr1is4ïc.
Q..vlo\ ~ tVw(o,~~l~) ) ~j Is o_ ~e.ro~meCLr"t Ga..ussia.Io'\ fQ..Y1doW\
Y~iJ,'~ t..J"rth VCU'i~t'lœ. 01. Il xj H~ = cr1.n.. U~i~ G~~~
-t~l bOtM'\J ) we. ~.e-+.Ii? ( I~~l '> C'"-fi" e.o~(P/.r)) ( exp (-
:::S/p.ln. vi~ ~ +ka.. lMÙ-QVl bO(A.r'\J 1 +hi~ ~'e~d.s
1P(c: c) ::::1l?(3 j •.t. 1 (xj)T~ 1 > cr -J2V\{01j(r/a)')
-6-~ournées de Statistique de la SFdS
r ) P~I: P ( l'~~I>oJl"P.Oa(p/â)' ~ ~ SIp= S_da'" da.f
To p"",ve +k Se~V\d Q.s,>ertt01l\ W'e.. ~S~U-Me. +hd- l.. IS
r~~se.J. T~ Oc.c~rJjV\~ +0 i) (.Je h.o...ye.
~ IlxT(Y-Xf)"oo ~ ~ IlXT~lloo ( \~cl ~'m~eJ.> +~Q..+ ~* is Q.. t~lo \-e so~~~. Th lAS»
\\rDS 111. ~ 1\ r~l\~u"i"~ -lk t,,-<=t +h ",:1- ~;jZ 0 C<-oAJ +k +n1Ul~le i"'4ueJt.~1
t.Ve ~e,-tIl Cf 1>S - ~*)(1")" Il,,, = Il r ~;')C II~
= Il ~J)511" - 1\ r~:Il"~ il ($* Il - 1l~1>S 1\
\" \:1" '1
~ Il ~:~ (L, - Il ~J~ Il,,~ Il (~*- f))$ )J* 111.
t_lI '(- X~L Il: -+ ÀIl~LIl:1 ~ ~ 1I1-x~*U~+\.11~* 111- •
R~Q.~~ y 10, x~*;-~ u:re. ~~.+.~ 11x( _X~L Il: + 4- ~TX(~~_~L) + ÀI\~L/11 ~ ~ H~*H'1.Slnce.. 1-k ~rst +e'M is >1' 0 (.Je ~~+
\11 ~L 11-1 ~ À II~"Il,, + ~ 1 ~TX (~tf-_~L) 1
~ ~ Ilr* IL, + ~ (1XT~noo' Il ~~- ~L)):L (J~JL+~,\\e,L4~~d)~ ~ J} f~'il" 4- l' )(JI ~ t(- - ~ L IIi. D'" é_.
- 1-00urnées de Statistique de la SFdS
:Diy\oIid e."e('~~~ b,Y À.) v:re ~e.+-
Ir ~L IL, !; Il ~*II-1 + ± IIrL- ~~ IL, Dv\ s,
~enœ) Il (~L- ~•.) (J*)C II~= 1/ f(LJ*), II~'" n f L II~- Ii FJL•••Il~
~ Il ~tE-/11 - Il ~ ~~ 111 + 1: Il ~L - ~~ 111
~ 1( (~~ - ~ L ) JJt 11-1 + S Il~ l. - ~* 1\ '1
:::.1- Il (~*- ~L) J* Il,, + t il (~l. - ~*)(J~)C )\1 •
~-e.Arï~~~~ +~ te.crns J ~e. ~e-t+k ~red r-esJ,-t.
y::: X ~*+ ~ ) ~ f'J ~ (0) (52 1\1\)
Y E lRn ~d X E tRnx.p Q..('e ~vvot.Jn. r* E IRf> j~ lM"~Vl.O'-ÙV\ •
Th~ diMe.V\s:.~ r is I~r~e. b(.4.+ !) Z IJ~ 1 1$ SM~1.~ . t.ll.e. <201~Wl~ ~ X ose (\.()rt'Yl~~~d: ~ 1\ XJ \\1. ~ .{,
:r>Ef. W.e.. CAl ( vv.u;:t CA,.~ cohe-r- e.n Ct- El ~ X -the. 'i1.C.G4 r'\ ~ ~d-
J = ~CU' j_ < XÀJ X~) = Mo.)( 1 (Xj)T, X~.d-4=~ ~ J-tl ~
CI~~ ) J '"s oltJ~s be..:t~ 0 ~J 1. Dc.-V\J ~~~ 0 i.t?X ~ oMko ~0 Y\ ~ l, l~ 1 Y Me.G\.h\.N('e.S ko~ Jose.. i,S X +0 ~o("'tlo'?,oV\ ~ 1N\a...+,j)(,
THE.OREMi[ L~t\ici Og 1 If f ~ ts )~he.n (Alth rr-oh~bi~.J-~sa.-h St" e-\ Il ~ DS- ~~ IlC><) (' 4>v
-3-~ournées de Statistique de la SFdS
pro..•.'jeJ -1-kv+ )., ~ ï5-J l Qo~(f/d)' .
I{) {~the..ç-m~) J> (_L Q.Al\d \ '4-cicr - ) l tl8 (pIs)'85 ~ ~
-th-e.t'\ ~J~ proba..bl~.).d ~ 1.- S" ~e.. ~~ve.
Il ~ Lo.sso - ~tt- Il \X) ~ 3À..
CONSEQUENCE I~ -them,",,·h.c.~.J eche.reV'lc.(. j~ sm~Ie.r+ha..Yl "'/.&5
th-e.n bofh -the ])5 Q.K'IJ the Lo.,,;)O CcV\."e-r~e to +~ +ru.e vwtQr
l'" 4!... e DO - nOl'IVl cd +I.e r<de (Co ~~ f) ')~. TL, Ls ver~ Jo,•.+0 -Hu.. pa..ra..me;tr-tc: ra..fe.. (V~)I\/.2.!
Proo{. We unll eN\l7 PlOye_. .J.k IYle9ualc:+(j f~ï iL. Lcuso) -the. Orle.
..for +k.e.. 1)$ L.~ ver~ SiW'l'.~r. We. ha..'1e
JI~L_ r~ll~ ~ II~XTX(rL- ~~)llcoi-II(~xTX-lp)(pL_~)t»)l~.OV\ +k tme. h~v\ 1
Il (4; 'x.Tx -lp ) (rL-~*)lloo= ~~ 1 ~ ~ <XÂ) X') (fL_ ~*)J1d l~d \
~ ma.X ~ ~ « x~J X ~ > 1· \(~ ~ ~~) j (d "~J \. "V' •
~J~ J> ·lIrL
- ~~II~.Acc.o,~ -to B'or l) Iii) } 'Ue h(;<.'1e \l ~L - ~~ ll~~4-\1 (~L_ r~)JMJL,~ 4-5 Il P.L- ~~ Il bO , The.tet-~1
IlrL - ~~llbQ ~ Jl~XïX(~L- ~~)11CX)+4sJllrL.-~~110Q.OV\. 4-~ o+Lu- bcA , ~~ pN)~~. a.K\ j Prop!2.) i) (,Je, ~ et-
Il ~ XTX(~L_B»~)Jl ' j_, IJxT(X~-l))l~+ 111xT~1100~ À+ à .1 1 ~ ~ ~ ~
-9-.Journées de Statistique de la SFdS
R~ ee.~s +0ItL
- ~••.n"" ~ Jf;- hY !Ir-~*\1 ""
or J e.'1t.Ù,v~-\- ~ )
Il t L - r~111'0 ~ 3> >... ~ 3 ~J(d.-4f>j)
pN)V',cled +ha,t y ~ A/Ss . ~
T~ i€.~+ CQ_yt b-e. t.ue.oI -to d~t.Ve ~ vGLficJ,l~- selec:.-hçm-
coVl~s+e.nC-d ~ t~ iresltotde.ol LC(sço V)+IWlQJor.
THEoREM.2. ASstM'Yle +h~t- f ~1is CU\.d If~ I>b\.. to\"
e.'1er'(l- d E J* · T~en., cf À '9.2 Yd(ô~lris) 1 ) -the.
~+i"""Q..i~
PRoOF I~ tri viç;J .
REMARK ~er V~joJol~-e.;,'i4 Se.lec~~- Co~s:.s+e...'\~ 1'e~(A....~
~.Jk LG\sso ~ ~ "])S ~"'Ia.:,\~blc. ~V\.. ih ~t-uo...k re1.a c;w\
-bk (cArol'\~') (fre.rre.SE.l'\;tcJoit.~ ctSS~ftl9'V\. l.i- is W"i!J~
~~Q..'" +k rv\t..\..+~oJ.. cot,e..r~c..e.. c:u:su..-r~Q\A bl.A..t r~vL-l.r~ ;V\..
o.dcL:.-kP"VI +0 Q..SS(.L.~.e.4ha..+ fu so~~ -to (1..) ls. ~,~)
W"~ Tk.ms A_~ 2 kolç{ ~ ~ SOUEM ~ (1.).
-1.0-
.Journées de Statistique de la SFdS
IV LASSO AND ITS PREDIC110N LOSS-
(0)ReccJl -t~Q..+ -the. L~sso i~~ so~.J..\9v\ Di -JI,.e. F~bJe~
& L E Ctrq rh i V\ {j_ Il'f _X B Il~ + \ Ilp..11 1 .1 a ~E IR? l..", r t, l" J
l1e. 30oJ, oJ tk~o..V\d +k n.e)(t seJ\~ lS. ~ ~.J-~d+kfre.J..\..cJ-I~ ~ç~ 0J ..l-k La.SSE> :
f...OL, ~~) e 4- IIx (fL_ ~l» \(.
TH e:.ORE M '3 [Ri~olle.+& \s~ba..kov '","lIf )., '>,.. Cf -J J. lO1(r/$) , """,01 4: IlXi Il>- 1. }<>(' e.ve.t"d i)ikevt.. ~::T(A-k. plDb~b; e-\J (J ~ i -~ vue... ~~ve.
t··et, fJ ~ ."f {~•..(FJ~*J + 4~11~lI"J (3)~é IR?
])'SC.USSiON
-1) ASSt.4.mi ~ +kf ~)f- t~ e~ 0... bow...ndeV\ eo1- nor W\ (loVith a.. 6ol-ttllJ
~VlJef~JEW\ 1- 9VI F (Â..V\d ho) t.Ve ~e.+tC"O'M C~) 0..\0\0\ ~-
S~a.r~ ..th~+
e~(tLI~*)~ 4 ÀU ~~ Il,, ~ 4\ E x Ilplf-/l7. ;: e/ G'~s eo~(p/$Y.
rv.J~ tU ).. I~ o.f .fk.._••.•..Jec-.,~ Q.p.o'à(r/sY. 1'\..
t\..
T~s ~&..o(..(jo:, W ~ec +~ e.><.CLC.+ sfCU"Ç,,~ COi\c::kf.\OVl.) ikpred.<.e..J.;C)vo ~'> ~ -Il. LCI.$sO i'i. Dt Otd... (;Q.,'àff/S) f )
- {i_-~ournées de Statistique de la SFdS
<.a4c.l. ~~ c~.J'e.J ,\ so.oc..v r~+e Il. TtL:s ra..te 15 vcJ.cd c.silhoJ
r9u.~rl.N\6- ~ ~lM'~"f~eM tîoWl +ke. Wla..fri'a( X_:G) 1~41~~~ C~) is. o..n..or~ ~u.a..ktd ~LJc. t"€WlcM.\1S
ffl~ f.vt (.11\ ~ co..c;e. 6>t i~x ~C-+ SpMs.1.td· If FE 1Rr~' L~ ~ ~- ~ra..rSe. vecto-r-) ~ +k preJ.l.c-hSVl Loss- e-J ~
c.vtll be, ct.-~ SW\~I (4) +hL F~cL.c.J.i(N\ ~SS ~tf u.r 40 ~rema.:u..ck-r- +mn 0t Q,cIe.r (s e.o~(fis) )-Y2_.
Proof of ~ ~RecaJl -I-h~ ~ +k tirs~-Q1Jer Cor\c;L:,.h~) ~e de.+
~ xTel- X1'L) E; À Slo~(rL.) ,Tbe.~} t.Je h.a..we
~(f~XT(Y-X~L) =- >-1I~lll,,-1~T xT(y-x~L.) ~ À. Il r ll~ \j ~E fRP
TRe~~('rte~u~{;V) ~'e.~J
1: (t-r)T x -r: (y- x ~~) ~ >.. (I\~\1<1. - lI~lll,,).
l?ep~Q.~n~ y b7 x r -r ~ ) (.Je oet-
4. LX(r- ~1.)1'.)( (~*- rL. ') ~ 4ï W- ~)TX\ + À (I/rI11-nrll~)
'We. L.l~e V\.()I.J"+k LJe..n.J.,+~
w-Lch ~Le.Us
-1.1-
~ournées de Statistique de la SFdS
--------- --- ~- -----~
t ...(rL, ~••.) ~ €" (~,~'t)+ ~ (fI:. ~)TXT~ +.n (II ~ 111.- W n~)
- f" (~,~L.) ( 4 )Ac.eerctva ~o Prof.et} i)) w-e .~r'lO~ ~i- t.Vi~ ~rob >,. -1- è
w-e. k"e_ /lXT~ lJôo ~ ; ~"'--. Uc;1~ -fk J~~d .~,~~)-tk~ 1.W\.~~ +k+
f", Ct, ~,..) ~ t..(~.t) + ~ w- ~Il,,-' nX:IIl"" -t .2>.(Il~ 14-Ifl~)~ ~~(~~~~) -\ol~II~L- ~}l1 + lÀ (II~ 1I-1-lIrll~)
~ Q" (r J ~") + 4 À- ~ r 1Id. V ~ té IR f
V FAST RATES UNDER "THE. R'[ ASSUM?TION-Wh~ X l~ c..lose +0 OA. Drih.oeJ0n.coJ. Medrix the. '\.I01AJ" rafe. Il
0t 1hm_ 3 Ca..Vl be SI~rlÀ.fiC~n.+~ tmpro"led.
DEF. Le..+ :rc {~)_._}p1 ~~J Co) 0 . We. $~ +h~ X Jt.(,ffd)~
tk ~:»\'("t'\I\r-t,-Qv\ RE(Co/:r') [RE = re.st('\c-tecl e.1~e.nvaJu.e.~J
q ~ SQ lMe CoVlS+~+ êIL '> 0 (.Je ~"'e.
VE (Rr IIv:r<: 11-1 ~ Co Il ~ 1l1.. =? ~ Il X v I~ 'q ~ J1~J: .
Prop~.k&v1 3 "I~ X Swk<ifQ.l') -+k yY\tA.:h.4.~ co),e.r-eVl~
CQvut..-kO'\l\ J ~ 1 4~ J ~\so 5Gl.-4>~€.J)el 5 (-t.-t- Co)7-
RE(Co/J) (.V-,,~ ~ J f'J c.~rJi~~.J..~ ~ 5 OvV\d ~th ~z: 1: .-1.3 -
~ournées de Statistique de la SFdS
\0 c.oMf~+-e) e:>v'le. S i~r~ c.-kC:~$ +~a....+-(;Wlpt:,e,<:. +h~+ (1-J (t\+Co)lS) '>,.. 1.
THEOR~M 4
Assv..me ~h~+X s~-h~fl~ RE (3.) J) lQï ~Qme :I C {~).") f\.Choosc- \ = l-) '-Q'(Hà~P/S)7 ~() ~or SOn1e.. SE (0,1). The.V\ ) (.c)it~
frob~b'l ~+d >....1-~ J (.Je.. ~A."e
e~(~~r*) ~ it\~ {1."(~1f*) + -fl- · ~ IJ\1 (5)~; f.r'-'; 0 T~ r
P('oof· Arp~ina -ik ~~t(1 CYU.4~~+d to (4'») cf. f()..d~4..3)
(...re. ~e.t-
eV&. (fL,f*) ~ ~"-(fJr*) +i Ilxï~"t>o·llrL-~ Il~+.2À(l\~ l\'1-llfLIl1)- ~ Il x (r -~L) 1\-1 .
- 14-~ournées de Statistique de la SFdS
Tc e.a..,>e.n..d-a...t;0V\) (..Je <;et v;: ~-~ l-, SiV\c..Q.. f:rc. -e: 0
~e. h~ve. Il ~ Il,,, - II~L 111 = llJ; ll~- nf~1I1-1I~~,111~Îlvr111-llvJ,.J11.
The-r-e.~~e.,
.e~(rL,~~)~ (\1.(~/~~)+ ~ UXi~ll()O l(yll-1+ JÀ (IIvJH-1-11vJcll",)- ~ IIXvll: .
1\'L '1iec.c d~ i)) 'Pt'Df 2.) ~th r\Ob ~.-1- S (.Je ha..;vliL Il XT~UC><)~ ~ •
T~ere~l'€)
R" Cf, () ~ LCfl") + À.( 311v;r \I~ - Il"1,111) - 4: IlXv Il: . (0E!(">-t CQ.~e. '_ 1/ YJc Il'1 )/ 311 VI \\1..
ln ·Hu~~c.a.se, (') LVV\r~ +hcv+ f\A.(~\~~) ~-L(~Jt~)
C\-no\ ~ d~r-u;\ i..~IA.~~ toJlQcN'"~.
Se~J C~e.- J) Y;r c lL1 < 3 H YJ 11-1~
ln.. Vle<..O" eJ -t-.k RtLs,:r) (À.~)l,.(,.l'V\p4;OVl) (..Je. h~"e.\ (3 IIYJ 11,1- Il VJ, II~) ~ 3~ 1)V;r IL,
~ 3 À. JiJT' 1IY'J \1'4
< f ).,~a!.:r1 + Il V;r U~de
~ ~ ~ l:rl + j_ IlXvll<4 ~ \\.. t.
C:>mb'~â +hi.s ~rtt..~~~ ~tt. (b) ure ~e.+iL. de,Wed rfsJ+. mG,r(\ n"I e.vJ.s
-1) Th.c. first te~+ ~""+k ~ri(,l+ o~ (5') ho.~ be.€Kt proved IV\
B.c.ke.! - Ritov- T'i'âbo..kov (0<3. The. ve.rsi<:W1 ~dh oows+a..V\+ 1-
i.~ ~V\+ ~t ~(~/~~)d +he l~HS€>~ (5) Ofr~ed (,v\.
- 1.5 -
~ournées de Statistique de la SFdS
koQ-+~ti.J Lu,nici & i5~b~ko" '11. Th~ proof W"e.. presen+ed he.re.
(.~ Ln~r·."'e.d 6, -the. 9V\e. tl1 S<A...n &. ~~CW\d 11.2...
~) Choo~Ylô À: 3C$-.J QosfP!s') lA)"e ~et .fki- w-~ prob. >/{-~
1.. (~l,f~) , w-.{ f t•.(j3, ~") + d . cr"s Îo"'lP/S) } .~ =0 V\.Je.
(.J~ C (..s Q.. ~+~..t ~ol s z: 1r) .
~) 1+ is. knoLJY\ ~h~f ~he bel)+ po:>s;bl-e. ra.:te. OY€r +kt. s-e-+ ~ ~/I
rO~s1.bk ~~~Q...-t.0f'5 IS 0t 9f'd<e.r sQ.o2(r!s) [LIA.t'\Ic.i 01J.n...
So Th.~ 4- ~kowc:; {h~ -t~e..Lo..')~o ISo r~e. Op+'Wl~ ur to
eo~- ta..d-~ c.ct.~ X sa..-h~~~~ 'RE ('.Q",cL:,ttOv\,
4) E'ieN\ {or MoJe..('O\..i~ V~(..(.e/) 9,f p) ~e Cow\p<A,.fa.JioV\o.1 (brn~xJd
0t (.,~eckiVld tk+ RE(CoJ!) ~/Js ~ ~"eAd J StA.~ ~GY+IJ \ ~ > l~ rro~lod.",,~ kÔe. A", ~lt~t\.~Jjve.. CoV\J..;,.J.;~
h(4.~ 6eeN\ pn::>rO'>~ ~7 :rtA.Cli+S~d lx. NeWliro"~t. (.20i1) I.Jlc.L.
hM -tk o..Jv~~~ 1\,~ verjfcJ:,l~ ~~~ F Qa.rd~ r·
VI RECENT RESULTS ON THE PREDICTION LOSS-We. ~tc:u-+ -thiS. sectle V1 J vJk:i.(.~ (..on+a.Z.,s ~OWle r~u.fis fro~
'J)~lo,.,ly~) Hebiri O\.V\cl LeJetef" (~O'H.)) b7 s+Ov-t,-, ~ n.L~",,-hve
~~+.- ~b-
00urnées de Statistique de la SFdS
F ;: Jvn ~
-1 -1 -1. -1. 'i -1. 1-- - .x = 1- -1 0 0 0 0 .-If f= 1- - .0 1..-1 0 0 0
0 - - .. . .<D 0 <D 0 •• 1 1. -1- 0
1-1- js c.'e.~ ..thaJ ~ Il xd Il: = i.. For (ÛM~.J.d 1 t..)~ rerlC\.~
~ c.w..Lvmf~"'" ~,J.!j .1'(0 16~) by ~i!.0R~e •••...J.•••...w-k.:t.J-, W\e~ .fk~f 1P (~~ ~ 1) Z ]P(fc; = -:i)-= 1-.PrOf ~oV1 4. For -e,ve<"d À. '> 0 ) ure. ho...ve.
/p( *" IIX(~L._ ~l»U~ ~ J}",. ) ~f·Comm~+ T~s. ~ sk.o<..e's ih...a..i for Some... ma...-t"'ic...rv.>
X iJ is Lmposs.tb l-e to Lmprove &'\Il +k \\sloW"" r~eJ) Il.
OVl-e COv'l ~e. tJuJ- r~ -fhL~ eX4rnr'e
- s~ 1s verd sW\ev\1 : s.•.;L
- {h~ eorreJ~.:IiQV1 l..S f\Qt Ve<"(J ea.rd~ : J> = 1ik fi -"'-or"" 6,f r* l~ ~lt\CJ~r~J~+ 9IA r\. ev\J 1
- t~e ~aJ; \le reM.&f- h.o~d5. for Q1I\.d À ') o. Fu-r+he("l'\1()1'(.
9V\e CAA\ e."Ie.vt Shol.J 0... s+rG>v\~e.r ~+ @J -tk forMlP ( ml ~ .i. Il X(~l.._ p.~'\ il1- ~ j_ \ ) .i .
À') 0 K. r \-) t_., ~ ,1&\1\.) :1;2.
- i1-~ournées de Statistique de la SFdS
D EF Le,} M clefit'le -fhe. ~i5~A:s
Wj = ~ Il (1.- n.:r\kJ Il,<.Vke. nj is ik or~lo~On~ proJ€CT91 evt+o 5r~ (x:r').
\Je s~ i~+ X $~-k~~iû,) +k c.J~h,t~J roVhf,,-.Hb'~~èJ~~~ lA. fV\ ft19V\
; V'I fV € e (:r)
WC (:r) (,t ttX" ~oW\e ôe ') 0 '-"'e. hç..v€1 :r 1 ~ Il X v It~ :} ~
Y\. (Il \Ir Il.,-Ife1.-iW' 1;.c 0 VJ~Il,, ) 2.
THEOREM 5"
Assu.Y't\e. -thcv+ X SoJ;Sfi~ WC Cr) ~Qr 50wae J c. f 1,. ..) P ~.
c~oos~ \:::. j"r:5-.jJ..t.o"le/sY' fQ-r SOWle Sé(O,1) . ThEA!)
(..cStt\"" proba....b i~Jd 0} ~e.~+ 1.- 2$ <..J~ ~~ve
e~($l-J s*) ~ ~V\f [eYl,(~1 fit) + 40-1. Ir) eo~Cp/s) )C r~ l} r ~: ~:r"~0 Vl. ,p,T J
wb rn =- -1 + ~ -t.1:,- .If, T eo~(p/s) 11\ ê1Z
ProQl Cf. 'Da..la..lya.VI e.t ~1. (2o~4).
RE MAR K I~ X SaJ-i,Stie6 R t: (3,:r) lot CA..1~o sa.-f.;~f\~
we_ Cr) tüt+1., +k,p_ S~me C-oVlf» +a..\'1'+ ce . T~ ooVl'le.rse (.5. Vl-Ot
tf'~ J S 0 Tt..~ 5" iS ~tr,d-lr s+roV\~er -tha..VI Th"", 4.
-4.S -
~ournées de Statistique de la SFdS
I",Jee.d } &)V\ +kt. OVte he:t.vtd) OY'l L (3)
(IIVj Il~- Il C1-1l.J).1< Gl Yj~ II-I)~Il YJ" II~ ~ 1.1" 1 l1Y.rIl~(À.Vld, tt,e..refo re. 1
{]"\ . IIx" \l~ Il xv Il ~n (Il "s111- 1( ( ~ - -i ...,.>S< 0 "j< \1'1 )'- ~ n /1 "':rll;:
O~ flu. o-t-k hOwV\d) V ES e (:r) ë.Wl~e~ +lJ-il v-rc. 1IJ\ ~ J. )( (\ ( 1-1 w)Jc 6) ys~ILl ~ ~ Il Vs I}j._ ~ 3 JlV:r 1/:1
whl.c.k ~WlrQ~ ~t-~Vl.f 1.T) . "X v Il~
t :+v b e (r) Vl (flvj Il,, - Il (1-1 (.J}J(. e Yj,- \l"),
~ f~~v: lIVJ' \L, ~ 311vJ 11"
REMARK
I", +ke. ~e @t TV- p~~3ed -Pea.st- S~(4a..t'e~ e;;,tiVl1~-k>r
{TV é 4h o...r~m, VI (.i. II y - ~ l" -t À I_1.l f. _ f, ) )1/ U .fiG IR "" c2 V1 t.. C)"" •• T~t ~~o
Co mr ...•..j;~ 'tT Y t~ e"l<M. ".J....v+ +0 eo"",p,,,,-k d X· ~L
~,tt. X ~ C~,.:] E 1Rh"~
U~ia +ktz.. Ia..f)+ the..cr€NV\ 1 (.Üe. ffla,Vl9-'5.e.d +0 pro,,~
th~+ ihiS Ma ..:tri)( X sa.A·hfi~ WC(:r) ~r ~~udf' .e.d ::r w-d-I, "- <?oVl s+a....... +- de 0+ arde.r ~~ VI •
r~s eea.cls L, ~ Ilt" -f' ll~ ~ Ci" û~2"~(f)' f.o~r1
for Cc;~ piec.e.- t.ViSlL Colt\<s+a...",t s,~~a..l ~*.-19 -
~ournées de Statistique de la SFdS