Primera Vance

8/13/2019 Primera Vance

1/19

1.13. Sufficiency

Definition: Let X have a family of possible distribution{F ixxi H} I, is suf-ficient for based on x, iff each condition distrbution of x given I = td notdepend on (except perhaps for a null set A with T(T A) = 0) t (Idea: baseddecisions on (minimal)sufficient statistic).Eg: x1, x2,...,xn idd b(i,p), P[0, 1]. Let T =

ni=1 xi, note T bin (n.P)

For t {0, 1, 2,...,n}, consider P(X=xT =t)P(X=xT =t) = P(X=x,T=t)

P(T=t) = P(X=x)I(xi{0,1}i=1,2,...,n,

ni=1xi=t)

(nt)pt(1p)nt

=n

x=1pxi (1p)xiI(xi{0,1}i=1,2,...,n,

ni=1xi=t)

(nt)pt(1p)nt

= pt(1p)ntI(xi{0,1}i=1,2,...,n,

ni=1xi=t)

(nt)pt(1p)nt

= I(xi{0,1}i=1,2,...,n,

ni=1xi=t)

(nt)

So, it doesnt depend on P, so, T is sufficient for p based on X.Eg 2: x1, x2,...,xn idd N(, 1).Candidate sufficient statistic T =

ni=1 xi.

Need to show P(X=xT =t) does not depend on for all t.

xt(xt)...3

Claim: can sow x1,...,xnT =t does not depend involve ... 1

E(ein

j=1

xj ujn

1

j=1

xj =t) =E(ei(

nj=1xjuj+un(t

n1j=1 xj))T =t)...2

1is true 2istrue 3istrue.

(Fisher- neyman criterion factorization criterion)Led X jave density fx(x, ), H. I = I(x) is sufficient for on X, ifffx(x, )can be expressed in the from:

fv(x, ) =q(TX)) h(x) (*)

, where h(X) doesnt depend on Valid for dersftles discrete & continous r.v.seven valid in more abstract settings.

eg: x1,...,xn idd B(I.P) P(0, 1)fx(x) =

ni=1p

xi(1 p)xiI(xi{0,1})

1


2/19

= Px

i.P

g(

xi.p)

(1 P)nxi I(h(x)

xi {0, 1}) xi is a sufficient statistic by fact ... crit.eg2:x1,...,xn idd N(, 1) Rfx(x) =

i= 1n 1

2exp{ (xi)22 }(2)

n2 e

12

i=1nx2i

h(x)

. e

i=1nxie12 n

2 g(

xi.)

I(x R2

xi is sufficient for basedd on X.Discete case see textproof:we prove when T is one-dimensional under regular conditions.Suppose () hdds, let T1 = T and T2,...,Tn be sucht that the transfomationfrom x1,...,xn to T1,...,Tn has a non.venishing Jacobian fo its inverse.

ft(ti, ) =fx(T1(t))

doesntinvolve T(T1) .

g(t1, ).h(T1(T))T(T1))

We wish to show to distribution of (x1,...,xn) given T1 = t1, does not involve.First show, FT2,T3,...,TnT1(T2, T3,...,TnT1) does not involve Key!

Then show the condition cif ofx1,...,xn given T1= t1 does not involve .

FT2,...,TnT1 = fT1,...,tn (T1,...,Tn)

int

...F

fT1,...,tn(T1,...,Tn)dt2...dtn

=g(t1,)//////h1F1(t).T(T1)

g(t1,)//////

...F

fT1,...,tn (T1,...,Tn)dt2...dtn

does not involve .

Conversely, wlog assumeT1, X2,...,Xnhave a joint density which can be obainedvia Jacobian from fx(xi, ). If XTi = ti is indep of, then x2,...,xnT1 = t1,is also independent of.

fx(x, ) =ft1,x2...xn(t1(x), x2...xn)

a function ot xi does not depend on J(T1)

use definition: fT1(t1(x).) g(,T)may depend on and depend on x through t1(x)

fx2,...,xnt1(x2,...,xnt1(x))Jh(x)

() hold.

Notes: (1) From the proof, w see that g(T(x).) is, except for a fa which mightdepend on x, but not on , the density of T.(2) if I is k.dimensional say I = (T1,...,Tk). We say that (T1,...,Tk) are jointlysufficient for based on X.(3) If I is sufficient for based on X and if S is a 1-1 function , then S is sufficientfor based on X also.(

xi)13 1

xi

(4) If is k-dimensional if I is suff for based on x, the the dim of I can b > or= or < then k.

2


3/19

eg: x1, x2,...,xn idd (0, ), where (0, ).

fx(x,)=

ni=1

1

I(0 0 depend on (U(0, )X)Eg: X poisson ()

fx(x, )xe

x! I(x Z+) = e

()

e

(log) Q()

XT(x) 1

X!I(x Z+)

r(x)

1 If X has a one-parameter exponential family of densities with the structurefunction T(x) also has a one- para... expon, family of densities (with structurefunction the identities) T= tx

ftt; =

()

r (t)eQ()t

2 If x1, x2,...,xn ae idd with common densities ftt; =

()

r (t)eQ()t,then

ni=1 T(xi) has an exponential family of densities & is sufficient (minimal)

for based on X.proof: Suff to prove in the case T(x)= X. Suppose thatx1 andx2are indepenttxi(x, ) =()ri(c)e

Q()X HLet Y= x1+ x2

fr(y, ) =

f1(y; )3f2(; )d= 1()2() r()=[()]2

eQ()Y

r1(y)r2()dByinduction, continuetodo

(x1+ x2) + x3, ...., (x1+ x2+ ... + xn)

Eventually, Y =n

i=1 xi will have exponential family of densities, with r() =[()]n r(y)= r r r r.... r

nr

3


4/19

Qr() =Q() T(Y)=Y

Proof: Fxi (x1,...,xn) = [()]nn

i=1 r1xie

Q()

ni=1

T(xi) sultian

1.15. Ifx1, x2, .., xnidd fx(x, ) =()r(x)e(Invertibleforidentifibility

Q()

structurefunction P(x) )

then

i=1 t(xi) is sufficient for based on X andn

i=1 t(xi) has an exponentialfamily of densities with structure function To(t) =tExample:x1, x2, .., xn idd (B](1, )), what is a good sufficient statistics.

fx(x, ) = (1 x)( 1)B(1.)

I(0< x 0, 2 >0textsuchstats (

ni=1 xi,

ni=1 x1)or(

ni=1log xi,

ni=1 x1)

4


5/19

(2)u(, ) maxxi(3)fx(x; ) =

ex

12 I(0< x < , (0, +)) (ni=1 x1, xn=n)

Hard to use the factorization criterion to show that T is not suff.To show notsufficient, we should use definition, the condition distribution of x given T =tdoes depend on for a non-tribial set ofts whether T(x) is suff depends on H.H1 H2 H3 H1 H2 H3

x1, x2,...,xniddN(,1), R x1, x2,...,xniddu(, ), >0x1, x2,...,xn x1, x2,...,xn

x1=n, x2=n,...,xn=n x1=n, x2=n,...,xn=nni=1 xi= n (xi= n, xn= n)

maxxi= max(x1=n, xn=n)

Minimal sufficient statistic = is minimal suff statistic (based on XF(x; )),if is sufficient for based on x and is a function of every suff statistic basedon x.

To identity a minimal suff. Stat, we need to consider the minimal suff parti-tion of x (the set of possible values of x)

For each x X, let

D(X) ={y = yX, fxy; ) =fx(x, ) k(y, x) indep of

}](Note: if y

D(X), then x D(y))This defines a partition of x.For example, x1, x2,...,xn idd B(1, ) x={0, 1} x= 26 = 64

X f(x; ) x= 64(1,1,1,1,1,1) 6 1(1,1,1,1,1,0) (1-)5 6

152015

6(0,0,0,0,0,0) (1-)6 1

partitions

D(x) is called the minimal suff partition.Any statistic function T(x) which is constant within cells & takes an differentvalues in different cells, defines T(x) a minimal statistic.How must x and y be related if they are to be in the same cell of the minimal

5


6/19

suff partition?

Must have fx(x, ) and fx(y, ) such that are is a constant mu of the others.Usually, can look at fx(x,)fx(y,)

eg: x1, x2,...,xn idd B (1, ) for x, y {0, 1}n

fx(x, )

fy(y, ) =

x1(1 )1x1 ...xn(1 )1xny1(1 )1y1 ...yn(1 )1yn =

1

ni=1xi

ni=1yi

This is indep of iffn

i=1 xi=n

i=1 yiConsider T(x)=

ni=1 xi

ni=1 xi is minimal suff for based on x.

eg: x1, x2,...,xn idd Cauchy (,1)

fx(x, ) =

ni=1

1

(1 + (xi )2) = fx(x, )

fy(y, ) = ni=1(1 + (yi

)2)n

i=1 (1 + (xi )2)

if y is permutation of x fx(x,)fy(y,)

is indep of. That means the minimal si.

so (x1=n, x2=n,...,xn=n) is a minimal suff stat.1.20 Nice properties of estimates.Untiasedness (all estimates based on bayesian are biased)XFx(x; ), H. Wish to estimate g() based on X.If Y=h(X) is a estimate with finite mean ()Can writeE(Y) =g()+by(), whereby() is called th bias of Y as an estimateofg()Ifby() = 0, H. Then Y is said to be an unbiased estimated ofg() (andconsequently, g() is estimable)

eg: x1, x2,...,xn idd N(, 1), RY = 1

n

ni=1 xi is an unbiased estimate of

More generally, ifx1, x2,...,xnidd with common mean, then

i=1 xi is an um-biased estimate ofIfx1, x2,...,xn are idd with common variance , then s

2 1n1

ni=1(xix)2

is an unbiased estimate of2.Invariance in estimationIf we use T as an estimate of g(). And then wish to estimate h(g()) we wouldconsider h(T) as a good estimate of h(g())eg :x1, x2,...,xn idd N(,

2) wish to estimate 2 by T= 1n1

ni=1(xix)2.

Instead we might wish to estimate . Consider

T but E(

t < )However

Tmultiple by a constant should be unbiased for . But the constant

are dependent on the density.Unbiasedness not invariant. If T is unbiased for g(), h(t) usually not unbiasedfor h(g()).

Unviased estimates are not unique.Ifx1, x2,...,xn idd N(, 1)x is unbiased for .

6


7/19

x1+2x2+3x3

x1, x1+x2

2 xn+1

2 =nare all unviased for .

IfT1, T2,...,Tk are unbiased est. of g() based on X. Thenni=1 aiTi is un-biased forg() providedni=1 ai= 1Somtimes, could have = ={unbiased est ofg()}eg: X bin(1,P), p(0,1) wish to estimate logp. Claim no unbiased est. exist.Suppose T= t(x) is an unviased est of p based on x.logp= Ep(T) =T(0)(1 p) + T(1)p= T(o) + (T(1) T(0))p is a linear fraetof P #.So, no matter what T your choose, logpis not a linear function of therefore, nounbiased est exists.Proposition: suppose xFx(x; ). H. Thas only a finite number a pos-sible value.Proof: For any est. T, we have only a finite bist of possible valuesT1,...,TNso, E(T) =ni=1 TiP(T =Ti)(l=N, TN=N) is bomeled, but () is unboun-ded, so no chace for E = () so proposition theorem holds.E. lehmannThey only unviased statistic may be unreasonable.eg: X p(), wish to estimate e2. Let T(X) be an unbiased statistic.

e2 =

j=0ej

j! T(j)

e2 = j=0 jj! T(j) for all >0e2 =

j=0

()jj!

=

j=0()j(1j)

j!

T(j) = (1)jj = 0, 1, 2,...=

1if j is even,

1if j is odd.

but e (0, 1),How we pick among unbiased estimate? Step i, sufficiency.Suppose that T, is unbrased est. of g() beased an X and that S is the (minimal)sufficient stat. For based on X.Define T2= E(T1S) obviausly its a function of s.Its also a statistic because T1 is a function of X, and S is suft stat. for .Usually, it is a estimate.Usually, if His convex.What mat unbiasedness? E(T2) =E(E(T1|s)) =E(T1)So T2 is also unbiased.If we are lucky contain only one est.CompletenessLet{F(x; ) H} be a family of distributions. The family is said to becomplete for any measureable function g.

E(g(x)) = 0, H P(g(x))= 0, H ()

The family is said to be boundedly complete if for any bounded g, (*) hold ifE(h(x)) = 0 can call h(x) an unbiased est. of.

7


8/19

The family is complete if there are no non.trivial unbiased est. of.

Completebounded completeSupose{F H} is complete & suppose g1(x) and g2(x) are unbiased est. ofh()g(x) =g1(x) g2(x) poEg: X(0, ), (0, )Consider suppose E(g(x)) = 0

01

g(x)dx= 0 () (ba

f(x)dx= 0(a, b)f(x) = 0a.e.x) g(x) = 0a.exIf differentiable for (*), then g() = 0, >0, g(x) = 0, x >0.

eg2: f(x; ) =()r(x)ex where < < Suppose Eg(x) = 0, (, ) int()r(x)e

xg(x)dx= 0, (, ) int

()exg(x)dx= 0,

(, )

This is just the laplace transform of g(x)r(x) is o(x.i.e), P(g(x)0) =0(, )

In generall, in a k.dim exponential family, the min, suff stat. based a sample of si-ze n (for any n) haz a complete family of distribution provided Q1(), Q2(),...,Qk(), in Hcontains an open set. (usually k-dim rectangle), otherwise often not.Eg:x1, x2,...,xn idd f(x; ) is this fx(x; ) complete?consider x1 x2.g(x) =x1 x2, Eg(x) = 0 p(g(x)0) = 1)fx(x, ) is notcomplete.

Questions Friday 1-23-10

1Rohaliji pag 364 example 9

f(x; ) = 1

I(x

2,

2

, >2) f(x; ) =

1

nI(x {x:

2 xn:n

2})(g(n: n, xn:n))?f(x; ) = 1

I(x

2NN(, 2)fx(x; ) = 122

e1

x 12x2 l12

Q1= 1 , Q2 is function ofQ1?Q2= 12

Definition. SufficiencyLet x= (x1, x2,...,xn) be a sample from

or have a family of possibles distribution?{Fx(x: ), H}

A statistics I = I(X) is SUFFICIENT for (or for the family of distributions)bured on x, iff each conditional distribution) of x given I=t is independent of(except perphaps for a null set A with P (TA) = 0forallthetain H)

January 11/09Class

8


9/19

Data: X=x

Model: XF(x; ), HGool:Given a model F(x; ) we have to decide a time value of bured x.We know HWe do not know wich is in H

Clasical (objetive) Bayesian (subjetive)

estimation

point,

intervalAditional information. fH(x) prior data minotion of

Hypothesis testingseleding ranking, ... fH

Inference: Trying to infer properly of a population bored on a sample propor-tion.1=

xn

if n is small me does mo preper it, if = 0 2 y 3 are good.

= 12 3 on are good.x1, x2,...,xn idd N(, 1)

Take sufficient statistic xWhenxN(, 1

n)

Now we can gerent a sample x1, x2,...,xn with the some distribution N(, 1)

January 11Class

On class of stimate of v rs

{(x) :}

Example(i) x1, x2,...,xn idd N(, 1), Rwe wish to estimate Candidates:x= 1

n

xi

or x n+12 :n

or x1or x21Nor x1+

2 ifa is uniknow the x1+

2 is not a statistics

(ii) x1, x2,...,xn idd N(1, 2), 1 R, 2 R+ we wis estimate (1, 2)Candidates: (x, s2)s2 = 1

n1 I(xi x)2

(xn+12

:n,(xn:n xn:n)2

16 )

Which estimate is best?Depend on how the estimate will be used depend on cant of error.

Might consider loss functions might considerer the corresponding risk:l(t,v) f(l(t, v))

cost of estimate v by t

* The problem is to write the loss function.

9


10/19

CLASS 02 January

* I =t is sufficient we can reproduce data without knodoge of the parameter, because the conditional distribution of x doues not depend on * I is a umpad of the somple, we would like to get the more compant. This is

the minimal sufficient statistics.* Definition: the some.* Example: x1, x2,...,xn iid B(1,p), p(0,1), let T =

xi note Ybin(n, p)

is T sufficient (m p bored on x?)For t {0, 1, 2,...,n}Considerp(X=xT =t) = p(X=x,T=t)

p(T=t)

= p(x=x)I(xi{0,1},i=1,2,...,n,

ni=1xi=t)

(nt)pt(1p)ntn=6, t=4, (1,1,1,1,0,0),

(1,1,1,1,0,1) 64. . . .. . . .. . . .

= 1(nt)

I(xi= 0, 1;

xi= t)

T is a sufficient for p based on x

Example 2 x1, x2,...,xn iid N(, 1)candidate sufficient statistic T = 2xi claim; 1

?

Finiser-Neyman criterio or factorization criterio.Eg1:x1, x2,...,xn iid B(1,p), p(0, 1)

fx(x) =

ni=1

pxi(n p)n2xiI(xi {0, iiN

}) =p

xi(1.p)n2xi g(

xi,p)

iiN+

})

eg2:x1, x2,...,xn iid N(, 1), R

fx(x) =n

i=112

e12 (xi)2 , I(xi R)

= ( 12

)ne12

(xi)2 , I(xi R)

= ( 12

)ne12

((xi)

22xi+2), I(xi R)== ( 1

2)ne

12

(xi)

22xi+

2), I(xi R)= ( 1

2)ne

12

(xi)

2

e122

xi + e

12

2), I(xi R)

Factorizacion criterioproof: OkWhy we use charadentic fuction?Notes (1)? osk profesor.

10


11/19

January 11 1.1

Date.x: (x1, x2)xRxRwx (w1, w2)(x1, x2)

x: 1x...xn R+(w1,...,wn)(x1,...,xn)

xFx(x; ) (x1, x2)f(x1,x2)(x1, x2; )x

Poisson x

fx(x; )

Fx(x)) = ()xNormal fx(x; , 2) = (, 2) H = H={: >0} = (, 2) H={(, 2) : R, 2 >0}

Eg: Tuss a coin n timesx = # of heads opeaning in the n times.

xbin(x, p) or X= xi xi

1 if head

0 otherwisenisgirul xibernoulli(p)

p= ? X=

xibin(n, p)= p A={p: 0 p 1} Note:

1. Estimate: real yef value of

2. Estimator, random variable

How should we estimate p?Pick some function of X to estimate P

T1= x

n

E(T1) =p()E[t1] =E

xn

= 1

nE[x] = np

n

var [t1] =var

xn

= 1

n2var(x)

= 1n2

var(1 p) = p(1p)n

January 11 class

T2= 1

e, t3= CIf P is close to

1e

we preper T2= 1

e

We prefer t1, t0, t2, t3 but we can not saw t1 is better than t2, t3. In some cir-cunstancest2, t3 might be clases to p.Let T4=

12

if xn 1

2< 0,02

xn

if xn 12> 0,02

Model: suppose x=x1, x2,...,xn has distfx(x; ), H,H= parameter space. is unknown except that H (K-dim parameter space).

11


12/19

Eg:

X= (x1,...,xn)xiN(, 2)= N(, 2)HK-dim parameter space(, 2) H={(, 2)R,

Let v= h(), some fuction of{ + 3= h(, 2) we wish to estimate v.Define x={v : v = h()} for some H. A estimate of v is any statistic (anyfunction of x) before normal is a struct of x (sometime often v= orv= )Identifilily assumption:

= Fx(x; ) Fx(x; )

Example: x= x0+ (, F)= parameterWhere xoN(, 1) RF

x0N(3, 1) x0N(2, 1)N(2, 1) N(3, 1)xN(5, 2) xN(5, 2)

Often x is a ramdon sample from F(x; ) Hx= (x1,...,xn) iddHparameter spacex possibles values for X. all function x to h ()eg:{xi}n1 idd N(, 1), Ru=? x, xn+1

2:n,x21+ x22,...eg:{xi}n1 idd N(, 2), R, 2 R

(, 2) =? (x, s2), (xn+12 :n

, R2

4 ),...

Which estimate should be used? T =? tsuchE(t h()2) is infinitely small.T h()Step (Fisher)- Cut out all non-informative aspect of the data. - Pick (x) to simplify withoutloss of infermation which does not depent to eg: xi idd N(, 1)xis sufficient for all based on (x1,...,xn).Sufficiency:Definition (Rohatzi)Let x= (x1,...,xn) be a sample from

{f :

H

}. A statistic T(T(x)) is suf-

ficient for or for the family of distributions {f : H}; if and only theconditional distribution of x, given T=t, does not depend on (except perhopsfor a null set A ,P{T A}= 0 for all ).Definition (Arnold)Let x have a family of possible distribution{F(x; )}A statistic T=T(x) is sufficient for based on x iff each conditional distributionof x given I = t is independent of (except perphops for a null set A with

12


13/19

P(T A) = 0, H)

What is on family?

Binomial (1,p){Fx(x, = p)}

Binomial (n,p){Fx(x, ) = (n, p)}

eg: x1,...,xn idd binomial (1,p). x= (x1,...,xn) have a family of possibles dis-tributions.

fx(x, = p}=

fx(x, = 0,5) fx(x, = 0,1)

fx(x, = 0,2) fx(x, = 0,8

...

fx(x) =fx1(x1) + ... + fxn(xn) xi are indep=Px1(1 p)1x1 ...Pxn(1 p)1xn=p

xi(1 p)n

xi I(xi {0, 1}i= 1, 2,...,n)

Let T=n

1 xibinomial (n,p) why?Theorem 1.The MGF uniquily determines a DF and, conversely, if the MGF exist, it isunique.

Mt=

xi(t) =E(et

xi)=E etx1+tx2+...+txn

=E etx1etx2 ...etxn

=E etx1Eetx2 ...Eetxn

=mx1(t)mx2(t)...mxn(t)mx(t) =? xbinomial (n,p)mx(t) =E(e

tx) =n

i=0 etxpx(X=x)

=n

i=0 etx

nx

px(1 p)nx

=n

i=0

nx

(etp)x(1 p)nx

Binomial theorem (a + b)n =sumnx=0

nx

axbnx

= (etp + (1 p))n

If x bin (1,p)Mx(t) = (et

p + (1 p))1

. ThenMt=

xi(t) =Mx1(t)Mx2(t)...Mxn(t)

= (etp + (1 p))0(etp + (1 p))1...(etp + (1 p))nBy theorem this is the GMF a variable x Bin (n,p)Therefore T =

xiBin (n,p)

for t {0, 1,...,n}Non is T sufficient for = p?

13


14/19

Note we here a x (x1,...,xn), xibinom(1,p). Here = Pand I =xi (a one

dimensional case).By definition we have:If P (X=xT=t) is a independet of , the T is sufficient for = p.We none the P(AB) = P(A

B)

P(B) ; P(B)> 0

P (X=xT=t) = P(X=x,T=t)P(T=t)

Notice: P (X=x , T=t) = P ({X=x}{T =t})What man the events{X=x} and{T =t}?X=x : R Rxxx= (x1, x2) : R R(x1, x2)(x1, x2).

..x(x1,...,xn) : R R or T=

xi: R R

{x= x}={(x1,...,xn)R :x(x1,...,xn) = (x1,...,xn)}{T = xi= t}={(x1,...,xn) R :T(x1,...,xn) = xi = t}{x= x}{T = xi= t}= x = xP{X=xT =t}= P{X=x} for t=0,1,...,nNowP(X=xT =t) = P(X=x,T=t)

P(t=T) = P(X=x)

P(t=T)I(xi= 0 or 1,

xi= t)

.P(X=x) =p{x1,...,xn)}= P{x1= x1, x2= x2,...,xn= xn} n

i=1 P(xi=xi) * xi is independent.

P(xi = xi) =? xibin (1,p)P(xi = xi) =P

xi

(i p)1

xi

P(x= x) =n

i=1 P(xi= xi) =n

i=1 Pxi(i p)1xi

=px1(1 p)1x1)px2(1 p)1x2)...pxn(1 p)1 xn)=p

xi(1 p)n

xi

=pt(1 p)nt t= xi. p(T=t) = ?T =

xibinomal (n,p)

p(T =t) =

nt

pt(1 p)nt

The for

xi= t, whe have t= 0,1,2,...,n

P(x= xT =t) = Pt(1p)nt(nt)pt(1p)nt

I(xi {1, 0}, witn

i=1 xi= t)

= 1(nt)

I(xi {1, 0}, wit

ni=1 xi = t)

Sino

t, P(x = x

T = t) does not depend on p. T is sufficient for p based on

x. eg. x= (x1,...,xn)x have a family of possible distribution{Fx(x, ) : H}Fx is a the distribution of xU (0, )T= Xn:n is sufficient for = based on x.P(X=xT =xn:n= t) doe not depend on ?P(X=xT xn:n= t) = P(xx,xn:n=t)P(xn:n=t)

14


15/19

. P(x x, xn:n = t) =?

The event{x

x, xn:n= t}={(x1,...,xn)

(x1,...,xn), xn:n= t}={x1 x1, x2 x2,...,xn xn, xn:n= t}= P(x1 x1)P(x2 x2)...P(xn xn) =Fx1 ...FxnFx(x) =? if x U(a,b)xfx(x) = 1ab I(a,b)(x)fx(x) =

xab I(x(a, b))

Here xU(o, )here Fx(x) =

x

ThenP(x x, xn:n= t) =Fx1(x1)...Fxn(xn)= x1

(x1)...

xn

(xn)= 1

n

xi

. P(xn:n= t) =P(x1 t, x2 t,...,xn t, )=P(x1 t)P(x2 t)...P(xn t)

=Fx1(t)Fx2(t)...Fxn(t)RememberFx(x) =

x

ifxu(0, )ThenFx(t) =

t

ifxu(0, )P(xn:n= t) =

t

t

... t

=

t

n

P(x xxn:n= t) =1

n

ni=1 xi

t

n I(xn:n= t)=

ni=1xi

tn , I(xn:n= t)

Thenxnn is sufficient statistic for = basul on x become P(x xxn:n= t)does not depend on .

MORE EXAMPLES AND EXERCISES

example 4 (1360) Let x1, x2 be idd P()Rvs. Then T= x1+ x2 is sufficientfor based on x= (x1, x2)P{X=xT =t}==P{X= (x1, x2)T =x1+ x2= t}=P{x1= x1, x2= x2)T =x1+ x2= t}=

P{x1=x1,x2=x2)T=x1+x2=t}

P(x1+x2=t) ift= x1+ x2, xi= 0, 1, 2,...

0 otherwise

= P{x1=x1,x2=tx1}P(x1+x2=t)

I (xi {0, 1,...} with2

i=1 xi= t)

x1 or x2 are independet the.P(x1= x1, x2= t x2) =P(x1= x1)P(x2= t x2)We know that P(X=x)= e

x

x! I(x0, 1,...)P(x1= x1, x2= t x2) = ex1x1! e

x2

x2!. P(x1+ x2= t) =?What is the distribution of T= x1+ x2?. By MGF (Moment Generety Function)

15


16/19

mt=x1+x2(t) =E(et(x1+x2)) =E(etx1+tx2)

=E(e

tx1

e

tx2

)=E(etx1)E(etx2) * x1 and x2 are independent=mx1(t)mx2(t)=mx1(t) =? mx2(t) =?We knoe that xpoiss(), mx2poiss() also if xpoiss().

mx(t) =E(etx) =

i=0

etxf(x)

mx(t) =

i=0 etx e

x

x!

=e

i=0(et)x

x!

we know that et =

i=0

xi

x!. Remember that taylor theorem

f(x) = i=0 f(i)(a)i! (x a)imx(t) =e

eet

=e(et1)

Thenmx1+x2(t) =mx1(t) + mx2(t)

=e(et1)e(e

t1) =e2(et1)

This is the GMF of Xpoisson(2, ). Therefore P(x1+ x2= t) = e2(2)t

t!

p{X=xT =t}= P{x1=x1,x2=tx1}p(x1+x2=t)

= P(x1=x1)p(x2=tx1)p(x1+x2=t)

e x1x1!

tx1

(tx1)! e2(2)t

t! IA(X)

A={(x1, x2) R :xi = 0, 1, 2,...; i= 1, 2; t= x1+ x2}e2

t

x1!(tx1)!

e22t t

t!

= t!2tx1!(tx1)! IA(X)

=

tx1

12

tIA(X)

Example 5 (pag 360)Let x1, x2 be idd P() Rvs, and consider the statistic T = x1+ x2.= H={: >0}x= (x1, x2), T =x1+ 2x2P(x= xT =t) =P(x1= x1, x2= x2x1+ 2x2= t)

TheoremFactorization criterionLet x have density fx(x; ) H.Then T= T(x) is sufficient for based on xif and only if (iff) fx(x; ) can be expresed in the form.

fx(x; ) =g ((x), )h(x) (x R)

Where h(.) is a nonnegative function ofx1,...,xn only and does not depend on on g(.) is a nonegative function on and I(x).The definition is true for discrete or absolute continous core and ever more abs-tract setting (Halms- savoge)

16


17/19

proof

Suppose I is dimensional. Suppose *.Let t1 = T an let t2,...,tn be such that the trom formation form x to T has ano-vanmisting Jacobian for its inversa.T :T(x) =tT1 :x = T1(t)T(x) =T(T1(t)) is a fuction oft1so Ft(t; ) =fx(t

1(t); )(j(T1))= g(t1; )h((T

1(t))(T1)

We will show the conditional density ft2,...,tnt1(t2,...,tnt1) does not dependon

ft2,...,tnt1(t2,...,tnt1) = f(t2,...,tn, t1)

ft1(t1) = ft(t; )

ft1(t1)

= g(t1; )h(t

1(t))(J(t1))t2

...

tng(t1; )h(t1(t))(J(t1))dt2...dtn

Notice:g (t1; ) is a constant for the integral, then we can canal onil ir with thenumeral.

= h(t1(t))(J(t1))

t2 ... tn h(t1(t))(J(t1))dt2...dtnThis expression does not depend on . Thent1is sufficient for based on x. Con-versely: Asumme that x1, x2,...,xn have density that can be obteined via Jaco-bian frmfx(x; ) ifxT1= t1is independent oft1then alsox2,...,xnT1= t1is independent of t1Why? on because (x2,...,xn) i a subset of (x1,...,xn)So fx(x; ) =ft1,x2,...,xn(t1(x), x2,...,xn; )JNotice:Jdoes not depend on only it is function ofxisfx(x; ) =ft1,x2,...,xn(t1(x), x2,...,xn)Jfx(x; ) =ft1(t1(x); )fx2,...,xnf(x2,...,xnt1(t1(x)))x1,...,xn are independent t1(x1,...,xn) =t1(x)Then t1(x) and (x2,...,xn) are independent. *** May depend on , depend on

x through t1(x) so * holds.fx(x; ) =g(t1(x); )h(x)fn discrete cases see rohotgis.Definition (Ancillary statistic)xfx(x; ), in H U(x) is anciallary based on x is the distribution of U(x)doesntinvolve .Note1 : Fromtheproofweseethatg(t(x);) is except for a factor which might

17


18/19

depend on x but not on , thedensityofT .

W hy?Example.1 x1,...,xn idd N(, 1)fx(x; ) =fxi(x, )=1

2(1)e

12 (

x11 )

2

...frac1

2(1)e

12 (

xn1 )

2

=

1

2(1)

ne

12

(xi)2

we have that

(xi )2 =

(xi x + x )2=

((xi x)2 + (xi x)(x ) + (x )2)=

(xi x)2 +

(xi x)(x ) +

x )2

fx(x; () =

12

ne

12

(xix)2 12

(x)2}

fx(x; () = 12n

e12

(xi

x)2

e12

(x

)2

*

(x )2 =n(x )2

Theng(t(x), ) =? h(x)=?let t(x) =x

g(t(x), ) =en2(x)2

h(t(x), ) =

12

ne

12

(x)2

Therefore T(x)= x is sufficient for all based on X.Note 2:If I is k-dimentional, say I = (t1,...,tk) we say that I = (t1,...,tk) are JOINSUFFICIENTfor based on X.EXAMPLE

textcircled2

x1,...,xn idd N(, 1)

t1= x1:nt2= x2:n.

.

.

tn= xn:n

Then (x1:n, x2:n,...,xn:n) are joint sufficient for based on xWhy? x is sufficient for .

textcircled3

x1,...,xn idd N(1, 2); 1, 2 UNKNOWN

t1= x1:nt2= x2:n

18


19/19

.

..

tn= xn:nWhy? . (x1:n, x2:n,...,xn:n) are joint sufficient for based on x.(x, s2) or (

xi,

x2i ) is sufficient for (1, 2) based on x.

why?Example 8

fx(x; 1, 2) = 1

2n2e 12

ni=1

(xi1)2

2

= 1

theta2

x2i

212

xi+ n

212

fx(x; 1, 2) =

1

2n2 e 122

x2i

e

122

xi

m212

g(t(x), ) =? h(x)=?

19

Primera Vance

Documents

Transcript of Primera Vance