Efficient estimation of conditional covariance matrices for dimension reduction - Solís, Loubes,...

7/31/2019 Efficient estimation of conditional covariance matrices for dimension reduction - Sols, Loubes, Marteau - JdS, Bruxelles 2012

1/44

Efficient estimation of conditional covariance matricesfor dimension reduction

Maikol Sols Chacon Jean-Michel Loubes Clement MarteauInstitut de Mathematiques de Toulouse

Universite Paul Sabatier

44e Journees de Statistiques

Bruxelles22 August 2012

Sols Chacon, Loubes, Marteau (IMT) Bruxelles, 22 August 2012 1 / 23
http://find/http://goback/


2/44

Outline

1 Introduction

2 Methodology

3 Estimation of quadratic functionals

4 Main results

5 Summary

http://find/


3/44

Introduction

Outline

1 Introduction

2 Methodology


4 Main results

5 Summary



4/44

Introduction

Background: Sliced Inverse Regression [Li, 1991]

Consider the problemY = (X) + ,

where X Rp, Y R and E [] = 0.Find s such that

Y = (1 X, . . . , KX, ),

where the s are unknown vectors in Rp, is independent of X and is an arbitrary function in RK+1. K

p.

The eigenvectors of CovE [X|Y] spans the same subspace that thes (effective dimension reduction directions or e.d.r.ds).

The objective is to estimate CovE [X|Y].

http://find/


5/44

I t d ti


6/44

Introduction




Y = (1 X, . . . , KX, ),


p.




Introduction


7/44

Introduction




Y = (1 X, . . . , KX, ),


p.




Introduction
http://goforward/http://find/http://goback/


8/44

Introduction

Previous work

Estimators for CovE [X|Y] and the e.d.r.ds can be found usingKernel estimators, [Zhu & Fang, 1996] and[Ferre & Yao, 2003,2005].

A combination of the nearest neighbor and SIR, [Hsing 1999].

Assumption that E [X|Y] has some parametric form,[Bura & Cook, 2001].

K-means, [Setodji & Cook,2004].

Transformation of SIR to least square form,[Cook & Ni, 2005].

etc.


Introduction


9/44

Introduction

Our idea to estimate CovE [X|Y]

The objective is to estimate directly CovE [X|Y], using a plug-in

method in a semiparametric framework.

We aim to estimate E E [X|Y]E [X|Y] as a quadratic functional(studied by [Da Veiga & Gamboa, 2012] and [Laurent, 1996]).

We will also focus on the efficient estimation of

E E [X|Y]E [X|Y]

.


Introduction


10/44

Introduction

Our idea to estimate CovE [X|Y]

The objective is to estimate directly CovE [X|Y], using a plug-in

method in a semiparametric framework.

We aim to estimate E E [X|Y]E [X|Y] as a quadratic functional(studied by [Da Veiga & Gamboa, 2012] and [Laurent, 1996]).

We will also focus on the efficient estimation of

E E [X|Y]E [X|Y]

.



11/44

Methodology


12/44

gy

Outline

1 Introduction

2 Methodology


4 Main results

5 Summary


Methodology
http://find/


13/44

gy

Preliminaries

1 Let f(xi, xj, y) be the joint distribution of (Xi, Xj, Y), then define

Tij(f) = EE [Xi|Y]E [Xj|Y]

.

2 In general, for f

L

2(dxi, dxj, dy), define the non-linear functional

f Tij(f) with Tij(f) having the formxif(xi, xj, y)dxidxj

f(xi, xj, y)dxidxj

xjf(xi, xj, y)dxidxj

f(xi, xj, y)dxidxj

f(xi, xj, y)dxidxjdy.

3 For an i.i.d sample (X(k)i , X(k)j , Y(k)), k = 1, . . . , n, we can build a

preliminary estimator f of f with a subsample n1 < n.


Methodology


14/44

Preliminaries



.

2 In general, for f

L



f(xi, xj, y)dxidxj


f(xi, xj, y)dxidxj





Methodology
http://find/


15/44

Preliminaries



.

2 In general, for f

L



f(xi, xj, y)dxidxj


f(xi, xj, y)dxidxj





Methodology
http://find/


16/44

Taylors development

Define the auxiliar function F : [0, 1] R asF(u) = Tij(uf + (1

u)f) with u

[0, 1].

We have F(1) = F(0) + F(0) + 12 F(0) + 16 F

()(1 )3 for some [0, 1].Notice that F(1) = Tij(f) and F(0) = Tij(f).


Methodology


17/44

Taylors development


u)f) with u

[0, 1].

We have F(1) = F(0) + F(0) + 12 F(0) + 16 F



Methodology


18/44

Taylors development


u)f) with u

[0, 1].

We have F(1) = F(0) + F(0) + 12 F(0) + 16 F



Methodology


19/44

Taylors development

We obtain,

Tij(f) = H1(f, xi, xj, y)f(xi, xj, y)dxidxjdy Linear Functional (LF)

+

H2(f, xi1, xj2, y)f(xi1, xj1, y)f(xi2, xj2, y)dxi1dxj1dxi2dxj2dy

Quadratic Functional (QF)+ n

Error



20/44

Methodology


21/44

Estimating (LF) and (QF)

The linear part is direct.

Define n2 = n n1. We estimate (LF) by

1

n2

n2

k=1H1

f, X

(k)i , X

(k)j , Y

(k)

.

Core of the work: Main issue.

To estimate (QF), we will build an estimator of

(f) =

(xi1, xj2, y)f(xi1, xj1, y)f(xi2, xj2, y)dxi1dxj1dxi2dxj2dy

where : R3 R is a bounded function.



22/44

Estimation of quadratic functionals


23/44

Estimation of(f): Projection method.

Assumption

SMn f f22 0 where SMn f = lMn alpl (pl is a basis ofL

2(dxi, dxj, dy)).supl/Mn |cl|2

2 |Mn| /n2 for cl a fixed sequence.We build an estimator of using a projection scheme with Bias equalto

(SMn f(xi1, xj1, y)f(xi1, xj1, y))(SMn f(xi2, xj2, y)f(xi2, xj2, y))(xi1, xj2, y)dxi1dxj1dxi2dxj2dy

(AMSE) E

n

2

= O

1

nSols Chacon, Loubes, Marteau (IMT) Bruxelles, 22 August 2012 13 / 23


24/44


25/44


26/44

Main results


27/44

Putting (LF) and (QF) together

T

(n)ij =

1

n2

n2

k=1H1(f, X

(k)i , X

(k)j , Y

(k))+Estimator of (QF)

The estimator of (QF) has two sums which depends on the basis plchosen for the projection.

The term n is negligible.


Main results ( )


28/44

Asymptotic normality of T(n)ijTheorem

Under some technical assumptions and |Mn| /n 0 when n . Wehave:

nT(n)ij

Tij(f)

D

N(0, Cij(f)) ,

and

limn

nET(n)ij Tij(f)2 = Cij(f),

where

Cij(f) = VarH1(f, Xi, Xj, Y)Conclusion: There is asymptotic normality.



29/44

Main results

S i i C R b d


30/44

Semi-parametric Cramer-Rao bound

Theorem

Under the same conditions as before, for any estimator T(n)ij of Tij(f) andevery family{Vr(f)}r>0 of neighborhoods of f we have

inf{Vr(f)}r>0

lim infn

supfVr(f)

nE

T

(n)ij Tij(f)

2 Cij(f).

Conclusion: The estimator is efficient.

Generalization via half-vectorization to matrices: T(f) = (Tij(f))pp andH

1(f) = (H1(f, xi, xj, y))pp.

n vech

T(n) T(f) D N(0, C(f)) ,C(f) = Cov

vech(H1(f))Sols Chacon, Loubes, Marteau (IMT) Bruxelles, 22 August 2012 17 / 23

Summary

O li


31/44

Outline

1 Introduction

2 Methodology


4 Main results

5 Summary


Summary

S


32/44

Summary

We use a plug-in method used by[Laurent, 1996, Da Veiga & Gamboa, 2012] to find CovE [X|Y].

Taylors development on EE [X|Y]E [X|Y]

(semi-parametric

framework)Projection method to estimate the non-linear part.

Asymptotically normal and efficient.

Generalization of the asymptotic normality to the whole matrix.

Non-adaptative: the method depends directly on the regularity of f.

Future work

Numerical results for this method (In progress).Analyze rates of convergence exploring other techniques like kernelestimators.


Summary

S


33/44

Summary



(semi-parametric





Future work



Summary

S


34/44

Summary



(semi-parametric





Future work



Summary

Summary


35/44

Summary



(semi-parametric





Future work



Summary

Summary
http://find/


36/44

Summary



(semi-parametric





Future work



Summary

Summary


37/44

Summary



(semi-parametric





Future work



Summary

Summary


38/44

Summary



(semi-parametric





Future work



Summary

Summary


39/44

Summary



(semi-parametric





Future work



Summary

Summary
http://find/


40/44

Summary



(semi-parametric





Future work



Th k


41/44

Thank youReferences

Da Veiga, S. & Gamboa, F. (2012).Efficient estimation of sensitivity indices.Arxiv Preprint ArXiv:1203.2899.

Laurent, B. (1996).Efficient estimation of integral functionals of a density.The Annals of Statistics, 24(2), 659681.

Li, K. C. (1991).Sliced inverse regression for dimension reduction.Journal of the American Statistical Association, 86(414), 316327.

Sols Chacon, M., Loubes, J. M., Marteau, C., & Da Veiga, S. (2011).Efficient estimation of conditional covariance matrices for dimensionreduction.

Summary

Hypothesis
http://find/


42/44

Hypothesis

Let(pl(xi, xj, y))lD an orthonormal basis ofL2(dxidxjdy) (Dcountable).

E= lD

alpl : (al)lD such that lD

alcl 2

< 1 L2(dxidxjdy)where (cl)lD is a fixed sequence.

f

E.

Let (Mn)n1 a subset sequence of D. For any n there is Mn such thatMn D. Denote by |Mn| the cardinal of Mn.


Summary

Assumptions
http://find/


43/44

Assumptions

1 For any n 1cmmere is Mn D such thatsupl/Mn |cl|2

2 |Mn| /n2. Moreover, f L2(dxdydz),(SMn f f)2 dxdydz 0 when n 0, where SMn f =

lMn

alpl.

2 supp f

[d1, b1]

[d2, b2]

[d3, b3] and

(x, y, z)

supp f,

0 < f(x, y, z) with , R.3 We can build f , such that > 0,

(x, y, z) supp f, 0 < f(x, y, z) + y 2 q +, l N, Eff flq C(q, l)nl1

for > 1/6 and a constant C(q, l) no depending of f E.


Summary

Appendix: Estimator (LF) and (QF)


44/44

pp ( ) (Q )

T(n)ij = 1n2n2k=1

H1(f, X(k)i , X

(k)j , Y

(k))

+1

n2(n2

1) lM

n2

k=k=1pl

X

(k)i , X

(k)j , Y

(k)

pl

xi, xj, Y(k)

H3

f, xi, xj, X(k)i , X

(k)j , Y

(k)

dxidxj

1n2(n2

1) l,lM

n2

k=k=1pl

X(k)i , X

(k)j , Y

(k)

pl

X(k)i , X

(k)j , Y

(k)

pl(xi1, xj1, y)pl(xi2, xj2, y)H2

f, xi1, xj2, y)dxi1dxj1dxi2dxj2dy.

where H3(f, xi1, xj1, xi2, xj2, y) = H2(f, xi1, xj2, y) + H2(f, xi2, xj1, y) andn2 = n

n1.


Efficient estimation of conditional covariance matrices for dimension reduction - Solís, Loubes,...

Documents

Transcript of Efficient estimation of conditional covariance matrices for dimension reduction - Solís, Loubes,...