A goodness-of-fit test based on the multiplier Bootstrap with application to left-truncated data

by Takeshi Emura

Joint work with Dr. Yoshihiko Konno,Japan Women’s University

1

Seminar at Institute of StatisticsSeminar at Institute of StatisticsNational University of KaohsiungNational University of Kaohsiung

Outlines• Goodness-of-fit test: Review• Multiplier Bootstrap: Review

• Goodness-of-fit with application toleft-truncated data1. Proposed method2. Simulation 3. Data analysis

2

Goodness-of-fit: Review•Parametric family:

•Underlying CDF =

•Testing

based on data

3

pF Rθθ ,}|{

F

FHvsFH :.: 10

FXX n ~...,,1

•Under , there exists such that .

is consistent

Test Statistics1) Kolmogorov-Smirnov statistic

2) Cramér-von Mises statistic

4

|,)()(ˆ|sup ˆ xFxFKkRx

θ

)(})()(ˆ{ ˆ2

ˆ xdFxFxFC θθ

),...,(ˆˆ1 nXXθθ

FH :0

θ θFF

)()(1)(ˆ

),()()(

1

ˆ

xFxXIn

xF

xFxFxFn

ii

θθ

•The null distribution of :

depend on both and

1) Table available in Lilliefors (1967 JASA)

2) Table available in Lilliefores (1969 JASA)

5

|)()(ˆ|sup ˆ xFxFKkRx

θ

)(})()(ˆ{ ˆ2

ˆ xdFxFxFC θθ

θ }|{ θθF

}0,|),({ 22 RN

}0|)({ Exp

•Parametric Bootstrap has been suggested to approximate the null distribution of

Under cerntain regularity conditions, the parametric bootstrap is theoretically and numerically justified:• Stute et al. (1993 Metrika): Continuous case• Henze (1996 Can. J. Stat): Discrete case• Genest & Remillard (2008 Annales de Inst. Poincare) : Easy-to-check regularity conditions

• p.279 of Asymptotic Statistics by van der Vaart (1998): Empirical process approach

6

|)()(ˆ|sup ˆ xFxFKkRx

θ

)(})()(ˆ{ ˆ2

ˆ xdFxFxFC θθ

Algorithm:

7

0.05 )(IB1 value-

if }|{:HReject :4 Step

)(})()(ˆ{

Compute :3 Step),,( from

)(I1)(ˆ and ˆ Calculate :2 Step

,...,1,~),,( Generate :1 Step

B

1b

)*(

0

ˆ2

ˆ)*()*(

)*()*(1

n

1i

)*(1

)*()*(

ˆ)*()*(

1

)(*)(*

CCP

F

xdFxFxFC

XX

xXn

xF

BbFXX

b

bb

bn

b

bbb

bn

b

bb

θ

θ

θ

θθ

θ

•The parametric Bootstrap is very time consuming,especially when is multivariate

•A tool to reduce the computational cost “Multiplier Bootstrap” or “weighted Bootstrap”become very popular in the recent literature

(Remillard & Scaillet 2009 JMVA; Kojadinovic & Yan 2011 Stat.& Computing;Bucher & Dette 2010 Stat&Prob letters; Kojadinovic & Yan, 2012 manusc.)

•I use 4 slide to explain what is “multiplier Bootstrap”

8

F

What is multiplier Bootstrap?• IID: • Empirical Process:

How to approximate the distribution ?• Re-sampling:

Bootstrap version:

9

FXX n ~,...,1

i

in tFtXIn

t )}()({1)(G

),...,(),...,( **11 nn XXXX

i

in tFtXIn

t )}(ˆ)({1)( **G i

i tXIn

tF )(1)(ˆ

What is multiplier Bootstrap?• Multiplier representation

• Multiplier Bootstrap is a modification :

10

} )/1...,,/1(, { lMultinomia~),...,( where

)}(ˆ)({1

)}(ˆ)({1)(

1

**

nnnZZ

tFtXIZn

tFtXIn

t

n

iii

iin

G

1Var(Z) 0,E[Z] Z,~),...,( 1 iid

nZZ

i

iin tFtXIZn

t )}(ˆ)({1)(MPG

Ref: Sec 3.6 of van der Vaart & Wellner (1996) Weak convergence & empirical processes

----- Original----- Multiplier Boot

11

)(MP tnG)(tnG

0.0 0.5 1.0 1.5 2.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

F_n(

t)

n=1000, X~Exp(1), Z~N(0,1)

12

ondistributi asymptotic same thehas )( and )(: oremlimit the central Multiplier

MP tt nn FF

i

iUn

T 1 :Statistic

• In principle, Multiplier Bootstrap is applicable to any statistics of the sum of iid terms:

i

iiUZn

T 1 : versionBootstrap MP *

( Ref: Sec 2.9: Multiplier Central Limit Theorem in the book:van der Vaart & Wellner (1996) Weak convergence & empirical processes )

From now on, I focus on applications to left-truncated data

13

Left-Truncated data• Vast literature: see

Book of Klein & Moeschberger 2nd ed. 2003

14

jjjj XLnjXL subject to)},,1( ),({

0 2 4 6 8 10

02

46

810

Truncated Normal Plot

L

X L~N(5,2), X~N(5,2), Cov(L,X)=0.5

• Mathematical Set up Population random variable;

Post-truncated data;

15

),( OO XL

! observed nothing , If OO XL

observed is ),(),(pair a , If jjOOOO XLXLXL

jjjj XLnjXL subject to)},,1( ),({

)Pr()(),(

)|,Pr(

from ...

OOXLOOOO

XLxlxlf

XLdxXdlL

diiOO

I

• Likelihood

• Maximum likelihood estimator

Consistency & Asymptotic normality follows when andare 3 times continuously differentiable (Emura & Konno 2012 CSDA)

16

.),()Pr((),(for density :

where

,),(()(

xl

OO

OO

jjj

n

dldxxlfXLcXLf

XLfcL

θ

θ

θ

θ)

θ)θ

0θθ

θ )(log :ˆ L

θ)(c θf

Some useful bivariate modelsExample 1: Bivariate t or bivariate normal

Example 2: Bivariate Poisson (Holgate, 1964 Biometrika) :

Example 3: Zero-modified Poisson (Dietz and Böhning, 2000 CSDA)

17

2/)2(22/1222 ),|,(1)2/(

}2/)2{()(),(

v

LXXL

vxlQ

vvvxlf Σμ

θ

),,,),,(( 22 vLXXLXL μθ

tvvXLcLXLX

LXOO -student of cdf :)(: ,:2

)Pr()(22

θ

l

w

wwxX

wlL

wwxwlexlf XL

0 !)!()!(),(

θ

0

)(!

)Pr()(l

V

lLOO lS

leXLc XL

θ

lv

vX

V velS

X

!)(

,2,1,0;1,0,!

)()1(),( 1

xlx

eppxlfXx

Xll

θ

XpeXLc OO 1)Pr()(θ

18

Let }|{ θθf be a given parametric family.

fH :0 against fH :1 .

Goodness-of-fit• Cramér-von Mises statistic

* I will not use Kolmogorov-Smirnov statistic since it is computationally more demanding.

19

j jj

jjjjj

xl

nxXlLxlF

cXLFXLF

xlFdcxlFxlFC

/),(),(ˆ where

,})ˆ(/),(),(ˆ{

),(ˆ})ˆ(/),(),(ˆ{

2ˆ

2ˆ

I

θ

θ

θ

θ

)Pr(( OO XLc θ)

.discrete ),(

;continuous),(),( ,

lu xvu

xvulu

vuf

dudvvufxlF

θ

θ

θ

OL

OX

Some useful bivariate modelsExample 1: Bivariate normal

Example 2: Bivariate Poisson (Holgate, 1964) :

Example 3: Zero-modified Poisson (Dietz and Böhning, 2000)

20

density normal Bivariate),( xlfθ

l

w

wwxX

wlL

wwxwlexlf XL

0 !)!()!(),(

θ

,2,1,0;1,0,!

)()1(),( 1

xlx

eppxlfXx

Xll

θ

dsssssxxlFLLl

LLXX

LLXXLL

LLXX

LLXX )(/

//

/),(/)(

222222

θ

l

u

x

uv

u

w

wwvX

wuL

wwvwuexlF XL

0 0 !)!()!(),(

θ

,1 if!/

;0 if )!/()1(),(

0

0

lpue

luepxlF x

u

xX

x

u

xX

X

X

θ

Parametric Bootstrap

21

Step 0: Calculate the statistic i

iiii cXLFXLFC 2ˆ })ˆ(/),(),(ˆ{ θθ

.

Step 1: Generate ),( )()( bj

bj XL which follows the truncated distribution of ),(ˆ xlFθ ,

subject to )()( bj

bj XL , for njBb ,,2,1,,,2,1 .

Step 2: Calculate },,2,1;{ )*( BbC b ,

where i

bbi

bi

bi

bi

bb cXLFXLFC b2)()()(

ˆ)()()()*( })ˆ(/),(),(ˆ{ )( θθ and where

),(ˆ )( xlF b and )(ˆ bθ are the empirical CDF and MLE based on

},,2,1);,({ )()( njXL bj

bj .

Step 3: Reject 0H at the 100 % significance level if

BCCB

b

b /)(1

)*(I .

Parametric Bootstrap

22

Table 1: Type I error rates of the the Cramér-von-Mises type goodness-of-fit test

at level based on 300 replications. The cut-off value is obtained by the

parametric bootstrap-based procedure based on 1,000 resamplings.

OOYX

-0.70 -0.35 0.00 0.35 0.70

10.0 0.113 0.090 0.107 0.123 0.083

05.0 0.056 0.040 0.047 0.060 0.050

01.0 0.013 0.007 0.007 0.023 0.013

2

2

2

2

16.8116.8119.6416.8119.6419.64

60.8262.63-120

~LX

LX

XLX

LXL

X

LO

O

NXL

•Some simulation results under bivariate normal model (Emura & Konno, 2012a Stat Papers)

•Parametric Bootstrap is computationally very intensive, especially in Step 2:

• Maximizing likelihood functions B times oftenencounter the problem of local maxima(especially serious when data are not from the null)

23

Step 2: Calculate },,2,1;{ )*( BbC b ,

where i

bbi

bi

bi

bi

bb cXLFXLFC b2)()()(

ˆ)()()()*( })ˆ(/),(),(ˆ{ )( θθ

and where

),(ˆ )( xlF b and )(ˆ bθ are the empirical CDF and MLE based on

},,2,1);,({ )()( njXL bj

bj .

Alternatively, we apply the “Multiplier Bootstrap” to approximate the distribution of

Motivation: Reduce the computational cost

24

jjjjj

xl

cXLFXLF

xlFdcxlFxlFC

2ˆ

2ˆ

})ˆ(/),(),(ˆ{

),(ˆ})ˆ(/),(),(ˆ{

θ

θ

θ

θ

Theorem 1 (Emura & Konno 2012 CSDA):

Conditional on data, only random quantities are

25

θθθθθθθg

θθgθIθ

θθ

θθ

θθθ

θθ

θ

/)()(,/),(),(

},)(),()(),({)(),(

),(}/)(){,()(/),(),();,(ˆ where

),1()ˆ;,(ˆ1})ˆ(/),(),(ˆ{

2

1

ˆ

ccxlFxlFcxlFcxlFcxl

lnixlcxlFxXlLxlV

oxlVn

cxlFxlFn

jnjjj

Pj

j

1)( ,0)(E with variablerandom be Let jjj ZVarZZ

},,2,1;{ njZ j

j

jj xlVZn

)ˆ;,(ˆ1 θ

26

Step 0: Calculate the statistic i

iiii cXLFXLFC 2ˆ })ˆ(/),(),(ˆ{ θθ

and matrix

)ˆ(ˆ θV .

Step 1: Generate njBbNZ bj ,,2,1,,,2,1);1,0(~)( .

Step 2: Calculate },,2,1;{ )( BbC b ,

where ||/)ˆ(ˆ)(|| )()( nC bb θVZ and ),,( )()(1

)( bn

bb ZZ Z .

Step 3: Reject 0H at the 100 % significance level if

BCCB

b

b /)(1

)(I .

22

2

2ˆ

)ˆ(ˆ)ˆ;,(ˆ1

by edapproximat is })ˆ(/),(),(ˆ{

ofon dsitributi therem,limit theo central multiplier By the

nXLVZ

n

cXLFXLFC

i jiijj

jjjjj

θVZθ

θθ

27

Let })ˆ(/),(),(ˆ{),( ˆ θθ cxlFxlFnxlGn and j j

bj

bn xlVZnxlG )ˆ;,(ˆ),( )(2/1)( θ

for Bb ,,2,1 .

Theorem 2: Suppose that (R1) through (R8) listed in Emura & Konno (2012 CSDA)

hold. Under 0H , and given B,

)...,,,()...,,,( )()1()()1( BBnnn GGGGGG θθθ

in )1(2}),{( BD , where θG is the mean zero Gaussian process whose

covariance for 2** ),(),(),,( xlxl is given as

),,()(),()(/),(),()(/),(

});,();,({}),(),,({**12****

****

xlIxlcxlFxlFcxxllF

xlVxlVExlGxlGCov jj

θθθθθ

θθ

gθgθθ

θθ

where ),min( baba , and )()1( ...,, BGG θθ are independent copies of θG .

Simulation under the null

28

60.8262.63-120

X

L

μ ,

2

2

2

2

16.8116.8119.6416.8119.6419.64

LX

LX

XLX

LXL

Σ

• Model for : Bivariate normal

• Generate data:

• Compute Cramér-von Mises statistic

• Compute Bootstrap version

),( OO XL

jjjj XLjXL subject to)}400,,1( ),({

j

jjjj cXLFXLFC 2ˆ })ˆ(/),(),(ˆ{ θθ

}1000,,2,1;*{ :Boot Parametric Usual}1000,,2,1;{ :Boot Multiplier

)(

)(

bCbC

b

b

29

Multiplier Boot vs. naïve Bootstrap with 400n

Elapsed time

(in second)

Resampling mean and

SD (in parenthesis)

Multiplier Parametric

Bootstrap Multiplier

Parametric

Bootstrap

OOYX =0.70 23.17 1402.31 0.0621 (0.0314) 0.0603 (0.0304)

OOYX =0.35 22.84 1463.01 0.0606 (0.0269) 0.0581 (0.0252)

OOYX =0.00 23.04 1539.29 0.0597 (0.0219) 0.0586 (0.0225)

OOYX =-0.35 23.37 1380.09 0.0598 (0.0234) 0.0577 (0.0209)

OOYX =-0.70 26.77 1296.32 0.0591 (0.0225) 0.0569 (0.0217)

Almost same null distribution

Power study

30

we generated data from the bivariate t-distribution (Lang et al., 1989) while we performed the goodness-of-fit test under the null hypothesis of the bivariate normal distribution.

Data analysisLaw school data in US(Efron & Tibshirani 1993, An Introduction to the Bootstrap)

• Average LSAT (the national law test)• Average GPA (graduate point average)

for N=82 American law schools

31

)GPA,(LSAT jj for Nj ...,,2,1

32

}82,...,1);LSAT,{GPA jjj

260 280 300 320 340

500

550

600

650

700

100*GPA

LSA

T

• We artificially define the truncation

33

! observed nothing ,900GPA100LSAT If (ii)sample theobserve we,900GPA100LSAT If (i)

Observed

Un observed

260 280 300 320 340

500

550

600

650

700

100*GPA

LSA

T

• We rewrite the truncation criterion

*

34

Observe }49,,2,1);,({ jXL jj ,

subject to jj XL , where jjL GPA100900 and jjX LSAT .

GPA100 - 900

LSAT where

GPA100 - 900 LSAT ,900 GPA 100LSAT

O

O

OO

LX

L X

35

560 580 600 620 640

500

550

600

650

700

900 - GPA

LSA

T

jj

jj

XLjXL

subject to}49,,2,1);,({

Un observed

bivariate normal fit ?

Computational time for the multiplier method = 1.25 secondComputational time for the parametric Bootstrap = 222.46 second

36

Multiplier

log(Cramer-von Mises statistic)

Freq

uenc

y

-4 -3 -2 -1 0 1

010

2030

40

p-value = 0.884

Parametric Bootstrap

log(Cramer-von Mises statistic)

Freq

uenc

y

-4 -3 -2 -1 0 1

020

4060

p-value = 0.645

Summary: Why multiplier reduce computational cost ?• The multiplier involves only arithmetic operations (multiply,

sum). On the other hand, the parametric Bootstrap involves nonlinear maximization to get

• Re-sampling from a truncated parametric model is involved:

On the other hand, the multiplier only requires generatingi.i.d. sequences

37

)(ˆ bθ

Accept-reject algorithm

(i) data ),( XL from the distribution function ),(ˆ xlFθ is generated;

(ii) if XL , we accept the sample and set ),(),( )()( XLXL bj

bj ;

otherwise we reject ),( XL and return to (i).

Thank you for your kind Thank you for your kind attentionattention• Bücher, A. and Dette, H. , 2010. A note on bootstrap approximations for the empirical copula process. Statist.

Probab. Lett. 80, 1925-1932.• Dietz, E., Böhning, D., 2000. On estimation of the Poisson parameter in zero-modified Poisson models.

Computational Statistics ＆ Data Analysis 34, 441-459.• Efron, B., Tibshirani, R. J., 1993. An Introduction to the Bootstrap. Chapman & Hall, London.• Emura, T., Konno, Y., 2012. A goodness-of-fit test for parametric models based on dependently truncated data,

Computational Statistics & Data Analysis 56, 2237-2250.• Emura, T., Konno, Y, 2012. Multivariate normal distribution approaches for dependently truncated data. Statistical

Papers 53 (No.1), 133-149.• Genest C., Remillard B., 2008. Validity of the parametric bootstrap for goodness-of-fit testing in semiparametric

models, Annales de Institut Henri Poicare –Probabilites et Statistiques 44, 1096-1127.• Henze N., 1996. Empirical-distribution-function goodness-of-fit tests for discrete models. Cand. J. Statist. 24, 81-93.• Lilliefors HW, 1967. On the Kolmogorov-Smirnov test for normality with mean and variance unknown. JASA 62

399-402.• Lilliefors HW, 1969. On the Kolmogorov-Smirnov test with exponential distribution with mean unknown. JASA 64

387-389.• Fang, K.-T., Kotz, S., Ng, K. W., 1990. Symmetric Multivariate and Related Distributions. Chapman & Hall, New York.• Holgate, 1964. Estimation of the bivariate Poisson distribution. Biometrika 51, 241-245.• Klein, J. P., Moeschberger, M. L., 2003. Survival Analysis: Techniques for Censored and Truncated Data. New York,

Springer.• Kojadinovic, I., Yan, J., Holmes, M., 2011. Fast large-sample goodness-of-fit tests for copulas. Statistica Sinica 21,

841-871.• Kojadinovic, I., Yan, J., 2011. A goodness-of-fit test for multivariate multiparameter copulas based on multiplier

central limit theorems. Statistics and Computing 21, 17-30.• Lang, K. L., Little, J. A., Taylor, J. M. G., 1989. Robust statistical modeling using the t distribution. Journal of the

American Statistical Association 84, 881-896.• Stute W et al. , 1993. Bootstrap based goodness-of-fit test. Metrika 40, 243-256.• van der Vaart. A. W., Wellner, J. A., 1996. Weak Convergence and Empirical Process, Springer Series in Statistics,

Springer-Verlag, New York.• J. P. Klein, M. L. Moeschberger, Survival Analysis: Techniques for Censored and Truncated Data, 2nd edition,

Springer, New York, 2003.38