Data-driven goodness-of-fit tests · Data-driven goodness-of-fit tests Mikhail Langovoy Institute...
Transcript of Data-driven goodness-of-fit tests · Data-driven goodness-of-fit tests Mikhail Langovoy Institute...
Data-driven goodness-of-fit tests
Mikhail Langovoy
Institute for Applied Mathematics,University of Bonn
September 1, 2008
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 1 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 2 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 3 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 4 / 34
Two Approaches
Constructing good tests is an essential problem of statistics.
Two approaches:
direct: "distance" between theoretical and empirical distribution isproposed as statistic
aiming optimality: construct tests which are asymptotically efficient(Neyman, Le Cam, Wald)
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 5 / 34
Two Approaches
Constructing good tests is an essential problem of statistics.
Two approaches:
direct: "distance" between theoretical and empirical distribution isproposed as statistic
aiming optimality: construct tests which are asymptotically efficient(Neyman, Le Cam, Wald)
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 5 / 34
Examples of Distance-Based Statistics
Kolmogorov - Smirnov:
Dn =√
n∥∥Fn − F
∥∥∞
Cramer - von Mises:
ω2n = n
∫ ∞−∞
(Fn(t)− F0(t)
)2 d F0(t)
many other
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 6 / 34
About Distance-Based Tests
These tests works
asymptotically optimal only in a few directions of alternatives
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 7 / 34
Another type: Neyman’s Statistic
hypothesis H0 : X ∼ U[0,1]
{φj}∞j=0 orthonormal basis of L2([0,1], λ)
Nk =k∑
j=1
{n−
12
n∑i=1
φj(Xi)
}2
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 8 / 34
Another type: Neyman’s Statistic
hypothesis H0 : X ∼ U[0,1]
{φj}∞j=0 orthonormal basis of L2([0,1], λ)
Nk =k∑
j=1
{n−
12
n∑i=1
φj(Xi)
}2
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 8 / 34
Another type: Neyman’s Statistic
hypothesis H0 : X ∼ U[0,1]
{φj}∞j=0 orthonormal basis of L2([0,1], λ)
Nk =k∑
j=1
{n−
12
n∑i=1
φj(Xi)
}2
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 8 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 9 / 34
Idea of Selection Rule
model dimension k was known fixed in advance
important: select the right model dimension!
incorrect choice can decrease the power of a test
Solution: incorporate the test statistic by some procedurechoosing the right dimension automatically by the data
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 10 / 34
Idea of Selection Rule
model dimension k was known fixed in advance
important: select the right model dimension!
incorrect choice can decrease the power of a test
Solution: incorporate the test statistic by some procedurechoosing the right dimension automatically by the data
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 10 / 34
Idea of Selection Rule
model dimension k was known fixed in advance
important: select the right model dimension!
incorrect choice can decrease the power of a test
Solution: incorporate the test statistic by some procedurechoosing the right dimension automatically by the data
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 10 / 34
Bonuses of data-driven score tests
Data-driven score tests are
asymptotically optimal in an infinite number of directions
show good overall performance in practice
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 11 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 12 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 13 / 34
Penalty
Definitionnested family of models Mk for k = 1, . . . , d(n)
d(n) control sequence
choose π(·, ·) : N× N→ R
assume
π(1,n) < π(2,n) < . . . < π(d(n),n) for all n
π(j ,n)− π(1,n)→∞ as n→∞ for j = 2, . . . ,d(n)
Call π(j ,n) a penalty attributed to model Mj and sample size n.
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 14 / 34
Penalty
Definitionnested family of models Mk for k = 1, . . . , d(n)
d(n) control sequence
choose π(·, ·) : N× N→ R
assume
π(1,n) < π(2,n) < . . . < π(d(n),n) for all n
π(j ,n)− π(1,n)→∞ as n→∞ for j = 2, . . . ,d(n)
Call π(j ,n) a penalty attributed to model Mj and sample size n.
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 14 / 34
Penalty
Definitionnested family of models Mk for k = 1, . . . , d(n)
d(n) control sequence
choose π(·, ·) : N× N→ R
assume
π(1,n) < π(2,n) < . . . < π(d(n),n) for all n
π(j ,n)− π(1,n)→∞ as n→∞ for j = 2, . . . ,d(n)
Call π(j ,n) a penalty attributed to model Mj and sample size n.
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 14 / 34
Penalty
Definitionnested family of models Mk for k = 1, . . . , d(n)
d(n) control sequence
choose π(·, ·) : N× N→ R
assume
π(1,n) < π(2,n) < . . . < π(d(n),n) for all n
π(j ,n)− π(1,n)→∞ as n→∞ for j = 2, . . . ,d(n)
Call π(j ,n) a penalty attributed to model Mj and sample size n.
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 14 / 34
Selection rule
Tk : testing validity of Mk
DefinitionA selection rule S for the sequence of statistics {Tk} is
S = min{
k : 1 ≤ k ≤ d(n); Tk − π(k ,n) ≥ Tj − π(j ,n), j = 1, . . . ,d(n)}
Call TS a data-driven test statistic for testing validity of the initial model.
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 15 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 16 / 34
Framework
X1,X2, . . . random variables, values in a measurable space X
X1, . . . ,Xm have joint distribution Pm ∈ Pm - for every m
function F acting from ⊗∞m=1 Pm = (P1,P2, . . .) to a known set Θ
F(P1,P2, . . .) = θ
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 17 / 34
Framework
X1,X2, . . . random variables, values in a measurable space X
X1, . . . ,Xm have joint distribution Pm ∈ Pm - for every m
function F acting from ⊗∞m=1 Pm = (P1,P2, . . .) to a known set Θ
F(P1,P2, . . .) = θ
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 17 / 34
Framework
H0 : θ ∈ Θ0 ⊂ Θ
HA : θ ∈ Θ1 = Θ \Θ0
observations Y1, . . . , Yn, values in a measurable space Y
not necessarily on the basis of X1, . . . , Xm !
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 18 / 34
Framework
H0 : θ ∈ Θ0 ⊂ Θ
HA : θ ∈ Θ1 = Θ \Θ0
observations Y1, . . . , Yn, values in a measurable space Y
not necessarily on the basis of X1, . . . , Xm !
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 18 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 19 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 20 / 34
Main concept
Definition
observations Y1, . . . , Yn, values in a measurable space Y
k fixed number
l = (l1, . . . , lk ) vector-function
li : Y→ R for i = 1, . . . , k are known Lebesgue measurable
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 21 / 34
Main concept
Definition
L = {E0[l(Y )]T l(Y )}−1
E0 is with respect to P0
P0 is the d.f. of some (fixed and known) random variable Y
Y has values in Y
assume
E0 l(Y ) = 0
L is well defined
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 22 / 34
Main concept
Definition
L = {E0[l(Y )]T l(Y )}−1
E0 is with respect to P0
P0 is the d.f. of some (fixed and known) random variable Y
Y has values in Y
assume
E0 l(Y ) = 0
L is well defined
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 22 / 34
Main concept
Definition
Tk =
{1√n
n∑j=1
l(Yj)
}L{
1√n
n∑j=1
l(Yj)
}T
Tk - statistic of Neyman’s type (NT-statistic)
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 23 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 24 / 34
New possibilities
l1, . . . , lk can be
some score functions
any other functions, depending on the problem
truncated, penalized or partial likelihood
possible to use l1, . . . , lk unrelated to any likelihood
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 25 / 34
New possibilities
l1, . . . , lk can be
some score functions
any other functions, depending on the problem
truncated, penalized or partial likelihood
possible to use l1, . . . , lk unrelated to any likelihood
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 25 / 34
New possibilities
l1, . . . , lk can be
some score functions
any other functions, depending on the problem
truncated, penalized or partial likelihood
possible to use l1, . . . , lk unrelated to any likelihood
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 25 / 34
Important applications
statistical inverse problems
rank tests for independence
semiparametric regression
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 26 / 34
Important applications
statistical inverse problems
rank tests for independence
semiparametric regression
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 26 / 34
Important applications
statistical inverse problems
rank tests for independence
semiparametric regression
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 26 / 34
Statistical inverse problems
Deconvolution
applications from signal processing to psychology
basic statistical inverse problem
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 27 / 34
Deconvolution
instead of Xi one observes Yi
Yi = Xi + εi
ε′is are i.i.d. with a known density h
Xi and εi are independent for each i
H0 : X has density f0
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 28 / 34
Deconvolution
instead of Xi one observes Yi
Yi = Xi + εi
ε′is are i.i.d. with a known density h
Xi and εi are independent for each i
H0 : X has density f0
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 28 / 34
Deconvolution
choose for every k ≤ d(n) an auxiliary parametric family {fθ}
θ ∈ Θ ⊆ Rk
f0 from this family coincides with f0 from the null hypothesis H0
the true F possibly has no relation to the chosen {fθ}
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 29 / 34
Deconvolution
l(y) =
∂∂θ
( ∫R fθ(s) h( y − s) ds
)∣∣∣θ=0∫
R f0(s) h( y − s) ds
define Tk as above ⇒ Tk is an NT-statistic
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 30 / 34
Outline
1 IntroductionMain ideasData-driven tests
2 General notionsSelection ruleFramework
3 NT-statisticsDefinitionsExamplesTheorems
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 31 / 34
Null distribution
Theorem{Tk} sequence of NT-statistics
S selection rule, with penalty of proper weight
large deviations of Tk are properly majorated
d(n) ≤ min{un ,mn} .
Then S = OP0(1) and TS = OP0(1).
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 32 / 34
Consistency condition
〈C〉 there exists integer K = K (P) ≥ 1 such that
EP l1(Y ) = 0, . . . ,EP lK−1(Y ) = 0, EP lK = CP 6= 0
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 33 / 34
Consistency theorem
Theorem{Tk} sequence of NT-statistics
S selection rule, with penalty of proper weight
the regularity assumptions are satisfied
d(n) = o(rn), d(n) ≤ min{un,mn}.
Then TS is consistent against any (fixed) alternative P satisfying (C).
M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 34 / 34