EMPIRICAL PROCESSES WITH APPLICATIONS TO STATISTICS€¦ · EMPIRICAL PROCESSES WITH APPLICATIONS...

EMPIRICAL PROCESSESWITH APPLICATIONSTO STATISTICS

GALEN R. SHORACK

JON A. WELLNER

University of Washington

JOHN WILEY & SONSNew York • Chichester • Brisbane • Toronto • Singapore

Contents

List of Tables xxxi

List of Special Symbols xxxiii

1. Introduction and Survey of Results 1

1. Definition of the Empirical Process and the InverseTransformation, 1

2. Survey of Results for ||Un||, 103. Results for the Random Functions Gn and L),, on [0,1], 134. Convergence of U,, in Other Metrics, 175. Survey of Other Results, 19

2. Foundations, Special Spaces and Special Processes 23

0. Introduction, 23

1. Random Elements, Processes, and Special Spaces, 24Projection mapping; Finite-dimensional subsets; Measurablefunction space; Random elements; Equivalent processes;Change of variable theorem; Borel and ball a-fields

2. Brownian Motions §, Brownian Bridge U, the UhlenbeckProcess, the Kiefer Process IK, the Brillinger Process, 29

Definitions; Relationships between the various processes;Boundary crossing probabilities for S and U; Reflectionprinciples; Integrals of normal processes

3. Weak Convergences, 43Definitions of weak convergence and weak compactness;weak convergence criteria on (D,3)); Weak convergencecriteria on more general spaces; The Skorokhod - Dudley -Wichura theorem; Weak convergence of functionals; Thekey equivalence; \\-/q\\ convergence; On verifying thehypotheses of Theorem 1; the fluctuation inequality; addi-tional weak convergence criteria on (D, 2); Conditions fora process to exist on (C, '#)

XV

XVI CONTENTS

4. Weak Convergence of the Partial-Sum Process Sn, 52Definition of Sn; Donsker's theorem that §„=£>§; TheSkorokhod construction form of Donsker's theorem;O'Reilly's theorem; Skorokhod's embedded partial sumprocess; Hungarian construction of the partial sum process;the partial sum process of the future observations

5. The Skorokhod Embedding of Partial Sums, 56The strong Markov property; Gambler's ruin problem;Choosing r so that S(r) = X; The embedding version of Sn;Strassen's theorem on rates of embedding; Extensions andlimitations; Breiman's embedding in which S is fixed

6. Wasserstein Distance, 62Definition of Wasserstein distance d2; Mallow's theorem;Minimizing the distance between rv's with given marginals;Variations

7. The Hungarian Construction of Partial Sums, 66The result; Limitations; Best possible rates; Other rates

8. Relative Compactness ~», 69Definition of'**; LILforiid N(0,1) rv's; LIL for Brownianmotion; Hartman-Wintner LIL and converse; MultivariateLIL in «** form; Tm approximation and Tm linearization;Criteria for establishing "*; <*•» mapping theorem

9. Relative Compactness of §(nl)/\fnbn, 79Definition of Strassen's limit class %; Properties of 3€;Strassen's theorem that S(nl)/Jnbn^*3€; Definition ofFinkelstein's limit class X; B(M, •)~+for the BrillingerprocessB

10. Weak Convergence of the Maximum of NormalizedBrownian Motion and Partial Sums, 82

Extreme value dfs; Darling and Erdos theorem with gen-eralizations

11. The LLN for iid rv's, 83Kolmogorov's SLLN; Feller's theorem; Theorems of Erdos,Hsu and Robbins, and Katz; Necessary and sufficient condi-tions for the WLLN

3. Convergence and Distributions of Empirical Processes 85

1. Uniform Processes and Their Special Construction, 85

Uniform empirical dfGn, empirical process Un, andquantileprocess Vn; Smoothed versions Gn, Un, and Vn; Identities;

CONTENTS

Weighted uniform empirical process Wn; Covariances, -»f.d.,and correlation pn = pn(c,\); Finite sampling process (orempirical rank process) Un, with identities; 3!2\ | [ -] | , (•),and BVI(0, 1); Applications: Kolmogorov-Smirnov,Cramer-von Mises, stochastic integrals, and simple linearrank statistics; The special construction of Vm Vn, Wn, Un

and Brownian bridges U, W; The special construction ofyohdWn=a\

lohdW; Glivenko-Cantelli theorem in the

uniform case; Generalizations to Dn(A); Uniform orderstatistics: the key relation, densities, moments

2. Definition of Some Basic Processes under GeneralAlternatives, 98

The empirical df ¥„, the average dfFn, and empirical process•/n(¥n~Fn); The quantile process; Reduction to [0,1] inthe case of continuous dfs; Xn, Yn, Zn, Un, and identities;Reduction to [0, 1] in the general case: associated array ofcontinuous rv's; Extended Glivenko-Cantelli theorem;Some change of variable results

3. Weak Convergence of the General Weighted EmpiricalProcess, 108

Definition and moments; The function vn; Weak convergence(=$) of Zn and its modulus of continuity; The special con-struction of Zn; Moment inequalities for Zn

4. The Special Construction for a Fixed Nearly Null Array, 119Notation for the reduced processes Xn and Zn; Nearly nullarrays; The special construction of Zn; Nearly null arrays;The special construction of Zn; The special construction for\\hdZn

5. The Sequential Uniform Empirical Process IKn, 131The definition of Kn and the Kieferprocess K; The Bickel-Wichura theorem that IKn=»IK

6. Martingales Associated with Un, Vn, Wn, Un, 132Martingales for Un, V,,, Wn, Rn divided by I-1; The Pyke-Shorack inequality, with analogs; Reverse martingales forIUn, Vn, Wn, Un divided by I; Submartingales for \\n(Gn-/)*V||; Reverse submartingales for | |(Gn-7)#i//| |; Sen'sinequality; Vanderzanden's martingales

7. A Simple Result on Convergence in || • /q\\ Metrics, 140

8. Limiting Distributions under the Null Hypothesis, 142Kolmogorov-Smirnov and Kuiper statistics; Renyi's statis-tics; Cramer-von Mises, Watson, Anderson-Darlingstatistics

xviii CONTENTS

4. Alternatives and Processes of Residuals 151

0. Introduction, 151

1. Contiguity, 152The key contiguity condition; Convergence of the centeringfunction; Convergence of the weighted empirical process En

on (-00,00) and the empirical rank process Un; Le Cam'srepresentation of the log likelihood ratio Ln under contiguity;Uniform integrability, ->Li and -*p of the rv's exp (LJ ; LeCam's third lemma; The Radon-Nikodym derivative ofU + A measure wrtli measure; miscellaneous contiguityresults

2. Limiting Distributions under Local Alternatives, 167Chibisov's theorem; An expansion of the asymptotic powerof the | | (G n - / ) + | | test

3. Asymptotic Optimality of Fn, 171Beran's theorem on the asymptotic optimality of Fn; State-ment of other optimality results

4. Limiting Distributions under Fixed Alternatives, 177Raghavachari' s theorem for supremum functional;Analogous result for integral functionals

5. Convergence of Empirical and Rank Processes under Con-tiguous Location, Scale, and Regression Alternatives, 181

Fisher information for location and scale; The contiguoussimple regression model, and its associated special construc-tion; The contiguous linear model with known scale; Thecontiguous scale model; The contiguous linear model withunknown scale; the main result

6. Empirical and Rank Processes of Residuals, 194The weighted empirical process of standardized residuals En;The empirical rank process of standardized residuals Rn;Convergence of En and Un; Classical and robust residuals,the Pierce and Kopecky idea; The estimated empirical processl)n; Testing the adequacy of a model

5. Integral Tests of Fit and Estimated Empirical Process 201


1. Motivation of Principal Component Decomposition, 203Statement of a problem; Principal component decompositionof a random vector; Principal component decomposition ofa process-heuristic treatment

CONTENTS XIX

2. Orthogonal Decomposition of Processes, 206Properties of kernels; Complete orthonormal basis for !£2;Mercer's theorem; Representations of covariance functionsvia Mercer's theorem; Orthogonal decomposition of X a laKac and Siegert; Distribution of J x 2 via decomposition

3. Principal Component Decomposition of Un, U and OtherRelated Processes, 213

Kac and Siegert decomposition of U; Durbin and Knottdecomposition ofl)n; DecompositionsofW\ and W2; Distri-bution of the components ofVn; Computing formula for W2

n;Testing natural Fourier parameters; Power of W2

n, A2,, and

other tests; Decomposition of\p\}for tp continuous on [0,1]

4. Principal Component Decomposition of the Anderson andDarling Statistic A2

n, 224Limiting distribution ofA2,; Anderson and Darling decompo-sition of Z and A; Computing formula for A1,

5. Tests of Fit with Parameters Estimated, 228Darling's theorem; An estimated empirical process L)„;Specialization to efficient estimates of location and scale

6. The Distribution of W2, W2n, A2, A2

n, and Other RelatedStatistics, 235

The Darling-Sukhatme theorem for K(s, t) = K(s, t) —£1" <Pi(s)<Pi(t); Tables of distributions; Normal, exponential,extreme value and censored exponential cases; Normalizedprincipal components of W2

n; A proof for 1 = 2 - (pep'

1. Confidence Bands, Acceptance Bands, and QQ, PP, and SPPlots, 247

8. More on Components, 250Asymptotic efficiency of tests based on components; Choosingthe best components; We come full circle

9. The Minimum Cramer-von Mises Estimate of Location, 254The Blackman estimator of location

6. Martingale Methods 258


1. The Basic Martingale Mn for Un, 264The cumulative hazard function A; Definition of the basicmartingale Mn; The key identity; The variance function V;Convergence of Mn to M = S( V) for a Brownian motion S;

XX CONTENTS

M = Z(F) for continuous Fand a particular Brownian motionZ; The predictable variation process {Mn); Discussion ofRebellodo's CLT; The exponential identity for vn(Fn - F);Extension to the weighted case of Wn

2. Processes of the Form ^Mn, <AUn(F), and iAWn(F), 273Convergence in \\-ili\\ metrics; F"1 and F+l are q-functions

3. Processes of the Form {!„ h dM,,, 276K B s J_co/idMn is a martingale; Evaluation of the predict-able variation of Kn; Existence of K = \_m h dM; Conver-gence of Kn to K in || • i/>|| metrics

4. Processes of the Form {!„ h dUn(F) and J l^ /i dWn(F), 282Reduction of \x_x h dWn(F) to the form l*_mh*dMn;Existence of J_M /) dU(F) and /_„,, /i dW(F); Convergencein || • i/»|| metrics; Some covariance relationships among theprocesses; Replacing h by hn

5. Processes of the Form [_x Mn dh, l_m Un(F) dh, and[_0OWn{F)dh, 289

Convergence of these processes in || • <//|| metrics

6. Reductions When F is Uniform, 291

7. Censored Data and the Product-Limit Estimator 293


The random censorship model; The product limit estimatorFn; The cumulative hazard function A and its estimator An;The processes Bn = Vn(An-A) and Xn = v/n(Fn-F); Thebasic martingale Mn

1. Convergence of the Basic Martingale Mn, 296The covariance function V; Convergence ofMn to M = S( V)

2. Identities Based on Integration by Parts, 300Representation of Bn and Xn/(1 - F) as integrals j 0 dMn;The exponential formula

3. Consistency of An and Fn, 304

4. Preliminary Weak Convergence => of Bn and Xm 306The Breslow-Crowley theorem

5. Martingale Representations, 310The predictable variation (Mn) of the basic martingale Mn;The predictable variation of Bn and Xn/(1 - F)

CONTENTS XXi

6. Inequalities, 316The Gill- Wellner inequality; Lenglart's inequality for locallysquare integrable martingales; Gill's inequality (product-limit version of Daniels and Chang inequalities)

7. Weak Convergence => of Bn and Xn in || / 4 HJ"-Metrics, 318Application of Rebolledo's CLT and Lenglart's inequality;Confidence bands

8. Extension to General Censoring Times, 325Convergence of Bn and Xn; The product-limit estimator isthe MLE

8. Poisson and Exponential Representations 334


1. The Poisson Process N, 334One-dimensional; Two-dimensional

2. Representations of Uniform Order Statistics, 335As partial sums of exponentials; As waiting times of aconditioned Poisson process; Normalized exponentialspacings; Lack of memory property

3. Representations of Uniform Quantile Processes, 337

4. Poisson Representations of U,,, 338Conditional, Chibisov, and Kac representations

5. Poisson Embeddings, 340The Poisson bridge; Representation of the sequential uniformempirical process

9. Some Exact Distributions 343


1. Evaluating the Probability that Gn Crosses a GeneralLine, 344

Dempster's formula; Daniels' theorem; Chang's theorem

2. The Exact Distribution of ||U*|| and the DKWInequality, 349

The Birnbaum and Tingey formula; Asymptotic expansions;Harter's approximation; Pyke's result for the smoothedempirical dfGn; The Dvoretzky-Kiefer- Wolfowitz (DKW)inequality

XXii CONTENTS

3. Recursions for P(g<Gn < h), 357Recursions of Noe, Bolshev, and Steck; Steck's formula;Ruben's recursive approach; The exact distribution of ||Un||using Ruben's table; Tables for | |U^/>/ / ( l - / ) | | and||U^/\/G,,(l —Gn) /ram Kotelnikova and Chmaladze

4. Some Combinatorial Lemmas, 376Andersens's lemma; Takdcs' lemma; Tusnddy's lemma;Another proof of Dempster's formula; Csdki and Tusnddy'slemma

5. The Number of Intersections of Gn with a General Line, 380The exact distribution of the number of intersections; Limit-ing distributions in various special cases

6. On the Location of the Maximum of U* and V*, 384The smoothed uniform empirical and quantile processes;Theorems of Birnbaum and Pyke, Gnedenko and Mihalevic,Kac, and Wellner

7. Dwass's Approach to Gn Based on Poisson Processes, 388Dwass's approach to Gn based on Poisson processes;Dwass's theorem with applications; Zeros and ladder pointsof Un; Crossings on a grid

8. Local Time of U,,, 398Definition of local time; The key representation; Some openquestions about the limit

9. The Two-Sample Problem, 401The Gnedenko and Korolyuk distribution; Application to thelimiting distribution of \\lit\\

10. Linear and Nearly Linear Bounds on the Empirical Distribution

Function G,, 404

0. Summary, 404

1. Almost Sure Behavior of £,,:(c with k Fixed, 407Kiefer's characterization ofP(€,,k s a,, i.o.); Robbins-Sieg-mund characterization of P(gn:k > a,, i.o.)

2. A Glivenko-Cantelli-type Theorem for | |(G f l-/)^| | , 410

Lai's SLLN for \$Gn-I)4>\\

3. Inequalities for the Distributions of ||G,,//|| and||//Gn | |L,, 412

Shorackand Wellner bound on P(\\I/Gn\\l:l^$; Wellner'sexponential bounds

CONTENTS XXIII

4. In-Probability Linear Bounds on Gn, 418

5. Characterization of Upper-Class Sequences for ||G,,//|| and||//Gn||J. :1, 420

Shorack and Wellner's characterization of upper classsequencesfor | |Gn / / | | and ||//Gn||fn:1; Chang's WLLNtyperesult for || • || ],~a-, Wellner's restriction to || • || \n with an -> 0;Mason's upper class sequences for ||n"(Gn —/)/[ / ( I - / ) ] ' " ! with 0 < i / < |

6. Almost Sure Nearly Linear Bounds on Gn and G^', 426Bounding Gn and 6^ ' between (1 ± e)tl±s; James' boundaryweight t/<= //(log2 (e

e/1) for ||i/'/Gn||jii.1; Nearly linearbounds with logarithmic terms

7. Bounds on Functions of Order Statistics, 428Mason's upper class result for max {ig(£n:f)/nan: 1 s f< k,,}with g \i and an /"; Strength of bundles of fibers; Determina-tion of the a.s. limsup of ||Gng||/an via Eg(g)

8. Almost Sure Behavior of Zn(a,,)/bn as a,,|0, 432Kiefer's theorem

9. Almost Sure Behavior of Normalized Quantiles as anl0, 435

11. Exponential Inequalities and || -/^||-Metric Convergence of U,,

and Vn 438


1. Universal Exponential Bounds for Binomial rv's, 439The exponential bounds for Binomial tail probabilities ofBennett, Bernstein, Hoeffding, and Wellner, Behavior of thefunctions $ and h; Constants /3* related to if and h;Extensions of the binomial exponential bounds to supremaof Un over neighborhoods of zero

2. Bounds on the Magnitude of ||U*/g||*, 445inequalities for F(| |U*/q| |Ss A); Corollary for q =-ft; Cor-responding inequalities for F( | |U/q| |aaA)

3. Exponential Bounds for Uniform Order Statistics, 453Exponential bounds for order statistic tail probabilities;Behavior of the functions i// and h; Bounds for absolutecentral moments of uniform order statistics; Extensions ofthe bounds to suprema of Vn over neighborhoods of 0

4. Bounds for the Magnitude of ||V*/q||*, 460Inequalities for P( || V*/ q \\ b

a a A); Corollary for q = -ft

CONTENTS

5. Weak Convergence of Un and Vn in ||-/<7|| Metrics, 461Chibisov's theorem; O'Reilly's theorem on convergence ofVn with respect to \\'/q\\; Other versions of the quantileprocess

6. Convergence of Un, Wn, Vn and Rn in Weighted 5£r

Metrics, 470

7. Moments of Functions of Order Statistics, 474Existence of moments of rv's; Existence of moments of orderstatistics; Anderson's moment expansions; Mason's boundson moments

8. Additional Binomial Results, 480Unimodality of the binomial distribution and relatedinequalities; Feller's inequalities; Large deviations, theBahadur and Rao theorem

9. Exponential Bounds for Poisson, Gamma, and BetaRV's, 484

Unimodality of the Poisson distribution; Large deviations;Exponential bounds for Poisson probabilities related to theBinomial bounds; Inequalities of Bohman and of Andersonand Samuels; Analogous results for the Gamma distribution

12. The Hungarian Constructions of Kn, Un and Vn 491

0. Introduction, 491Skorokhod constructions o/Un and Vn again; The sequentialuniform empirical process Kn

1. The Hungarian Construction of Kn, 493The Hungarian construction ofKn at rate (log n)2/\fn; Theother Hungarian construction of Un at rate (log n)/\fn

2. The Hungarian Renewal Construction of Vn, 496The Brillinger process; The Hungarian renewal constructionof Vn using partial sums of exponentials

3. A Refined Construction of Un and Vn, 499

4. Rate of Convergence of the Distribution of Functionals, 502Rate (log n)/\fn is possible for Lipschitz functionals withbounded density

13. Laws of the Iterated Logarithm Associated with U,, and V,, 504


1. A LIL for ||Un||, 504

CONTENTS XXV

Smirnov's LIL for ||U*||; Chung's characterization of upperclass sequences; Boundary crossing probabilities; A remarkon an Erdos theorem

2. A Maximal Inequality for ||U*/g||*, 510James' inequality for general q; Shorack's refinement forq = \Tt

3. Relative Compactness *•» of Un and Vn, 512Definition of ~»; Finkelstein's theorem that l)n/bn*»;Cassels' theorem; Reseated Kieferprocess K(n, • ) / v n b n ^ ;An alternative proof of Finkelstein's theorem based on theHungarian construction

4. Relative Compactness of Dn in || /q\\ -Metrics, 517James' theorem

5. The Other LIL for | | U j , 526Mogulskii' s theorem

6. Extension to General F, 530

14. Oscillations of the Empirical Process 531


1. The Oscillation Moduli (o, a>, and a> of U and S, 533The modulus of continuity co and Levy's theorem, Themodulus 6>; An exponential bound for &>(a); The Bickel andFreedman bound on Ew(a); The order ofa>(a); An exponen-tial bound for u>(a)

2. The Oscillation Moduli of UB, 542The modulus of continuity a>n; The Lipschitz \ modulus a>n;Stute's theorem on the order of ion(an) for "regular" an;Behavior of oin(an) on the boundary sequences; The sametheorems hold for con(an) and -fcTn(i)n(an); The Mason, Well-ner, Shorack exponential bound on con; The associatedmaximal inequality ofStute; The martingale y/ncon(a), n >1; A proof using the conditional Poisson representation

3. A Modulus of Continuity for the Kiefer Process Kn, 558The modulus of continuity con of Bn = K(n, • )/\fn; Behaviorof (on(an) on the upper boundary sequences; A maximalinequality; The Lipschitz ^-modulus u>n of Bn

4. The Modulus of Continuity Again, via the HungarianConstruction, 567

XXVi CONTENTS

A partial theorem for "regular" sequences an; A full theoremfor an on the upper boundary

5. Exponential Inequalities for Poisson Processes, 569Inequalities for centered Poisson processes; The modulus ofcontinuity of Ks, Ms, and Ns; Inequalities for the Poissonbridge; Another proof of Chibisov's theorem; Convergenceof rescaled Gn to a Poisson process

6. The Modulus of Continuity Again, via PoissonEmbedding, 578

The centered Poisson process Ks, the Poisson bridge Ms, andthe final Ns; Theorems of Section 2 reproved via Poissonembedding; Summary; A proof using Poisson embedding

7. The Modulus of Continuity of Vn, 581

15. The Uniform Empirical Difference Process Dn = Un + Vn 584


1. The Uniform Empirical Difference Process Dn, 584Definition of D,,; The key picture; Kiefer's theorems; Asimple proof establishing the correct order of magnitude; Theorder of O,,/q

2. The Integrated Empirical Difference Process, 594Vervaat's theorem; The parameters estimated version

16. The Normalized Uniform Empirical Process Zn and the NormalizedUniform Quantile Process 597


1. Weak Convergence of | | Z j , 597Definition of Zn and its natural limit Z; Relationshipsbetween Z, S, and the Uhlenbeck process X; Darling andErdos' limit theorem for ||S/vT||o; Jaeschke's analogouslimit theorem for ||Zn||; Eicker's theorem for the quantileversion; Representation of j'0Zn(t) dt as a sum of 0-meaniid rv's

2. The a.s. Rate of Divergence of ||Zt||i /2, 603Csdki's theorems; Shorack's theorem

3. Almost Sure Behavior of ||Z*||yn2 with an \ 0 , 609

Csorgo and Revesz theorem; Proof of Singh's theorem

4. The a.s. Divergence of the Normalized Quantile Process 615

CONTENTS xxvii

17. The Uniform Empirical Process Indexed by Intervals and

Functions 621


1. Bounds on the Magnitude of ||IUn/q||«((Iifc), 621

2. Weak Convergence of Un in | |7g| |« Metrics, 625

3. Indexing by Continuous Functions via Chaining, 630

18. The Standardized Quantile Process Qn 637

0. Introduction, 637Summary; Recollection of earlier results for Vn

1. Weak Convergence of the Standardized Quantile ProcessQn, 638

Convergence in distribution of sample quantiles; TheHdjek- Bickel theorem on weak convergence of Qn on[a, b]c (0, 1); Shorack's theorem on \\ • /q\\ convergence ofQn to V; The Csorgo and Resesz condition on f

2. Approximation of Q,, by Vn with Applications, 645Csorgo and Revesz determination of the rate at which \\Qn —Vn || goes to 0; Extension of Kiefer- Bahadur and Finkelsteintheorems to Qn; Miscellaneous applications; Parzen'sobservation on the Csorgo and Revesz condition on f;Mason's SLLN

3. Asymptotic Theory of the Q-Q Plot, 652Limiting distribution of Doksum'sprocess; Confidence bandsfor A = G - ' o F - 7

4. Weak Convergence => of the Product — Limit Quantile Pro-cess Vn> 657

19. L-Statistics 660


1. Statement of the Theorems, 660Basic idea of the proof; The assumptions; CLT, LIL, SLLN;Functional CLT and LIL for past and future; Simplifyingthe mean; Better rates of embedding for a specific scorefunction

2. Some Examples of L-statistics, 670Mean, median, and sign test; The likelihood ratio statistic;Pitman efficiency; Nonstandard examples

xxviii CONTENTS

3. Randomly Trimmed and Winsorized Means, 678

Ordinary trimmed and Winsorized means; The metricallysymmetrized and Winsorized mean; Other examples

4. Proofs, 688

20. Rank Statistics 695

0. Linear Rank Statistics, 695

1. The Basic Martingale Mm 695

Definition of Mn; Convergence of Mn

2. Processes of the Form ¥„ = Jo hn dUn in the Null Case, 699

Definition of' T „ ; Convergence of J„

3. Contiguous Alternatives, 704

Efficiency, asymptotic linearity, and rank estimators

4. The Chernoff and Savage Theorem, 715

5. Some Exercises for Order Statistics and Spacings, 717

21. Spacings 720


1. Definitions and Distributions of Uniform Spacings, 720

2. Limiting Distributions of Ordered Uniform Spacings, 725

3. Renewal Spacings Processes, 727

Normalized, ordered, and weighted renewal spacings proces-ses; Convergence to limiting processes

4. Uniform Spacings Processes, 731

Normalized, ordered, and weighted uniform spacings proces-ses; Convergence in || • / qf |[ metrics

5. Testing Uniformity with Functions of Spacings, 733

Testing an iid sample; Testing a renewal process for exponen-tiality; Testing for exponentiality

6. Iterated Logarithms for Spacings, 741

22. Symmetry 743

1. The Empirical Symmetry Process S n and the Empirical RankSymmetry Process Rn, 743

Definitions of the absolute empirical process, the empiricalsymmetry process §„, and the empirical rank symmetry pro-

CONTENTS XXiX

cess Un; Identities relating these processes to lin; Conver-gence theorems

2. Testing Goodness of Fit for a Symmetric DF, 746The symmetric estimator F* of Fsatisfies \/n||(F* - F ) # | | =||IUn'||/2; Supremum, integral, and components tests

3. The Processes under Contiguity, 751

4. Signed Rank Statistics under Symmetry, 753Asymptotic normality; Representation of the limiting rv;Asymptotic normality under contiguous alternatives

5. Estimating an Unknown Point of Symmetry, 757Estimators based on signed rank statistics; Estimators basedon variants of the Cramer-von Mises statistic

6. Estimating the DF of a Symmetric Distribution withUnknown Point of Symmetry, 759

Schuster's results; Boos' statistic

23. Further Applications 763

1. Bootstrapping the Empirical Process, 763

2. Smooth Estimates of F, 764

3. The Shorth, 767

4. Convergence of [/-Statistic Empirical Processes, 771

5. Reliability and Econometric Functions, 775

24. Large Deviations 781


1. Bahadur Efficiency, 781Exact slope; Bahadur's theorem

2. Large Deviations for Supremum Tests of Fit, 783Large deviations of binomial rv's; The key function </»2(<) =—log (f(l —0); -A version of Abrahamson's theorem

3. The Kullback-Leibler Information Number, 789Elementary properties

4. The Sanov Problem, 792Sanov's conclusion; Hoadley's theorem; The Groeneboom,Oosteroff, and Ruymgaart extension; Reformulation of Sec-tion 2 in the spirit of Sanov's conclusion

XXX CONTENTS

25. Independent but not Identically Distributed Random Variables 796


1. Extensions of the DKW Inequality, 796Bretagnolle's inequality and exponential bound

2. The Generalized Binomial Distribution, 804Hoeffding's inequalities for the probability distribution of asum of independent Bernoulli rv's; Feller's varianceinequality

3. Bounds on Fn, 807Van Zuijlen's inequalities for P(||Fn/F7|| s A) and

:

4. Convergence of Xn, Yn, and Zn with respect to ||/<j||Metrics, 809

=> ofZn in \\-1q\\-metrics; => of Xn and Yn in \\-/q\\;Comparison inequalities; Another natural reduction of theempirical process; An inequality of Marcus and Zinn; TheMarcus-Zinn exponential bound for the weighted empiricalprocess Zn

5. More on L-statistics, 821The CLT for L-statistics of independent but not identicallydistributed rv's; Stigler's variance comparisons

26. Empirical Measures and Processes for General Spaces


1. Glivenko-Cantelli Theorems via the Vapnik-ChervonenkisIdea, 827

Vapnik-Chervonenkis classes of sets; The Vapnik-Cher-vonenkis exponential bound; Bounds on the growth functionm<e(r); Examples of VC classes of sets; Equivalence of -*a s

and -»pforDn(<$)

2. Glivenko-Cantelli Theorems via Metric Entropy, 835Entropy conditions; Pollard's entropy bound for functions;The Blum-DeHardt Glivenko-Cantelli theorem; The Pol-lard-Dudley Glivenko-Cantelli theorem

3. Weak and Strong Approximations to the Empirical ProcessZn, £37

The Gaussian limit process; Functional Donsker and strong-invariance classes of functions; A general theorem of Dudleyand Philipp; Dudley's CLT; Pollard's CLT

CONTENTS XXXi

A. Appendix A: Inequalities and Miscellaneous 842


1. Simple Moment Inequalities, 842Basic, Markov, Chebyshev, Jensen, Liapunov, Cr, Cauchy-Schwarz, Minkowski, estimating E\X\

2. Maximal Inequalities for Sums and a MinimalInequality, 843

Kolmogorov, Monotone, Hdjek-Renyi, Levy, Skorokhod,Menchoff; Maximal inequality for the Poisson process; Weaksymmetrization, Levy; Mogulskii's minimal inequality; Thecontinuous monotone inequality of Gill- Wellner

3. Berry-Esseen Inequalities, 848Berry-Esseen, with generalizations; Cramer's expansion;Esseen's lemma; Stein's CLT, a special case

4. Exponential Inequalities and Large Deviations, 850Mill's ratio for normal rv's; Variations on P(S,,/sn > A) =exp (—A2/2); Bennett, Hoeffding, and Bernstein inequalities;Kolmogorov's exponential bounds; Large deviation theoremsof Chernoff and others; The Poisson example; Properties ofmoment-generating functions

5. Moments of Sums, 857von Bahr inequality; rth mean convergence equivalence;Burkholder's inequality; Marcinkiewicz and Zygmundequivalences, with variations; Hornich's inequality

6. Borel-Cantelli Lemmas, 859Borel-Cantelli, Renyi, and other variations

7. Miscellaneous Inequalities, 860Events lemma, Bonferoni inequality; Anderson's inequality

8. Miscellaneous Probabilistic Results, 862Moment convergence; -*p is equivalent to -»o-s. on subsequen-ces; Cramer- Wold device; Formulas for means and covari-ances; Tail behavior of F when moments exist; Vitali'stheorem; Scheffe's theorem with applications

9. Miscellaneous Deterministic Results, 864Stirling's formula; Euler's constant; Some implications ofconvergence of series and integrals; A discussion of the sub-sequence n, = (exp (aj/logj)) used in upper class proofs;Integration by parts

XXXil CONTENTS

10. Martingale Inequalities, 869Functions of martingales; Doob's inequality with variations;The Birnbaum-Marshall inequalities; Submartingale conver-gence theorem

11. Inequalities for Reversed Martingales, 874Inequalities; Reverse submartingale convergence theorem

12. Inequalities in Higher Dimensions, 876Wichura's inequality; The Shorack and Smythe inequality

13. Finite-Sampling Inequalities, 878Hoeffding's bound

14. Inequalities for Processes, 878

B. Appendix B: Martingales and Counting Processes 884

1. Basic Terminology and Definitions, 884

2. Counting Processes and Martingales, 886Examples; Compensators; Doob-Meyer decomposition fora counting process N = M + A; The predictable variationprocess (M> = | ( 0 . ] ( l -A / \ ) dA; Formulas for the com-pensator A; Continuity of A and quasi-left-continuity of N

3. Stochastic Integrals for Counting Processes, 890Martingale transform theorems; The predictable variationprocess of a martingale transform; Martingale representationtheorem

4. Martingale Inequalities, 892Lenglart's inequality; Burkholder-Davis-Gundy inequality

5. Rebolledo's Martingale Central Limit Theorem, 894The ARJ conditions; Relationships among the ARJ condi-tions; A central limit theorem for local martingales; A centrallimit theorem for locally square integrable martingales

6. A Change of Variable Formula and Exponential Semimar-tingales, 896

The Ito, Doleans-Dade, Meyer formula; The exponential ofa semimartingale; Examples; A useful exponential supermar-tingale

References 901

Author Index 923

Subject Index 927

EMPIRICAL PROCESSES WITH APPLICATIONS TO STATISTICS€¦ · EMPIRICAL PROCESSES WITH APPLICATIONS...

Documents

Transcript of EMPIRICAL PROCESSES WITH APPLICATIONS TO STATISTICS€¦ · EMPIRICAL PROCESSES WITH APPLICATIONS...