Concentration Inequalities for Random Matrices · Concentration Inequalities for Random Matrices M....

Concentration Inequalities for RandomMatrices

M. Ledoux

Institut de Mathematiques de Toulouse, France

exponential tail inequalities

classical theme in probability and statistics

quantify the asymptotic statements

central limit theorems

large deviation principles

classical exponential inequalities

sum of independent random variables

Sn =1√n

(X1 + · · ·+ Xn)

0 ≤ Xi ≤ 1 independent

P(Sn ≥ E(Sn) + t

)≤ e−t

2/2, t ≥ 0

Hoeffding’s inequality

same as for Xi standard Gaussian

central limit theorem

Sn =1√n

(X1 + · · ·+ Xn)

P(Sn ≥ E(Sn) + t

)≤ e−t

2/2, t ≥ 0

Sn =1√n

(X1 + · · ·+ Xn)

P(Sn ≥ E(Sn) + t

)≤ e−t

2/2, t ≥ 0

measure concentration ideas

asymptotic geometric analysis

V. Milman (1970)

Sn =1√n

(X1 + · · ·+ Xn)

F (X ) = F (X1, . . . ,Xn), F : Rn → R Lipschitz

Gaussian sample

independent random variables

V. Milman (1970)

Sn =1√n

(X1 + · · ·+ Xn)

Gaussian sample

V. Milman (1970)

Sn =1√n

(X1 + · · ·+ Xn)

Gaussian sample

V. Milman (1970)

Sn =1√n

(X1 + · · ·+ Xn)

Gaussian sample

V. Milman (1970)

Sn =1√n

(X1 + · · ·+ Xn)

Gaussian sample

concentration inequalities

Sn =1√n

(X1 + · · ·+ Xn)

F (X ) = F (X1, . . . ,Xn), F : Rn → R 1-Lipschitz

X1, . . . ,Xn independenty standard Gaussian

P(F (X ) ≥ E

(F (X )

)+ t)≤ e−t

2/2, t ≥ 0

0 ≤ Xi ≤ 1 independent, F 1-Lipschitz and convex

P(F (X ) ≥ E

(F (X )

)+ t)≤ 2 e−t

2/4, t ≥ 0

M. Talagrand (1995)

Sn =1√n

(X1 + · · ·+ Xn)

P(F (X ) ≥ E

(F (X )

)+ t)≤ e−t

2/2, t ≥ 0

P(F (X ) ≥ E

(F (X )

)+ t)≤ 2 e−t

2/4, t ≥ 0

M. Talagrand (1995)

Sn =1√n

(X1 + · · ·+ Xn)

P(F (X ) ≥ E

(F (X )

)+ t)≤ e−t

2/2, t ≥ 0

0 ≤ Xi ≤ 1 independent, F 1-Lipschitz

and convex

P(F (X ) ≥ E

(F (X )

)+ t)≤ 2 e−t

2/4, t ≥ 0

M. Talagrand (1995)

Sn =1√n

(X1 + · · ·+ Xn)

P(F (X ) ≥ E

(F (X )

)+ t)≤ e−t

2/2, t ≥ 0

P(F (X ) ≥ E

(F (X )

)+ t)≤ 2 e−t

2/4, t ≥ 0

M. Talagrand (1995)

Sn =1√n

(X1 + · · ·+ Xn)

P(F (X ) ≥ E

(F (X )

)+ t)≤ e−t

2/2, t ≥ 0

P(F (X ) ≥ E

(F (X )

)+ t)≤ 2 e−t

2/4, t ≥ 0

M. Talagrand (1995)

Sn =1√n

(X1 + · · ·+ Xn)

P(F (X ) ≥ E

(F (X )

)+ t)≤ e−t

2/2, t ≥ 0

P(F (X ) ≥ E

(F (X )

)+ t)≤ 2 e−t

2/4, t ≥ 0

M. Talagrand (1995)

empirical processes

X1, . . . ,Xn independent with values in (S ,S)

F collection of functions f : S → [0, 1]

Z = supf ∈F

n∑i=1

f (Xi )

Z Lipschitz and convex

concentration inequalities on

P(∣∣Z − E(Z )

∣∣ ≥ t), t ≥ 0

empirical processes

Z = supf ∈F

n∑i=1

f (Xi )

P(∣∣Z − E(Z )

∣∣ ≥ t), t ≥ 0

empirical processes

Z = supf ∈F

n∑i=1

f (Xi )

P(∣∣Z − E(Z )

∣∣ ≥ t), t ≥ 0

empirical processes

Z = supf ∈F

n∑i=1

f (Xi )

P(∣∣Z − E(Z )

∣∣ ≥ t), t ≥ 0

Z = supf ∈F

n∑i=1

f (Xi )

|f | ≤ 1, E(f (Xi )) = 0, f ∈ F

P(|Z −M| ≥ t

)≤ C exp

(− t

σ2 + M

)), t ≥ 0

C > 0 numerical constant, M mean or median of Z

σ2 = supf ∈F∑n

i=1 E(f 2(Xi ))

M. Talagrand (1996)

P. Massart (2000)

S. Boucheron, G. Lugosi, P. Massart (2005)

P.-M. Samson (2000) (dependence)

Z = supf ∈F

n∑i=1

f (Xi )

|f | ≤ 1, E(f (Xi )) = 0, f ∈ F

P(|Z −M| ≥ t

)≤ C exp

(− t

σ2 + M

)), t ≥ 0

σ2 = supf ∈F∑n

i=1 E(f 2(Xi ))

M. Talagrand (1996)

P. Massart (2000)

Z = supf ∈F

n∑i=1

f (Xi )

|f | ≤ 1, E(f (Xi )) = 0, f ∈ F

P(|Z −M| ≥ t

)≤ C exp

(− t

σ2 + M

)), t ≥ 0

σ2 = supf ∈F∑n

i=1 E(f 2(Xi ))

M. Talagrand (1996)

P. Massart (2000)

Z = supf ∈F

n∑i=1

f (Xi )

|f | ≤ 1, E(f (Xi )) = 0, f ∈ F

P(|Z −M| ≥ t

)≤ C exp

(− t

σ2 + M

)), t ≥ 0

σ2 = supf ∈F∑n

i=1 E(f 2(Xi ))

M. Talagrand (1996)

P. Massart (2000)

Z = supf ∈F

n∑i=1

f (Xi )

|f | ≤ 1, E(f (Xi )) = 0, f ∈ F

P(|Z −M| ≥ t

)≤ C exp

(− t

σ2 + M

)), t ≥ 0

σ2 = supf ∈F∑n

i=1 E(f 2(Xi ))

M. Talagrand (1996)

P. Massart (2000)

numerous applications

• geometric functional analysis

• discrete and combinatorial probability

• empirical processes

• statistical mechanics

• random matrix theory

recent studies of

random matrix and random growth models

new asymptotics

common, non-central, rate (mean)1/3

universal limiting Tracy-Widom distribution

random matrices, longest increasing subsequence,

random growth models, last passage percolation...

recent studies of

new asymptotics

recent studies of

new asymptotics

recent studies of

new asymptotics

recent studies of

new asymptotics

sample covariance matrices

multivariate statistical inference

principal component analysis

population (Y1, . . . ,YN)

Yj vectors (column) in RM (characters)

Y = (Y1, . . . ,YN) M × N matrix

sample covariance matrix Y Y t (M ×M)

(independent) Gaussian Yj : Wishart matrix models

Y = (Y1, . . . ,YN) M × N matrix

is Y Y t a good approximation of the

population covariance matrix

E(Y Y t) ?

M finite

NY Y t → E(Y Y t) N →∞

M infinite ?

M = M(N) → ∞ N →∞

N∼ ρ ∈ (0,∞) N →∞

E(Y Y t) ?

M finite

M infinite ?

M = M(N) → ∞ N →∞

N∼ ρ ∈ (0,∞) N →∞

E(Y Y t) ?

M finite

M infinite ?

M = M(N) → ∞ N →∞

N∼ ρ ∈ (0,∞) N →∞

E(Y Y t) ?

M finite

M infinite ?

M = M(N) → ∞ N →∞

N∼ ρ ∈ (0,∞) N →∞

E(Y Y t) ?

M finite

M infinite ?

M = M(N) → ∞ N →∞

N∼ ρ ∈ (0,∞) N →∞

Y = (Y1, . . . ,YN) M × N matrix

Y = (Yij)1≤i≤M,1≤j≤N

Yij independent identically distributed

(real or complex)

E(Yij) = 0, E(Y 2ij ) = 1

Wishart model : Yj standard Gaussian in RM

numerous extensions

Y = (Y1, . . . ,YN) M × N matrix

(real or complex)

E(Yij) = 0, E(Y 2ij ) = 1

numerous extensions

Y = (Y1, . . . ,YN) M × N matrix

(real or complex)

E(Yij) = 0, E(Y 2ij ) = 1

numerous extensions

Y = (Y1, . . . ,YN) M × N matrix

(real or complex)

E(Yij) = 0, E(Y 2ij ) = 1

numerous extensions

Y = (Y1, . . . ,YN) M × N matrix

Y = (Yij)1≤i≤M,1≤j≤N iid E(Yij) = 0, E(Y 2ij ) = 1

center of interest : eigenvalues 0 ≤ λN1 ≤ · · · ≤ λNM

of Y Y t (M ×M non-negative symmetric matrix)√λNk singular values of Y

λNk =λNkN

eigenvalues of1

NY Y t

spectral measure1

M∑k=1

δλNk

asymptotics M = M(N) ∼ ρN N →∞

Y = (Y1, . . . ,YN) M × N matrix

of Y Y t (M ×M non-negative symmetric matrix)

√λNk singular values of Y

λNk =λNkN

eigenvalues of1

NY Y t

spectral measure1

M∑k=1

δλNk

Y = (Y1, . . . ,YN) M × N matrix

λNk =λNkN

eigenvalues of1

NY Y t

spectral measure1

M∑k=1

δλNk

Y = (Y1, . . . ,YN) M × N matrix

λNk =λNkN

eigenvalues of1

NY Y t

spectral measure1

M∑k=1

δλNk

Y = (Y1, . . . ,YN) M × N matrix

λNk =λNkN

eigenvalues of1

NY Y t

spectral measure1

M∑k=1

δλNk

Y = (Y1, . . . ,YN) M × N matrix

λNk =λNkN

eigenvalues of1

NY Y t

spectral measure1

M∑k=1

δλNk

Marchenko-Pastur theorem (1967)

asymptotic behavior of the spectral measure (λN

k = λNk /N)

M∑k=1

δλNk→ ν Marchenko-Pastur distribution

dν(x) =(

1− 1

)+δ0 +

ρ 2πx

√(b − x)(x − a) 1[a,b]dx

a = a(ρ) =(1−√ρ

)2b = b(ρ) =

(1 +√ρ)2

k = λNk /N)

M∑k=1

dν(x) =(

1− 1

)+δ0 +

ρ 2πx

√(b − x)(x − a) 1[a,b]dx

a = a(ρ) =(1−√ρ

)2b = b(ρ) =

(1 +√ρ)2

k = λNk /N)

M∑k=1

dν(x) =(

1− 1

)+δ0 +

ρ 2πx

√(b − x)(x − a) 1[a,b]dx

a = a(ρ) =(1−√ρ

)2b = b(ρ) =

(1 +√ρ)2

k = λNk /N)

M∑k=1

dν(x) =(

1− 1

)+δ0 +

ρ 2πx

√(b − x)(x − a) 1[a,b]dx

a = a(ρ) =(1−√ρ

)2b = b(ρ) =

(1 +√ρ)2

k = λNk /N)

M∑k=1

dν(x) =(

1− 1

)+δ0 +

ρ 2πx

√(b − x)(x − a) 1[a,b]dx

a = a(ρ) =(1−√ρ

)2b = b(ρ) =

(1 +√ρ)2

Marchenko-Pastur theorem

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

global regime

large deviation asymptotics of the spectral measure

fluctuations of the spectral measure

M∑k=1

[f(λNk)−∫R f dν

]→ G Gaussian variable

f : R→ R smooth

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

global regime

M∑k=1

f : R→ R smooth

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

global regime

M∑k=1

f : R→ R smooth

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

global regime

M∑k=1

f : R→ R smooth

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

local regime

behavior of the individual eigenvalues

spacings (bulk behavior)

extremal eigenvalues (edge behavior)

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

local regime

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

local regime

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

local regime

extremal eigenvalues

largest eigenvalue λNM = max1≤k≤M λNk

λNM =λNMN→ b(ρ) =

(1 +√ρ)2

M ∼ ρN

λNM =λNMN

→ b(ρ) =(1 +√ρ)2

M ∼ ρN

(1 +√ρ)2

M ∼ ρN

k = λNk /N)

M∑k=1

dν(x) =(

1− 1

)+δ0 +

ρ 2πx

√(b − x)(x − a) 1[a,b]dx

a = a(ρ) =(1−√ρ

)2b = b(ρ) =

(1 +√ρ)2

M ∼ ρN

fluctuations around b(ρ)

complex or real Gaussian (Wishart matrices)

FTW C. Tracy, H. Widom (1994) distribution

K. Johansson (2000), I. Johnstone (2001)

(1 +√ρ)2

M ∼ ρN

(1 +√ρ)2

M ∼ ρN

(1 +√ρ)2

M ∼ ρN

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

(1 +√ρ)2

M ∼ ρN

M2/3N−1[λNM − b(ρ)N

]→ C (ρ)FTW

(1 +√ρ)2

M ∼ ρN

]→ C (ρ)FTW

(1 +√ρ)2

M ∼ ρN

]→ C (ρ)FTW

(complex) FTW(s) = exp

(−∫ ∞s

(x − s)u(x)2dx

), s ∈ R

u′′ = 2u3 + xu Painleve II equation

density

(complex) FTW(s) = exp

(−∫ ∞s

(x − s)u(x)2dx

), s ∈ R

u′′ = 2u3 + xu Painleve II equation

density

mean ' −1.77

FTW(s) ∼ e−s3/12 as s → −∞

1− FTW(s) ∼ e−4s3/2/3 as s → +∞

density (similar for real case)

(1 +√ρ)2

M ∼ ρN

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

Gaussian (Wishart matrices)

completely solvable models

determinantal structure

orthogonal polynomial analysis

asymptotics of Laguerre orthogonal polynomials

C. Tracy, H. Widom (1994)

extension to non-Gaussian matrices

A. Soshnikov (2001-02)

moment method E(Tr((YY t)p

))L. Erdos, H.-T. Yau (2009-12) (and collaborators)

local Marchenko-Pastur law

T. Tao, V. Vu (2010-11)

Lindeberg comparison method

symmetric matrices

L. Erdos, H.-T. Yau (2009-12) (and collaborators)

T. Tao, V. Vu (2010-11)

symmetric matrices

T. Tao, V. Vu (2010-11)

symmetric matrices

T. Tao, V. Vu (2010-11)

symmetric matrices

(brief) survey of recent approaches to

non-asymptotic exponential inequalities

quantify the limit theorems

spectral measure

catch the new rate (mean)1/3

from the Gaussian case to non-Gaussian models

spectral measure

two main questions and objectives

tail inequalities for the spectral measure

P( M∑

f (λNk ) ≥ t

P( M∑

f (λNk ) ≥ t

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

global regime

M∑k=1

f : R→ R smooth

P( M∑

f (λNk ) ≥ t

tail inequalities for the extremal eigenvalues

P(λNM ≥ b(ρ) + ε

P( M∑

f (λNk ) ≥ t

(1 +√ρ)2

M ∼ ρN

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

P( M∑

f (λNk ) ≥ t

Wishart matrices

more general covariance matrices

P( M∑

f (λNk ) ≥ t

Wishart matrices

measure concentration tool

F = F (Y Y t) = F (Yij)

satisfactory for the global regime

less satisfactory for the local regime

specific functionals

eigenvalue counting function

extreme eigenvalues

F = F (Y Y t) = F (Yij)

extreme eigenvalues

F = F (Y Y t) = F (Yij)

extreme eigenvalues

F = F (Y Y t) = F (Yij)

extreme eigenvalues

F = F (Y Y t) = F (Yij)

extreme eigenvalues

P( M∑

f (λNk ) ≥ t

Wishart matrices

A. Guionnet, O. Zeitouni (2000)

f : R→ R smooth (Lipschitz)

X = (Xij)1≤i ,j≤M M ×M symmetric matrix

eigenvalues λ1 ≤ · · · ≤ λM

F : X → Tr f (X ) =M∑k=1

f (λk) Lipschitz

with respect to the Euclidean structure on M ×M matrices

convex if f is convex

F : X → Tr f (X ) =M∑k=1

f (λk) Lipschitz

F : X → Tr f (X ) =M∑k=1

f (λk) Lipschitz

F : X → Tr f (X ) =M∑k=1

f (λk) Lipschitz

F : X → Tr f (X ) =M∑k=1

f (λk) Lipschitz

Sn = 1√n

(X1 + · · ·+ Xn)

P(F (X ) ≥ E

(F (X )

)+ t)≤ e−t

2/2, t ≥ 0

P(F (X ) ≥ E

(F (X )

)+ t)≤ 2 e−t

2/4, t ≥ 0

M. Talagrand (1995)

Gaussian entries Yij

f : R→ R such that f (x2) 1-Lipschitz

P( M∑

[f (λNk )− E

(f (λNk )

)]≥ t

)≤ C (ρ) e−t

2/C(ρ), t ≥ 0

compactly supported entries Yij

f : R→ R such that f (x2) 1-Lipschitz and convex

P( M∑

[f (λNk )− E

(f (λNk )

)]≥ t

)≤ C (ρ) e−t

2/C(ρ), t ≥ 0

P( M∑

[f (λNk )− E

(f (λNk )

)]≥ t

)≤ C (ρ) e−t

2/C(ρ), t ≥ 0

P( M∑

[f (λNk )− E

(f (λNk )

)]≥ t

)≤ C (ρ) e−t

2/C(ρ), t ≥ 0

M∑k=1

δλNk→ ν on

(a(ρ), b(ρ)

)M ∼ ρN

global regime

M∑k=1

f : R→ R smooth

non-Lipschitz functions f

typically f = 1I , I ⊂ R interval

M∑k=1

f(λNk)

= #{λNk ∈ I

}= NI counting function

Wishart matrices (determinantal structure)

I interval in (a, b)

1√logM

[NI − E(NI )

P(NI − E(NI ) ≥ t

)≤ C e−ct log(1+t/ logM), t ≥ 0

Var(NI

)= O(logM)

M∑k=1

f(λNk)

= #{λNk ∈ I

1√logM

[NI − E(NI )

Var(NI

)= O(logM)

M∑k=1

f(λNk)

= #{λNk ∈ I

1√logM

[NI − E(NI )

Var(NI

)= O(logM)

M∑k=1

f(λNk)

= #{λNk ∈ I

1√logM

[NI − E(NI )

Var(NI

)= O(logM)

M∑k=1

f(λNk)

= #{λNk ∈ I

1√logM

[NI − E(NI )

Var(NI

)= O(logM)

M∑k=1

f(λNk)

= #{λNk ∈ I

1√logM

[NI − E(NI )

Var(NI

)= O(logM)

non-Gaussian covariance matrices

comparison with Wishart model

partial results

localization results L. Erdos, H.-T. Yau (2009-12)

Lindeberg comparison method T. Tao, V. Vu (2010-11)

Var(NI

)= O(logM)

S. Dallaporta, V. Vu (2011)

)≤ C e−ct

δ, t ≥ C logM, 0 < δ ≤ 1

T. Tao, V. Vu (2012)

partial results

Var(NI

)= O(logM)

)≤ C e−ct

δ, t ≥ C logM, 0 < δ ≤ 1

T. Tao, V. Vu (2012)

partial results

Var(NI

)= O(logM)

)≤ C e−ct

δ, t ≥ C logM, 0 < δ ≤ 1

T. Tao, V. Vu (2012)

partial results

Var(NI

)= O(logM)

)≤ C e−ct

δ, t ≥ C logM, 0 < δ ≤ 1

T. Tao, V. Vu (2012)

M∑k=1

f(λNk)

= #{λNk ∈ I

1√logM

[NI − E(NI )

Var(NI

)= O(logM)

P( M∑

f (λNk ) ≥ t

Wishart matrices

P( M∑

f (λNk ) ≥ t

Wishart matrices

fluctuations of the largest eigenvalue

M2/3[λNM − b(ρ)

]→ C (ρ)FTW M ∼ ρN

(1 +√ρ)2

M ∼ ρN

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

M2/3[λNM − b(ρ)

finite M inequalities

at the (mean)1/3 rate

reflecting the tails of FTW

bounds on Var( λNM)

M2/3[λNM − b(ρ)

(Gaussian) Wishart matrix Y Y t

λNM = max1≤k≤M

λNk = sup|v |=1

|Y v |2

sNM =√λNM Lipschitz of the Gaussian entries Yij

Gaussian concentration

P(sNM ≥ E

+ t)≤ e−M t2/C , t ≥ 0

E(sNM) ∼√

correct large deviation bounds (t ≥ 1)

λNM = max1≤k≤M

λNk = sup|v |=1

|Y v |2

P(sNM ≥ E

+ t)≤ e−M t2/C , t ≥ 0

E(sNM) ∼√

λNM = max1≤k≤M

λNk = sup|v |=1

|Y v |2

P(sNM ≥ E

+ t)≤ e−M t2/C , t ≥ 0

E(sNM) ∼√

λNM = max1≤k≤M

λNk = sup|v |=1

|Y v |2

P(sNM ≥ E

+ t)≤ e−M t2/C , t ≥ 0

E(sNM) ∼√

λNM = max1≤k≤M

λNk = sup|v |=1

|Y v |2

P(sNM ≥ E

+ t)≤ e−M t2/C , t ≥ 0

E(sNM) ∼√b(ρ)

λNM = max1≤k≤M

λNk = sup|v |=1

|Y v |2

P(sNM ≥ E

+ t)≤ e−M t2/C , t ≥ 0

E(sNM) ∼√b(ρ)

λNM = max1≤k≤M

λNk = sup|v |=1

|Y v |2

P(sNM ≥ E

+ t)≤ e−M t2/C , t ≥ 0

E(sNM) ∼√b(ρ)

does not fit the small deviation regime t = s M−2/3

extreme eigenvalues

alternate tools

Riemann-Hilbert analysis (Wishart matrices)

tri-diagonal representations (Wishart and β-ensembles)

moment methods (Wishart and non-Gaussian matrices)

extreme eigenvalues

alternate tools

extreme eigenvalues

alternate tools

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

P(λNM ≤ b(ρ) + s M−2/3

)→ FTW(C s)

bounds for Wishart matrices

tri-diagonal representation

B. Rider, M. L. (2010)

)≤ C e−Mε3/2/C , 0 < ε ≤ 1

P(λNM ≤ b(ρ)− ε

)≤ C e−Mε3/C , 0 < ε ≤ b(ρ)

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

P(λNM ≤ b(ρ) + s M−2/3

)→ FTW(C s)

B. Rider, M. L. (2010)

)≤ C e−Mε3/2/C , 0 < ε ≤ 1

)≤ C e−Mε3/C , 0 < ε ≤ b(ρ)

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

P(λNM ≤ b(ρ) + s M−2/3

)→ FTW(C s)

B. Rider, M. L. (2010)

)≤ C e−Mε3/2/C , 0 < ε ≤ 1

)≤ C e−Mε3/C , 0 < ε ≤ b(ρ)

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

P(λNM ≤ b(ρ) + s M−2/3

)→ FTW(C s)

B. Rider, M. L. (2010)

)≤ C e−Mε3/2/C , 0 < ε ≤ 1

)≤ C e−Mε3/C , 0 < ε ≤ b(ρ)

P(λNM ≤ b(ρ) + s M−2/3

)→ FTW(C s)

)≤ C e−Mε3/2/C , 0 < ε ≤ 1

)≤ C e−Mε3/C , 0 < ε ≤ b(ρ)

fit the Tracy-Widom asymptotics (ε = s M−2/3)

1− FTW(s) ∼ e−s3/2/C (s → +∞)

FTW(s) ∼ e−s3/C (s → −∞)

Var( λNM) = O( 1

P(λNM ≤ b(ρ) + s M−2/3

)→ FTW(C s)

)≤ C e−Mε3/2/C , 0 < ε ≤ 1

)≤ C e−Mε3/C , 0 < ε ≤ b(ρ)

1− FTW(s) ∼ e−s3/2/C (s → +∞)

FTW(s) ∼ e−s3/C (s → −∞)

Var( λNM) = O( 1

P(λNM ≤ b(ρ) + s M−2/3

)→ FTW(C s)

)≤ C e−Mε3/2/C , 0 < ε ≤ 1

)≤ C e−Mε3/C , 0 < ε ≤ b(ρ)

1− FTW(s) ∼ e−s3/2/C (s → +∞)

FTW(s) ∼ e−s3/C (s → −∞)

Var( λNM) = O( 1

P(λNM ≤ b(ρ) + s M−2/3

)→ FTW(C s)

)≤ C e−Mε3/2/C , 0 < ε ≤ 1

)≤ C e−Mε3/C , 0 < ε ≤ b(ρ)

1− FTW(s) ∼ e−s3/2/C (s → +∞)

FTW(s) ∼ e−s3/C (s → −∞)

Var( λNM) = O( 1

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

b(ρ) =(1 +√ρ)2

λNM = λNM/N, M = M(N) ∼ ρN

(√MN)1/3

(√M +

√N)4/3

(λNM − (

√M +

√N)2

)→ FTW

N + 1 ≥ M 0 < ε ≤ 1

P(λNM ≥ (

√M +

√N)2(1 + ε)

)≤ C e

−√MN ε3/2( 1√

ε∧(

)1/4)/C

P(λNM ≤ (

√M +

√N)2(1− ε)

)≤ C e−MN ε3( 1

ε∧(

)1/2)/C

M2/3[λNM − b(ρ)

]→ C (ρ)FTW

b(ρ) =(1 +√ρ)2

λNM = λNM/N, M = M(N) ∼ ρN

(√MN)1/3

(√M +

√N)4/3

(λNM − (

√M +

√N)2

)→ FTW

N + 1 ≥ M 0 < ε ≤ 1

P(λNM ≥ (

√M +

√N)2(1 + ε)

)≤ C e

−√MN ε3/2( 1√

ε∧(

)1/4)/C

P(λNM ≤ (

√M +

√N)2(1− ε)

)≤ C e−MN ε3( 1

ε∧(

)1/2)/C

bi and tri-diagonal representation

χN 0 0 · · · · · · 0

χ(M−1) χN−1 0 0 · · ·...

0 χ(M−2) χN−3 0. . .

...... 0

. . .. . .

. . . 0... · · · . . . χ2 χN−M+2 00 · · · · · · 0 χ1 χN−M+1

χ(N−1), . . . , χ1, χ(M−1), . . . , χ1 independent chi-variables

B Bt same spectrum as Y Y t (Wishart)

H. Trotter (1984), A. Edelman, I. Dimitriu (2002)

extension to β-ensembles

χN 0 0 · · · · · · 0

χ(M−1) χN−1 0 0 · · ·...

0 χ(M−2) χN−3 0. . .

...... 0

. . .. . .

. . . 0... · · · . . . χ2 χN−M+2 00 · · · · · · 0 χ1 χN−M+1

χN 0 0 · · · · · · 0

χ(M−1) χN−1 0 0 · · ·...

0 χ(M−2) χN−3 0. . .

...... 0

. . .. . .

. . . 0... · · · . . . χ2 χN−M+2 00 · · · · · · 0 χ1 χN−M+1

χN 0 0 · · · · · · 0

χ(M−1) χN−1 0 0 · · ·...

0 χ(M−2) χN−3 0. . .

...... 0

. . .. . .

. . . 0... · · · . . . χ2 χN−M+2 00 · · · · · · 0 χ1 χN−M+1

χN 0 0 · · · · · · 0

χ(M−1) χN−1 0 0 · · ·...

0 χ(M−2) χN−3 0. . .

...... 0

. . .. . .

. . . 0... · · · . . . χ2 χN−M+2 00 · · · · · · 0 χ1 χN−M+1

bounds for non-Gaussian entries

))O. Feldheim, S. Sodin (2010)

largest eigenvalue (symmetric, subGaussian entries)

)≤ C e−M ε3/2/C , 0 < ε ≤ 1

below the mean ?

necessary for variance bounds

)≤ C e−M ε3/2/C , 0 < ε ≤ 1

below the mean ?

)≤ C e−M ε3/2/C , 0 < ε ≤ 1

below the mean ?

)≤ C e−M ε3/2/C , 0 < ε ≤ 1

below the mean ?

variance level

Var( λNM) = O( 1

S. Dallaporta (2012)

variance level

Var( λNM) = O( 1

S. Dallaporta (2012)

smallest eigenvalue

soft edge M = M(N) ∼ ρN, ρ < 1

a(ρ) =(1−√ρ

P(λN1 ≤ a(ρ)− ε

)≤ C e−M ε3/2/C , 0 < ε ≤ 1

P(λN1 ≥ a(ρ) + ε

)≤ C e−M ε3/C , 0 < ε ≤ a(ρ)

Wishart matrices B. Rider, M. L. (2010)

smallest eigenvalue

soft edge M = M(N) ∼ ρN, ρ < 1

a(ρ) =(1−√ρ

P(λN1 ≤ a(ρ)− ε

)≤ C e−M ε3/2/C , 0 < ε ≤ 1

P(λN1 ≥ a(ρ) + ε

)≤ C e−M ε3/C , 0 < ε ≤ a(ρ)

Wishart matrices B. Rider, M. L. (2010)

smallest eigenvalue

hard edge M = N, ρ = 1

a(ρ) =(1−√ρ

P(λN1 ≤

)≤ C

√ε+ C e−cN

large families of covariance matrices

M. Rudelson, R. Vershynin (2008-10)

smallest eigenvalue

hard edge M = N, ρ = 1

a(ρ) =(1−√ρ

P(λN1 ≤

)≤ C

√ε+ C e−cN

large families of covariance matrices

M. Rudelson, R. Vershynin (2008-10)

Concentration Inequalities for Random Matrices · Concentration Inequalities for Random Matrices M....

Documents

Transcript of Concentration Inequalities for Random Matrices · Concentration Inequalities for Random Matrices M....