Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse...
Transcript of Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse...
![Page 1: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/1.jpg)
Some words on Stochastic Eigen-Analysis
Alan Edelman Raj Rao
Dept of MathematicsComputer Science & AI Laboratories
MITJuly 10, 2006
![Page 2: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/2.jpg)
Some words on Stochastic Eigen-Analysis
Alan Edelman Raj Rao
Dept of MathematicsComputer Science & AI Laboratories
MITJuly 10, 2006
![Page 3: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/3.jpg)
Just when you thought mathematics just about
wrapped up …
1. Orthogonal Polynomials & Special Functions
2. Convolutions3. Stochastic Differential Operators
![Page 4: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/4.jpg)
Just when you thought mathematics just about
wrapped up …
1. Orthogonal Polynomials & Special Functions
2. Convolutions3. Stochastic Differential Operators
![Page 5: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/5.jpg)
Pre & Early Computer Days
The Bateman Manuscript
Project
![Page 6: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/6.jpg)
The web era
![Page 7: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/7.jpg)
The SEA era
Ahead of its time Orthogonal Polynomials & Random Matrices:
A Riemann-Hilbert Approach MOPS: Dumitriu
Koev
Anshelevich (Free Meixner poly.)
Chikuse (Statistics on manifolds)
![Page 8: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/8.jpg)
Just when you thought mathematics just about
wrapped up …
1. Orthogonal Polynomials & Special Functions
2. Convolutions3. Stochastic Differential Operators
![Page 9: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/9.jpg)
Classical Convolutions
![Page 10: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/10.jpg)
Plus
-3 -2 -1 0 1 2 30
0.05
0.1
0.15
0.2
0.25
0.3
0.35
x
Prob
abili
ty
0 0.5 1 1.5 2 2.5 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
Pro
babi
lity
Y=randn(n,2n)B=Y*Y’
zm2+(2z-1)m+2=0
+
X =randn(n,n)A=X+X’
m2+zm+1=0
-2 -1 0 1 2 3 40
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
Pro
babi
lity
A+Bm3+(z+2)m2+(2z-1)m+2=0
![Page 11: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/11.jpg)
Times
-3 -2 -1 0 1 2 30
0.05
0.1
0.15
0.2
0.25
0.3
0.35
x
Prob
abili
ty
0 0.5 1 1.5 2 2.5 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
Pro
babi
lity
X =randn(n,n)A=X+X’
m2+zm+1=0
Y=randn(n,2n)B=Y*Y’
zm2+(2z-1)m+2=0
*
-2 -1 0 1 2 3 40
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
Pro
babi
lity
A*Bm4z2-2m3z+m2+4mz+4=0
-3 -2 -1 0 1 2 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x
Pro
babi
lity
![Page 12: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/12.jpg)
The convolutions (Free Prob)
![Page 13: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/13.jpg)
Spectrum of Sample Covariance Matrix
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 40
0.2
0.4
0.6
0.8
1
1.2
1.4
x
Pro
babi
lity
c = 0.5c = 0.1
- Convolution is highly non-linear
- Density is function of Sensors/Snapshots, eig(R)
- Symbolic package (RMTool) to compute density
. Moments (canonically) in closed form!
1 3
0.4
0.6
eig(R) Marcenko-Pastur SCM spectrum
0 1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x
Pro
babi
lity
c = 0.5c = 0.1
![Page 14: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/14.jpg)
Eigenvalues of true covariance matrix
Eigenvalues blurred (free convolution)
R^ = R1/2 W(c) R1/2
Two distinct subspaces
Blurring of eigenvaluesbecause of insufficient sample support
![Page 15: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/15.jpg)
Free “Deconvolving” of a singleSample Covariance Matrix
There is no structure visible to the eye, but the subspace structure can be deduced by free deconvolution
Eigenvalues blur because of limited data
![Page 16: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/16.jpg)
“Convolution” <-> “Deconvolution”
• Model based– moment matching + “second” order freeness
(with Speicher + Mingo)– parametric
• Non-model based– Stieltjes transform-to-resolvent matching– Connection to Lanczos, GMRES– (with Per-Olof Persson)– non-parametric
![Page 17: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/17.jpg)
Free probability in SEA’06
• Speicher (Survey)• Chatterjee (Rate of convergence)• Speicher + Mingo (Fluctuations & 2nd order
freeness)• Anshelevich (Free Meixner polynomials)• Demni (Processes)• Burda (Free Levy matrices)• Kargin (Large deviations)• Rashidi Far (Operator values free probability)
![Page 18: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/18.jpg)
Just when you thought mathematics just about
wrapped up …
1. Orthogonal Polynomials & Special Functions
2. Convolutions3. Stochastic Differential Operators
![Page 19: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/19.jpg)
Stochastic Operators
• Ito & Stratonovich• Many recent methods• Whole Field of stochastic differential
equations–MATLAB SPEAK:
• rand + “\” well studied• rand + “eig” missing
![Page 20: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/20.jpg)
Stochastic Operator Limit
,
N(0,2)χχN(0,2)χ
χN(0,2)χχN(0,2)
nβ21~H
β
β2β
2)β(n1)β(n
1)β(n
βn
⎟⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜⎜
⎝
⎛
−−
−
,dWβ
2x dxd
2
2
+−
,Gβ
2HH nnβn +≈ ∞
… … …
![Page 21: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/21.jpg)
21
Those betas• Real Numbers: x β=1• Complex Numbers: x+iy β=2• Quaternions: x+iy+jz+kw β=4• β=2½? x+iy+jz
![Page 22: Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators](https://reader033.fdocuments.us/reader033/viewer/2022043017/5f397d7486653d22b143f774/html5/thumbnails/22.jpg)
Other (Math) Talks in SEA’06
• Appearance of “universal” distributions– Kuijlaars, Baik, Johnstone, El Karoui, Dieng,
Sutton, Rider, Sasamoto, Seba
• Causal sets, airplane boarding– Bachmat
• Complex systems– Ergün, Sethna, Timme
• Principal component analysis– Paul, Onatski
• Applications & more!