Insights into Large Complex Systems via Random Matrix Theory · Akemann et al. (eds) [2011]The...

Insights into Large Complex Systemsvia Random Matrix Theory

Lu Wei

SEAS, Harvard

06/23/2016 @UTC Institute for Advanced Systems Engineering

University of Connecticut

Lu Wei Random Matrix Theory and Large Complex Systems 1 / 19

Random Matrix Theory





milestones

Wishart distribution (Wishart [1928])Semicircle law (Wigner [1955])Marchenko-Pastur law (Marchenko-Pastur [1967])Tracy-Widom law (Tracy-Widom [1990s])KPZ universality class (Johansson [2000s])

tools from almost all branches of mathematics and physicsAkemann et al. (eds) [2011] The Oxford Handbook of Random MatrixTheory. Oxford University Press

applications: biology, data sciences, economics, information theory,machine learning, wireless communications,. . .




milestonesWishart distribution (Wishart [1928])

Semicircle law (Wigner [1955])Marchenko-Pastur law (Marchenko-Pastur [1967])Tracy-Widom law (Tracy-Widom [1990s])KPZ universality class (Johansson [2000s])






milestonesWishart distribution (Wishart [1928])Semicircle law (Wigner [1955])

Marchenko-Pastur law (Marchenko-Pastur [1967])Tracy-Widom law (Tracy-Widom [1990s])KPZ universality class (Johansson [2000s])






milestonesWishart distribution (Wishart [1928])Semicircle law (Wigner [1955])Marchenko-Pastur law (Marchenko-Pastur [1967])

Tracy-Widom law (Tracy-Widom [1990s])KPZ universality class (Johansson [2000s])






milestonesWishart distribution (Wishart [1928])Semicircle law (Wigner [1955])Marchenko-Pastur law (Marchenko-Pastur [1967])Tracy-Widom law (Tracy-Widom [1990s])

KPZ universality class (Johansson [2000s])






milestonesWishart distribution (Wishart [1928])Semicircle law (Wigner [1955])Marchenko-Pastur law (Marchenko-Pastur [1967])Tracy-Widom law (Tracy-Widom [1990s])KPZ universality class (Johansson [2000s])







tools from almost all branches of mathematics and physics

Akemann et al. (eds) [2011] The Oxford Handbook of Random MatrixTheory. Oxford University Press



Product of Random Matrices


joint works with Akemann, Hero, Kieburg, Liu, Tarokh, Zhang, Zheng



Products of Random Matrices

Crisanti et al. [1993] Products of Random Matrices in StatisticalPhysics. Springer

Müller [2002] On the asymptotic eigenvalue distribution ofconcatenated vector-valued fading channels, IEEE Trans. Inf. Theory

y = Hx + w, where H = Hn · · ·H2H1

H1 H2 Hn

Tx. Rx. cluster 1 cluster 2 cluster n-1

x y

w

an earlier attempt via eigenvalues and singular values relation




Crisanti et al. [1993] Products of Random Matrices in StatisticalPhysics. Springer

Müller [2002] On the asymptotic eigenvalue distribution ofconcatenated vector-valued fading channels, IEEE Trans. Inf. Theory

y = Hx + w, where H = Hn · · ·H2H1

H1 H2 Hn

Tx. Rx. cluster 1 cluster 2 cluster n-1

x y

w

an earlier attempt via eigenvalues and singular values relation∗

∗W., Zheng, Tirkkonen, Hämäläinen [2013] On the ergodic mutual information of multiple cluster scattering MIMOchannels, IEEE Commun. Lett.




H = Hn · · ·H2H1

n = 1 – Wishart-Laguerre ensemble Bronk [1965]

p (λ1, . . . , λm) ∝ det2(λj−1

k

) m∏i=1

e−λi

n arbitraryAkemann, Kieburg, W. [2013] Singular value correlationfunctions for products of Wishart random matrices, J. Phys. A

p (λ1, . . . , λm) ∝ det(λj−1

k

)det (fj(λk ))

fj(x) = Gm,00,m

(x

∣∣∣∣∣ −0, . . . , 0, j − 1

)= 1

2πı

∮L du x−uΓm−1(u)Γ(u + j − 1)




H = Hn · · ·H2H1

n = 1 – Wishart-Laguerre ensemble Bronk [1965]

p (λ1, . . . , λm) ∝ det2(λj−1

k

) m∏i=1

e−λi

n arbitrary†

p (λ1, . . . , λm) ∝ det(λj−1

k

)det (fj(λk ))

fj(x) = Gm,00,m

(x

∣∣∣∣∣ −0, . . . , 0, j − 1

)= 1

2πı

∮L du x−uΓm−1(u)Γ(u + j − 1)

†Akemann, Kieburg, W. [2013] Singular value correlation functions for products of Wishart random matrices, J. Phys. A



Applications to Capacity Analysis

ergodic capacity Akemann-Kieburg-W. [2013]

E[ m∑

i=1

log (1 + γλi)

]

outage capacity of orthogonal space-time codes, i.e., distribution ofm∑

i=1

λi

outage capacity of double-cluster channels, i.e., distribution ofm∑

i=1

log (1 + γλi) for n = 2





E[ m∑

i=1

log (1 + γλi)

]

outage capacity of orthogonal space-time codes‡, i.e., distribution ofm∑

i=1

λi

outage capacity of double-cluster channels, i.e., distribution ofm∑

i=1


‡W., Zheng, Corander, Taricco [2015] On the outage capacity of orthogonal space-time block codes over multi-clusterscattering MIMO channels, IEEE Trans. Commun.





E[ m∑

i=1

log (1 + γλi)

]

outage capacity of orthogonal space-time codes‡, i.e., distribution ofm∑

i=1

λi

outage capacity of double-cluster channels§, i.e., distribution ofm∑

i=1


‡W., Zheng, Corander, Taricco [2015] On the outage capacity of orthogonal space-time block codes over multi-clusterscattering MIMO channels, IEEE Trans. Commun.

§Zheng, W., Speicher, Müller, Hämäläinen, Corander On the fluctuation of mutual information of double-clusterscattering MIMO channels: A free probability approach, IEEE Trans. Inf. Theory, under revision, arXiv:1502.05516



Spiked Products of Random Matrices: Phase Transitions

H = Σ1/2Hn · · ·H2H1

natural generalization E[HH†

]∝ Σ = diag (σ1, . . . , σm) – spikes

n = 1 – Baik-Ben Arous-Péché [2005] Phase transition of the largesteigenvalue for nonnull complex sample covariance matrices, Ann.Probab.

λ1 from Tracy-Widom to Gaussian (BBP phase transition)critical value σcrit = 2

n arbitraryLiu, W., Zhang Singular values for spiked products ofcomplex Ginibre matrices

λ1 BBP phase transitioncritical value σcrit = n + 1




H = Σ1/2Hn · · ·H2H1


]∝ Σ = diag (σ1, . . . , σm) – spikes


λ1 from Tracy-Widom to Gaussian (BBP phase transition)

critical value σcrit = 2






H = Σ1/2Hn · · ·H2H1


]∝ Σ = diag (σ1, . . . , σm) – spikes








H = Σ1/2Hn · · ·H2H1


]∝ Σ = diag (σ1, . . . , σm) – spikes



n arbitrary¶


¶Liu, W., Zhang Singular values for spiked products of complex Ginibre matrices




H = Σ1/2Hn · · ·H2H1


]∝ Σ = diag (σ1, . . . , σm) – spikes



n arbitrary¶

λ1 BBP phase transition

critical value σcrit = n + 1





H = Σ1/2Hn · · ·H2H1


]∝ Σ = diag (σ1, . . . , σm) – spikes



n arbitrary¶





Phase Transitions

50

40

30

λ1

20

10

020

15

10

σ1

5

0.1

0.2

0.3

0

0.4

dens

ity o

f λ

1

MIMO radar detection; community detection,. . .



Phase Transitions

50

40

30

λ1

20

10

020

15

10

σ1

5

0.1

0.2

0.3

0

0.4

dens

ity o

f λ

1

MIMO radar detection‖;

community detection,. . .

‖W., Zheng, Hero, Tarokh Scaling laws and phase transitions for target detection in MIMO radar, ITW’16



Phase Transitions

50

40

30

λ1

20

10

020

15

10

σ1

5

0.1

0.2

0.3

0

0.4

dens

ity o

f λ

1

MIMO radar detection‖; community detection,. . .

‖W., Zheng, Hero, Tarokh Scaling laws and phase transitions for target detection in MIMO radar, ITW’16


Other Large Complex Systems

Applications to Other Large Complex Systems


Signal Processing

Signal Processing

joint works with Dharmawansa, Liang, McKay, Tirkkonen


Signal Processing

Signal Detection

cognitive radio - a solution to spectrum underutilization problem bydynamic spectrum access

unlicensed users are allowed to opportunistically use the frequencybands that are not heavily occupied by licensed users

awareness of spectrum usage information via spectrum sensing

ym×1

= Hm×p

x + w

Ym×N

= (y1, y2, . . . , yN) =⇒{H0 : noise

H1 : signal + noise

R = YY† data sample covariance; E noise sample covariance


Signal Processing

Signal Detection

p > 1 but unknown∗

TST =det (R)(1m tr(R)

)m

low signal-to-noise ratio†

TJ =tr(R2)

tr2(R)

unknown noise covariance‡

TW =det (E)

det (R + E)

∗W., Tirkkonen [2012] Spectrum sensing in the presence of multiple primary users, IEEE Trans. Commun.

†W., Dharmawansa, Tirkkonen [2013] Multiple primary user spectrum sensing in the low SNR regime, IEEE Trans.Commun.‡W., Tirkkonen, Liang [2014] Multi-source signal detection with arbitrary noise covariance, IEEE Trans. Signal Process.


Signal Processing

Signal Detection



)m


TJ =tr(R2)

tr2(R)


TW =det (E)

det (R + E)

∗W., Tirkkonen [2012] Spectrum sensing in the presence of multiple primary users, IEEE Trans. Commun.†W., Dharmawansa, Tirkkonen [2013] Multiple primary user spectrum sensing in the low SNR regime, IEEE Trans.

Commun.

‡W., Tirkkonen, Liang [2014] Multi-source signal detection with arbitrary noise covariance, IEEE Trans. Signal Process.


Signal Processing

Signal Detection



)m


TJ =tr(R2)

tr2(R)


TW =det (E)

det (R + E)

∗W., Tirkkonen [2012] Spectrum sensing in the presence of multiple primary users, IEEE Trans. Commun.†W., Dharmawansa, Tirkkonen [2013] Multiple primary user spectrum sensing in the low SNR regime, IEEE Trans.

Commun.‡W., Tirkkonen, Liang [2014] Multi-source signal detection with arbitrary noise covariance, IEEE Trans. Signal Process.


Coding Theory

Coding Theory

joint works with Corander, Pitaval, Tirkkonen


Coding Theory

Coding Theory

fundamental issue: cardinality and minimum distance tradeoff

a code with cardinality |C|, C ={

G1,G2, . . . ,G|C|}⊂ G

minimum distance, r = min{||Gi − Gj ||

∣∣ Gi ,Gj ∈ C, i 6= j}

1µ (B (r))︸︷︷︸

Gilbert-Varshamov bound

≤ |C| ≤ 1µ (B (r/2))︸︷︷︸

Hamming bound

metric ball, B (r) ={

G ∈ G∣∣ d(

G,G′)≤ r

}, G

′ ∈ Gvolume of metric ball, µ (B (r)) =

∫d(G,G′)≤r f (G) dG


Coding Theory

Coding Theory



G1,G2, . . . ,G|C|}⊂ G


∣∣ Gi ,Gj ∈ C, i 6= j}

1µ (B (r))︸︷︷︸


≤ |C| ≤ 1µ (B (r/2))︸︷︷︸

Hamming bound


G ∈ G∣∣ d(

G,G′)≤ r

}, G

′ ∈ G

volume of metric ball, µ (B (r)) =∫

d(G,G′)≤r f (G) dG


Coding Theory

Coding Theory



G1,G2, . . . ,G|C|}⊂ G


∣∣ Gi ,Gj ∈ C, i 6= j}

1µ (B (r))︸︷︷︸


≤ |C| ≤ 1µ (B (r/2))︸︷︷︸

Hamming bound


G ∈ G∣∣ d(

G,G′)≤ r

}, G

′ ∈ Gvolume of metric ball, µ (B (r)) =

∫d(G,G′)≤r f (G) dG


Coding Theory

Volume of Metric Balls


Coding Theory

Unitary Group

µ (B (r)) ∝∫· · ·∫||U−In||F≤r

∏1≤j<k≤n

∣∣∣eıθj − eıθk∣∣∣2 n∏

i=1

dθi

Han-Rosenthal [2006] Unitary space-time constellation analysis: anupper bound for the diversity, IEEE Trans. Inf. Theory

limiting behaviorW., Pitaval, Corander, Tirkkonen From random matrixtheory to coding theory: Volume of a metric ball in unitary group,IEEE Trans. Inf. Theory, submitted, arXiv:1506.07259 as n→∞

µ (B (r)) ' 12

erf(n)− 12

erf

(n − r2

2

)

super-exponential rate of convergence O (n−cn)CLT of linear statistics of unitary group


Coding Theory

Unitary Group

µ (B (r)) ∝∫· · ·∫||U−In||F≤r

∏1≤j<k≤n


i=1

dθi


limiting behavior∗ as n→∞

µ (B (r)) ' 12

erf(n)− 12

erf

(n − r2

2

)


∗W., Pitaval, Corander, Tirkkonen From random matrix theory to coding theory: Volume of a metric ball in unitary group,IEEE Trans. Inf. Theory, submitted, arXiv:1506.07259


Coding Theory

Unitary Group

µ (B (r)) ∝∫· · ·∫||U−In||F≤r

∏1≤j<k≤n


i=1

dθi



µ (B (r)) ' 12

erf(n)− 12

erf

(n − r2

2

)

super-exponential rate of convergence O (n−cn)

CLT of linear statistics of unitary group



Coding Theory

Unitary Group

µ (B (r)) ∝∫· · ·∫||U−In||F≤r

∏1≤j<k≤n


i=1

dθi



µ (B (r)) ' 12

erf(n)− 12

erf

(n − r2

2

)




Coding Theory

Grassmann Manifold

µ (B (r)) ∝∫· · ·∫

0≤xi≤1∑pi=1 xi≤r

∏1≤j<k≤p

(xj − xk )2p∏

i=1

xn−p−qi (1− xi)

q−p dxi

Barg-Nogin [2002] fixed p, q, large n; Dai et al. [2008] r ≤ 1

limiting behavior as n, p, q →∞

µ (B (r)) ' 12

erf(

b√2a

)− 1

2erf

(b − r2√

2a

)

CLT of linear statistics of Jacobi ensemblemoments from Painlevé V


Coding Theory

Grassmann Manifold

µ (B (r)) ∝∫· · ·∫


∏1≤j<k≤p

(xj − xk )2p∏

i=1


q−p dxi


limiting behavior† as n, p, q →∞

µ (B (r)) ' 12

erf(

b√2a

)− 1

2erf

(b − r2√

2a

)


†Pitaval, W., Tirkkonen, Corander Volume of metric balls in high-dimensional complex Grassmann manifolds, IEEETrans. Inf. Theory, submitted, arXiv:1508.00256


Coding Theory

Grassmann Manifold

µ (B (r)) ∝∫· · ·∫


∏1≤j<k≤p

(xj − xk )2p∏

i=1


q−p dxi



µ (B (r)) ' 12

erf(

b√2a

)− 1

2erf

(b − r2√

2a

)

CLT of linear statistics of Jacobi ensemble

moments from Painlevé V



Coding Theory

Grassmann Manifold

µ (B (r)) ∝∫· · ·∫


∏1≤j<k≤p

(xj − xk )2p∏

i=1


q−p dxi



µ (B (r)) ' 12

erf(

b√2a

)− 1

2erf

(b − r2√

2a

)




Conclusion

Large Complex Systems


Wireless Communications

Statistical Physics

Information Theory

Signal Processing

Machine Learning

Data Sciences

?


Insights into Large Complex Systems via Random Matrix Theory · Akemann et al. (eds) [2011]The...

Documents

Transcript of Insights into Large Complex Systems via Random Matrix Theory · Akemann et al. (eds) [2011]The...