Eigenvalues of Symmetrix Hierarchical Matrices

MAX PLANCK INSTITUTE

FOR DYNAMICS OF COMPLEX

TECHNICAL SYSTEMS

MAGDEBURG

20 February 2012

Eigenvalue Algorithms for SymmetricHierarchical Matrices

Thomas Mach

Max Planck Institute for Dynamics of Complex Technical SystemsComputational Methods in Systems and Control Theory

Max Planck Institute Magdeburg Thomas Mach, Eigenvalue Algorithms for Symmetric H-Matrices

Hierarchical (H-)Matrices LR Algorithm Slicing Algorithm PINVIT Conclusions

Eigenvalue Problem

Definition

The pair (λ, v) ∈ R× Rn is called an eigenpair of the symmetricmatrix M = MT ∈ Rn×n, if

Mv = vλ.

The set Λ(M) = {λ|∃v : (λ, v) eigenpair of M} is the spectrumof M.

Similarity Transformation

Λ(M) = Λ(P−1MP) ∀P invertible



Classification of Eigenvalue Problems[Golub, Van der Vorst ’00]

Is M real or complex?

M ∈ Rn×n

Special properties (symmetric, Hermitian, skew-symmetric orunitary)?symmetric: M = MT

Further structure? Yeah.M ∈ H(TI× I, k) ⇒ see next slide

Which eigenvalues required?some (inner) or all eigenvalues




Is M real or complex?M ∈ Rn×n








Special properties (symmetric, Hermitian, skew-symmetric orunitary)?

symmetric: M = MT








Further structure?

Yeah.M ∈ H(TI× I, k) ⇒ see next slide







Further structure? Yeah.

M ∈ H(TI× I, k) ⇒ see next slide








Which eigenvalues required?

some (inner) or all eigenvalues



H-Matrices [Hackbusch 1998]

Some dense matrices, e.g. BEM or FEM, can be approximated byH-matrices in a data-sparse manner.

hierarchical tree TI block H-tree TI× I

I = {1, 2, 3, 4, 5, 6, 7, 8}

{1, 2, 3, 4} {5, 6, 7, 8}

{1, 2} {3, 4} {5, 6} {7, 8}

{1}{2}{3}{4}{5}{6}{7}{8}

1 1 1 12 2 2 23 3 3 34 4 4 45 5 5 56 6 6 67 7 7 78 8 8 8

1 1 1 12 2 2 23 3 3 34 4 4 45 5 5 56 6 6 67 7 7 78 8 8 8

dense matrices, rank-k-matrices

rank-k-matrix: Ma×b = ABT , A ∈ Rn×k ,B ∈ Rm×k (k � n,m),



H-Matrices [Hackbusch 1998]

A1

BT1

+ A2

BT2

= A1 A2

BT1

BT2

22

3 3

7 10

3 7

3 10

19 10

10 31

14 8

11 11

14 11

8 11

19 10

10 31 11

11 31

11 9

9 16 12

11 1611 8

9 16 11

11 16

61

9 73 3

8

11 11

9 3

7 3

11

8 11

25 10

10 19 11

11 31

8 5

11 8

8 15 8 12

116 5

15 613

13

7 5

13 8

8 11 8 11

11

6 5

15 6

8

11 8

8 15

5 8 11

12

6 15

5 613

13

7

13 8

8 11

5 811

11

6 15

5 6

6110 103

6 14

10 3 6

10 14

25 10 6

10

6

19 10

10 31

15 9

9 1110

10 16

15 9

8 1110

10 16

5110 10

7 9 73 3

10 7

10

9 3

7 3

25 11

11

25 10

10 19

11 811 8

8 15 9

8 1512 13

13

10 7

13 8

8 11 9

9 15 1119

20

10 7

13 8

9

11 8

8 15 11 12

13

9 7

13 8

9

13 8

8 1111

10

11 8

8 15 8

9 15

7 12 13

13

10

13 8

8 11 8

9 15

7 1120

19

9

13 9

8

11 8

8 15

7 11 12

129

13 9

8

13 8

8 11

7 11

39 10

10 25

3 7

3 10 10

7 10 63 3

7 10 7

10

10

6

22

3 3

7 10

3 7

3 10

19 10

10 31

15 10

11

11 9

9 16

15 11

10

11 8

9 16

34 10

10 25

13 10

7 11

13 7

10 11 61

6 5

13 6 11

12

8 5

11 8

8 15 812

13

6 5

13 6 11

11 23

6 13

5 6 12

118

11 8

8 15

5 813

12

6 13

5 6 10

11 23

20 9

9 39

9 7

10

3 7

3 10

9 10

7

3 3

7 1061

15 10

1015 9

9 11

15 10

1015 9

8 11

20 9

9 34

9 7

10 13

9 10

7 1351

adaptive rank k(ε)

storage NSt,H(T , k) = O(n log n k(ε))

complexity of approximate arithmetic

MHv O(n log n k(ε))+H,−H O(n log n k(ε)2)

∗H,HLU(·), (·)−1H O(n (log n)2 k(ε)2)

Hierarchical Matrices

H(TI× I, k) ={M ∈ RI× I

∣∣ rank (Ma×b) ≤ k ∀a× b admissible}



Special Case: H`-Matrices [Hackbusch 1998]

A8BT8

B8AT8

A4BT4

B4AT4

A12BT12

B12AT12

A2BT2

A6BT6

A10BT10

A14BT14

B2AT2

B6AT6

B10AT10

B14AT14

F1

F3

F5

F7

F9

F11

F13

F15

Structure of a symmetric H3(k)-matrix.



Hlib

Hlib [Borm, Grasedyck, et al.]

We use the Hlib (www.hlib.org) for theH-arithmetic operations and some examples out ofthe library for testing the eigenvalue algorithm.


www.hlib.org


Eigenvalues of Symmetric H-Matrices

M = MT ∈ H(T , k)⇓

ΛH(M) = {λ1, . . . , λn} in O(n2 (log n)α kβ)

{λi} ∈ ΛH(M) in O(n (log n)α kβ)?

dense: M + N , Mv in O(n2) and Λ(M) in O(n3)



LR Cholesky Algorithm

QR-like Algorithm



LR Cholesky Algorithm [Rutishauser 1958]

LR-Cholesky Transformation

for i = 1, . . . do

LiLTi = Mi

Mi+1 = LTi Liend

limi→∞

Mi = diag (λ1, λ2, . . . , λn) ∈ H(T , 0)

∀i : Mi − µiI symmetric positive definite

H-LR-Cholesky Transformation

for i = 1, . . . do

Li = H-Cholesky factorization(Mi − µiI)Mi+1 = LTi ∗H Li + µiI

endshift strategy

deflation





for i = 1, . . . do

LiLTi = Mi ⇒ Li = MiL

−Ti

Mi+1 = LTi Li = LTi MiL−Ti

end

limi→∞

Mi = diag (λ1, λ2, . . . , λn) ∈ H(T , 0)



for i = 1, . . . do


endshift strategy

deflation





for i = 1, . . . do

LiLTi = Mi − µiI

Mi+1 = LTi Li + µiIend

limi→∞

Mi = diag (λ1, λ2, . . . , λn) ∈ H(T , 0)



for i = 1, . . . do


endshift strategy

deflation





for i = 1, . . . do

LiLTi = Mi − µiI


limi→∞

Mi = diag (λ1, λ2, . . . , λn) ∈ H(T , 0)



for i = 1, . . . do


end

shift strategy

deflation





for i = 1, . . . do

LiLTi = Mi − µiI


limi→∞

Mi = diag (λ1, λ2, . . . , λn) ∈ H(T , 0)



for i = 1, . . . do


endshift strategy

deflation



Example - H-Fill-In

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

Matrix FEM16 (∆2,h, 16 inner discr. points).



Example - H-Fill-In

32 10

10 32

4

5 6

4 5

6

32 9

9 32

8

6 8

8

68

32 10

10 32

6

5 4

6 5

4

32 10

10 32

Matrix FEM16 (∆2,h, 16 inner discr. points), after 1 step.



Example - H-Fill-In

32 10

10 32

4

7 7

4 7

7

32 10

10 32

8

7 8

8

78

32 11

11 32

7

7 4

7 7

4

32 10

10 32

Matrix FEM16 (∆2,h, 16 inner discr. points), after 2 steps.



Example - H-Fill-In

32 10

10 32

4

7 7

4 7

7

32 10

10 32

8

8 8

8

88

32 11

11 32

7

8 4

7 8

4

32 10

10 32




Example - H-Fill-In

32 10

10 32

4

7 7

4 7

7

32 11

11 32

8

9 8

8

98

32 12

12 32

7

10 4

7 10

4

32 11

11 31




Example - H-Fill-In

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

4

4

4

4

4

4

4

4

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

4

4

4

4

4

4

4

4

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

Matrix FEM32 (∆2,h, 32 inner discr. points).



Example - H-Fill-In

32 11

11 32

4

10 11

4 10

11

32 14

14 32

8

184 4

15 8

8

18

4 15

4 8

32 14

14 32

11 5

17 14

11 17

5 14

32 20

20 32

4

8 4

188 7

9 5

15 13 10

4 8

4

18

89 15

5 13

7 10

32 14

14 32

7 4

10 12

7 10

4 12

32 11

11 32

4 4

14 7 5

143 3

11 4

4 14

4 7 14

53 11

3 4

32 20

20 32

14 6

18 13

14 18

6 13

32 15

15 32

8

16 8

26 8

16 8

8

16

8

268

16

8

32 15

15 32

11 4

18 14

11 18

4 14

32 20

20 32

4 3

11 3 4

188 7

17 9

4 11

3 3 18

48 17

7 9

32 15

15 32

11 6

11 9

11 11

6 9

32 20

20 32

7 5

12 4

14 8 7

22 4

8 4

712 14

4 8

5 7 224 8

4

32 19

19 32

14 4

20 13

14 20

4 13

32 15

15 32

6 4

15 4

21 8

6 15

4 4 21

8

32 18

18 32

11

14 4

11 14

4

32 11

11 32

Matrix FEM32 (∆2,h, 32 inner discr. points), after 1 step.



Example - H-Fill-In

32 11

11 32

4

22 13

4 22

13

32 22

22 32

12

434 4

25 17

12

43

4 25

4 17

32 28

28 32

24 32

30 25

24 30

32 25

32 32

32 32

4

16 4

7310 9

30 32

32 32 38

4 16

4

73

1030 32

32 32

9 38

32 32

32 32

23 30

31 32

23 31

30 32

32 25

25 32

31 30

32 32 46

5732 31

29 32

31 32

30 32 57

4632 29

31 32

32 32

32 32

32 32

32 32

32 32

32 32

32 32

32 32

8

38 8

1048

40 8

8

38

8

1048

40

8

32 32

32 32

32 32

32 32

32 32

32 32

32 32

32 32

30 31

32 30 42

6332 32

32 32

30 32

31 30 63

4232 32

32 32

32 31

31 32

32 31

31 28

32 31

31 28

32 32

32 32

33 10

32 30

32 30 13

75 4

18 4

3332 32

30 30

10 13 754 18

4

32 32

32 32

32 23

32 30

32 32

23 30

32 31

31 32

19 6

25 7

48 8

19 25

6 7 48

8

32 28

28 32

14

20 4

14 20

4

32 11

11 17




Computation Time

101 102 103 104 105 10610−2

10−1

100

101

102

103

104

105

106

107

Dimension

CP

Uti

me

ins

H-LR algorithmLAPACK dsyev

O(n2 (log2 n)2)

O(n3)

Name n # iterations

FEM8 64 101FEM16 256 556FEM32 1 024 2 333FEM64 4 096 10 320



TheoremAdaption of [Fasino ’05/Plestenjak, Van Barel, Van Camp ’08]

M = diag (d) +∑r

i=1

(tril(uiv

Ti

)+ triu

(viu

Ti

))

Structure Preservation of dpss Matrices

Let M be a symmetric positive definite diagonal plus semiseparablematrix, with a decomposition as in the definition. The Choleskyfactor L of M = LLT can be written in the form

L = diag(d)

+∑r

i=1 tril(ui v

Ti

).

Multiplying the Cholesky factors in reverse order gives the nextiterate N = LTL of the LR Cholesky algorithm. The matrix N hasthe same form as M,

N = diag(d)

+∑r

i=1

(tril(ui v

Ti

)+ triu

(vi u

Ti

)).



TheoremAdaption of [Fasino ’05/Plestenjak, Van Barel, Van Camp ’08]

M = diag (d) +∑r

i=1

(tril(uiv

Ti

)+ triu

(viu

Ti

))Structure Preservation of dpss Matrices

Let M be a symmetric positive definite diagonal plus semiseparablematrix, with a decomposition as in the definition. The Choleskyfactor L of M = LLT can be written in the form

L = diag(d)

+∑r

i=1 tril(ui v

Ti

).

Multiplying the Cholesky factors in reverse order gives the nextiterate N = LTL of the LR Cholesky algorithm. The matrix N hasthe same form as M,

N = diag(d)

+∑r

i=1

(tril(ui v

Ti

)+ triu

(vi u

Ti

)).



Proof Idea

N = LTL =

(diag

(d)

+∑i

tril(ui v

Ti

))T

(diag

(d)

+∑i

tril(ui v

Ti

))

ui =(Z + diag

(d))

ui , with

Zp,: =∑j

vj |p[0 · · · 0 uj |p uj |p+1 · · · uj |n

]tril (N,−1) =

∑i

tril((

diag(d)ui + Zui)vTi ,−1

)=∑i

tril(ui v

Ti ,−1

)N is a dpss matrix.



Structure of u and v

M = diag (d) +r∑

i=1

tril(uiv

Ti

)+ . . .

N = diag (d) +r∑

i=1

tril(ui v

Ti

)+ . . .

vi =

0...0∗...∗0...0

vi =

0...0∗...∗*...*

ui =

0...0∗...∗0...0

ui =

*...*∗...∗0...0



Hierarchical Matrices

rank (Ma:b,c:d) = k Ma:b,c:d = ABT

uTir =[0 · · · 0 AT

j ,r 0 · · · 0]

vTir =[0 · · · 0 BT

j ,r 0 · · · 0], r = 1, . . . , k

uTir =[∗ · · · ∗ ∗ 0 · · · 0

]vTir =

[0 · · · 0 ∗ ∗ · · · ∗

]

tril(ui v

Ti

)=

00 00 0 00 0 0 00 ∗ ∗ 0 00 ∗ ∗ 0 0 00 0 0 0 0 0 0

tril

(ui v

Ti

)=

00 ∗0 ∗ ∗0 ∗ ∗ ∗0 ∗ ∗ ∗ ∗0 ∗ ∗ ∗ ∗ ∗0 0 0 0 0 0 0

The structure of hierarchical matrices is not preserved under LRCholesky transformations.



Example - H-Fill-In

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

4

4

4

4

4

4

4

4

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

4

4

4

4

4

4

4

4

32 8

8 32

4

4

4

4

32 8

8 32

8

8

8

8

32 8

8 32

4

4

4

4

32 8

8 32

32 11

11 32

4

10 11

4 10

11

32 14

14 32

8

184 4

15 8

8

18

4 15

4 8

32 14

14 32

11 5

17 14

11 17

5 14

32 20

20 32

4

8 4

188 7

9 5

15 13 10

4 8

4

18

89 15

5 13

7 10

32 14

14 32

7 4

10 12

7 10

4 12

32 11

11 32

4 4

14 7 5

143 3

11 4

4 14

4 7 14

53 11

3 4

32 20

20 32

14 6

18 13

14 18

6 13

32 15

15 32

8

16 8

26 8

16 8

8

16

8

268

16

8

32 15

15 32

11 4

18 14

11 18

4 14

32 20

20 32

4 3

11 3 4

188 7

17 9

4 11

3 3 18

48 17

7 9

32 15

15 32

11 6

11 9

11 11

6 9

32 20

20 32

7 5

12 4

14 8 7

22 4

8 4

712 14

4 8

5 7 224 8

4

32 19

19 32

14 4

20 13

14 20

4 13

32 15

15 32

6 4

15 4

21 8

6 15

4 4 21

8

32 18

18 32

11

14 4

11 14

4

32 11

11 32



H`-Matrices

A8BT8

A4BT4

A12BT12

A2BT2

A6BT6

A10BT10

A14BT14

B8AT8

B4AT4

B12AT12

B2AT2

B6AT6

B10AT10

B14AT14

F1

F3

F5

F7

F9

F11

F13

F15

⇒ rank bounded by `k instead of k⇒ total storage required by the low-rank parts of M is increasedonly from 2nk` to 2nk `(`−1)2



H`-Matrices

I

F1

F3

F5

F7

F9

F11

F13

F15




H`-Matrices

I

I

I

I

I

I

I

F1

F3

F5

F7

F9

F11

F13

F15




H`-Matrices

I

II

II

I

I

I

I

F1

F3

F5

F7

F9

F11

F13

F15




H`-Matrices

I

II

II

II

II

II

II

F1

F3

F5

F7

F9

F11

F13

F15




H`-Matrices

I

II

II

III

III

III

III

F1

F3

F5

F7

F9

F11

F13

F15




Computation Time H`-Matrices

10−1

101

103

105

107

CP

Uti

me

ins

H-LR H`(1)

H-LR H`(2)dsyev

O(n2 log2 n)

O(n3)

O(n2)

102 103 104102103104105

#It

erat

ion

s

# It. H`(1)

# It. H`(2)

O(n)



Slicing the Spectrum

Slicing the Spectrum



Bisectioning [Parlett ’80]

-6 -4 -2 -1 0 1 2 4 6

b0 − a0 ≈ 2 ‖M‖2∣∣∣λi − λi ∣∣∣ < ε ⇔ bn − an < 2ε

bi+1 − ai+1 = 12 (bi − ai )

⇒ O (log2(‖M‖2 /ε))




λ3 = ?

-6 -4 -2 -1 0 1 2 4 6

-6 -4 -2 -1 0 1 2 4 6

b0 − a0 ≈ 2 ‖M‖2∣∣∣λi − λi ∣∣∣ < ε ⇔ bn − an < 2ε

bi+1 − ai+1 = 12 (bi − ai )

⇒ O (log2(‖M‖2 /ε))




λ3 = ?

-6 -4 -2 -1 0 1 2 4 6

-6 -4 -2 -1 0 1 2 4 6

a0 = −6.5 b0 = 5.5

b0 − a0 ≈ 2 ‖M‖2∣∣∣λi − λi ∣∣∣ < ε ⇔ bn − an < 2ε

bi+1 − ai+1 = 12 (bi − ai )

⇒ O (log2(‖M‖2 /ε))




λ3 = ?

-6 -4 -2 -1 0 1 2 4 6

a0 = −6.5 b0 = 5.5

-6 -4 -2 -1 0 1 2 4 6

a0 = −6.5 b0 = 5.5µ1 = −0.5

b0 − a0 ≈ 2 ‖M‖2∣∣∣λi − λi ∣∣∣ < ε ⇔ bn − an < 2ε

bi+1 − ai+1 = 12 (bi − ai )

⇒ O (log2(‖M‖2 /ε))




λ3 = ?

-6 -4 -2 -1 0 1 2 4 6

a0 = −6.5 b0 = 5.5µ1 = −0.5

-6 -4 -2 -1 0 1 2 4 6

a0 = −6.5 b0 = 5.5µ1 = −0.5

ν(−0.5) = 4

b0 − a0 ≈ 2 ‖M‖2∣∣∣λi − λi ∣∣∣ < ε ⇔ bn − an < 2ε

bi+1 − ai+1 = 12 (bi − ai )

⇒ O (log2(‖M‖2 /ε))




λ3 = ?

-6 -4 -2 -1 0 1 2 4 6

a0 = −6.5 b0 = 5.5µ1 = −0.5

ν(−0.5) = 4

-6 -4 -2 -1 0 1 2 4 6

a1 = −6.5 b1 = −0.5µ2 = −3.5

ν(−3.5) = 2

b0 − a0 ≈ 2 ‖M‖2∣∣∣λi − λi ∣∣∣ < ε ⇔ bn − an < 2ε

bi+1 − ai+1 = 12 (bi − ai )

⇒ O (log2(‖M‖2 /ε))




λ3 = ?

-6 -4 -2 -1 0 1 2 4 6

a1 = −6.5 b1 = −0.5µ2 = −3.5

ν(−3.5) = 2

-6 -4 -2 -1 0 1 2 4 6

a2 = −3.5 b2 = −0.5µ3 = −2

ν(−2) = 4

b0 − a0 ≈ 2 ‖M‖2∣∣∣λi − λi ∣∣∣ < ε ⇔ bn − an < 2ε

bi+1 − ai+1 = 12 (bi − ai )

⇒ O (log2(‖M‖2 /ε))




λ3 ∈ [−3.5,−2.75], λ3 = −3.125

-6 -4 -2 -1 0 1 2 4 6

a2 = −3.5 b2 = −0.5µ3 = −2

ν(−2) = 4

-6 -4 -2 -1 0 1 2 4 6

a3 = −3.5 b3 = −2µ4 = −2.75

ν(−2.75) = 3

b0 − a0 ≈ 2 ‖M‖2∣∣∣λi − λi ∣∣∣ < ε ⇔ bn − an < 2ε

bi+1 − ai+1 = 12 (bi − ai )

⇒ O (log2(‖M‖2 /ε))



Evaluation of ν(µ)

Sylvester’s Law of Inertia

Each matrix M is congruent to a matrix

diag(−Iν , Irank(M)−ν , 0n−rank(M)

),

where ν is the number of negative eigenvalues. The triple

(ν, rank (M)− ν, n − rank (M))

is called the inertia of M.

M = LDLT ⇒ ν(M) = ν(D)

M − µI = LµDµLTµ ⇒ ν(µ) = ν(M − µI ) = ν(Dµ)



Complexity

Flops per factorization (for H`-matrices):

O(nk2 (log n)4

).

Factorization per eigenvalue:

O (log(‖M‖2 /ε)) .

Flops per eigenvalue:

O(nk2 (log n)4 log ‖M‖2 /ε

).

Slicing the whole spectrum:

O(n2k2 (log n)4 log(‖M‖2 /ε)

).



Absolute Error for H5(1)-Matrix

0 100 200 300 400 500 600 700 800 900 1 000

10−11

10−10

10−9

10−8

10−7

Eigenvalue

Ab

solu

teer

ror

abs. error12εev

Absolute error |λi − λi | for the 1 024× 1 024 H5(1)-matrix, εev = 10−8.



CPU Time for 10 Eigenvalues

104 105 106100

101

102

103

Dimension

Tim

ein

sO(n (log n)4) O(n (log n)2) O(n log n)

O(n) Computation Time

Computation times for 10 eigenvalues of H`(1)-matrices (` = 8, . . . , 15).



Parallelization

-6 -4 -2 -1 0 1 2 4 6

a0 = −6.5 b0 = 5.5µ1 = −0.5

ν(−0.5) = 4

-6 -4 -2 -1 0 1 2 4 6

a11 = −6.5 b11 = −0.5 = a21µ12 = −3.5

b21 = 5.5

µ22 = 2.5

ν(−3.5) = 2 ν(2.5) = 9



Parallelization Speedup

OpenMP:

Name n t1 core in s t1c/t2c t1c/t4c t8 core t1c/t8c

H2 r1 128 0.33 1.83 3.30 0.06 5.50H4 r1 512 9.44 1.94 3.67 1.43 6.60H6 r1 2 048 219.28 1.91 3.64 33.88 6.47H8 r1 8 192 4 022.80 1.87 3.44 676.57 5.95H10 r1 32 768 49 012.24 1.93 3.18 10 006.60 4.90



Parallelization Speedup

Open MPI: (H9(1) ∈ R16 384×16 384)

No. of Processes t in s Speedup Efficiency

1 16 564.58 1.00 1.002+1 8 340.22 1.99 0.664+1 4 044.13 4.10 0.826+1 2 678.95 6.18 0.88

11+1 1 494.33 11.08 0.9223+1 713.80 23.21 0.9635+1 476.44 34.77 0.9647+1 364.30 45.47 0.9595+1 188.92 87.68 0.91

191+1 100.61 164.64 0.86287+1 71.86 230.50 0.80383+1 61.91 267.56 0.70



H-Matrices

Shifting affects the structure.

The LDLT factorization is of almost linear complexity only forthe original H-matrix and not necessarily for the shifted ones.

For H`-matrices we use the exact LDLT factorization.For general H-matrices: The truncation introduces errors inthe LDLT factorization — the computed D may have anotherinertia⇒ some eigenvalues may lie outside the computed interval⇒ larger errors



Preconditioned Inverse Iteration

Preconditioned InverseIteration for Hierarchical

Matrices



Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]

Definition

The function

µ(x) = µ(x ,M) =xTMx

xT x

is called the Rayleigh quotient.

Minimize the Rayleigh quotient by a gradient method:

xi+1 := xi − α∇µ(xi ), ∇µ(x) =2

xT x(Mx − xµ(x)) ,

+ preconditioning ⇒ update equation:

xi+1 := xi − B−1 (Mxi − xiµ(xi )) .



Preconditioned Inverse Iteration[Knyazev, Neymeyr 2009]

xi+1 := xi − B−1 (Mxi − xiµ(xi ))

If

M ∈ Rn×n symmetric positive definite and

B−1 approximates the inverse of M, such that∥∥I − B−1M∥∥M≤ c < 1,

then Preconditioned INVerse ITeration (PINVIT) converges andthe number of iterations is independent of n.



Algorithm and Complexity

The number of iterations is independent of matrix size n.

H-PINVIT

Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)

Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H

O(n (log n)2 k (c)2)

R := MX0 − X0µ, µ = XT0 MX0

for (i := 1; ‖R‖F > ε; i + +) do

Xi := Orthogonalize(Xi−1 − B−1R

)

O(n (log n) k (c)2)

R := MXi − Xiµ, µ = XTi MXi

O(n (log n) k (c))

end

The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).

Competitive to MATLAB® eigs.

Expensive.



Algorithm and Complexity

The number of iterations is independent of matrix size n.

H-PINVIT

Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)

Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT

0 MX0

for (i := 1; ‖R‖F > ε; i + +) do

Xi := Orthogonalize(Xi−1 − B−1R

)O(n (log n) k (c)2)

R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))

end

The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).

Competitive to MATLAB eigs.

Expensive.



CPU Time

FE

M3D

4

FE

M3D

8

FE

M3D

16

FE

M3D

32

FE

M3D

64

FE

M3D

128

10−2

10−1

100

101

102

103

104C

PU

tim

ein

sH-Cholesky PINVITMATLAB eigs

10−2

10−1

100

101

102

103

104

64 512

409

6

3276

8

262

144

207

915

2



CPU Time

out

ofm

emor

y

FE

M3D

4

FE

M3D

8

FE

M3D

16

FE

M3D

32

FE

M3D

64

FE

M3D

128

10−2

10−1

100

101

102

103

104C

PU

tim

ein

sH-Cholesky PINVITMATLAB eigs

10−2

10−1

100

101

102

103

104

64 512

409

6

3276

8

262

144

207

915

2



CPU Time

out

ofm

emor

y

FE

M3D

4

FE

M3D

8

FE

M3D

16

FE

M3D

32

FE

M3D

64

FE

M3D

128

10−2

10−1

100

101

102

103

104C

PU

tim

ein

sH-Cholesky PINVITMATLAB eigsH-Cholesky decomposition

10−2

10−1

100

101

102

103

104

64 512

409

6

3276

8

262

144

207

915

2



CPU Time

out

ofm

emor

y

FE

M3D

4

FE

M3D

8

FE

M3D

16

FE

M3D

32

FE

M3D

64

FE

M3D

128

10−2

10−1

100

101

102

103

104C

PU

tim

ein

sH-Cholesky PINVITMATLAB eigsH-Cholesky decompositionSlicing the Spectrum

10−2

10−1

100

101

102

103

104

64 512

409

6

3276

8

262

144

207

915

2



Folded Spectrum Method

Folded Spectrum Method [Wang, Zunger 1994]

Mσ = (M − σI)2

Mσ is s.p.d., if M is s.p.d. and σ 6= λi .

(x − σ)2

The condition number of (M − σI )2 is large.⇒ The computation of M−1σ is more expensive.⇒ M−1σ has larger local ranks.⇒ M−1σ v is more expensive.

Multiple eigenvalues of Mσ may lead to incomplete subspaceinformation.

⇒ vTMv/vT v does not approximate λ.



Folded Spectrum Method

Folded Spectrum Method [Wang, Zunger 1994]

Mσ = (M − σI)2

Mσ is s.p.d., if M is s.p.d. and σ 6= λi .

The condition number of (M − σI )2 is large.

⇒ The computation of M−1σ is more expensive.

⇒ M−1σ has larger local ranks.

⇒ M−1σ v is more expensive.

Multiple eigenvalues of Mσ may lead to incomplete subspaceinformation.

⇒ vTMv/vT v does not approximate λ.



Conclusions

Conclusions



Slicing the Spectrum vs. LR Cholesky

102 103 104 105

10−210−1

1

10102103104105106

Tim

ein

s

Slicing the spectrumLR Cholesky algorithmLAPACK dsyev

102 103 104 10510−1010−610−2

102

Acc

ura

cy

H`(1)-matrices (` = 1, . . . , 10)

all eigenvalues



Slicing the Spectrum vs. LR Cholesky

102 103 104 105

10−210−1

1

10102103104105106

Tim

ein

s

Slicing the spectrumLR Cholesky algorithmLAPACK dsyev

102 103 104 10510−1010−610−2

102

Acc

ura

cy



Slicing the Spectrum vs. H-PINVIT

102 103 104 105 106

10−2

10−1

1

10

102

103

104

Tim

ein

s

Slicing the spectrumHPINVITMATLAB eigs

HPINVIT iterationHPINVIT preconditioner

102 103 104 105 10610−1010−610−2

102

Acc

ura

cy

H`(1)-matrices (` = 1, . . . , 15)

three smallest eigenvalues



Slicing the Spectrum vs. H-PINVIT

102 103 104 105 106

10−2

10−1

1

10

102

103

104

Tim

ein

s

Slicing the spectrumHPINVITMATLAB eigs

HPINVIT iterationHPINVIT preconditioner

102 103 104 105 10610−1010−610−2

102

Acc

ura

cy



Conclusions

Three algorithms:H-LR Cholesky algorithm: efficient only forH`-matrices, but expensive otherwise

Slicing the spectrum: efficient only forH`-matrices, good parallelizable

H-PINVIT: efficient for the smallest eigenvalue ofpositive definite H-matrices, computation of innereigenvalues is difficult

Thank you for your attention.


Eigenvalues of Symmetrix Hierarchical Matrices

Technology

Transcript of Eigenvalues of Symmetrix Hierarchical Matrices