Laplacian Matrices of...

Laplacian Matrices of Graphs

Daniel A. Spielman Dept. of Computer Science

Program in Applied Mathematics Yale Institute for Network Science

MTNS, July 7, 2014

Outline

Examples of graphs and networks Physical metaphors for graphs and the Laplacian Some applications Algorithms and theorems

Graphs and Networks V: a set of vertices (nodes) E: a set of edges an edge is a pair of vertices

Dan

Donna Allan

Gary Maria

Nikhil

Shang-Hua

A small social network

Graphs and Networks

A big social network

Theorem: There is no nice picture of this network

from: http://nrc.uchsc.edu/STATES/united-states-map.jpg

Examples of Graphs

Nearest neighbor graphs of points

Nearest neighbor graphs from data

Cartoon Graphs

Path

Grid

Examples of Graphs

8

5

1

7 3 2

9 4 6

10

8

5

1

7 3 2

9 4 6

10

Dan

Amy Allan

Gary Maria

Nikhil

Shang-‐Hua

Examples of Graphs

How to understand large-scale structure

Use physical metaphors Edges as rubber bands Edges as resistors

Examine processes

Diffusion of gas Spilling paint

Identify structures

Communities Hierarchy

Nail down some verEces, let rest seIle

View edges as rubber bands or ideal linear springs spring constant 1 (for now)

Graphs as Spring Networks


View edges as rubber bands or ideal linear springs spring constant 1 (for now)


When stretched to length potenEal energy is

`

�2/2

Nail down some verEces, let rest seIle.

Physics: posiEon minimizes total potenEal energy

subject to boundary constraints (nails)

x(a)a

1

2

X

(a,b)2E

(x(a)� x(b))2



Energy minimized when free verEces are averages of neighbors

x(a)a

�x(a) =1

da

X

(a,b)2E

�x(b)


is degree of , the number of attached edges da a

If nail down a face of a planar 3-‐connected graph, get a planar embedding!

Tutte’s Theorem ‘63

3-‐connected: cannot break graph by cuYng 2 edges

Tutte’s Theorem ‘63

Tutte’s Theorem ‘63 3-‐connected: cannot break graph by cuYng 2 edges

Graphs as Resistor Networks

View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow.

1V

0V


View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow. Induced voltages minimize

X

(a,b)2E

(v(a)� v(b))2

Subject to fixed voltages (by battery)


View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow.

1V

0V

0.5V

0.5V

0.625V 0.375V


View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow. Current flow measures strength of connection between endpoints. More short disjoint paths lead to higher flow.

Learning on Graphs [Zhu-‐Ghahramani-‐Lafferty ’03]

Infer values of a function at all vertices from known values at a few vertices.

Minimize

Subject to known values

X

(a,b)2E

(x(a)� x(b))2

0

1

Learning on Graphs [Zhu-‐Ghahramani-‐Lafferty ’03]

Infer values of a function at all vertices from known values at a few vertices.

Minimize

Subject to known values

X

(a,b)2E

(x(a)� x(b))2

0

1 0.5

0.5

0.625 0.375

The Laplacian quadratic form X

(a,b)2E

(x(a)� x(b))2

x

TLx =

X

(a,b)2E

(x(a)� x(b))2

The Laplacian matrix of a graph

There is a symmetric matrix L so that

Weighted Graphs

Edge assigned a non-negative real weight measuring strength of connection spring constant 1/resistance

wa,b 2 R

x

TLx =

X

(a,b)2E

wa,b(x(a)� x(b))2

(a, b)


x

TLx =

X

(a,b)2E

wa,b(x(a)� x(b))2

1! 2!

3!4!


To minimize subject to boundary constraints, set derivative to zero. Solve equation of form

Lx = b

x

TLx =

X

(a,b)2E

wa,b(x(a)� x(b))2

Can solve Laplacian equations absurdly quickly

Lx = b

Asymptotically best algorithm (today) takes time

O(mplog n log(1/✏))

Where m is number of non-zeros and n is dimension (Cohen-Kyng-Pachocki-Peng-Rao ’14)

Can solve Laplacian equations absurdly quickly

Lx = b



Where m is number of non-zeros and n is dimension (Cohen-Kyng-Pachocki-Peng-Rao ’14)

Good code: LAMG (lean algebraic multigrid) – Livne-Brandt CMG (combinatorial multigrid) – Koutis

Measuring boundaries of sets

Boundary: edges leaving a set

S S



S

0 0

0 0

0 0

1

1 0

1 1

1 1 1

0

0 0

1

S

Characteristic Vector of S:

x(a) =

(1 a in S

0 a not in S



S

0 0

0 0

0 0

1

1 0

1 1

1 1 1

0

0 0

1

S

Characteristic Vector of S:

x(a) =

(1 a in S

0 a not in S

x

TLx =

X

(a,b)2E

(x(a)� x(b))

2= |boundary(S)|

Eigenvalues and Eigenvectors A n-by-n symmetric matrix has n real eigenvalues and eigenvectors such that

�1 � �2 · · · � �n

v1, ..., vn

Lvi = �ivi

Spectral Graph Theory A n-by-n symmetric matrix has n real eigenvalues and eigenvectors such that

�1 � �2 · · · � �n

v1, ..., vn

These eigenvalues and eigenvectors tell us a lot about a graph!

Lvi = �ivi

Spectral Graph Drawing [Hall ‘70]

31 2

4

56 7

8 9

Arbitrary Drawing

Spectral Graph Drawing Plot vertex at draw edges as straight lines

[Hall ‘70]

(v2(a), v3(a))a

31 2

4

56 7

8 9

12

4

5

6

9

3

8

7Arbitrary Drawing

Spectral Drawing

A Graph

Drawing of the graph using v2, v3

Plot vertex at a (v2(a), v3(a))

The Airfoil Graph, original coordinates

The Airfoil Graph, spectral coordinates

Spectral drawing of Streets in Rome

Spectral drawing of Erdos graph: edge between co-authors of papers

Dodecahedron

Best embedded by first three eigenvectors

Graph Partitioning

Spectral Graph Partitioning

for some

[Donath-Hoffman ‘72, Barnes ‘82, Hagen-Kahng ‘92]

S = {a : v2(a) � t} t

Spectral Graph Partitioning

Performance guarantees come from Cheeger’s Inequality

[Donath-Hoffman ‘72, Barnes ‘82, Hagen-Kahng ‘92]

Spectral Image Segmentation (Shi-Malik ‘00)

200 400 600 800 1000 1200

100

200

300

400

500

600

700

800

900


2 4 6 8 10 12 14 16

2

4

6

8

10

12


2 4 6 8 10 12 14 16

2

4

6

8

10

12

edge weight

The second eigenvector

50 100 150 200 250 300

50

100

150

200

Second eigenvector cut

50 100 150 200 250 300

50

100

150

200

Third Eigenvector

50 100 150 200 250 300

50

100

150

200

50 100 150 200 250 300

50

100

150

200

Fourth Eigenvector

50 100 150 200 250 300

50

100

150

200

50 100 150 200 250 300

50

100

150

200

Laplacian Linear Systems Lx = b

Algorithms

Solve in Eme where = number of non-‐zeros entries of L

O(m log

c m)

m

(S-‐Teng ’04, ‘14)


Algorithms

Solve in Eme where = number of non-‐zeros entries of L Emes for -‐approximate soluEon.

O(m log

c m)

m

log(1/�) ✏

��x� L�1b��L �

��L�1b��L

(S-‐Teng ’04, ‘14)


Algorithms

Solve in Eme where = number of non-‐zeros entries of L Emes for -‐approximate soluEon.

O(m log

c m)

m

log(1/�) ✏

��x� L�1b��L �

��L�1b��L

kxkL =px

TLx

(S-‐Teng ’04, ‘14)

Algorithms

Two main ingredients: Graph Sparsifiers

Low Stretch Spanning Trees

(S-‐Teng ’04, ‘14)

Approximating Graphs

for all

A graph H is an -approximation of G if ✏

1

1 + � xTLHx

xTLGx 1 + �

x

Approximating Graphs

for all x

A very strong notion of approximation Preserves all electrical and spectral properties Preserves boundaries of every set

A graph H is an -approximation of G if ✏

1

1 + � xTLHx

xTLGx 1 + �

Sparsifying Graphs

for all 1

1 + � xTLHx

xTLGx 1 + �

x

(Batson-S-Srivastava ’09) Every graph G has an -approximation H with edges

A graph H is an -approximation of G if

n(2 + ✏)2/✏2

✏

✏

for all 1

1 + � xTLHx

xTLGx 1 + �

x

(Batson-S-Srivastava ’09) Every graph G has an -approximation H with edges (S-Srivastava ‘08) Can quickly find one with edges by sampling according to effective resistances

A graph H is an -approximation of G if

n(2 + ✏)2/✏2

Sparsifying Graphs ✏

✏

Cn log n/✏2

If , kAk < 1 (I �A)�1 = I +A+A2 +A3 + · · ·

=Y

i�0

(I +A2i)

Solving Equations (S-‐Peng ‘14)

If , kAk < 1 (I �A)�1 = I +A+A2 +A3 + · · ·

=Y

i�0

(I +A2i)

If , kAk < 1 A2i ! 0

⇡kY

i=0

(I +A2i)

Sparsify

Solving Equations (S-‐Peng ‘14)

I +A2i

Spanning trees of graphs

Connected No cycles Have a unique path between every two verEces

Connected No cycles Have a unique path between every two verEces

Spanning trees of graphs

The stretch of a spanning tree

stT (G) =X

(a,b)2E

path-lengthT (a, b)

(Alon-‐Karp-‐Peleg-‐West ’91)

path-‐len 5

stT (G) =X

(a,b)2E

path-lengthT (a, b)


path-‐len 3

stT (G) =X

(a,b)2E

path-lengthT (a, b)


path-‐len 1

stT (G) =X

(a,b)2E

path-lengthT (a, b)


stT (G) =X

(a,b)2E

path-lengthT (a, b)

For every G there is a T with stT (G) O(m log n log log n)


(Elkin-‐Emek-‐S-‐Teng ‘04, Abraham-‐Bartal-‐Neiman ’08, Abraham-‐Neiman ‘12)

stT (G) =X

(a,b)2E

path-lengthT (a, b)

For every G there is a T with stT (G) O(m log n log log n)

And, we can compute it in Eme O(m log n log log n)

(Elkin-‐Emek-‐S-‐Teng ‘04, Abraham-‐Bartal-‐Neiman ’08, Abraham-‐Neiman ‘12)


Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13

Want the flow saEsfying demands that minimizes

Kirchoff: is the flow saEsfying demands such that

for all cycles C in the graph

X

(a,b)2C

fa,bra,b = 0,

X

(a,b)2E

f2a,bra,b


Push flow around a cycle unEl it saEsfies KCL. This decreases the energy. To decide which cycles, choose a low-‐stretch tree. Each non-‐tree edge determines a cycle.

X

(a,b)2C

fa,bra,b = 0,


Choose off-‐tree edge with prob

Push flow around that cycle to minimize energy and saEsfy Kirchoff.

With right data structure, each operaEon takes logarithmic Eme.

Solve in Eme O(m log

2 m log logm log ✏�1)

pa,b ⇠ path-lengthT (a, b)

1.0

1.0

1.0

1.0 1.0

1.0 1.0

1.0

0 0


3.0 3.0

Start with a simple flow

1.0

1.0

1.0

1.0 1.0

1.0 1.0

1.0

0 0


3.0 3.0

Minimize energy on a cycle

0.8

0.8

0.8

1.2 1.2

1.0 1.0

1.0

0 0


3.0 3.0

Minimize energy on a cycle and saEsfy Kirchoff

0.8

0.8

0.8

1.2 1.13

1.0 1.07

1.0

0 0.07


3.0 3.0

Minimize energy on another cycle

0.8

0.8

0.8

1.13 1.13

1.07 1.07

1.0

0.07 0.07


3.0 3.0


0.8

0.8

0.8

1.13 1.13

1.07 1.07

0.71

0.36 0.36


3.0 3.0


0.83

0.83

0.83

1.24 1.24

0.93 0.93

0.62

0.31 0.31


3.0 3.0

Number of iteraEons is proporEonal to the stretch

Fast Laplacian Solvers



(Cohen-Kyng-Pachocki-Peng-Rao ’14)

Good code: LAMG (lean algebraic multigrid) – Livne-Brandt CMG (combinatorial multigrid) – Koutis

A powerful computational primitive!

My web page on: Laplacian linear equations, sparsification, local graph clustering, low-stretch spanning trees, and so on.

My class notes from “Graphs and Networks” and “Spectral Graph Theory”

Lx = b, by Nisheeth Vishnoi

To learn more

Laplacian Matrices of...

Documents

Transcript of Laplacian Matrices of...