Spectral Graph Theory

Aaron Mishtal

April 27, 2016

1 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

2 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

3 / 36

OverviewWhat is spectral graph theory?

I The study of eigenvalues and eigenvectors of graphs.

I The multiset of eignevalues of a matrix is called its spectrum.

I If the matrix corresponds to a graph G, then the spectrum ofthe matrix is also called the spectrum of G.

I A graph’s spectrum provides insight into its structure andproperties that it possesses.

4 / 36

OverviewAn analogy

I Astronomers discover a new star, and want to learn about itscomposition.

I It’s obviously too far away to gather physical samples, and fartoo dangerous even if they could get there.

I Instead, they can observe the light emitted by the star, itsspectrum.

I They have models that tell them what the spectrum of a starshould look like depending on its temperature, and moremodels that relate temperature to composition.

stars ↔ graphsprisms, telescopes, models ↔ linear algebra, graph theory

5 / 36

OverviewAn analogy

I Astronomers discover a new star, and want to learn about itscomposition.

I It’s obviously too far away to gather physical samples, and fartoo dangerous even if they could get there.

I Instead, they can observe the light emitted by the star, itsspectrum.

I They have models that tell them what the spectrum of a starshould look like depending on its temperature, and moremodels that relate temperature to composition.

stars ↔ graphsprisms, telescopes, models ↔ linear algebra, graph theory

6 / 36

OverviewAn analogy

I Astronomers discover a new star, and want to learn about itscomposition.

I It’s obviously too far away to gather physical samples, and fartoo dangerous even if they could get there.

I Instead, they can observe the light emitted by the star, itsspectrum.

I They have models that tell them what the spectrum of a starshould look like depending on its temperature, and moremodels that relate temperature to composition.

stars ↔ graphsprisms, telescopes, models ↔ linear algebra, graph theory

7 / 36

OverviewAn analogy

I Astronomers discover a new star, and want to learn about itscomposition.

I It’s obviously too far away to gather physical samples, and fartoo dangerous even if they could get there.

I Instead, they can observe the light emitted by the star, itsspectrum.

I They have models that tell them what the spectrum of a starshould look like depending on its temperature, and moremodels that relate temperature to composition.

stars ↔ graphsprisms, telescopes, models ↔ linear algebra, graph theory

8 / 36

OverviewAn analogy

I Astronomers discover a new star, and want to learn about itscomposition.

I It’s obviously too far away to gather physical samples, and fartoo dangerous even if they could get there.

I Instead, they can observe the light emitted by the star, itsspectrum.

I They have models that tell them what the spectrum of a starshould look like depending on its temperature, and moremodels that relate temperature to composition.

stars ↔ graphsprisms, telescopes, models ↔ linear algebra, graph theory

9 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

10 / 36

Linear Algebra PrimerEigenvalues and Eigenvectors

Let A ∈ Rn×n be a real, square matrix with n rows and n columns.

The matrix A represents a linear transformation betweenn-dimensional vectors.

In other words, given x ∈ Rn, there exists a y ∈ Rn such that

Ax = y

If y = λx for some scalar λ, then x is an eigenvector of A withcorresponding eigenvalue λ. We can rewrite the above equation as

Ax = λx

11 / 36

Linear Algebra PrimerSo what?

I So... eigenvalues and eigenvectors just let us rewrite anequation?

I Yes! But this rewriting shows exactly what makeseigenvectors interesting.

I Linear transformations typically both rotate and scale vectorswhen they are applied.

I Eigenvectors are special in that they are only scaled, and theireigenvalue is the scaling factor.

12 / 36

Linear Algebra PrimerVisual Example

Consider applying a linear transformation to every point of a 2Dgrid.

The red vector (and the pixel it points to) gets rotated andstretched.But the blue vector is an eigenvector, since its direction remainsunchanged.

13 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

14 / 36

HistoryEigenvalues and Eigenvectors: 1700s

I Originally arose in the study of physicalsystems.

I Leonhard Euler noted the importance ofprincipal axes in the rotation of rigid bodies.

I Joseph-Louis Lagrange later discovered thatthese principle axes are the eigenvectors ofthe inertia matrix.

15 / 36

HistoryEigenvalues and Eigenvectors: 1800s

I Augustin-Louis Cauchy extended andgeneralized the work of Euler and Lagrange.Referred to eigenvalues as characteristicroots.

I Joseph Fourier extended the existing work todifferential equations, allowing him to solvethe heat equation in 1822.

I Eigenvalues and eigenvectors of matricesstart to be studied for their own sake.

I Cauchy proves that real symmetric matriceshave real eigenvalues.

16 / 36

HistoryEigenvalues and Eigenvectors: 1900s

I David Hilbert first used the German wordeigen, meaning “own”, “inherent”, or“characteristic”, in reference to eigenvaluesin 1904, which eventually replaced the thenpopular term of proper values.

I Hilbert extended the concept to infinitematrices.

I First numerical algorithms for findingeigenvalues developed by Richard von Misesin 1929 (power method).

17 / 36

HistorySpectral Graph Theory

I Appeared as a branch of algebraic graph theory in the 1950sand 1960s.

I Early work focused on using the adjacency matrix, whichlimited initial results to regular graphs.

I Research was independently begun in quantum chemistry, aseigenvalues of graphical representation of atoms correspond toenergy levels of electrons.

I In 1973, Czech mathematician Miroslav Fielder developed theconcept of algebraic connectivity (2nd eigenvector ofLaplacian). The corresponding eigenvector is named after him.

18 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

19 / 36

Some More Linear Algebra

I In general, eigenvalues and eigenvectors can have imaginarycomponents.

I Real, symmetric n × n matrices will always have n realeigenvalues, and a corresponding set of n linearly independenteigenvectors.

I Eigenvectors are not unique. A scalar multiple of aneigenvector is also an eigenvector.

I The spectrum of a matrix is generally a multiset. Eigenvaluescan be repeated.

20 / 36

Matrix Representation of Graphs

Assume di is the degree of vertex i . There are three matrixrepresentations are commonly used:

I The adjacency matrix

Aij ={

1 if i and j are adjacent0 otherwise

I The Laplacian matrix

Lij =

{di if i = j−1 if i and j are adjacent0 otherwise

I The normalized Laplacian matrix

Lij =

1 if i = j and di 6= 0− 1√

didj

if i and j are adjacent

0 otherwise

21 / 36

Adjacency Matrix

I Adjacency matrices were the first matrices to be explored inspectral graph theory.

I They work nicely with regular graphs, but results proveddifficult to generalize to general graphs.

I We’ll denote the eigenvalues of the adjacency matrix Acorresponding to graph G as

α1 ≥ α2 ≥ · · · ≥ αn

22 / 36

Laplacian Matrix

I The eigenvalues of the Laplacian and adjacency matricesbehave similarly as long as the graph is regular.

I Once the Laplacian became popular, spectral graph theorybecame much more widely applicable.

I The vector of all ones is always an eigenvector of L, withcorresponding eigenvalue of 0.

I We’ll denote the eigenvalues of the Laplacian matrix Lcorresponding to graph G as

0 = λ1 ≤ λ2 ≤ · · · ≤ λn

23 / 36

Normalized Laplacian Matrix

I A degree-normalized version of the Laplacian.

I Closely related to the walk or diffusion matrix of a graph,another matrix representation of a graph that reflects thedynamics of a random walk on.

I We won’t be discussing this matrix in detail.

24 / 36

What can we learn from a graph’s spectrum?Basic Facts

I α1 is at least as large as the average degree, and no largerthan the maximum degree.

I If G is connected, then α1 > α2.

I The vector of all ones is an eigenvector of A with eigenvalueα1 iff G is an α1−regular graph.

I The number of times 0 appears in the spectrum of L is equalto the number of connected components of G .

I λn is at most twice the maximum degree of a vertex.

I αn = −α1 iff G is bipartite.

25 / 36

What can we learn from a graph’s spectrum?How to draw the graph.

I We can use the second and third eigenvectors to ascoordinates for vertices, which can often lead to nice visualrepresentations of graphs.

I Doesn’t always work out!

26 / 36

Algebraic Connectivity

I λ2 grows as the graph is more connected.

I We can use the corresponding eigenvector to partition thegraph.

I Suppose v2 is the eigenvector of L corresponding to λ2, theFiedler vector.

I Fiedler proved that if we select a value t ≤ 0, then the set ofvertices i such that v2(i) ≥ t form a connected component.

27 / 36

Isospectral Graphs

I It’s possible for two non-isomorphic graphs to have the samespectrum.

I The graphs may look very different, but share fundamentalproperties.

I A graph’s spectrum is invariant to relabling of vertices, so ofcourse isomorphic graphs are also isospectral.

28 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

29 / 36

Applications

I Gaining insight into diffusion processes on a graph.

I Identifying important vertices (PageRank), bottlenecks, andother graph structures.

I Spectral clustering of data using a similarity matrix of thedata.

I Putting bounds on graph properties that can be efficientlycomputed.

I Many applications in physics and other sciences in general.

30 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

31 / 36

Open Problems

I Conjecture: If A is the adjacency matrix of a d-regulargraph, then there is a symmetric signing of A (replacing +1entries with −1) so that the resulting matrix has alleigenvalues of magnitude at most 2

√d − 1.

I Are almost all graphs determined by their spectrum?

32 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

33 / 36

Homework Problems

1. Find the (Laplacian) spectrum of the Peterson graph.

2. Find the (Laplacian) spectrum of Kn.

3. Prove that a graph’s spectrum is invariant under relabelingthe vertices.

34 / 36

Outline

Overview

Linear Algebra Primer

History

Theory

Applications

Open Problems

Homework Problems

References

35 / 36

References

I http://en.wikipedia.org/wiki/Spectral_graph_theory

I http://en.wikipedia.org/wiki/https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors

I http://www.cs.yale.edu/homes/spielman/561/

I http://www.math.ucsd.edu/~fan/research/revised.html

I http://www.math.ucsd.edu/~fan/cbms.pdf

36 / 36