spectral clustering between friends

One of these things is not like the other…

spectral clustering (a la Ng-Jordan-Weiss)

data similarity graphedges have weights w(i,j)

e.g.

the Laplacian

diagonal matrix D

Normalized Laplacian:

energy

Normalized Laplacian:

spectral embedding

Normalized Laplacian:

Compute first k eigenvectors: v1, v2 , …, vk

clustering

Run k–means to cluster the points

spectral clustering

Sidi, et. al. 2011 [TelAviv-SFU]

Many, many variants…

it’s amazin

g!it’s mediocre!

it’s antiquated

Many opinions

… what to prove?

why should spectral clustering work?

spectral embedding

k perfect clusters

graph expansion

Expansion: For a subset S µ V, define

E(S) = set of edges with one endpoint in S.

S

graph expansion

Expansion: For a subset S µ V, define

E(S) = set of edges with one endpoint in S.

S1

Theorem [Cheeger70, Alon-Milman85, Sinclair-Jerrum89]:

¸2

2· ½G (2) ·

p2̧ 2

½G (k) = minfmaxÁ(Si ) : S1;S2; : : : ;Sk µ V disjointg

k-way expansion constant:

S2

S3

S4

“most important result in spectral graph theory” -- Wikipedia

Miclo’s conjecture

Higher-order Cheeger Conjecture [Miclo 08]:

¸k

2· ½G (k) · C(k)

p¸k

for some C(k) depending only on k.

For every graph G and k 2 N, we have

[Lee-OveisGharan-Trevisan 2012]:

True with

This bound for C(k) is tight.

Algorithm of Ng-Jordan-Weiss works, changing the last step.

S1

S2

S3

S4

the clustering step

Run k–means to cluster the points

we do random projection

random space partition

Miclo’s conjecture

Higher-order Cheeger Conjecture [Miclo 08]:

¸k

2· ½G (k) · C(k)

p¸k

for some C(k) depending only on k.

For every graph G and k 2 N, we have

[Lee-OveisGharan-Trevisan 2012]:

True with

This bound for C(k) is tight.

Algorithm of Ng-Jordan-Weiss works, changing the last step.

S1

S2

S3

S4

hybrid algorithms

Suppose the data has some nice low-dimensional structure

Spectral embedding could losethat information:Back in a high-dimensional space

hybrid algorithms

Suppose the data has some nice low-dimensional structure

Use spectral embedding distances to deform the data

Do clustering on transformed data set

unraveling the mysteries of complexity

the unique games conjecture

Consider linear equations in two variables, modulo a prime p

Variables: x1, x2, …, xn

x12 + x2 = 4

x4 – 3 x7 = 1

x9 + 8 x12 = 9…

If there exists a solution that satisfies 99% of the equations,can you find one that satisfies 10%?

Conjectured to be NP-hard [Khot 2002]

a spectral attack

Construct a graph with one vertex for every variable, and anedge whenever two variables occur in the same constraint.

x12 + x2 = 4

x4 – 3 x7 = 1

x9 + 8 x12 = 9…A “good” solution to the equations implies a partition of thegraph into p nice clusters!

a spectral attack

Higher-order Cheeger Theorem:

For every graph G and k 2 N, we haveS1

S2

S3

S4

Unnecessary for large k:

[Arora-Barak-Steurer 2010]

A better asymptotic dependence would disprove the UGC.