Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with...
-
Upload
natalie-stevenson -
Category
Documents
-
view
215 -
download
0
Transcript of Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with...
![Page 1: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/1.jpg)
Institute for Advanced Study, April 16 2012
Sushant SachdevaPrinceton University
Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi
Linear Time Graph Partitioningvia Fast Simulation of Random walks
![Page 2: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/2.jpg)
INTRODUCTIONTheory of near-linear time algorithms
![Page 3: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/3.jpg)
Why linear-time algorithms?
Input graphs and data sets are massive.e.g. Web graph (1011 nodes), Social networks (109 nodes)
Super-linear algorithms are impractical.
© 2004–2012 Michael K. Bergman.
Need time algorithms for fundamental problems:e.g. MAX-MATCHING, s-t MINCUT/MAXFLOW, Balanced Graph Partitioning
Approximate solutions maybe?
(1-²) approx., time[Duan and Pettie]
(1-²) approx., time[Christiano et al.]
Spectral approx., time
[This talk]
![Page 4: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/4.jpg)
GRAPH PARTITIONING
![Page 5: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/5.jpg)
Conductance
Given a undirected, unweighted graph
S Conductance of
[Cheeger, Alon-Milman]
[Spielman-Teng] allows us to approximate ¸2(L) and find such a cut in
time
Assume G is d-regularSpectral approximation
Conductance of G
Fundamental quantity in Markov Chains, Riemannian
Manifolds
![Page 6: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/6.jpg)
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Applications to clustering, image segmentation, community detection,
primitive for divide-and-conquerAlgorithm Techniqu
eDistinguishes ≤ ° and
Running Time
Recursive Eigenvector
Spectral
Use Cheeger to find a cut and
remove it. Rinse and repeat.
Can take time. Too slow.
![Page 7: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/7.jpg)
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Algorithm Technique
Distinguishes ≤ ° and
Running Time
Recursive Eigenvector
Spectral
[Leighton-Rao] Flow [AK,OSVV,CKM+]
[Arora-Rao-Vazirani]
SDP [AK,She,CKM+]
[Madry]
![Page 8: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/8.jpg)
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Algorithm Technique Distinguishes ≤ ° and
Running Time
Recursive Eigenvector
Spectral
[Spielman-Teng] Local Random walks
[Andersen-Peres] Evolving Sets
[Orecchia-Vishnoi] SDP
![Page 9: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/9.jpg)
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Algorithm Technique Distinguishes ≤ ° and
Running Time
Recursive Eigenvector
Spectral
[Spielman-Teng] Local Random walks
[Andersen-Peres] Evolving Sets
[Orecchia-Vishnoi] SDP
[This talk] Simulating Random walks
![Page 10: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/10.jpg)
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Algorithm Technique
Distinguish ≤ ° and
Running Time
Recursive Eigenvector
Spectral
[Leighton-Rao] Flow [AK,OSVV,CKM+]
[Arora-Rao-Vazirani]
SDP [AK,She,CKM+]
[Madry]
[This talk] Simulating random walks
![Page 11: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/11.jpg)
Overview
1. Use continuous-time random walks to give an algorithm
2. Show how to simulate continuous-time random walks fast
![Page 12: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/12.jpg)
Graph Cuts and Random Walks
Intimate connection between Graph cuts and Random walks.
[Alon-Milman][Mihail][Lovasz-Simonovits][Spielman-Teng][Anderson-Chung-Lang][Andersen-Peres][Orecchia-Vishnoi][this talk]…
[Mihail] Given the distribution of a random walk that has mixed, can find a cut S with
in time.
Usual Random Walks on a graph
Page Rank Random walksEvolving Sets Random WalksContinuous Time Random Walks
![Page 13: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/13.jpg)
Bird’s eye view1. Simulate random walks (one or several)
3. Else, modify the graph
Regular walk / Page Rank walk /Evolving Sets /Continuous-time walks
2. Try to cut the graph using the random walksIf there’s no low conductance cut, we get a certificate.If you find a balanced low conductance cut, we’re done. To remove the unbalanced low conductance cuts
Remove the set
Add edges across the cut
Using matchings /flows /stars
Threshold cuts/SDP rounding
![Page 14: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/14.jpg)
Continuous Time Random Walks
Thm. We can reduce Balanced Separator to simulating
heat-kernel random walks with
Given initial distribution ,
Random walk of length l ~ Poisson(t)
Heat-Kernel random walk
Why heat-kernel random walk?
e-x is a nice function. Track progress using Golden Thompson inequality [AK]Don’t know such a result for other random walks
Techniques from SDPs[OV], exponential updates aka Matrix Mult. Weights [AK]
Previous results had a poly(1/°) dependence
![Page 15: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/15.jpg)
Algorithm
Wej
An embedding in Random walk from vertex i
Case 1: All vectors are small
Long vectors are random walks that
haven’t mixed
Wei
All random walks have mixed well.
No small cuts.
Ensures mixing across cuts with
conductance more than °
![Page 16: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/16.jpg)
Algorithm
Case 2: Walks have not mixed
Wej
WeiRandom Projection
+Threshold Cut
“Cheeger Rounding”
If we find a balanced cut of small conductance , we’re
done.ELSE
![Page 17: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/17.jpg)
Algorithm
Case 2b: Walks have not mixed (and we didn’t find a balanced cut)
Wej
WeiWe can find a ball cut S
S
S is the union of “all unbalanced low
conductance cuts”
![Page 18: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/18.jpg)
Algorithm
Case 2b: Walks have not mixed (and we didn’t find a balanced cut)
Wej
WeiWe can find a ball cut S
SAdd edges from all vertices outside the ball to all inside
Soft removal of unbalanced cuts
O(log n) rounds
For efficiency, we use JL lemma and work with
O(log n) dimensional embeddings. [AK]Goal: Given L,¿,v. Compute Goal: Given L,¿ ,v. Approximate
![Page 19: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/19.jpg)
SIMULATING RANDOM WALKS
![Page 20: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/20.jpg)
Approximating Heat-Kernel
Goal: Given ¿, L, v, ±, find u s.t.
Taylor Series Approx.?
Need
Too Slow
Heat-Kernel random walk required by
algorithm
Small error suffices for the algorithm Used in
previous algorithms
![Page 21: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/21.jpg)
Approximating Heat-Kernel
Better polynomials?
YES!!
[Thm] There exists a polynomial p of degree such that
It suffices to approximate e-x on
[Thm] Given G that has a balanced cut of conductance , we can find one of conductance
in time Better guarantee, same
running time as [Andersen-Peres]
Still not near-linear!
Goal: Given ¿, L, v, ±, find u s.t.
![Page 22: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/22.jpg)
Approximating Heat-Kernel
[Thm] Any polynomial such that requires degree
Even better polynomials?
Not Really!
Simple proof using Markov’s theorem
Goal: Given ¿, L, v, ±, find u s.t.
![Page 23: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/23.jpg)
Beyond PolynomialsRational Functions![Saff-Schönhage-Varga] There is a degree k polynomial pk such that
Infinite Interval!
Geometric Decay!Need
degree
Assume: 1. We knew pk explicitly 2. ST computation was exact
We’re DONE!
Two Issues: 1. [SSV] result is existential
2. ST computation is approximate
Approximating up to error ±
Computing vectors
Can use [Spielman-Teng]![Thm] Given ¿, L, v, ±, we can compute u such
that in
time
[Spielman-Teng] (Informal) We can approximate (cI+L)-1y in
time.
![Page 24: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/24.jpg)
LANCZOS’ METHOD“But we don’t know the polynomial.”
![Page 25: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/25.jpg)
Lanczos’ MethodLet
Define f so that
Goal: Approximate f(B)v as well as any degree k poly.
Lanczos Method
Existence of a good approximate poly suffices
Small degree implies efficiency
From real approximation to matrices
![Page 26: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/26.jpg)
Lanczos’ Method
Goal: Approximate f(B)v as well as any degree k poly.
Observe
p – any degree k polynomial
Krylov Subspace · of
order k
Let be an orthonormal basis for ·
Let T be the operator B restricted to · (k+1) x (k+1) matrix
For any degree k polynomial p,
Let Projection on · Our guess:
T is a much smaller matrix!
![Page 27: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/27.jpg)
Error Guarantee
Goal: Bound the norm of the error
Let the error be
Equal!
Since T is B restricted to ·,Spectrum(T) is bounded by Spectrum(B)
* error of best degree k polynomial approximation
Cost = k mult. by B + Construct basis + Diagonalize T
for our setting
Suppose p(x) is a good degree k approximation to f
Bounded by r on Spectrum(B)
Bounded by r on Spectrum(T)
Don’t even need to know the polynomial!
![Page 28: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/28.jpg)
HANDLING APPROXIMATE COMPUTATION
What about the error?
![Page 29: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/29.jpg)
We bound the error
Why Lanczos is insufficient
Multiplication withis only approximate
Subspace · is approximate.& Operator T is
approximate
T is not even symmetric!
We lose nice spectral
properties
[ST] computation is only approximate
We symmetrize.Compute approximation with
We needed to compute an orthonormal basis for
![Page 30: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/30.jpg)
Sketch of Error AnalysisBound
Bound BoundBound
Polynomial part
Error part
Rational approx. result
BoundBound sum of coefficients of
pk
Bound spectrum
of
Error Analysis + Lanczos method = Fast random walks
Fast random walks + Algorithm = Fast Balanced Separator
![Page 31: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/31.jpg)
Conclusions
• Reduced balanced separator to simulating heat-kernel random walks
• Approximated the heat kernel random walk in time.
• Can be used as primitives for designing near-linear time algorithms.
![Page 32: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/32.jpg)
Open Questions
• Other applications of fast matrix exponentiation
• Linear-time Graph decomposition?• Linear time algorithms for small set
expansion?• Linear time algorithm for
approximation for Balanced Separator
![Page 33: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/33.jpg)
Thank you!
![Page 34: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/34.jpg)
An Important Tool[Christiano et al.]
s-t MAXFLOW s-t MINCUT
Electrical Flows
Solve Linear Systems
Lx=y
L is a graph Laplacian
Solving Laplacian Linear Systems
Direct Methods
Gaussian Elimination /
Cholesky Decomposition
Iterative methods
Too slow. O(n3) time
Hopeless
Conjugate Gradient
Approximate method.
time
[Spielman-Teng] (Informal) We can approximate L-1y in time.
Other Applications – Approximating second eigenvector [Spielman-Teng], Cover times [Ding-Lee-
Peres], Generating random spanning trees[Madry-Kelner], Graph Sparsification [Spielman-Srivastava]
Balanced graph partitioning,Simulating continuous-time random walks
[This talk]
![Page 35: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/35.jpg)
Beyond Polynomials
Rational Functions!
[Saff-Schönhage-Varga] There is a degree k polynomial pk such that
Infinite Interval!
Geometric Decay!Small required
degree.
L1 bound on [0,1)
L1 bound on [-1,1]
L1 bound on [-1,1]
L2bound on [-1,1]
Change IntervalWrite function as an
integralCauchy Schwarz
Assume: 1. We knew pk explicitly 2. ST Solver was exact
We’re DONE!
Two Issues: 1. We don’t know pk explicitly 2. ST Solver
is approximate
![Page 36: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/36.jpg)
Faster SimulationPolynomials don’t suffice, then, how do we simulate the random walk fast?
![Page 37: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/37.jpg)
Lanczos Method
Goal: Given symmetric B, v, k, and function f;approximate f(B)v as well as any degree k poly
Observe
p – any degree k polynomial
Krylov Subspace · of
order kLet be an orthonormal basis for ·
k-Identity Projection on
·Let
For any degree k polynomial p,
Our guess:
T is a much smaller matrix!
![Page 38: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/38.jpg)
Error Guarantee
Goal: Bound the norm of the error
Write Any degree
k polynomial
Equal!
Since and the columns of V are orthnormal,
The columns of V are orthnormal
True for any p!!
As good as the best degree k polynomial!
Cost = k mult. by B + Construct basis + Diagonalize T
if B is a Laplacian
![Page 39: Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi Linear Time Graph.](https://reader037.fdocuments.us/reader037/viewer/2022103100/56649e935503460f94b98b8b/html5/thumbnails/39.jpg)
Using Lanczos for our Algorithm
Goal: Given A, wanted to approximate as
Use Lanczos with
suffices.
Still need to address the issue of approximate multiplication with