Fast Linear Solvers for Laplacian Systems - UCSD Research...
Transcript of Fast Linear Solvers for Laplacian Systems - UCSD Research...
![Page 1: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/1.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Fast Linear Solvers for Laplacian SystemsUCSD Research Exam
Olivia Simpson
University of California, San Diego
November 8, 2013
![Page 2: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/2.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
![Page 3: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/3.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Definitions
Let G = (V ,E ) be an undirected edge-weighted graph on nvertices and m edges. We can generalize an unweighted graphto the case where all edge weights are equal to 1.The degree of a vertex v ∈ V is the sum of the weights of theedges adjacent to it,
dv =∑u∼v
w(u, v).
The degree matrix is the diagonal matrix whose entries are thedegrees of the vertices,
(D)vv = dv .
![Page 4: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/4.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Definitions
The adjacency matrix is the n × n matrix which is the weightw(e) in the entries corresponding to an edge,
(A)uv =
{w(u, v) if {u, v} ∈ E ,
0 otherwise.
The Laplacian is the matrix
L = D − A.
![Page 5: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/5.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Example
For the followingsimple graph,
L =
2 0 0 0 00 2 0 0 00 0 4 0 00 0 0 1 00 0 0 0 3
−
0 0 1 0 10 0 1 0 11 1 0 1 10 0 1 0 01 1 1 0 0
=
2 0 −1 0 −10 2 −1 0 −1
−1 −1 4 −1 −10 0 −1 1 0
−1 −1 −1 0 3
![Page 6: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/6.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Properties of the Laplacian
• Symmetric
• Diagonally dominant
• All zero row-sums
• Non-positive off-diagonal entries
• Clean quadratic form:
xTLx =∑u∼v
(x(u)− x(v))2
![Page 7: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/7.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Consensus
A network of n decision-making agents
• edges are communicationchannels
• xi is the decision value ofeach agent
• values can be influenced bycommunicating neighbors
• two agents vi , vj are said toagree when xi = xj
Goal: all agents reach a common decision value, called the con-sensus
![Page 8: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/8.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Consensus
A network of n decision-making agents
The Laplacian potential is a mea-sure of disagreement in the system,
ΨG (x) =1
2xTLx
=1
2
∑vi∼vj
(xi − xj)2.
all agents agree ⇔ ΨG (x) = 0.
Goal: find the vector x which minimizes the Laplacian poten-tial [OM03]
![Page 9: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/9.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Why Laplacian Systems?
Laplacian Systems arise in a number of natural contexts,
• Characterizing the motion of coupled oscillators [HS08]
• Approximating Fiedler eigenvectors [ST04]
• ...
• Computing effective resistance in an electricalnetwork [Kir1847]
![Page 10: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/10.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
• Edges are wires in the network
• Edge weights correspond to the conductance of each wire
• Goal: set voltages x1, x2, x3 on the vertices to create aflow of electrical current
![Page 11: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/11.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Ohm’s law: take conductance(e) = 1/resistance(e), then
current(e) = conductance(e)× |voltage(u)− voltage(v)|
Consider the flow of current from V 1. By Ohm’s law:
current(V 1,V 2) = 1(voltage(V 1)− voltage(V 2))
current(V 1,V 3) = 2(voltage(V 1)− voltage(V 3))
![Page 12: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/12.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Kirchhoff’s law [Kir1847]: For every point in the network,
netflow(v) := flowin(v)− flowout(v) = 0,
except at the injection point, netflow = 1, and the extractionpoint, netflow = −1.
![Page 13: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/13.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
So, at vertex V 1 the voltage x1 must satisfy:
netflow(V 1) = current(V 1,V 2) + current(V 1,V 3)
1 = 1(x1 − x2) + 2(x1 − x3)
1 = 3x1 − x2 − 2x3
![Page 14: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/14.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Applying the same rules at V 2 and V 3 yields the following sys-tem of equations:
3x1 − x2 − 2x3 = 1
−x1 + 2x2 − x3 = 0
−2x1 − x2 + 3x3 = −1
![Page 15: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/15.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Or, in matrix form, 3 −1 −2−1 2 −1−2 −1 3
x1
x2
x3
=
10−1
![Page 16: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/16.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
Or, in matrix form, 3 −1 −2−1 2 −1−2 −1 3
x1
x2
x3
=
10−1
A system in the Laplacian of the network.
![Page 17: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/17.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Electrical Networks
When current is injected at V 1 and extracted at V 3, the solutionvector x can be used to compute the effective resistance of theedge (V 1,V 3),
R(V 1,V 3) = |x1 − x3|
![Page 18: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/18.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Consensus Again
[OM03] A connected networkof decision-making agents
• edges are communicationchannels
• xi is the decision value ofeach agent
• values can be influenced bycommunicating neighbors
• two agents vi , vj are said toagree when xi = xj
Goal: all agents reach a common decision value, called theconsensus
![Page 19: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/19.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Consensus Again
[OM03] A connected networkof decision-making agents
Suppose each agent evolves theirdecision value according to thedistributed linear protocol,
xi (t) =∑vj∼vi
xj(t)− xi (t).
Then the solution to the system
x = −Lx , x(0) ∈ Rn
is the vector of decision values as afunction of t.
Goal: all agents reach a common decision value, called theconsensus
![Page 20: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/20.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
![Page 21: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/21.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Directly
Given a system of linear equations, Ax = b, we can solve thesystem directly using Gaussian elimination.
Gaussian elimination takes O(n3) time in general.
- [CW90] show the exponent can be as small as 2.376
- [Wil12] improves this to 2.3727
![Page 22: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/22.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Iterative Methods
Iterative methods for solving systems Ax = b are based on asequence of increasingly better approximations.
The idea is to construct a sequencex (0), x (1), . . . , x (i), . . . , x (N), . . . that converges to the truesolution, x ,
limk→∞
x (k) = x .
In practice, the iterative process is stopped after N iterationswhen
||x (N) − x || < ε
for any vector norm and a prescribed error parameter ε.
![Page 23: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/23.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Iterative Methods
Richardson’s Method([Young53],[GV61])
An iterative method that improves approximations using theresidual error, b − Ax (i), at each step:
x (i+1) = x (i) + (b − Ax (i))
= b + (I − A)x (i)
![Page 24: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/24.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
![Page 25: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/25.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
![Page 26: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/26.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
![Page 27: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/27.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
![Page 28: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/28.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
![Page 29: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/29.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:
B−1Ax = B−1b.
What makes a matrix B a good preconditioner for A?
1 It is a very good approximation of the matrix
2 It reduces the number of iterations
3 B can be computed quickly
4 Systems in B can be solved quickly
![Page 30: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/30.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
Preconditioned Richardson’s Method([Young53],[GV61])
x (i+1) = b + (I − A)x (i)
![Page 31: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/31.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
Preconditioned Richardson’s Method([Young53],[GV61])
x (i+1) = B−1b + (I − B−1A)x (i)
![Page 32: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/32.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Preconditioned Iterative Methods
Preconditioned Richardson’s Method([Young53],[GV61])
x (i+1) = B−1b + (I − B−1A)x (i)
The term B−1b involves solving a system in B.
![Page 33: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/33.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Since the solution x is not known, different criteria must beused to determine the value N for which
||x (N) − x || < ε.
![Page 34: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/34.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Since the solution x is not known, different criteria must beused to determine the value N for which
||x (N) − x || < ε.
- One criterion uses the residual vector, r (i) = b − Ax (i)
- The minimum iterations N should satisfy
||x − x (N)||||x ||
≤ κ(A)||r (N)||||b||
≤ εκ(A),
where κ(A) = ||A−1|| · ||A|| is the condition number of A for anymatrix norm || · || [QRS07].
![Page 35: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/35.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Since the solution x is not known, different criteria must beused to determine the value N for which
||x (N) − x || < ε.
In the case of preconditioned methods, this becomes
||B−1r (N)||||B−1r (0)||
≤ ε.
In particular, this means the rate of convergence will depend onhow quickly systems in B can be solved.
![Page 36: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/36.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Preconditioned Chebyshev method will find solutions withabsolute error ε in time
O(mS(B) log(κ(A)/ε)√κ(A,B)), [GO88]
where S(B) is the time required so solve a system in B and
κ(A,B) =
(max
x :Ax 6=0
xTAx
xTBx
)(max
x :Ax 6=0
xTBx
xTAx
).
![Page 37: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/37.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Convergence of Iterative Methods
Factors:
• minimum number of iterations N, depends on κ(A)
• time to solve the system in B
In general, worst case time bounds are O(mn).
![Page 38: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/38.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
![Page 39: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/39.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A First Nearly-Linear Time Solver
Spielman and Teng [ST04] presented the first nearly-lineartime algorithm for solving systems of equations in symmetric,diagonally-dominant (SDD) systems.
The success of the ST-solver can be ascribed to twoinnovations:
1 enhancing a one-level iterative solver using recursion
2 using a preconditioning matrix based on a subgraph
![Page 40: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/40.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A First Nearly-Linear Time Solver
Spielman and Teng [ST04] presented the first nearly-lineartime algorithm for solving systems of equations in symmetric,diagonally-dominant (SDD) systems.
The success of the ST-solver can be ascribed to twoinnovations:
1 enhancing a one-level iterative solver using recursion
2 using a preconditioning matrix based on a subgraph
![Page 41: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/41.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A First Nearly-Linear Time Solver
Spielman and Teng [ST04] presented the first nearly-lineartime algorithm for solving systems of equations in symmetric,diagonally-dominant (SDD) systems.
The success of the ST-solver can be ascribed to twoinnovations:
1 enhancing a one-level iterative solver using recursion
2 using a preconditioning matrix based on a subgraph
![Page 42: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/42.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Recursive Solver
The ST-solver addresses the problem of solving systems in thepreconditioning matrix B with recursion:
solve(A,b):
if dimension-check(A) and sparse(A):
return PrecChebyshev(A,b)
B = precondition(A)
A’ = reduce(B)
solve(A’,b)
• Compute B, the preconditioner for A
• Reduce the system in B to a system in A1, a refinedversion of A
• Recursively solve the system in A1
• At the base, use the Preconditioned Chebyshev method
![Page 43: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/43.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Recursive Solver
The ST-solver addresses the problem of solving systems in thepreconditioning matrix B with recursion:
A→ B → A1 → B1 → · · · → Ak
• Compute B, the preconditioner for A
• Reduce the system in B to a system in A1, a refinedversion of A
• Recursively solve the system in A1
• At the base, use the Preconditioned Chebyshev method
![Page 44: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/44.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Recursive Solver
Two methods are used at each level of the recursion.
reduce(B): Bi → Ai+1
- Reduce the preconditioned matrix by greedily removing rowsand columns with at most two non-zero entries- Done with a partial Cholesky decomposition
precondition(A): Ai → Bi
- Sparsify the graph associated to A to obtain a subgraph H- Set B to be a matrix of H
![Page 45: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/45.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Graph Preconditioners
[GMZ95]: There is a linear time transformation:
Ax = b SDD system
⇓Lx ′ = b′ Laplacian system
For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.
![Page 46: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/46.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Graph Preconditioners
[GMZ95]: There is a linear time transformation:
Ax = b SDD system
⇓Lx ′ = b′ Laplacian system
For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.
A→ B → A1 → B1 → · · · → Ak
![Page 47: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/47.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Graph Preconditioners
[GMZ95]: There is a linear time transformation:
Ax = b SDD system
⇓Lx ′ = b′ Laplacian system
For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.
LA → LB → LA1 → LB1 → · · · → LAk
![Page 48: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/48.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Graph Preconditioners
[GMZ95]: There is a linear time transformation:
Ax = b SDD system
⇓Lx ′ = b′ Laplacian system
For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.
G → H → G1 → H1 → · · · → Gk
![Page 49: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/49.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Spanning Trees
[Vaidya91]: subgraphs serve as good preconditioners
• maximum weight spanning tree as a preconditioning base
• solves SDD systems with non-positive off-diagonal entriesof degree d in time
O(dn)1.75 log(κ(A)/ε)
[ST04]: subgraphs serve as good preconditioners
• a better base tree uses an edge measure called stretch
![Page 50: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/50.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
The detour forced by traversing T instead of G
![Page 51: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/51.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
The detour forced by traversing T instead of G
e = {V 1,V 4},w(e) = 1
![Page 52: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/52.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
The detour forced by traversing T instead of G
A spanning tree, T
![Page 53: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/53.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
The detour forced by traversing T instead of G
e = {V 1,V 4}, Tree path: {V 1,V 3}, {V 3,V 5}, {V 5,V 4}
![Page 54: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/54.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Stretch of an Edge
Let T be a spanning tree of a weighted graph G , and let theweight of an edge e = {u, v} be denoted by w(e). Definew ′(e) = 1/w(e), which is the resistance of the edge, re .
Let e1, e2, . . . , ek be the unique path in T from u to v . Thenthe stretch of the edge by T is defined
stretchT (e) =
∑ki=1 w ′(e1)
w ′(e)= 1/re
k∑i=1
rei
The total stretch of a graph G by the tree T is the sum of thestretch of all the off-tree edges.
![Page 55: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/55.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Low Stretch Spanning Trees
Low stretch spanning tree (LSST) for the preconditioning base↓
A “spine” that keeps resistance low
LSSTs for preconditioners had been implemented before([AKPW95, BH01, BH03]), but combining the high qualitypreconditioners with a recursive solver amounted to analgorithm for solving SDD systems in nearly-linear time.
![Page 56: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/56.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Low Stretch Spanning Trees
Low stretch spanning tree (LSST) for the preconditioning base↓
A “spine” that keeps resistance low
LSSTs for preconditioners had been implemented before([AKPW95, BH01, BH03]), but combining the high qualitypreconditioners with a recursive solver amounted to analgorithm for solving SDD systems in nearly-linear time.
![Page 57: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/57.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
ST-Solver
[ST04]: Given a system of equations Ax = b in a symmetric,diagonally dominant matrix, the ST-solver computes a vector xsatisfying
||Ax − b|| < ε and ||x − x || ≤ ε
in time O(m logO(1) m), where the exponent is a large constant.
![Page 58: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/58.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A Better Sparsifier
Sparsifier of the ST-solver
1 Compute an LSST T of the graph G
2 Reweight the edges of T by a constant factor k , call thereweighted tree T ′
3 Replace T in G by T ′
4 H ← T ′
5 Add off-tree edges to H by sampling with probabilitiesrelated to vertex degree
![Page 59: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/59.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A Better Sparsifier
Sparsifier of Koutis et al. [KMP10, KMP11]
1 Compute an LSST T of the graph G
2 Reweight the edges of T by a constant factor k , call thereweighted tree T ′
3 Replace T in G by T ′
4 H ← T ′
5 Add off-tree edges to H by sampling with probabilitiesrelated to effective resistance
![Page 60: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/60.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A Better Sparsifier
pro: Probabilities related to effective resistance yield sparsifierswith few edges [SS11]con: Computing effective resistances involves solving a systemof linear equations
This is problem is avoided by instead using upper bounds foredge probabilities,
pe ≥ weRe ,
where Re is the effective resistance of the edge.
The sparsifier of [KMP11] improved the expected time boundto O(m log2 n log(1/ε)) for an approximate solution withabsolute error bounded by ε.
![Page 61: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/61.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
A Better Sparsifier
pro: Probabilities related to effective resistance yield sparsifierswith few edges [SS11]con: Computing effective resistances involves solving a systemof linear equations
This is problem is avoided by instead using upper bounds foredge probabilities,
pe ≥ weRe ,
where Re is the effective resistance of the edge.
The sparsifier of [KMP11] improved the expected time boundto O(m log2 n log(1/ε)) for an approximate solution withabsolute error bounded by ε.
![Page 62: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/62.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Summary of Methods Discussed
Gaussian elimination O(n3)[CW90] O(n2.376)[Wil12] O(n2.3727)
Prec Chebyshev [GO88] O(mS(B) log(κ(A)/ε)√κ(A,B))
[Vaidya91] O(dn)1.75 log(κ(A)/ε)
[ST04] O(m logO(1) m)[KMP11] O(m log2 n log(1/ε))
![Page 63: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/63.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
![Page 64: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/64.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Boundary of a Subset
Let S be a subset of vertices ina graph.
S = {S1,S2,S3}
![Page 65: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/65.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Boundary of a Subset
Let S be a subset of vertices ina graph.
The vertex boundary of S is theset of vertices not in S whichborder S
δ(S) = {v /∈ S : {v , u} ∈ E for some u ∈ S}.
![Page 66: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/66.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Boundary of a Subset
Let b be a vector over the ver-tices of a graph.
Then a vector x over the satis-fies the boundary condition of bfor a subset S when
x(v) = b(v) ∀v ∈ δ(S).
δ(S) = {v /∈ S : {v , u} ∈ E for some u ∈ S}.
![Page 67: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/67.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Leader-Following Formation
• a network with a leader, anda group of agents
• values xi correspond toposition
• leader moves independently ofagents
Goal: design a distributed protocol for agents to follow the leader
![Page 68: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/68.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Leader-Following Formation
The leader is the boundary of thesubset of agents.
So our solution should:- Respect the leader’s position (fixthe solution on the boundary)- Use local information among theagents (compute the solution onthe subset)
Goal: design a distributed protocol for agents to follow the leader
![Page 69: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/69.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Leader-Following Formation
[NC10]: Consider a multi-agent system of n agents and oneleader.
1 The dynamics of the leader is
x0 = Ax0,
which is independent.
2 The dynamics of each agent is
xi =∑vj∼vi
xj − xi ,
and the vector x of agentpositions is given by
x = −Lx .
![Page 70: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/70.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Satisfying the Boundary Condition
Consider a linear system Lx = b. Suppose there exists a subsetS such that
• the induced subgraph on S is connected
• the boundary δ(S) is non-empty
• support(b) ⊆ δ(S)
• x satisfies the boundary condition b:
x(v) = b(v), ∀v ∈ δ(S).
Goal: find the solution x restricted to S .That is, the solution will satisfy
x(v) =
{1dv
∑u∼v x(u) if v ∈ S
b(v) if v ∈ δ(S).
![Page 71: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/71.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Linear Systems withBoundary Conditions
Main tools and techniques:
• Fast computation of a heat kernel pagerank vector
• Expressing a Laplacian linear system with boundaryconditions in terms of the heat kernel of the graph
• Approximate the solution by a sum of heat kernelpagerank vectors
![Page 72: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/72.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Walks on a Graph
Consider P = D−1A as a random walk matrix.
P =
0 0 1/2 0 1/20 0 1/2 0 1/2
1/4 1/4 0 1/4 1/40 0 1 0 0
1/3 1/3 1/3 0 0
![Page 73: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/73.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Walks on a Graph
Consider P = D−1A as a random walk matrix.
P =
0 0 1/2 0 1/20 0 1/2 0 1/2
1/4 1/4 0 1/4 1/40 0 1 0 0
1/3 1/3 1/3 0 0
![Page 74: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/74.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Walks on a Graph
Consider P = D−1A as a random walk matrix.
P =
0 0 1/2 0 1/20 0 1/2 0 1/2
1/4 1/4 0 1/4 1/40 0 1 0 0
1/3 1/3 1/3 0 0
When f is a probability distribution vector, f TPk is thedistribution after k random walk steps.
Define ∆ to be the Laplace operator, defined ∆ = I − P.
![Page 75: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/75.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
![Page 76: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/76.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
![Page 77: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/77.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
![Page 78: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/78.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
![Page 79: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/79.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,
ρt,f = f T e−t∆ = e−t∞∑k=0
tk
k!f TPk .
• If f is a starting distribution over the vertices, then f TPk
will be the distribution after k random walk steps
• The sum of the coefficients satisfies
∞∑k=0
e−ttk
k!= e−t · et = 1
• Then if k steps of a P random walk are taken withprobability e−t t
k
k!
• ⇒ ρt,f is the expected distribution of the process
![Page 80: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/80.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
![Page 81: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/81.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
? Avoid an exponential sum by taking samples.
![Page 82: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/82.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
? Avoid an exponential sum by taking samples.◦ Control error by drawing enough samples r(ε)
![Page 83: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/83.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
? Avoid an exponential sum by taking samples.◦ Control error by drawing enough samples r(ε)
? Avoid long walk processes by sampling truncated randomwalks.
![Page 84: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/84.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
? Avoid an exponential sum by taking samples.◦ Control error by drawing enough samples r(ε)
? Avoid long walk processes by sampling truncated randomwalks.◦ Control contribution lost in later step by stopping after enoughsteps K (ε)
![Page 85: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/85.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Q: A good way to compute the heat kernel pagerank?
A: Draw enough samples of the random process to get theexpected value.
Let pk be the probability of taking k random walk steps,
pk = e−ttk
k!.
Let X be the distribution over values of k.• Then obtain a good approximation by taking the average of rrandom walks of length at most K .• The variables r ,K are both in terms of a prescribed errorparameter, ε.
![Page 86: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/86.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Computing Heat Kernel Pagerank
Let the output of the algorithm be ρt,f . Then, to achieve
(1− ε)ρt,f [v ]− ε ≤ ρt,f [v ] ≤ (1 + ε)ρt,f [v ],
we choose
r ← 16
ε3log s,
K ← log(ε−1)
log log(ε−1).
Then, assuming that (1) taking a random walk step and (2)sampling from a distribution take constant time, the runtime ofthe heat kernel pagerank computation is
r · K =16
ε3log s · log(ε−1)
log log(ε−1)= O
( log s log(ε−1)
ε3 log log ε−1
).
![Page 87: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/87.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Linear Systems withBoundary Conditions
Main tools and techniques:
• Fast computation of a heat kernel pagerank vector
• Expressing a Laplacian linear system with boundaryconditions in terms of the heat kernel of the graph
• Approximate the solution by a sum of heat kernelpagerank vectors
![Page 88: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/88.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Linear Systems with BoundaryConditions
Recall what it means for a solution vector x to satisfy theboundary condition in the system Lx = b.
S = {v ∈ V : b[v ] = 0},
x(v) =
{1dv
∑u∼v x(u) if v ∈ S
b(v) if v ∈ δ(S).
![Page 89: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/89.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Linear Systems with BoundaryConditions
Given a graph G and a vector b of boundary conditions,• Let S be the subset of s = |S | vertices
S = {v ∈ V : b[v ] = 0}.
• Let AδS be the s × |δ(S)| matrix A with columns restricted tothe vertices of δ(S) and rows to vertices of S .• Let LS , AS and DS be L, A and D with rows and columnsrestricted to vertices of S .• Let xS be the solution vector over the vertices of S , and letbδS be b over δ(S).
![Page 90: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/90.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Linear Systems with BoundaryConditions
Then we would like to find a vector x ∈ Rs satisfying:
DSxS = ASxS + AδSbδS
or, equivalently:
xS = (DS − AS)−1(AδSbδS)
= L−1S (AδSbδS),
The inverse L−1S exists when the induced subgraph on S is
connected and the boundary of S is non-empty.
![Page 91: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/91.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Linear Systems with BoundaryConditions
Then we would like to find a vector x ∈ Rs satisfying:
DSxS = ASxS + AδSbδS
or, equivalently:
xS = (DS − AS)−1(AδSbδS)
= L−1S (AδSbδS),
The inverse L−1S exists when the induced subgraph on S is
connected and the boundary of S is non-empty.
![Page 92: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/92.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Normalized Laplacian
The normalized Laplacian is defined
(L)uv =
1 if u = v ,−1√dudv
if u ∼ v ,
0 otherwise.
• Degree-nomalized version of L,
L = D−1/2LD−1/2
• Symmetric version of ∆ = I − P,
L = D1/2∆D−1/2
![Page 93: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/93.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Normalized Laplacian
When G has no isolated vertex and the matrix D is invertible,
Lx = b and Lx ′ = b′
the system in L can be deduced from the system in L bysetting x ′ = D1/2x and b′ = D−1/2b.
Then the solution x ∈ Rs satisfying the boundary conditionshould satisfy
x ′S = L−1S (AδSb′δS)
![Page 94: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/94.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
The Normalized Laplacian
When G has no isolated vertex and the matrix D is invertible,
Lx = b and Lx ′ = b′
the system in L can be deduced from the system in L bysetting x ′ = D1/2x and b′ = D−1/2b.
Then the solution x ∈ Rs satisfying the boundary conditionshould satisfy
x ′S = L−1S (AδSb′δS)
x ′S = L−1S b1.
And computing b1 takes time proportional to the sum of vertexdegrees in δ(S) := vol(δ(S)).
![Page 95: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/95.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Dirichlet Heat Kernel
The Dirichlet heat kernel is defined
HS,t = e−tLS = D1/2S e−t∆S D
−1/2S .
Lemma ([CS13])
![Page 96: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/96.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Dirichlet Heat Kernel
The Dirichlet heat kernel is defined
HS,t = e−tLS = D1/2S e−t∆S D
−1/2S .
Lemma ([CS13])
Let S be a strict subset of vertices of a graph G and let theinduced subgraph on S be connected. Then,
L−1S =
∫ ∞0HS,t dt.
![Page 97: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/97.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Dirichlet Heat Kernel
The Dirichlet heat kernel is defined
HS,t = e−tLS = D1/2S e−t∆S D
−1/2S .
Lemma ([CS13])
Let S be a strict subset of vertices of a graph G and let theinduced subgraph on S be connected. Then,
L−1S =
∫ ∞0HS,t dt.
x ′S = L−1S b1 =
∫ ∞0HS,tb1 dt.
![Page 98: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/98.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Linear Systems withBoundary Conditions
Main tools and techniques:
• Fast computation of a heat kernel pagerank vector
• Expressing a Laplacian linear system with boundaryconditions in terms of the heat kernel of the graph
• Approximate the solution by a sum of heat kernelpagerank vectors
![Page 99: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/99.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving Linear Systems withBoundary Conditions
• Approximate the solution
1 express the solution as an improper integral of heat kernelpagerank
2 approximate with a definite integral by limiting the range3 approximate by taking a finite sum of heat kernel pagerank
vectors4 approximate each term by sampling truncated random
walks
![Page 100: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/100.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′,
x ′S =
(∫ ∞0
ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
![Page 101: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/101.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′: x ′S =
( ∫∞0 ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
Proof.
x ′S =
∫ ∞0HS ,tb1 dt from Lemma
x ′S =
∫ ∞0
bT1 HS,t dt HS ,t symmetric
x ′S =
∫ ∞0
bT1 (D
1/2S e−t∆S D
−1/2S ) dt definition of HS,t
x ′S =
∫ ∞0
bT1 D
1/2S e−t∆S dt D
−1/2S
![Page 102: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/102.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′: x ′S =
( ∫∞0 ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
Proof.
x ′S =
∫ ∞0HS ,tb1 dt from Lemma
x ′S =
∫ ∞0
bT1 HS,t dt HS ,t symmetric
x ′S =
∫ ∞0
bT1 (D
1/2S e−t∆S D
−1/2S ) dt definition of HS,t
x ′S =
∫ ∞0
bT1 D
1/2S e−t∆S dt D
−1/2S
![Page 103: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/103.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′: x ′S =
( ∫∞0 ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
Proof.
x ′S =
∫ ∞0HS ,tb1 dt from Lemma
x ′S =
∫ ∞0
bT1 HS,t dt HS ,t symmetric
x ′S =
∫ ∞0
bT1 (D
1/2S e−t∆S D
−1/2S ) dt definition of HS,t
x ′S =
∫ ∞0
bT1 D
1/2S e−t∆S dt D
−1/2S
![Page 104: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/104.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Approximating with Heat Kernel
ClaimFor Lx ′ = b′: x ′S =
( ∫∞0 ρt,f dt
)D−1/2S , f = bT
1 D1/2S .
Proof.
x ′S =
∫ ∞0HS ,tb1 dt from Lemma
x ′S =
∫ ∞0
bT1 HS,t dt HS ,t symmetric
x ′S =
∫ ∞0
bT1 (D
1/2S e−t∆S D
−1/2S ) dt definition of HS,t
x ′S =
∫ ∞0
bT1 D
1/2S e−t∆S︸ ︷︷ ︸ dt D
−1/2S
ρt,f
![Page 105: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/105.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Finding the Solution
Let x1 = x ′S D1/2S . We can compute
x1 =
∫ ∞0
ρt,f dt, f = bT1 D
1/2S .
by a number of approximations,
1. Ignore the tail, take the integral to a finite T ,
x1 ≈∫ T
0ρt,f dt
![Page 106: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/106.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Finding the Solution
Let x1 = x ′S D1/2S . We can compute
x1 =
∫ ∞0
ρt,f dt, f = bT1 D
1/2S .
by a number of approximations,
2. Discretize with a finite Riemann sum for small intervals T/N,
x1 ≈N∑j=1
ρjT/N,f · T/N
![Page 107: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/107.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Finding the Solution
Let x1 = x ′S D1/2S . We can compute
x1 =
∫ ∞0
ρt,f dt, f = bT1 D
1/2S .
by a number of approximations,
3. Obtain a good approximation by finitely many samples ofρjT/N,f over values of t.
for r times:
draw j from the interval [1,N]
compute ApproxHKPR(G, jT/N, f)
![Page 108: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/108.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving the System with BoundaryConditions
[CS13] show that to achieve an approximation vector xsatisfying
||x ′ − x || ≤ O(ε(1 + ||b||)),
the number of samples needed is r = ε−2(log s + log(ε−1)),where s = |S |.
Then using that the time to compute a heat kernel pagerankvector is
O( log s log(ε−1)
ε3 log log ε−1
),
and assuming additional preprocessing time proportional tovol(δS), the main result of [CS13] is the following.
![Page 109: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/109.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving the System with BoundaryConditions
[CS13] show that to achieve an approximation vector xsatisfying
||x ′ − x || ≤ O(ε(1 + ||b||)),
the number of samples needed is r = ε−2(log s + log(ε−1)),where s = |S |.Then using that the time to compute a heat kernel pagerankvector is
O( log s log(ε−1)
ε3 log log ε−1
),
and assuming additional preprocessing time proportional tovol(δS), the main result of [CS13] is the following.
![Page 110: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/110.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving the System with BoundaryConditions
Theorem (Boundary Solver [CS13])
For a graph G and a linear system Lx = b, assume:
• x is required to satisfy the boundary condition b
• S = V \ support(b) and s = |S |• the boundary of S is nonempty
• the induced subgraph on S is connected.
![Page 111: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/111.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Solving the System with BoundaryConditions
Theorem (Boundary Solver [CS13])
Then the approximate solution x output by the boundary solversatisfies the following with probability ≥ 1− ε.
1 ||x − x || ≤ O(ε(1 + ||b||))
2 The running time is
O((log s)2(log(ε−1))2
ε5 log log(ε−1)
)with additional preprocessing time O(vol(δ(S))).
![Page 112: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/112.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Overview
1 Introduction
2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian
3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier
4 Solving Linear Systems with Boundary Conditions UsingRandom Walks
Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System
5 Future Directions
![Page 113: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/113.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Future Directions
Capitalize on speed.
• Use fast linear solvers to improve existing graph algorithms
• Interior point algorithms [SD08]• Learning problems [ZHS05]• Graph partitioning [ST04],[ACL06]• Sparsification [ST04],[SS11]
• Find more applications for these linear solvers
• Find more applications for heat kernel pagerank• Expected distribution of random walks• Vertex similarity → graph distance
![Page 114: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/114.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
ReferencesNoga Alon, Richard M. Karp, David Peleg, and Douglas West (1995)
A Graph-Theoretic Game and its Applications to the k-SeverProblem.
SIAM J. Computing, 21(1), 78–100.
Reid Andersen, Fan Chung, and Kevin Lang (2006)
Local Graph Partitioning Using PageRank Vectors.
FOCS’06, 475–486.
Erik Boman and Bruce Hendrickson (2001)
On Spanning Tree Preconditioners.
Sandia National Laboratories Manuscript.
Erik Boman and Bruce Hendrickson (2003)
Support Theory for Preconditioning.
SIAM J. Matrix Analysis and Applications, 25(3), 694–717.
Sergey Brin and Lawrence Page (1998)
The Anatomy of a Large-Scale Hypertextual Web Search Engine.
Computer Networks and ISDN Systems, 30(1), 107–117.
![Page 115: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/115.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
Fan Chung and Olivia Simpson (2013)
Solving Linear Systems with Boundary Conditions using Heat KernelPagerank.
WAW’13.
Don Coppersmith and Shmuel Winograd (1990)
Matrix Multiplication via Arithmetic Progressions.
J. Symbolic Computation, 9(3), 251–280.
Samuel I. Daitch and Daniel A. Spielman (2008)
Faster Approximate Lossy Generalized Flow Via Interior PointAlgorithms.
STOC’08, 451–460.
Gene H. Golub and Michael L. Overton (1988)
The Convergence of Inexact Chebyshev and Richardson IterativeMethods for Solving Linear Systems.
Numerische Mathematik, 53(5), 571–593.
![Page 116: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/116.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
Gene H. Golub and Richard S. Varga (1961)
Chebyshev Semi-Iterative Methods, Successive OverrelaxationIterative Methods, and Second Order Richardson Iterative Methods.
Numerical Math. 3, 147–168.
Keith D. Gremban, Gary L. Miller, and Marco Zagha (1995)
Performance Evaluation of a New Parallel Preconditioner.
International Parallel Processing Symposium ’95, 65–69.
Aric Hagberg and Daniel A. Schult (2008)
Rewiring Networks for Synchronization.
Chaos: An Interdisciplinary Journal of Nonlinear Science, 18(3).
Gustav Kirchhoff (1847)
Uber die Auflosung der Gleichungen, auf welche man bei derUntersuchung der linearen Vertheilung galvanischer Strome gefuhrtwird.
Annalen der Physik, 148(12), 497–508.
![Page 117: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/117.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
Ioannis Koutis, Gary L. Miller, and Richard Peng (2010)
Approaching Optimality for Solving SDD Linear Systems.
FOCS’10, 235–244.
Ioannis Koutis, Gary L. Miller, and Richard Peng (2011)
A Nearly-m log n Time Solver for SDD Linear Systems.
FOCS’11, 590–598.
Ioannis Koutis, Gary L. Miller, and Richard Peng (2012)
A Fast Solver for a Class of Linear Systems.
Communications of the ACM, 55(10), 99–107.
Wei Ni and Daizhan Cheng (2010)
Leader-Following Consensus of Multi-Agent Systems Under Fixed andSwitching Topologies.
Systems & Control Letters, 59(3), 209–217.
Reza Olfati-Saber and Richard Murray (2003)
Consensus Protocols for Networks of Dynamic Agents.
American Control Conference ’03, 951–956.
![Page 118: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/118.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
Daniel A. Spielman and Nikhil Srivastava (2011)
Graph Sparsification by Effective Resistances.
SIAM J. Computing, 40(6), 1913–1926.
Daniel A. Spielman and Shang-Hua Teng (2004)
Nearly-Linear Time Algorithms for Graph Partitioning, GraphSparsification, and Solving Linear Systems.
STOC’04, 81–90.
Pravin M. Vaidya (1991)
Solving Linear Equations with Symmetric Diagonally DominantMatrices by Constructing Good Preconditioners.
UIUC manuscript based on a talk.
Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri (2007)
Numerical Mathematics, vol 37.
Virginia Vassilevska Williams (2012)
Multiplying Matrices Faster than Coppersmith-Winograd.
STOC’12, 887–898.
![Page 119: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/119.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
References
David Young (1950)
On Richardson’s Method for Solving Linear Systems with PositiveDefinite Matrices.
J. Math. and Physics 32, 243–255.
Denyong Zhou, Jiayuan Huang, and Bernhard Sch’olkopf (2005)
Learning From Labeled and Unlabeled Data on a Directed Graph
International Conference on Machine Learning 1036–1043.
![Page 120: Fast Linear Solvers for Laplacian Systems - UCSD Research Examcseweb.ucsd.edu/~osimpson/RE_presentation.pdf · Simpson Introduction Background Information Laplacians Laplacian Systems](https://reader034.fdocuments.us/reader034/viewer/2022050600/5fa756ea89e0b3513f3b9e24/html5/thumbnails/120.jpg)
SolvingLaplacianSystems
OliviaSimpson
Introduction
BackgroundInformation
Laplacians
LaplacianSystems
Fast LinearSolvers
PreviousMethods
ST-Solver
A BetterSparsifier
RandomWalks
BoundaryConditions
Heat KernelPagerank
Variation
Boundary Solver
FutureDirections
Thank You