Laplacian Paradigm 2 - sachdevasushant.github.io · Laplacian Paradigm 2.0 8:40-9:10: Merging...

LaplacianParadigm2.08:40-9:10:MergingContinuousandDiscrete(RichardPeng)9:10-9:50:BeyondLaplacianSolvers(AaronSidford)9:50-10:30:ApproximateGaussianElimination(Sushant Sachdeva)10:30-11:00:coffeebreak11:00-12:00:AnalysisusingmatrixMartingales(RasmusKyng)12:00-14:00 lunch14:00-15:00GraphStructureviaEliminations(AaronSchild)

Website:bit.ly/laplacian2

MergingtheContinuousandDiscrete

RichardPengOct6,2018

Outline

• GraphsandLaplacians• BuildingBlocks• LaplacianParadigm2.0

LargeNetworks• Datamining:centrality,clustering…• Image/videoprocessing:segmentation,denoising …• Scientificcomputing:stress,fluids,waves…

• \ (linearsystemsolve)• CVX(convexoptimization)• Eigenvectorsolvers

GraphsandMatricesHighperformancecomputing:non-zerosó edges,design/analyzematrixalgorithmsusinggraphtheory

nrows/columnsO(m)non-zeros

nverticesmedges

2-1-1-110-101

GraphsandMatricesHighperformancecomputing:non-zerosó edges,design/analyzematrixalgorithmsusinggraphtheory

nrows/columnsO(m)non-zeros

1

1

nverticesmedges

2-1-1-110-101

graphLaplacianmatrixL• Diagonal:degrees• Off-diagonal:-edgeweights

SourceofLaplacians2-1-1-110-101


d-Regulargraphs:L =dI – A,A:adjacencymatrix

SourceofLaplacians

Graphcuts:xTLx =∑u~v wuv(xu - xv)2

2-1-1-110-101



xb=0

xa=1

xc=1

(1-0)2=1

(1-1)2=0

xindicatorvectorofcutè weightofcut

SourceofLaplacians

L =BTWBwhereB isedge-vertexincidencematrix

Graphcuts:xTLx =∑u~v wuv(xu - xv)2

2-1-1-110-101



xb=0

xa=1

xc=1

(1-0)2=1

(1-1)2=0

xindicatorvectorofcutè weightofcut

[Spielman Teng `04]Input:graphLaplacianL

vectorbOutput:vectorxs.t. Lx ≅ bRuntime:O(mlogO(1)nlog(1/ε))

OriginoftheLaplacianParadigm



[Cohen-Kyng-Miller-Pachocki-P-Rao-Xu`14]:≤1/2




[Cohen-Kyng-Miller-Pachocki-P-Rao-Xu`14]:≤1/2


Wallclock:m≤107 in≤20s

TheLaplacianParadigm

Directlyrelated:Elliptic systems

Fewiterations:Eigenvectors,Heatkernels

Manyiterations/modifyalgorithmGraphproblemsImageprocessing

Outline


Lx =b asagraphproblemx:voltagevectorsDual:electricalflowf

Unifiedformulation:minf withresdiual b ‖f‖p:• p=2:solvingLx =b• p=1:shortestpath/transshipment• p=∞:max-flow/min-cut

DirectMethods(combinatorial)

M(2) ß Eliminate(M(1),i1)M(3) ß Eliminate(M(2),i2)…

Repeatedlyremoveverticesbycreatingequivalentgraphsontheirneighborhoods

DirectMethods(combinatorial)

M(2) ß Eliminate(M(1),i1)M(3) ß Eliminate(M(2),i2)…

• Parallelgraphalgorithms• Matrixmultiplication/densesolves• Sparsified squaring

Repeatedlyremoveverticesbycreatingequivalentgraphsontheirneighborhoods

IterativeMethods(numerical)

SolveAx=b byxß x - (Ax – b)

Fixedpoint:Ax – b =0



• SimpleB:B =I,manyiterations• B =A:1iteration,butsameproblem


Preconditioning:SolveB-1Ax =B-1b by:xß x - B-1(Ax – b)



• Krylov spacemethods/PCG• Convexoptimizationalgorithms

• SimpleB:B =I,manyiterations• B =A:1iteration,butsameproblem


Preconditioning:SolveB-1Ax =B-1b by:xß x - B-1(Ax – b)

HardinstancesDirectmethodscreatetoomuchfillonhighlyconnectedgraphs

Iterativemethodstaketoomanyiterationspaths



Still`easy’bythemselves

Easyforiterativemethods Easyfordirectmethods



Musthandlebothsimultaneously,butavoidpayingniterations⨉mperiteration

Still`easy’bythemselves

Easyforiterativemethods Easyfordirectmethods

Hybridalgorithms(aka.v1.0)• Scientificcomputing:iChol,multigrid• [Vaidya`89]preconditionwithgraphs

Hybridalgorithms(aka.v1.0)• Scientificcomputing:iChol,multigrid• [Vaidya`89]preconditionwithgraphs

Focus:howtocombine• [Gemban-Miller`96]:spectralgraphtheory• [Spielman-Teng `04]:spectral(ultra-)sparsify

Key“glue”:sparsification[Spielman-Teng `04]: foranyG,canfindHwithO(nlogO(1)n)edgess.t. xTLGx ≈ xTLHx∀x


• Combinatorialparameter:#edges• Numericalparameter:approximations


• Combinatorialparameter:#edges• Numericalparameter:approximations

[Spielman-Srivatava`08]:samplebyeffectiveresistancesgivesHwithO(nlogn)edges

Outline


s

Max-Flowproblem

t

Maximumnumberofdisjoints-tpaths

Applications:• Routing• Scheduling

Recall:minf withresdiual b ‖f‖p:• p=2:solvingLx =b• p=∞:max-flow/min-cut

s

Max-Flowproblem

t

Maximumnumberofdisjoints-tpaths

Applications:• Routing• Scheduling

Dual: separatesandtbyremovingfewestedges

Applications:• Partitioning• Clustering

Recall:minf withresdiual b ‖f‖p:• p=2:solvingLx =b• p=∞:max-flow/min-cut

HybridAlgorithmsforMax-Flow[Daitch-Spielman `08][Christiano-Kelner-Madry-Spielman-Teng `10]:[Lee-Sidford `14]Max-flow/Min-cutvia(several)electricalflows

s t

Repeataboutm1/3 iters• Solvelinearsystems• Re-adjustedgeweights


s t


[Madry `10][Racke-Shah-Taubig `14]:cutapproximator /obliviousroutingO(no(1))-approx.inO(m1+o(1))


s t



[Lee-Rao-Srivastava`13][Sherman`13,`17][Kelner-Lee-Orecchia-Sidford `14]:Preconditioning,(1+ε)-approx


s t



[Lee-Rao-Srivastava`13][Sherman`13,`17][Kelner-Lee-Orecchia-Sidford `14]:Preconditioning,(1+ε)-approx

[P`16]:recurse themintoeachother:O(mlog41n),optimisticallymlog6n

LaplacianParadigm2.0

NewIntermediatestructures/theoremsmotivatedbytheoverallalgorithms

Motivatedbythegoalofhybridalgorithms,modifydirectanditerativemethods

Sparsified/ApproximateGaussianElimination

ExamplesDirectedgraphs/asymmetricmatrices

F

C A[CF]

SC(A,C)

A[CC]

A[FF] A[FC]

c

b

a

1

1

21

Partitioning/LocalizationsofRandomWalks

UnderthehoodMatrix(martingale) concentration

V1 V2

G[V1] Sc(G,V2)

Interactionswithdatastructures

NotcoveredL

[Kelner-Orecchia-Sidford-Zhu `13][Nanongkai-Saranuk `17][Wulff-Nilsen`17][Durfee-Kyng-Peebles-Rao-Sachdeva `17]

G

H

events

MatrixZoofromScientificComputing

[Boman-Hendrickson-Vavasis `04][Kyng-Lee-P-Sachdeva-Spielman `16][Kyng-Zhang `17][Kyng-P-Schweiterman-Zhang `18]

Questions

Approximateeliminationsbeyondspectralcondition#

Non-linear(preconditioned)iterativemethods

[Adil-Kyng-P-Sachdeva `19]:p-normiterativerefinment

Unreasonableeffectivenessofpcg(ichol(A),b),multigrid

Wist list Direct Iterative HybridConvex functions ? J JJ??Arbitrary values J ? ?LJ!Dynamic/streaming J L J???

SolversinPractice[Kyng-Rao-Sachdeva `15] we suggest rerunning the program afew times… An alternate solver based on iChol is provided…

Questions:• Precision• (pseudo)deterministic

Laplacian Paradigm 2 - sachdevasushant.github.io · Laplacian Paradigm 2.0 8:40-9:10: Merging...

Documents

Transcript of Laplacian Paradigm 2 - sachdevasushant.github.io · Laplacian Paradigm 2.0 8:40-9:10: Merging...