Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University...

29
Spectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018.

Transcript of Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University...

Page 1: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

SpectralGraphTheory

SocialandTechnologicalNetworks

Rik Sarkar

UniversityofEdinburgh,2018.

Page 2: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Spectralmethods

• Understandingagraphusingeigen valuesandeigen vectorsofthematrix

• Wesaw:• Ranksofwebpages:componentsof1steigenvectorofsuitablematrix

• Pagerank orHITSarealgorithmsdesignedtocomputetheeigen vector

• Randomwalksandlocalpageranks helpinunderstandingcommunitystructure

Page 3: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Laplacian

• L=D– A[Disthediagonalmatrixofdegrees]

• Aneigen vectorhasonevalueforeachnode• Weareinterestedinpropertiesofthesevalues

2

664

1 �1 0 0�1 2 �1 00 �1 2 �10 0 �1 1

3

775 =

2

664

1 0 0 00 2 0 00 0 2 00 0 0 1

3

775�

2

664

0 1 0 01 0 1 00 1 0 10 0 1 0

3

775

Page 4: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Laplacian

• L=D– A[Disthediagonalmatrixofdegrees]

• Symmetric.RealEigenvalues.• Rowsum=0.Singularmatrix.Atleastoneeigenvalue=0.

• Positivesemidefinite.Non-negativeeigen values

2

664

1 �1 0 0�1 2 �1 00 �1 2 �10 0 �1 1

3

775 =

2

664

1 0 0 00 2 0 00 0 2 00 0 0 1

3

775�

2

664

0 1 0 01 0 1 00 1 0 10 0 1 0

3

775

Page 5: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Laplacian andrandomwalks

• Supposewearedoingarandomwalkonagraph• Letu(i)betheprobabilityofthewalkbeingatnodei– E.g.initiallyitisatstartingnodes– After10steps,probabilityishighernears,lowatnodesfartheraway

– Question:Howdoestheprobabilitychangewithtime?

– Thisprobabilitydiffuseswithtime.Likeheatdiffuses

Page 6: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Laplacian matrix

• Imagineasmallanddifferentquantityofheatateachnode(say,inametalmesh)

• wewriteafunctionu:u(i)=heatati• Thisheatwillspreadthroughthemesh/graph• Question:howmuchheatwilleachnodehaveafterasmallamountoftime?

Page 7: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Heat diffusion

• Supposenodesi andjareneighbors– Howmuchheatwillflowfromi toj?

Page 8: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Heat diffusion

• Supposenodesi andjareneighbors• Inashorttime,howmuchheatwillflowfromi toj?

• Proportionaltothegradient:(u(i)- u(j))*∆𝑡– Letuskeep∆𝑡fixed,andwritejust(u(i)- u(j))

• thisissigned:negativemeansheatflowsintoi

Page 9: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Heat diffusion

• Ifi hasneighborsj1,j2….• Thenheatflowingoutofi is:

=(u(i)- u(j1))+(u(i)- u(j2))+(u(i)- u(j3))+…=degree(i)*u(i)- u(j1)- u(j2)- u(j3)- ….

• HenceL=D- A

Page 10: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

The heat equation

• Thenetheatoutflowofnodesinatimestep• Thechangeinheatdistributioninasmalltimestep– Therateofchangeofheatdistribution

@u

@t= L(u)

Page 11: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Thesmoothheatequation

• ThesmoothLaplacian:

• Thesmoothheatequation:

�f =@f

@t

Page 12: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Heatflow

• Willeventuallyconvergetov[0]:thezeroth eigenvector,witheigen value�0 = 0

v[0]=const forthe chain

Page 13: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Eigenvectors

• Othereigen vectors• Encodevariouspropertiesofthegraph• Havemanyapplications

Page 14: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Application1:Drawingagraph(Embedding)

• Problem:Computerdoesnotknowwhatagraphissupposedtolooklike

• Agraphisajumbleofedges

• Consideragridgraph:• Wewantitdrawnnicely

Page 15: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Graphembedding

• Findpositionsforverticesofagraphinlowdimension(comparedton)

• Commonobjective:Preservesomepropertiesofthegraphe.g.approximatedistancesbetweenvertices.Createametric– Usefulinvisualization– Findingapproximatedistances– Clustering

• Usingeigen vectors– Oneeigen vectorgivesxvaluesofnodes– Othergivesy-valuesofnodes…etc

Page 16: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Drawwithv[1]andv[2]

• Supposev[0],v[1],v[2]…areeigenvectors– Sortedbyincreasingeigenvalues

• PlotgraphusingX=v[1],Y=v[2]

• Producesthegrid

Page 17: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Intuitions:the1-Dcase

• Supposewetakethejth eigen vectorofachain

• Whatwouldthatlooklike?• Wearegoingtoplotthechainalongx-axis• Theyaxiswillhavethevalueofthenodeinthejth eigen vector

• Wewanttoseehowtheseriseandfall

Page 18: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Observations

• j=0

• j=1

• j=2

• j=3

• j=19

Page 19: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

For Allj

• Lowonesatbottom

• Highonesattop

• Codeonwebpage

Page 20: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Observations

• InDim 1grid:– v[1]ismonotone– v[2]isnotmonotone

• Indim2grid:– bothv[1]andv[2]aremonotoneinsuitabledirections

• Forlowvaluesofj:– Nearbynodeshavesimilarvalues• Usefulforembedding

Page 21: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Application2:Colouring

• Colouring:Assigncolours tovertices,suchthatneighboringverticesdonothavesamecolour– E.g.Assignmentofradiochannelstowirelessnodes.Goodcolouring reducesinterference

• Idea:Higheigen vectorsgivedissimilar valuestonearbynodes

• Useforcolouring!

Page 22: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Application3:Cuts/segmentation/clustering

• Findthesmallest‘cut’• Asmallsetofedgeswhoseremovaldisconnectsthegraph

• Clustering,communitydetection…

Page 23: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Clustering/communitydetection

• v[1]tendstostretchthenarrowconnections:discriminatesdifferentcommunities

Page 24: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Clustering:communitydetection

• Morecommunities• Spectralembeddingneedshigherdimensions

• Warning:itdoesnotalwaysworksocleanly

• Inthiscase,thedataisverysymmetric

Page 25: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

ImagesegmentationShi&malik’00

Page 26: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Laplacian

• ChangedimpliedbyLonanyinputvectorcanberepresentedbysumofactionofitseigenvectors(wesawthislasttimeforMMT)

• v[0]istheslowestcomponentofthechange– Withmultiplierλ0=0– Thesteadystatecomponent

• v[1]isslowestnon-zerocomponent– withmultiplierλ1

Page 27: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Spectralgap• λ1– λ0

• Determinestheoverallspeedofchange• Iftheslowestcomponentv[1]changesfast– Thenoverallthevaluesmustbechangingfast– Fastdiffusion

• If theslowest componentis slow– Convergencewill beslow

• Examples:– Expanders have largespectral gaps– Grids anddumbbellshave smallgaps~1/n

Page 28: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Application4:isomorphismtesting

• Eigenvaluesbeingdifferentimpliesgraphsaredifferent

• Thoughnotnecessarilytheotherway

Page 29: Spectral Graph TheorySpectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2018. Spectral methods • Understanding a graph using eigenvalues

Spectralmethods• Wideapplicabilityinsideandoutsidenetworks• Relatedtomanyfundamentalconcepts

– PCA– SVD

• Randomwalks,diffusion,heatequation…• Resultsaregoodmanytimes,butnotalways• Relativelyhardtoproveandunderstandproperties• Inefficient:eig.computationcostlyonlargematrix• (Somewhat)efficientmethodsexistformorerestricted

problems– e.g.whenwewantonlyafewsmallest/largesteigen vectors


Spectral Theory of Unsigned and Signed Graphs Applications to …jean/spectral-graph-notes.pdf · Spectral Theory of Unsigned and Signed Graphs Applications to Graph Clustering: a

Spectral Theory of Unsigned and Signed Graphs Applications to …jean/spectral-graph-notes.pdf · Spectral Theory of Unsigned and Signed Graphs Applications to Graph Clustering: a

Graph spectral techniques in computer sciences

Graph spectral techniques in computer sciences

Spectral Graph Theory, Linear Solvers, and Applications

Spectral Graph Theory, Linear Solvers, and Applications

Algorithm Design Using Spectral Graph Theory

Algorithm Design Using Spectral Graph Theory

Graph spectral analysis/Graph spectral clustering and its application to metabolic networks

Graph spectral analysis/Graph spectral clustering and its application to metabolic networks

Graph Partitioning II: Spectral Methods - Carlos …Spectral graph theory The study of the eigenvalues and eigenvectors of a graph matrix – Adjacency matrix – Laplacian matrix

Graph Partitioning II: Spectral Methods - Carlos …Spectral graph theory The study of the eigenvalues and eigenvectors of a graph matrix – Adjacency matrix – Laplacian matrix

epluse.tceptt.com · 2.01.2016  · Spectral graph theory is a mathematical theory in which linear algebra and graph theory meet. For any graph matrix M we can build a spectral graph

epluse.tceptt.com · 2.01.2016  · Spectral graph theory is a mathematical theory in which linear algebra and graph theory meet. For any graph matrix M we can build a spectral graph

Graph Partitioning and Spectral Methods

Graph Partitioning and Spectral Methods

Fast Regression Algorithms Using Spectral Graph Theory

Fast Regression Algorithms Using Spectral Graph Theory

Parallel Spectral Graph Partitioning

Parallel Spectral Graph Partitioning

Graph Reduction with Spectral and Cut Guarantees

Graph Reduction with Spectral and Cut Guarantees

Spectral Graph Theory - University of Connecticut › ... › SpecGraphTheory.pdf · D. J. Kelleher Spectral graph theory. Spectral Theorem Spectral Theorem If Ais a real symmetric

Spectral Graph Theory - University of Connecticut › ... › SpecGraphTheory.pdf · D. J. Kelleher Spectral graph theory. Spectral Theorem Spectral Theorem If Ais a real symmetric

Technical Report Spectral Clustering of Large Advertiser Datasets … · 2014-03-17 · 2.4 Spectral graph partitioning Spectral graph partitioning problem can be formulated as a

Technical Report Spectral Clustering of Large Advertiser Datasets … · 2014-03-17 · 2.4 Spectral graph partitioning Spectral graph partitioning problem can be formulated as a

Chapter 18 Spectral Graph Drawingcis515/cis515-15-graph-drawing.pdf · Spectral Graph Drawing 18.1 Graph Drawing and Energy Minimization Let G =(V,E)besomeundirectedgraph.Itisoftende-sirable

Chapter 18 Spectral Graph Drawingcis515/cis515-15-graph-drawing.pdf · Spectral Graph Drawing 18.1 Graph Drawing and Energy Minimization Let G =(V,E)besomeundirectedgraph.Itisoftende-sirable

Spectral Graph Theory - The University of Edinburgh · 2017-10-31 · Spectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2017. Project •

Spectral Graph Theory - The University of Edinburgh · 2017-10-31 · Spectral Graph Theory Social and Technological Networks Rik Sarkar University of Edinburgh, 2017. Project •

1-Jonathan Spectral Graph Theory

1-Jonathan Spectral Graph Theory

CS 598: Spectral Graph Theory. Lecture 1

CS 598: Spectral Graph Theory. Lecture 1

Spectral Graph Clustering

Spectral Graph Clustering

Introduction to Spectral Graph Theory and Graph Clusteringweb.cs.ucdavis.edu/~bai/ECS231/ho_clustering.pdf · Introduction to Spectral Graph Theory and Graph Clustering Chengming

Introduction to Spectral Graph Theory and Graph Clusteringweb.cs.ucdavis.edu/~bai/ECS231/ho_clustering.pdf · Introduction to Spectral Graph Theory and Graph Clustering Chengming

Spectral Clustering with Graph Neural Networks for Graph …Spectral Clustering with Graph Neural Networks for Graph Pooling eigenvalues, and O 2R K is an orthogonal transforma- tion

Spectral Clustering with Graph Neural Networks for Graph …Spectral Clustering with Graph Neural Networks for Graph Pooling eigenvalues, and O 2R K is an orthogonal transforma- tion