SpectralGraphTheory

SocialandTechnologicalNetworks

Rik Sarkar

UniversityofEdinburgh,2018.

Spectralmethods

• Understandingagraphusingeigen valuesandeigen vectorsofthematrix

• Wesaw:• Ranksofwebpages:componentsof1steigenvectorofsuitablematrix

• Pagerank orHITSarealgorithmsdesignedtocomputetheeigen vector

• Randomwalksandlocalpageranks helpinunderstandingcommunitystructure

Laplacian

• L=D– A[Disthediagonalmatrixofdegrees]

• Aneigen vectorhasonevalueforeachnode• Weareinterestedinpropertiesofthesevalues

2

664

1 �1 0 0�1 2 �1 00 �1 2 �10 0 �1 1

3

775 =

2

664

1 0 0 00 2 0 00 0 2 00 0 0 1

3

775�

2

664

0 1 0 01 0 1 00 1 0 10 0 1 0

3

775

Laplacian

• L=D– A[Disthediagonalmatrixofdegrees]

• Symmetric.RealEigenvalues.• Rowsum=0.Singularmatrix.Atleastoneeigenvalue=0.

• Positivesemidefinite.Non-negativeeigen values

2

664

1 �1 0 0�1 2 �1 00 �1 2 �10 0 �1 1

3

775 =

2

664

1 0 0 00 2 0 00 0 2 00 0 0 1

3

775�

2

664

0 1 0 01 0 1 00 1 0 10 0 1 0

3

775

Laplacian andrandomwalks

• Supposewearedoingarandomwalkonagraph• Letu(i)betheprobabilityofthewalkbeingatnodei– E.g.initiallyitisatstartingnodes– After10steps,probabilityishighernears,lowatnodesfartheraway

– Question:Howdoestheprobabilitychangewithtime?

– Thisprobabilitydiffuseswithtime.Likeheatdiffuses

Laplacian matrix

• Imagineasmallanddifferentquantityofheatateachnode(say,inametalmesh)

• wewriteafunctionu:u(i)=heatati• Thisheatwillspreadthroughthemesh/graph• Question:howmuchheatwilleachnodehaveafterasmallamountoftime?

Heat diffusion

• Supposenodesi andjareneighbors– Howmuchheatwillflowfromi toj?

Heat diffusion

• Supposenodesi andjareneighbors• Inashorttime,howmuchheatwillflowfromi toj?

• Proportionaltothegradient:(u(i)- u(j))*∆𝑡– Letuskeep∆𝑡fixed,andwritejust(u(i)- u(j))

• thisissigned:negativemeansheatflowsintoi

Heat diffusion

• Ifi hasneighborsj1,j2….• Thenheatflowingoutofi is:

=(u(i)- u(j1))+(u(i)- u(j2))+(u(i)- u(j3))+…=degree(i)*u(i)- u(j1)- u(j2)- u(j3)- ….

• HenceL=D- A

The heat equation

• Thenetheatoutflowofnodesinatimestep• Thechangeinheatdistributioninasmalltimestep– Therateofchangeofheatdistribution

@u

@t= L(u)

Thesmoothheatequation

• ThesmoothLaplacian:

• Thesmoothheatequation:

�f =@f

@t

Heatflow

• Willeventuallyconvergetov[0]:thezeroth eigenvector,witheigen value�0 = 0

v[0]=const forthe chain

Eigenvectors

• Othereigen vectors• Encodevariouspropertiesofthegraph• Havemanyapplications

Application1:Drawingagraph(Embedding)

• Problem:Computerdoesnotknowwhatagraphissupposedtolooklike

• Agraphisajumbleofedges

• Consideragridgraph:• Wewantitdrawnnicely

Graphembedding

• Findpositionsforverticesofagraphinlowdimension(comparedton)

• Commonobjective:Preservesomepropertiesofthegraphe.g.approximatedistancesbetweenvertices.Createametric– Usefulinvisualization– Findingapproximatedistances– Clustering

• Usingeigen vectors– Oneeigen vectorgivesxvaluesofnodes– Othergivesy-valuesofnodes…etc

Drawwithv[1]andv[2]

• Supposev[0],v[1],v[2]…areeigenvectors– Sortedbyincreasingeigenvalues

• PlotgraphusingX=v[1],Y=v[2]

• Producesthegrid

Intuitions:the1-Dcase

• Supposewetakethejth eigen vectorofachain

• Whatwouldthatlooklike?• Wearegoingtoplotthechainalongx-axis• Theyaxiswillhavethevalueofthenodeinthejth eigen vector

• Wewanttoseehowtheseriseandfall

Observations

• j=0

• j=1

• j=2

• j=3

• j=19

For Allj

• Lowonesatbottom

• Highonesattop

• Codeonwebpage

Observations

• InDim 1grid:– v[1]ismonotone– v[2]isnotmonotone

• Indim2grid:– bothv[1]andv[2]aremonotoneinsuitabledirections

• Forlowvaluesofj:– Nearbynodeshavesimilarvalues• Usefulforembedding

Application2:Colouring

• Colouring:Assigncolours tovertices,suchthatneighboringverticesdonothavesamecolour– E.g.Assignmentofradiochannelstowirelessnodes.Goodcolouring reducesinterference

• Idea:Higheigen vectorsgivedissimilar valuestonearbynodes

• Useforcolouring!

Application3:Cuts/segmentation/clustering

• Findthesmallest‘cut’• Asmallsetofedgeswhoseremovaldisconnectsthegraph

• Clustering,communitydetection…

Clustering/communitydetection

• v[1]tendstostretchthenarrowconnections:discriminatesdifferentcommunities

Clustering:communitydetection

• Morecommunities• Spectralembeddingneedshigherdimensions

• Warning:itdoesnotalwaysworksocleanly

• Inthiscase,thedataisverysymmetric

ImagesegmentationShi&malik’00

Laplacian

• ChangedimpliedbyLonanyinputvectorcanberepresentedbysumofactionofitseigenvectors(wesawthislasttimeforMMT)

• v[0]istheslowestcomponentofthechange– Withmultiplierλ0=0– Thesteadystatecomponent

• v[1]isslowestnon-zerocomponent– withmultiplierλ1

Spectralgap• λ1– λ0

• Determinestheoverallspeedofchange• Iftheslowestcomponentv[1]changesfast– Thenoverallthevaluesmustbechangingfast– Fastdiffusion

• If theslowest componentis slow– Convergencewill beslow

• Examples:– Expanders have largespectral gaps– Grids anddumbbellshave smallgaps~1/n

Application4:isomorphismtesting

• Eigenvaluesbeingdifferentimpliesgraphsaredifferent

• Thoughnotnecessarilytheotherway

Spectralmethods• Wideapplicabilityinsideandoutsidenetworks• Relatedtomanyfundamentalconcepts

– PCA– SVD

• Randomwalks,diffusion,heatequation…• Resultsaregoodmanytimes,butnotalways• Relativelyhardtoproveandunderstandproperties• Inefficient:eig.computationcostlyonlargematrix• (Somewhat)efficientmethodsexistformorerestricted

problems– e.g.whenwewantonlyafewsmallest/largesteigen vectors