SpectralGraphTheory

SocialandTechnologicalNetworks

RikSarkar

UniversityofEdinburgh,2017.

Project

•  Proposalfeedbacktoday/tomorrow– Pleasesharewithyourteammates!

•  ProjectguidelinesandIpsareuponthewebpage

Project-teams

•  Brainstorminteams.Submityourownproject•  Theteamistohelpyouthinkabouttheproject,discussspecificissues

•  Treatyourteammate’sprojectasanyotherbookorpaper–youcanreference/useit,butcannotclaimcredit!

•  Youarefreetodiscusswithanybody.Givecreditforsignificantideas.

Project--wriIng

•  Donotkeepitfortheend!•  Asyougo,putinplots,pictures,diagramsinthedocument.Youcanchange/removethemlater

•  Putinsmallparagraphs,descripIonsastheyoccurtoyou–youwillnotrememberthisonthelastday.

•  Rememberthethoughts,discussions,problems,ideasasyougoalong.ThiswillhelpyoutowriteaninteresIngreport.

Project--wriIng

•  Donotkeepitfortheend!•  Asyougo,putinplots,pictures,diagramsinthedocument.Youcanchange/removethemlater

•  Putinsmallparagraphs,descripIonsastheyoccurtoyou–youwillnotrememberthisonthelastday.

•  Rememberthethoughts,discussions,problems,ideasasyougoalong.ThiswillhelpyoutowriteaninteresIngreport.

Topics

•  Aretheretopicsyouwouldlikedisucssedinclass?LetmeknowonPiazza

Spectralmethods

•  Understandingagraphusingeigenvaluesandeigenvectorsofthematrix

•  Wesaw:•  Ranksofwebpages:componentsof1steigenvectorofsuitablematrix

•  PagerankorHITSarealgorithmsdesignedtocomputetheeigenvector

•  Today:otherwaysspectralmethodshelpinnetworkanalysis

Laplacian

•  L=D–A[Disthediagonalmatrixofdegrees]

•  Aneigenvectorhasonevalueforeachnode•  WeareinterestedinproperIesofthesevalues

2

664

1 �1 0 0�1 2 �1 00 �1 2 �10 0 �1 1

3

775 =

2

664

1 0 0 00 2 0 00 0 2 00 0 0 1

3

775�

2

664

0 1 0 01 0 1 00 1 0 10 0 1 0

3

775

Laplacian

•  L=D–A[Disthediagonalmatrixofdegrees]

•  Symmetric.RealEigenvalues.•  Rowsum=0.Singularmatrix.Atleastoneeigenvalue=0.

•  PosiIvesemidefinite.Non-negaIveeigenvalues

2

664

1 �1 0 0�1 2 �1 00 �1 2 �10 0 �1 1

3

775 =

2

664

1 0 0 00 2 0 00 0 2 00 0 0 1

3

775�

2

664

0 1 0 01 0 1 00 1 0 10 0 1 0

3

775

ApplicaIon1:Drawingagraph(Embedding)

•  Problem:Computerdoesnotknowwhatagraphissupposedtolooklike

•  Agraphisajumbleofedges

•  Consideragridgraph:•  Wewantitdrawnnicely

Graphembedding•  FindposiIonsforverIcesofagraphinlowdimension(comparedton)

•  CommonobjecIve:PreservesomeproperIesofthegraphe.g.approximatedistancesbetweenverIces.Createametric– UsefulinvisualizaIon–  Findingapproximatedistances–  Clustering

•  Usingeigenvectors– Oneeigenvectorgivesxvaluesofnodes– Othergivesy-valuesofnodes…etc

Drawwithv[1]andv[2]

•  Supposev[0],v[1],v[2]…areeigenvectors– Sortedbyincreasingeigenvalues

•  PlotgraphusingX=v[1],Y=v[2]

•  Producesthegrid

IntuiIons:the1-Dcase

•  Supposewetakethejtheigenvectorofachain

•  Whatwouldthatlooklike?•  Wearegoingtoplotthechainalongx-axis•  Theyaxiswillhavethevalueofthenodeinthejtheigenvector

•  Wewanttoseehowtheseriseandfall

ObservaIons•  j=0

•  j=1

•  j=2

•  j=3

•  j=19

ForAllj

ObservaIons

•  InDim1grid:–  v[1]ismonotone–  v[2]isnotmonotone

•  Indim2grid:–  bothv[1]andv[2]aremonotoneinsuitabledirecIons

•  Forlowvaluesofj:– Nearbynodeshavesimilarvalues•  Usefulforembedding

ApplicaIon2:Colouring•  Colouring:AssigncolourstoverIces,suchthatneighboringverIcesdonothavesamecolour–  E.g.Assignmentofradiochannelstowirelessnodes.Goodcolouringreducesinterference

•  Idea:Higheigenvectorsgivedissimilarvaluestonearbynodes

•  Useforcolouring!

ApplicaIon3:Cuts/segmentaIon/clustering

•  Findthesmallest‘cut’•  Asmallsetofedgeswhoseremovaldisconnectsthegraph

•  Clustering,communitydetecIon…

Clustering/communitydetecIon

•  v[1]tendstostretchthenarrowconnecIons:discriminatesdifferentcommuniIes

Clustering:communitydetecIon

•  MorecommuniIes•  Spectralembeddingneedshigherdimensions

•  Warning:itdoesnotalwaysworksocleanly

•  Inthiscase,thedataisverysymmetric

ImagesegmentaIonShi&malik’00

Laplacianmatrix

•  ImagineasmallanddifferentquanItyofheatateachnode(say,inametalmesh)

•  wewriteafuncIonu:u(i)=heatati•  Thisheatwillspreadthroughthemesh/graph•  QuesIon:howmuchheatwilleachnodehavealerasmallamountofIme?

•  “heat”canberepresentaIveoftheprobabilityofarandomwalkbeingthere

Heatdiffusion

•  Supposenodesiandjareneighbors– Howmuchheatwillflowfromitoj?

Heatdiffusion

•  Supposenodesiandjareneighbors•  Howmuchheatwillflowfromitoj?•  ProporIonaltothegradient:– u(i)-u(j)–  thisissigned:negaIvemeansheatflowsintoi

Heatdiffusion

•  Ifihasneighborsj1,j2….•  Thenheatflowingoutofiis:

=u(i)-u(j1)+u(i)-u(j2)+u(i)-u(j3)+…=degree(i)*u(i)-u(j1)-u(j2)-u(j3)-….

•  HenceL=D-A

TheheatequaIon

•  ThenetheatflowoutofnodesinaImestep•  ThechangeinheatdistribuIoninasmallImestep– TherateofchangeofheatdistribuIon

@u

@t= L(u)

ThesmoothheatequaIon

•  ThesmoothLaplacian:

•  ThesmoothheatequaIon:

�f =@f

@t

Heatflow

•  Willeventuallyconvergetov[0]:thezerotheigenvector,witheigenvalue

•  v[0]isaconstant:nomoreflow!

�0 = 0

v[0]=const

Laplacian•  ChangedimpliedbyLonanyinputvectorcanberepresentedbysumofacIonofitseigenvectors(wesawthislastImeforMMT)

•  v[0]istheslowestcomponentofthechange– WithmulIplierλ0=0

•  v[1]isslowestnon-zerocomponent– withmulIplierλ1

Spectralgap•  λ1–λ0

•  Determinestheoverallspeedofchange•  Iftheslowestcomponentv[1]changesfast–  Thenoverallthevaluesmustbechangingfast–  Fastdiffusion

•  Iftheslowestcomponentisslow–  Convergencewillbeslow

•  Examples:–  Expandershavelargespectralgaps–  Gridsanddumbbellshavesmallgaps~1/n

ApplicaIon4:isomorphismtesIng

•  Eigenvaluesdifferentimpliesgraphsaredifferent

•  Thoughnotnecessarilytheotherway

Spectralmethods•  Wideapplicabilityinsideandoutsidenetworks•  Relatedtomanyfundamentalconcepts

–  PCA–  SVD

•  Randomwalks,diffusion,heatequaIon…•  ResultsaregoodmanyImes,butnotalways•  RelaIvelytoproveproperIes•  Inefficient:eig.computaIoncostlyonlargematrix•  (Somewhat)efficientmethodsexistformorerestricted

problems–  e.g.whenwewantonlyafewsmallest/largesteigenvectors