Diffusion on a Graph - Department of Mathematicsbertram/lectures/Diffusion.pdf · diffusion rate,...
Transcript of Diffusion on a Graph - Department of Mathematicsbertram/lectures/Diffusion.pdf · diffusion rate,...
-
DiffusiononaGraph
-
Diffusion
Duetorandommotion,moleculesofahighconcentrationwilltendtoflowtowardsaregioninspacewheretheconcentrationislower.
Examples:Adyeinjectedintosolutionspreadingthroughacontainer,orheatspreadingfromaregionofhightemperaturetoaregionoflowertemperature.
-
TheDiffusionEquation
x=0 x=L
Considerdiffusioninonedimension(x)overtime(t)andletu(x,t)betheconcentrationofthesubstancethatisdiffusing.Then
𝜕𝑢𝜕𝑡 = 𝐷
𝜕&𝑢𝜕𝑥&
isthediffusionequationwithdiffusioncoefficientD.Onewouldalsoneedtosupplyinitialvaluesforu,u(x,0)=u0(x),andboundaryconditions ateachboundary.
-
TheDiffusionEquationTodescribediffusioninadomainwithmorethanonedimension,thesecondpartialderivativeoperatorisreplacedwiththeLaplacianoperator.Thenthediffusionequationis,
𝜕𝑢𝜕𝑡 = 𝐷𝛻
&𝑢
whereinthreedimensions
𝛻& =𝜕&
𝜕𝑥& +𝜕&
𝜕𝑦& +𝜕&
𝜕𝑧&
Laplacianoperator
-
DiffusiononaGraphWhatifthediffusingsubstancemovesalongedgesofagraphfromnodetonode?Inthiscase,thedomainisdiscrete,notacontinuum.
Letcbethediffusionrateacrosstheedge,thentheamountofsubstancethatmovesfromnodejtonodei overatimeperioddtisc 𝑢, − 𝑢. 𝑑𝑡 andfromnodei tonodejisc 𝑢. − 𝑢, 𝑑𝑡.So
𝑑𝑢.𝑑𝑡 = 𝑐 𝑢, − 𝑢.𝑑𝑢,𝑑𝑡 = 𝑐(𝑢. − 𝑢,)
ui ujc
-
DiffusiononaGraphDiffusiontoandfromnodei musttakeintoconsiderationallnodesinthegraph.Theconnectivityofthegraphisencodedintheadjacencymatrix.Hereweassumethatweareworkingwithasimplegraph.
𝑑𝑢.𝑑𝑡 = 𝑐𝐴.4 𝑢4 − 𝑢. + 𝑐𝐴.& 𝑢& − 𝑢. + ⋯+ 𝑐𝐴.6(𝑢6 − 𝑢.)
𝑑𝑢.𝑑𝑡 = 𝑐7𝐴.,(𝑢, − 𝑢.)
6
,84
or
-
DiffusiononaGraph
𝑑𝑢.𝑑𝑡 = 𝑐7𝐴.,𝑢, − 𝑐𝑢.7𝐴.,
6
,84
6
,84
Rewritingthelastexpression,
Degreeofnodei,di
= 𝑐7𝐴.,𝑢, − 𝑐𝑢.𝑑.
6
,84
WenowmakeuseoftheKronecker delta,𝛿.,
𝛿., = :0, if𝑖 ≠ 𝑗1, if𝑖 = 𝑗
-
DiffusiononaGraph
𝑑𝑢.𝑑𝑡 = 𝑐7𝐴.,𝑢, − 𝑐7𝛿.,𝑢,𝑑,
6
,84
6
,84
c𝑢.𝑑. = 𝑐 ∑ 𝛿.,𝑢,𝑑,6,84so
Definethen-dimensionalvector 𝑢 =𝑢4…𝑢6
Then c∑ 𝐴.,𝑢, = 𝑐 𝐴𝑢 .6,84
Nextdefinethe𝑛×𝑛degreematrix 𝐷 =𝑑4 0 00 … 00 0 𝑑6
Then c∑ 𝛿.,𝑢,𝑑, = 𝑐 𝐷𝑢 .6,84
-
TheGraphLaplacian
𝑑𝑢.𝑑𝑡 = 𝑐7𝐴.,𝑢, − 𝑐7𝛿.,𝑢,𝑑,
6
,84
6
,84
so
becomes𝑑𝑢𝑑𝑡 = 𝑐𝐴𝑢 − 𝑐𝐷𝑢
= 𝑐 𝐴 − 𝐷 𝑢
WenowdefinetheGraphLaplacianmatrix,
𝐿 ≡ 𝐷 − 𝐴
Theequationfordiffusiononagraphisthen
𝑑𝑢𝑑𝑡 + 𝑐𝐿𝑢 = 0
or 𝑑𝑢𝑑𝑡 + 𝑐 𝐷 − 𝐴 𝑢 = 0
-
TheGraphLaplacian
What’sinsideofL?
𝐿., = J𝑑., if𝑖 = 𝑗
−1, if𝑖 ≠ 𝑗andthereisanedge0, if𝑖 ≠ 𝑗andthereisnoedge
IsLsymmetric? Yes,why?
-
SolvingtheGraphDiffusionEquation
𝑑𝑢𝑑𝑡 + 𝑐𝐿𝑢 = 0
ThisisalinearsystemofODEs,soitissolvable.Also,sinceL issymmetricithasrealeigenvaluesandorthogonaleigenvectors,�⃗�., i= 1,⋯ , 𝑛.
Nowwritethesolutionasalinearcombinationoftheseeigenvectors,notingthatthecoefficientschangeovertime:𝑢 = ∑ 𝑎.(𝑡)�⃗�.6.84 .
InsertthisintotheODE, 7𝑑𝑎.𝑑𝑡 �⃗�. +7𝑐𝑎.𝐿�⃗�. = 0
6
.84
6
.84
7𝑑𝑎.𝑑𝑡 + 𝑐𝑎.𝜆. �⃗�. = 0
6
.84
where𝜆. isaneigenvalueofL.
-
SolvingtheGraphDiffusionEquation
Nowtaketheinnerproductofbothsidesofthelastequationwitheachoftheeigenvectors,recallingthattheyformanorthogonalset.Thisleadston differentialequationsforthecoefficients𝑎.(𝑡).
YZ[Y\+ 𝑐𝜆.𝑎. = 0 ,i=1,…,n
TheseODEsareuncoupledandlinear,sotheyhavesimpleexponentialsolutions:
𝑎. 𝑡 = 𝑎.(0)𝑒^_`[\
where𝑎.(0) istheinitialvalueofthecoefficient.
Sinceeachcoefficienthassuchasolution,thenbythesuperpositionprinciple,alinearcombinationoftheseisalsoasolution.Thus,thegeneralsolutiontothegraphdiffusiondifferentialequationis
𝑢 𝑡 =7𝑎.(0)𝑒^_`[\�⃗�.
6
.84
SpectralSolution
-
SolvingtheGraphDiffusionEquation
Howdowefindtheinitialvaluesofthecoefficients?
Usetheinitialdistributionofu amongthenodes.
𝑢 0 =7𝑎.(0)�⃗�.6
.84
Nowtaketheinnerproductofbothsideswithaneigenvector
𝑢 0 a �⃗�, = 𝑎,(0) �⃗�,&
𝑎, 0 =𝑢(0) a �⃗�,�⃗�,
&
-
SpectralPropertiesoftheGraphLaplacian
Byspectralproperties,wemeanpropertiesoftheeigenvaluesandeigenvectors.
SinceL issymmetric,itseigenvaluesarerealanditseigenvectorsareorthogonal
IsL singularornon-singular?
Lookatanyrowi. Thediagonalelementisthedegreeofthenode,𝑑..Alltheotherelementsareeither0or,foreachedge,-1.Thereareexactly𝑑. ofthese,soifyousumacrossanyrowofL youget𝑑. − 𝑑. = 0. Thisistrueforanyoftherows.Sothesumofallcolumnsofthematrixis0.Therefore,L issingular.Thatis,ithasatleastonezeroeigenvalue.Callit𝜆4 = 0.
-
SpectralPropertiesoftheGraphLaplacian
Whatistheeigenvectorassociatedwiththezeroeigenvalue?Thatis,thevector�⃗�4 suchthat𝐿�⃗�4 = 0?
Itmustbeavectorof1s,1.Why?
Because,𝐿1 isthesumofthecolumnsofL,whichweknowequals0.
DoesL haveanynegativeeigenvalues?
SoL hasnon-negativeeigenvalues,whichiscalledpositivesemidefinite.
Supposethat𝜆& < 0.Thentheterminthespectralsolution
𝑎&(0)𝑒^_`e\�⃗�& → ∞
whichweknowcan’thappen(thinkaboutspreadingdie,doesitsconcentrationgotoinfinityanywhereinthedomain?)
as 𝑡 → ∞
-
SpectralPropertiesoftheGraphLaplacianSupposethatthegraphhastwocomponents.Howisthatreflectedintheeigenvalues?
Labelthenodessothatthefirst𝑛4correspondtoonecomponentandthelast𝑛& = 𝑛 − 𝑛4 correspondtotheothercomponent.ThisresultsinablockdiagonalgraphLaplacianmatrix
L =
L1
L20
0
n1
n2
-
SpectralPropertiesoftheGraphLaplacian
L =
L1
L20
0
n1
n2
Define �⃗�4 = (1,1,1,⋯ , 0,0,0,0⋯)
n1
�⃗�& = (0,0,0,⋯ , 1,1,1,1⋯)
n2
𝐿�⃗�& = 𝐿&1 = 0Then 𝐿�⃗�4 = 𝐿41 = 0 and
So�⃗�4and�⃗�& arebotheigenvectorsofL with0eigenvalues,andL hastwo0eigenvalues.
-
SpectralPropertiesoftheGraphLaplacian
Ingeneral,thenumberof0eigenvaluesofthegraphLaplacianisequaltothenumberofcomponentsofthegraph.
OnecanordertheeigenvaluesofL fromsmallesttolargest.Then
𝜆4 = 0
If 𝜆& ≠ 0 thenthegraphisconnected
If 𝜆& = 0 thenthegraphisdisconnected
So𝜆&iscalledthealgebraicconnectivityofthegraph.
-
AsymptoticSolution
𝑢 𝑡 =7𝑎.(0)𝑒^_`[\�⃗�.
6
.84
Recallthatthespectralsolutiontothegraphdiffusionequationis
Supposethatthegraphisconnected.Then𝜆4 = 0 andallothereigenvaluesarepositive.Therearen termsinthesolutionabove.Whathappenstothesetermsas𝑡 → ∞?
Allapproach0,exceptforthefirstterm,whichisindependentoftime.Sotheasymptoticsolutionisjustthefirsttermofthespectralsolution,
𝑢h = 𝑎4 0 �⃗�4
-
AsymptoticSolution
𝑢h = 𝑎4 0 �⃗�4
But 𝑎4 0 �⃗�4 =𝑢(0) a �⃗�4�⃗�4 &
1 = 𝑢4 0 +⋯+ 𝑢6(0)
𝑛 1
So 𝑢h =𝑢4 0 +⋯+ 𝑢6(0)
𝑛 1
Howcanweinterpretthisphysically?
Inthelongterm,eachnodeintheconnectedgraphgetsthesameshareofthedye(orwhateverisdiffusing),whichisequaltothetotalamountinitiallypresentdividedbythenumberofnodes.
-
AsymptoticSolutionThiscanalsobederiveddirectlyfromthediffusionequation
𝑑𝑢𝑑𝑡 + 𝑐𝐿𝑢 = 0
Settimederivativeto0,
𝐿𝑢h = 0
SotheequilibriumvectorisaneigenvectorofthegraphLaplaciancorrespondingtothe0eigenvalue,whichiswhatwejustsawusingthedifferentapproach.
-
AsymptoticSolutionWecanrewritethisequilibriumequationbydeconstructingthegraphLaplacian
𝐿𝑢h = 0
or
(D-A)𝑢h = 0
D𝑢h = 𝐴𝑢h
𝑢h = 𝐷^4𝐴𝑢h
Holdonnow,isD invertible? Yes,aslongasthegraphisconnected
-
TheFilterMatrixDefineanewmatrix,W,whichI’llcallafiltermatrix
𝑊 ≡ 𝐷^4𝐴
𝑢h = 𝑊𝑢h
Then
Howcanweinterpretthis?Onewayistothinkaboutthefollowingfirst-order linearrecursionor differenceequation:
𝑢jk4 = 𝑊𝑢j
startingfromtheinitialvector𝑢l.Whathappensifyouiterateforever?
Thenultimatelyifthesystemconverges(anditwill),thek+1iteratewillbethesameasthekiterate.Thisgivestheequilibriumequationabove.Theequilibriumvector𝑢h isthereforethefixedpointoftherecursionwiththefiltermatrix.It’swhatyougetifyoukeepperformingthefilteringoverandoveragain.
-
TheFilterMatrix
Since 𝑢h = 𝐷^4𝐴𝑢h
𝑢h,. =∑ 𝐴.,𝑢h,,6,84∑ 𝐴.,6,84
wherewenotethat𝐴.. = 0whentherearenoself-edges.
If𝑢h isthesameateachnode,thenthisequationissatisfied.Sotheequilibriumvalueateachnodeistheaverageoftheinitialamountofdiffusiblesubstance.Thisisthepropertyofdiffusion,andmoreparticularly,theLaplacianoperator.Thusthename“graphLaplacian”forL.
-
HeterogeneousDiffusion
Sofarwehavethoughtofthegraphasanunweightedgraph.Thatis,diffusionbetweenanypairofnodeshasthesamerate,c.Thisiscalledhomogeneousdiffusion.Moregenerally,eachedgecanhaveitsowndiffusionrate,whichiscalledheterogeneousdiffusion.Sonowtheadjacencymatrixhasweightsasitselements(or0s),and
𝐿., = J𝑑., if𝑖 = 𝑗
−𝑐.,, if𝑖 ≠ 𝑗andthereisanedge0, if𝑖 ≠ 𝑗andthereisnoedge
wheredegreedi isthesumoftheweightededgesatnodei and𝑐., istheweightconnectingnodesi andj.
𝑢h,. =∑ 𝐴.,𝑢h,,6,84∑ 𝐴.,6,84
Again,
Asolutiontothisis𝑢h,. = 𝑢h foreachnodeinthenetwork.Thatis,onceagain,atequilibriumthenodesequallydividetheinitialamountofsubstance.
-
ImageProcessing
-
PixelatedImageasaNetwork
Eachpointonthegridhasagreylevel:0=white,1=black,withshadesofgreycorrespondingtointermediatevalues.Thesegridpointsarethenodes ofanetworkandtheirgreylevelisthevalueofu atthatnode.
-
PixelatedImageasaNetwork
Whataretheedges?
Mostgenerally,connecteachnodepairwithanedge,butmaketheedgesweighted(𝑐.,)bytheaffinity oftheconnectednodes.Here,affinitymeanshowsimilartheyare.
Similarcouldmeanlocation(nearestneighborsgethighestaffinity).Oritcouldmeanthegreylevelofthepixels(similaru valuesofnodes).Ifitisbasedonlocation,thentheedgeweightsarefixed.Ifbasedongreylevel,edgeweightsarefunctionsoftheu values.
-
ModifyinganImageWiththeFilterMatrix
Let𝑢l bethegreylevelvaluesoftheoriginalimage.Imageprocessingmultipliesthatvectorbythefiltermatrixtogetaproductvector,𝑢4 = 𝑊𝑢l.Ittypicallydoesthismorethanonce,applyingtherecursionrelationwesawearlier,𝑢jk4 = 𝑊𝑢j.
TheeffectthishasontheimagedependsonW.Iftheaffinitiesarechosentobeneighborsinphysicalspace,thentheeffectofthefilteringwillbetoaverageoutdifferencesamongneighbors.Theimagebecomessmootherandblurrier.
-
ModifyinganImageWiththeFilterMatrix
Iftheaffinitiesarechosenaccordingtosimilarityingreylevel,thenapplyingthefilterwillhavetheeffectofmakingsimilarpixelsmoresimilar,whichtendstosharpentheimage.Inthiscase,theweights𝑐.,andthereforetheelementsofWareupdatedwitheachiterationoftherecursionformula.
Inpractice,combinationsofaffinitybasedonlocationandbasedongreylevelsareused,anditerationstopswhensomemeasureofimagegoodnesshasbeenreached.
-
ForaGreatVideoonImageProcessing
Thefollowinglinkhasanhour-longlecturebyaresearcherfromGoogleonhowthegraphLaplacianisusedinimageprocessing:
https://www.youtube.com/watch?v=_ItmFYCr7ag
-
TheEnd