This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC

GraphSignalProcessingusingDeepNeuralNetworks

JayaramanJ.Thiagarajan

2

JointWorkWith

Rushil AnirudhLLNL

BhavyaKailkhuraLLNL

PrasannaSattigeriIBMResearch

Karthi RamamurthyIBMResearch

3

GraphsareEverywhere

Graphics Social Networks Medicine

Graphscancapturecomplexrelationalcharacteristicsforanalysisandinference

4

UndirectedGraphs– BasicNotations

5

SignalsonaGraphcanbeasInterestingastheGraphItself

Analyzecharacteristicsofthesignalbyexploitingthegraph

structure

§ Examples:— BrainnetworksandfMRIsignals— Generegulatorynetworksandgeneexpressionlevels— Socialnetworksandinformationcascades

6

ConnectiontoFourierAnalysis– EigenvectorsasFrequencies

7

§ UsingtheeigenvectorsoftheLaplacian,wecandefine

GraphFourierTransform

8

GraphFilteringistheKeytoDesigningGraphConvolutionalNeuralNetworks

9

GraphConvolutionalNetworksLearnaSequenceofGraphFilters

Chebyshev Approximation

10

Problem:UsingGraphCNNforAutismSpectrumDisorderClassification

Neuropathologystudiesmapvariationsinbrainfunctionalitytoclinicalmeasures

Population Graph Design Signal ConstructionNon-imaging features(e.g.

gender/site)orcombinationofimagingandnon-imaging

features

Statisticsfrom imagingfeatures(e.g.fMRI)

11

Fine-TuningGraphCNNsisHighlyChallengingduetotheirSensitivitytoGraphDesign

Population graph

G-CNN

G-CNN

G-CNN

Features

Features

Features

Consensus Class Label

Random Graph Ensemble

Predictive Modeling

Softmax Output

N-dimensional feature per node

Weadoptabootstrappingapproachthatperformsdropoutintheinputlayerofthenetwork

12

Results:AutismBrainImagingDataExchangeInitiative§ 872patientsfrom20differentsites

§ SignalConstruction:Upper-Triangularpartofcovariancematrix+PCA

BaselinesLinear SVM 64.71Kernel SVM 65.72

Ensemble G-CNN(Gender, Site) 67.7

Linear Kernel + (Gender, Site) 67.7

Graph Kernel + (Gender, Site) 68.28

BootstrappedGraphCNNsimprovethepredictionaccuracyonthechallengingABIDEdataset

13

§ Samesignaldefinedondifferentgraphscanleadtocompletelydifferentresults.

Problem:NeighborhoodGraphConstructionisSensitivetoSampleDensityandNoise

14

Idea1:GraphAuto-EncoderscanRecovertheOptimalEmbeddingsforData

§ Auto-encodersaimtominimizethereconstructionloss

§ The“reconstructionnature”ofspectralclusteringcanbeviewedasanauto-encoder

§ UsingL2loss– HiddenfeatureswillconvergetothesmallesteigenvectorsoftheLaplacianmatrix

Laplacian Matrix

15

Idea2:SpectralEmbeddingwithQuantileLossEnablesRobustSparsificationofGraphs

PiecewiseLinearQuadraticLosstypicallyusedtoexploreheterogeneousdatasets

Low-rank decomposition of the similarity matrix

16

Idea2:SpectralEmbeddingwithQuantileLossEnablesRobustSparsificationofGraphs

PiecewiseLinearQuadraticLosstypicallyusedtoexploreheterogeneousdatasets

Low-rank decomposition of the similarity matrix

Whenthequantileparameterislarge:§ Positiveresidualswillbepenalizedlesser,i.e.theapproximationwill

underestimatethesimilarityfunction

Whenthequantileparameterissmall:§ Returnsadensegraphwithoverestimatededgestrengths

17

ConditionalQuantilesoftheSimilarityFunctionRevealtheImportanceofEdges

Number of edges at 0.1 quantile

Highest quantile at which edge persists

Minimum probability for any edge

Rate of Decay Edge Probabilities

18

§ Givenafewlabeledexamplesandaneighborhoodgraph,propagatelabelstoallsamples– Greedyrandomwalk

UsingtheLocally-ScaledGraphsinSemiSupervisedLabelPropagation

ExtremeCase– 1LabeledNode

19

§ Graphsignalprocessingprovidesaconvenientframeworkforawidevarietyofdataanalysisproblems.

§ Canbehighlyeffectiveinfusingimagingandnon-imagingfeaturesinaneuralnetworksetting

§ Generalizeddeepnetworksfordataonnon-uniformgrids– thenewkidontheblock

§ Useofappropriatelossfunctionscanleadtorobustgraphconstructions

§ Appliestoproblemsinsupervised,unsupervised,semi-supervised,transferlearning…

Conclusions

ContactJayaramanJ.ThiagarajanCenterforAppliedScientificComputingLawrenceLivermoreNationalLaboratoryEmail:jjayaram@llnl.gov

Questions?