Deep Learning for GraphsTrends & Open Questions
Federico Errica
The What
2
● Node u → Entity
● Edge (u,v) → Relationship
● Generalizations: Multigraphs & Hypergraphs
The What (cont.)
3
● Representation Learning on graphs○ Vertex & Graph embeddings
● Supervised○ Vertex/Graph classification/regression
● Unsupervised○ Link Prediction○ Clustering○ Maximum Likelihood Estimation
● Generative○ Molecule generation
A visual example (Graph t-SNE)
4
Leow, Yao Yang, Thomas Laurent, and Xavier Bresson. GraphTSNE: A Visualization Technique for Graph-Structured Data. ICLR Workshop (2019).https://leowyy.github.io/graphtsne
The Why
5
RnxmHave → fun → with → Machine → Learning
Flat Sequences
Trees
Graphs
The Why (cont.)
6
● Handle cyclic structures ○ No recursion!
● Variable size
● Variable shape
● Efficiency
● No more feature engineering○ i.e. kernel methods
Me
The feature engineering guy
The How (in a nutshell)● Neighborhood Aggregation to the rescue● Use layering to spread context between vertices
● How can we aggregate neighbors?● How many layers do we need?
7
Resemblance to CNNs● Convolution as neighborhood aggregation
○ On regular grids
● Layers increases the local receptive field of each vertex
8
A review of some works
9
NN4G (Micheli, TNNLS 2009)● Constructive approach
○ Cascade Correlation
● Aggregation function○ Sum
● The first spatial DGN!
10
CGMM (Bacciu, Errica & Micheli, ICML 2018)● A deep stack of probabilistic layers
● Unsupervised constructive approach
● Switching Parent approximation○ Borrowed from
Hidden Tree Markov Models ● It works well
○ State-of-the-art accuracycompared to GNNs
● CGMM exploits layering
11
ℓ
GraphESN (Gallicchio & Micheli, IJCNN 2010)● Does not require training but for the output layer
● Let the Reservoir reach convergence
● Train a linear readout
12
DiffPool (Ying et al., 2018)● Differentiable Pooling technique
13
Robust Comparisons (Bacciu, Errica, Micheli & Podda, ICLR 2020)
● Popular model are not reproducible
● We tried to solve this problem
● Some optimized on test 👿
● We ran 47k experimentsto fix this
14
Y: yes N: no A: ambiguous - : not provided
Theses and Projects (for students that look for a challenge 🤓 )
15
● Unsupervised Criterion to automatically select layers● Unsupervised/Probabilistic pooling strategy● CGMM extensions
○ Supervised CGMM version○ Apply to node classification tasks → (Project or Thesis)○ Automatic selection of # of states○ Use more powerful aggregation functions
● Design a new GNN and compare it against sequence/tree/graph models○ e.g. in NLP
● Interpretable/explainable DGNs● Few-shot learning with DGNs● For Projects:
○ Implement a Graph Neural Network (using Pytorch Geometrics, Deep Graph Library).
Thank you!
16
You can reach out to me via:
Email: [email protected]
Office: Room 328, Department of Computer Science
Website: http://pages.di.unipi.it/errica/
Interested? Check out “A Gentle Introduction to Deep Learning for Graphs” (pre-print on arXiv).
Powered by CIML
mailto:[email protected]://pages.di.unipi.it/errica/Top Related