Deep Learning for Network Biology - Stanford...
Transcript of Deep Learning for Network Biology - Stanford...
![Page 1: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/1.jpg)
Deep Learning for Network BiologyMarinka Zitnik and Jure Leskovec
Stanford University
1Deep Learning for Network Biology --
snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
![Page 2: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/2.jpg)
This Tutorial
snap.stanford.edu/deepnetbio-ismb
ISMB 2018July 6, 2018, 2:00 pm - 6:00 pm
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 2
![Page 3: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/3.jpg)
This Tutorial
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 3
1) Node embeddings§ Map nodes to low-dimensional embeddings§ Applications: PPIs, Disease pathways
2) Graph neural networks§ Deep learning approaches for graphs§ Applications: Gene functions
3) Heterogeneous networks§ Embedding heterogeneous networks§ Applications: Human tissues, Drug side effects
![Page 4: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/4.jpg)
4
Part 2: Graph Neural
Networks
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
Some materials adapted from:• Hamilton et al. 2018. Representation Learning on
Networks. WWW.
![Page 5: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/5.jpg)
f ( )=Embedding Nodes
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 5
Intuition: Map nodes to d-dimensional embeddings such that similar nodes in the graph are embedded close together
Disease similaritynetwork
2-dimensional nodeembeddings
![Page 6: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/6.jpg)
Embedding Nodes
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 6
Goal: Map nodes so that similarity in the embedding space (e.g., dot product) approximates similarity in the network
Input network d-dimensional embedding space
![Page 7: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/7.jpg)
Embedding Nodes
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 7
similarity(u, v) ⇡ z>v zuGoal:
Need to define!
Input network d-dimensional embedding space
![Page 8: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/8.jpg)
Two Key Components
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 8
§ Encoder: Map a node to a low-dimensional vector:
§ Similarity function defines how relationships in the input network map to relationships in the embedding space:
enc(v) = zvnode in the input graph
d-dimensional embedding
Similarity of u and vin the network
dot product between node embeddings
similarity(u, v) ⇡ z>v zu
![Page 9: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/9.jpg)
So Far: Shallow Encoders
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 9
Shallow encoders:§ One-layer of data
transformation§ A single hidden layer
maps node 𝑢 to embedding 𝒛& via function 𝑓, e.g., 𝒛& = 𝑓 𝒛), 𝑣 ∈ 𝑁. 𝑢
![Page 10: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/10.jpg)
Shallow Encoders
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 10
§ Limitations of shallow encoding:§ O(|V|) parameters are needed:
§ No sharing of parameters between nodes§ Every node has its own unique embedding
§ Inherently “transductive”: § Cannot generate embeddings for nodes that
are not seen during training§ Do not incorporate node features:
§ Many graphs have features that we can and should leverage
![Page 11: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/11.jpg)
Deep Graph Encoders
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 11
§ Next: We will now discuss deep methods based on graph neural networks:
§ Note: All these deep encoders can be combined with similarity functions from the previous section
enc(v) = multiple layers of non-linear transformation of graph structure
![Page 12: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/12.jpg)
Deep Graph Encoders
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 12
…
![Page 13: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/13.jpg)
Idea: Convolutional NetworksCNN on an image:
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 13
Goal is to generalize convolutions beyond simple latticesLeverage node features/attributes (e.g., text, images)
![Page 14: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/14.jpg)
From Images to NetworksSingle CNN layer with 3x3 filter:
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 14
End-to-end learning on graphs with GCNs Thomas Kipf
Convolutional neural networks (on grids)
5
(Animation by Vincent Dumoulin)
Single CNN layer with 3x3 filter:
Image GraphTransform information at the neighbors and combine it§ Transform “messages” ℎ0 from neighbors: 𝑊0ℎ0§ Add them up: ∑ 𝑊0ℎ0�
0
![Page 15: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/15.jpg)
Real-World GraphsBut what if your graphs look like this?
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 15
End-to-end learning on graphs with GCNs Thomas Kipf
or this:
Graph-structured data
6
… …
…
Input
Hidden layer Hidden layer
ReLU
Output
ReLU
What if our data looks like this?
End-to-end learning on graphs with GCNs Thomas Kipf
or this:
Graph-structured data
6
… …
…
Input
Hidden layer Hidden layer
ReLU
Output
ReLU
What if our data looks like this?
or this:
§ Examples:Biological networks, Medical networks, Social networks, Information networks, Knowledge graphs, Communication networks, Web graph, …
![Page 16: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/16.jpg)
A Naïve Approach§ Join adjacency matrix and features§ Feed them into a deep neural net:
§ Issues with this idea:§ 𝑂(𝑁) parameters§ Not applicable to graphs of different sizes§ Not invariant to node ordering
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 16End-to-end learning on graphs with GCNs Thomas Kipf
A B C D EABCDE
0 1 1 1 0 1 01 0 0 1 1 0 01 0 0 1 0 0 11 1 1 0 1 1 10 1 0 1 0 1 0
Feat
A naïve approach
8
• Take adjacency matrix and feature matrix
• Concatenate them
• Feed them into deep (fully connected) neural net
• Done?
Problems:
• Huge number of parameters • No inductive learning possible
?A
C
B
D
E
[A,X]
![Page 17: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/17.jpg)
Outline of This Section
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 17
1.Basics of deep learning for graphs
2.Graph convolutional networks
3.Biomedical applications
![Page 18: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/18.jpg)
18
Basics of Deep Learning for Graphs
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
Based on material from:• Hamilton et al. 2017. Representation Learning on Graphs: Methods and
Applications. IEEE Data Engineering Bulletin on Graph Systems.• Scarselli et al. 2005. The Graph Neural Network Model. IEEE Transactions
on Neural Networks. • Kipf et al., 2017. Semisupervised Classification with Graph Convolutional
Networks. ICLR.
![Page 19: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/19.jpg)
Setup
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 19
§ Assume we have a graph 𝐺:§ 𝑉 is the vertex set§ 𝑨 is the adjacency matrix (assume binary)§ 𝑿 ∈ ℝ=×|@| is a matrix of node features
§ Biologically meaningful node features:– E.g., immunological signatures, gene
expression profiles, gene functional information§ No features:
– Indicator vectors (one-hot encoding of a node)
![Page 20: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/20.jpg)
ExamplesProtein-protein interaction networks in different tissues, e.g., blood, substantia nigra
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 20
WNT1RPT6
Node feature: Associations of proteins with midbrain development
Node feature: Associations of proteins with angiogenesis
![Page 21: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/21.jpg)
Graph Convolutional NetworksGraph Convolutional Networks:
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 21
Problem: For a given subgraph how to come with canonical node orderingLearning convolutional neural networks for graphs. M. Niepert, M. Ahmed, K. Kutzkov ICML. 2016.
Learning Convolutional Neural Networks for Graphs
a sequence of words. However, for numerous graph col-lections a problem-specific ordering (spatial, temporal, orotherwise) is missing and the nodes of the graphs are notin correspondence. In these instances, one has to solve twoproblems: (i) Determining the node sequences for whichneighborhood graphs are created and (ii) computing a nor-malization of neighborhood graphs, that is, a unique map-ping from a graph representation into a vector space rep-resentation. The proposed approach, termed PATCHY-SAN,addresses these two problems for arbitrary graphs. For eachinput graph, it first determines nodes (and their order) forwhich neighborhood graphs are created. For each of thesenodes, a neighborhood consisting of exactly k nodes is ex-tracted and normalized, that is, it is uniquely mapped to aspace with a fixed linear order. The normalized neighbor-hood serves as the receptive field for a node under consider-ation. Finally, feature learning components such as convo-lutional and dense layers are combined with the normalizedneighborhood graphs as the CNN’s receptive fields.
Figure 2 illustrates the PATCHY-SAN architecture whichhas several advantages over existing approaches: First, itis highly efficient, naively parallelizable, and applicable tolarge graphs. Second, for a number of applications, rang-ing from computational biology to social network analysis,it is important to visualize learned network motifs (Miloet al., 2002). PATCHY-SAN supports feature visualiza-tions providing insights into the structural properties ofgraphs. Third, instead of crafting yet another graph kernel,PATCHY-SAN learns application dependent features with-out the need to feature engineering. Our theoretical contri-butions are the definition of the normalization problem ongraphs and its complexity; a method for comparing graphlabeling approaches for a collection of graphs; and a resultthat shows that PATCHY-SAN generalizes CNNs on images.Using standard benchmark data sets, we demonstrate thatthe learned CNNs for graphs are both efficient and effec-tive compared to state of the art graph kernels.
2. Related WorkGraph kernels allow kernel-based learning approaches suchas SVMs to work directly on graphs (Vishwanathan et al.,2010). Kernels on graphs were originally defined as sim-ilarity functions on the nodes of a single graph (Kondor& Lafferty, 2002). Two representative classes of kernelsare the skew spectrum kernel (Kondor & Borgwardt, 2008)and kernels based on graphlets (Kondor et al., 2009; Sher-vashidze et al., 2009). The latter is related to our work,as it builds kernels based on fixed-sized subgraphs. Thesesubgraphs, which are often called motifs or graphlets, re-flect functional network properties (Milo et al., 2002; Alon,2007). However, due to the combinatorial complexity ofsubgraph enumeration, graphlet kernels are restricted to
... ...neighborhood graph construction
convolutional architecture
node sequence selection
graph normalization
Figure 2. An illustration of the proposed architecture. A nodesequence is selected from a graph via a graph labeling procedure.For some nodes in the sequence, a local neighborhood graph is as-sembled and normalized. The normalized neighborhoods are usedas receptive fields and combined with existing CNN components.
subgraphs with few nodes. An effective class of graphkernels are the Weisfeiler-Lehman (WL) kernels (Sher-vashidze et al., 2011). WL kernels, however, only sup-port discrete features and use memory linear in the num-ber of training examples at test time. PATCHY-SAN usesWL as one possible labeling procedure to compute re-ceptive fields. Deep graph kernels (Yanardag & Vish-wanathan, 2015) and graph invariant kernels (Orsini et al.,2015) compare graphs based on the existence or count ofsmall substructures such as shortest paths (Borgwardt &Kriegel, 2005), graphlets, subtrees, and other graph in-variants (Haussler, 1999; Orsini et al., 2015). In con-trast, PATCHY-SAN learns substructures from graph dataand is not limited to a predefined set of motifs. More-over, while all graph kernels have a training complexityat least quadratic in the number of graphs (Shervashidzeet al., 2011), which is prohibitive for large-scale problems,PATCHY-SAN scales linearly with the number of graphs.
Graph neural networks (GNNs) (Scarselli et al., 2009) area recurrent neural network architecture defined on graphs.GNNs apply recurrent neural networks for walks on thegraph structure, propagating node representations until afixed point is reached. The resulting node representationsare then used as features in classification and regressionproblems. GNNs support only discrete labels and performas many backpropagation operations as there are edges andnodes in the graph per learning iteration. Gated Graph Se-quence Neural Networks modify GNNs to use gated recur-rent units and to output sequences (Li et al., 2015).
Recent work extended CNNs to topologies that differ fromthe low-dimensional grid structure (Bruna et al., 2014;Henaff et al., 2015). All of these methods, however, assumeone global graph structure, that is, a correspondence of thevertices across input examples. (Duvenaud et al., 2015)perform convolutional type operations on graphs, develop-ing a differentiable variant of one specific graph feature.
End-to-end learning on graphs with GCNs Thomas Kipf
or this:
Graph-structured data
6
… …
…
Input
Hidden layer Hidden layer
ReLU
Output
ReLU
What if our data looks like this?
![Page 22: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/22.jpg)
Our Approach
Learn how to propagate information across the graph to compute node features
22Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
Determine node computation graph
Propagate andtransform information
𝑖
Idea: Node’s neighborhood defines a computation graph
Semi-Supervised Classification with Graph Convolutional Networks. T. N. Kipf, M. Welling, ICLR 2017
![Page 23: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/23.jpg)
Idea: Aggregate Neighbors
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 23
Key idea: Generate node embeddingsbased on local network neighborhoods
INPUT GRAPH
TARGET NODE B
DE
F
CA
B
C
D
A
A
A
C
F
B
E
A
![Page 24: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/24.jpg)
Idea: Aggregate Neighbors
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 24
Intuition: Nodes aggregate information from their neighbors using neural networks
INPUT GRAPH
TARGET NODE B
DE
F
CA
B
C
D
A
A
A
C
F
B
E
A
Neural networks
![Page 25: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/25.jpg)
Idea: Aggregate Neighbors
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 25
Intuition: Network neighborhood defines a computation graph
Every node defines a computation graph based on its neighborhood!
![Page 26: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/26.jpg)
Deep Model: Many Layers
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 26
§ Model can be of arbitrary depth:§ Nodes have embeddings at each layer§ Layer-0 embedding of node u is its input feature, i.e. xu.
INPUT GRAPH
TARGET NODE B
DE
F
CA
B
C
D
A
A
A
C
F
B
E
A
xA
xB
xC
xExF
xA
xA
Layer-2
Layer-1Layer-0
![Page 27: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/27.jpg)
Aggregation Strategies
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 27
INPUT GRAPH
TARGET NODE B
DE
F
CA
B
C
D
A
A
A
C
F
B
E
A
?
?
?
?
What’s in the box!?
§ Neighborhood aggregation: Key distinctions are in how different approaches aggregate information across the layers
![Page 28: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/28.jpg)
Neighborhood Aggregation
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 28
INPUT GRAPH
TARGET NODE B
DE
F
CA
B
C
D
A
A
A
C
F
B
E
A
§ Basic approach: Average information from neighbors and apply a neural network
1) average messages from neighbors
2) apply neural network
![Page 29: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/29.jpg)
Average of neighbor’s previous layer embeddings
The Math: Deep Encoder
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 29
§ Basic approach: Average neighbor messages and apply a neural network
Initial 0-th layer embeddingsare equal to node features
Embedding after K layers of neighborhood
aggregation
Non-linearity (e.g., ReLU)
Previous layer embedding of vh0
v = xv
hkv = �
0
@Wk
X
u2N(v)
hk�1u
|N(v)| +Bkhk�1v
1
A , 8k 2 {1, ...,K}
zv = hKv
![Page 30: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/30.jpg)
Training the Model
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 30
Need to define a loss function on the embeddings!
How do we train the model to generate embeddings?
𝒛C
![Page 31: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/31.jpg)
Model Parameters
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 31
We can feed these embeddings into any loss function and run stochastic gradient descent to train the weight parameters
trainable weight matrices (i.e., what we learn) h0
v = xv
hkv = �
0
@Wk
X
u2N(v)
hk�1u
|N(v)| +Bkhk�1v
1
A , 8k 2 {1, ...,K}
zv = hKv
![Page 32: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/32.jpg)
Unsupervised Training
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 32
§ Train in an unsupervised manner:§ Use only the graph structure§ “Similar” nodes have similar embeddings
§ Unsupervised loss function can be anything from the last section, e.g., a loss based on§ Random walks (node2vec, DeepWalk, struc2vec)§ Graph factorization§ Node proximity in the graph
![Page 33: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/33.jpg)
Unsupervised: Example
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 33
Image from: Rhee et al. 2017. Hybrid Approach of Relation Network and Localized Graph Convolutional Filtering for Breast Cancer Subtype Classification. arXiv.
![Page 34: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/34.jpg)
Supervised Training
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 34
Directly train the model for a supervised task (e.g., node classification)
Safe or toxic drug?
Safe or toxic drug?
E.g., a drug-drug interaction network
![Page 35: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/35.jpg)
Supervised: Example
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 35
Graph neural network applied to gene-gene interaction graph to predict gene expression level
Single gene inference task byadding nodes based on their distance from the node we want to predict
Image from: Dutil et al. 2018. Towards Gene Expression Convolutions using Gene Interaction Graphs. arXiv.
![Page 36: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/36.jpg)
Training the Model
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 36
Directly train the model for a supervised task (e.g., node classification)
Encoder output:node embedding
Classification weights
Node class label
Safe or toxic drug?
L =X
v2V
yv log(�(z>v ✓)) + (1� yv) log(1� �(z>v ✓))
![Page 37: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/37.jpg)
Model Design: Overview
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 37
1) Define a neighborhood aggregation function
2) Define a loss function on the embeddings
𝒛C
![Page 38: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/38.jpg)
Model Design: Overview
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 38
3) Train on a set of nodes, i.e., a batch of compute graphs
![Page 39: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/39.jpg)
Model Design: Overview
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 39
4) Generate embeddingsfor nodes
𝒛C
𝒛D
𝒛E
![Page 40: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/40.jpg)
Summary So Far
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 40
§ Recap: Generate node embeddings by aggregating neighborhood information§ We saw a basic variant of this idea§ Key distinctions are in how different approaches
aggregate information across the layers
§ Next: Describe state-of-the-art graph neural network
![Page 41: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/41.jpg)
Outline of This Section
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 41
1.Basics of deep learning for graphs
2.Graph convolutional networks
3.Biomedical applications
![Page 42: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/42.jpg)
42
Graph Convolutional Networks
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
Based on material from:• Hamilton et al., 2017. Inductive Representation Learning on Large Graphs.
NIPS.
![Page 43: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/43.jpg)
GraphSAGE
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 43
INPUT GRAPH
TARGET NODE B
DE
F
CA
B
C
D
A
A
A
C
F
B
E
A
???
?
?
?
So far we have aggregated the neighbor messages by taking their (weighted) average
Can we do better?
![Page 44: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/44.jpg)
INPUT GRAPH
TARGET NODE B
DE
F
CA
B
C
D
A
A
A
C
F
B
E
A
GraphSAGE: Idea
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 44
hkv = �
�⇥Ak · agg({hk�1
u , 8u 2 N(v)}),Bkhk�1v
⇤�
Any differentiable function that maps set of vectors in 𝑁(𝑢) to a single vector
![Page 45: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/45.jpg)
§ Simple neighborhood aggregation:
§ GraphSAGE:
GraphSAGE Aggregation
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 45
generalized aggregation
Concatenate self embedding and neighbor embedding
hkv = �
�⇥Wk · agg
�{hk�1
u , 8u 2 N(v)}�,Bkh
k�1v
⇤�
hkv = �
0
@Wk
X
u2N(v)
hk�1u
|N(v)| +Bkhk�1v
1
A
![Page 46: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/46.jpg)
Variants of Aggregation
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 46
Mean: Take a weighted average of neighbors
Pool: Transform neighbor vectors and apply symmetric vector function
LSTM: Apply LSTM to reshuffled of neighbors
agg =X
u2N(v)
hk�1u
|N(v)|
agg = LSTM�[hk�1
u , 8u 2 ⇡(N(v))]�
element-wise mean/maxagg = �
�{Qhk�1
u , 8u 2 N(v)}�
![Page 47: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/47.jpg)
Summary So Far
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 47
Key idea: Generate node embeddingsbased on local neighborhoods
§ Nodes aggregate “messages” from their neighbors using neural networks
𝑣
hkv = �
0
@Wk
X
u2N(v)
hk�1u
|N(v)| +Bkhk�1v
1
Ahkv = �
0
@Wk
X
u2N(v)
hk�1u
|N(v)| +Bkhk�1v
1
A
hkv = �
0
@Wk
X
u2N(v)
hk�1u
|N(v)| +Bkhk�1v
1
A
![Page 48: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/48.jpg)
More on Graph Neural Nets
48Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
Attention-based neighborhood aggregation:§ Graph attention networks (Hoshen, 2017; Velickovic et al., 2018;
Liu et al., 2018)
Embedding edges and entire graphs:§ Graph neural nets with edge embeddings (Battaglia et al., 2016;
Gilmer et. al., 2017)§ Embedding entire graphs (Duvenaud et al., 2015; Dai et al., 2016;
Li et al., 2018)
Spectral approaches to graph neural networks:§ Spectral graph CNN & ChebNet (Bruna et al., 2015;
Defferrard et al., 2016)
Hyperbolic geometry and hierarchical embeddings:§ Hierarchical relations (Nickel et al., 2017; Nickel et al., 2018)
![Page 49: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/49.jpg)
Outline of This Section
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 49
1.Basics of deep learning for graphs
2.Graph convolutional networks
3.Biomedical applications
![Page 50: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/50.jpg)
50
Application: Tissue-specific Protein
Function Prediction
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
Material based on:• Zitnik and Leskovec. 2017. Predicting Multicellular Function through
Multilayer Tissue Networks. ISMB.• Hamilton et al., 2017. Inductive Representation Learning on Large Graphs.
NIPS.
![Page 51: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/51.jpg)
[Greene et al. 2015, Yeger & Sharan 2015, GTEx and others]Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 51
Why Protein Functions?Knowledge of protein functions in different
tissues is essential for:§ Understanding human biology§ Interpreting genetic variation§ Developing disease treatments
![Page 52: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/52.jpg)
52
Biotechnological limits & rapid growth of sequence data: most proteins can only be
annotated computationally
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
Why Predicting Protein Functions?
![Page 53: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/53.jpg)
Protein Function Prediction
53
CDC3
CDC16CLB4
RPN3RPT1
RPT6
UNK1
UNK2
CDC3
CDC16CLB4
RPN3RPT1
RPT6
UNK1
Cellproliferation
Cell cycle
UNK2
Machine Learning
This is a multi-label node classification taskDeep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
![Page 54: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/54.jpg)
What Does My Protein Do?Goal: Given a protein and a tissue, predict the protein’s functions in that tissue
Proteins×Functions×Tissue𝑠 → [0,1]
𝑊𝑁𝑇1×(Midbraindevelopment, Substantianigra) → 0.9RPT6×(Angiogenesis, Blood) → 0.05
Midbrain development
WNT1
Substantia nigra tissue
Angiogenesis
RPT6
Blood tissue
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 54
![Page 55: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/55.jpg)
Existing Research§ Guilty by association: protein’s function is
determined based on who it interacts with § No tissue-specificity
§ Protein functions are assumed constantacross organs and tissues:§ Functions in heart are the same as in skin
Lack of methods for predicting protein functions in different biological contexts
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 55
![Page 56: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/56.jpg)
Challenges§ Tissues are related to each other:
§ Proteins in biologically similar tissues have similar functions
§ Proteins are missing in some tissues§ Little is known about tissue-specific
protein functions:§ Many tissues have no annotations
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 56
![Page 57: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/57.jpg)
Approach1. Represent every tissue with a separate
protein-protein interaction graph:§ Protein function prediction is a multi-label node
classification task§ Each protein can have 0, 1, or more functions
(labels) in each tissue2. Learn protein embeddings:
§ Use PPI graphs and labels to train GraphSAGE:§ Learn how to embed proteins in each tissue:
– Aggregate neighborhood information– Share parameters in the encoder
§ Use inductive learning!Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 57
![Page 58: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/58.jpg)
Inductive Learning of Tissues
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 58
INPUT GRAPH
B
DE
F
CA
Compute graph for node A Compute graph for node B
shared parameters
shared parameters
WkThis image cannot currently be
§ The same aggregation parameters are shared for all nodes:§ Can generalize to unseen nodes§ Can make predictions on entirely unseen
graphs (tissues)!
Neural model for node A Neural model for node B
![Page 59: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/59.jpg)
Inductive Learning of Tissues
59
1. Train on a protein-protein interaction graph from one tissue 2. Generate embeddings and make predictions for newly
collected data about a different tissue
Train on forebrain tissue Generalize to blood tissue
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
Midbrain development
WNT1
Angiogenesis
RPT6
Inductive node embedding generalize to entirely unseen graphs
![Page 60: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/60.jpg)
Data and Setup§ Data:
§ Protein-protein interaction (PPI) graphs, with each graph corresponding to a different human tissue
§ Use positional gene sets, motif gene sets, and immunological signatures from MSigDB as node features§ Feature data is very sparse (42% of nodes have no features)§ This makes leveraging neighborhood information critical
§ Use Gene Ontology annotations as labels § Setup:
§ Multi-label node classification:§ Each protein can have 0, 1, or more functions (labels) in each tissue
§ Train GraphSAGE on 20 tissue-specific PPI graphs§ Generate new embeddings “on the fly”§ Make prediction on entirely unseen graphs (i.e., new tissues)
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 60
![Page 61: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/61.jpg)
Annotating New Tissues§ Transfer protein functions to an unannotated tissue§ Task: Predict functions in target tissue without access to
any annotation/label in that tissue
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 61
§ GraphSAGE significantly outperforms the baseline approaches
§ LSTM- and pooling-based aggregators outperform mean- and GCN-based aggregators
Unsup. – unsupervised; Sup. – fully supervised GraphSAGEF1 – scores are in [0,1], higher is better
![Page 62: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/62.jpg)
Outline of This Section
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018 62
1.Basics of deep learning for graphs
2.Graph convolutional networks
3.Biomedical applications
![Page 63: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/63.jpg)
63
PhDStudents
Post-DoctoralFellows
Funding
Collaborators
IndustryPartnerships
ClaireDonnat
MitchellGordon
DavidHallac
EmmaPierson
HimabinduLakkaraju
RexYing
TimAlthoff
WillHamilton
BaharanMirzasoleiman
MarinkaZitnik
MicheleCatasta
SrijanKumar
StephenBach
RokSosic
ResearchStaff
AdrijanBradaschia
DanJurafsky,Linguistics,StanfordUniversityChristianDanescu-Miculescu-Mizil,InformationScience,CornellUniversityStephenBoyd,ElectricalEngineering,StanfordUniversityDavidGleich,ComputerScience,PurdueUniversityVSSubrahmanian,ComputerScience,UniversityofMarylandSarahKunz,Medicine,HarvardUniversityRussAltman,Medicine,StanfordUniversityJochen Profit,Medicine,StanfordUniversityEricHorvitz,MicrosoftResearchJonKleinberg,ComputerScience,CornellUniversitySendhill Mullainathan,Economics,HarvardUniversityScottDelp,Bioengineering,StanfordUniversityJensLudwig,HarrisPublicPolicy,UniversityofChicago
GeetSethi
AlexPorter
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018
![Page 64: Deep Learning for Network Biology - Stanford Universitysnap.stanford.edu/deepnetbio-ismb/slides/deepnetbio-part2-gcn.pdf · Graph neural networks (GNNs) (Scarselli et al., 2009) are](https://reader033.fdocuments.us/reader033/viewer/2022042302/5ecd217651f84567ae76a7fd/html5/thumbnails/64.jpg)
64
Many interesting high-impact projects in Machine Learning and Large Biomedical Data
Applications: Precision Medicine & Health, Drug Repurposing, Drug Side Effect modeling, Network Biology, and many more
Deep Learning for Network Biology -- snap.stanford.edu/deepnetbio-ismb -- ISMB 2018