Weisfeiler-Lehman Graph Kernel (JMLR 2011)and

Neighborhood Hash Graph Kernel (ICDM 2009)

Presenter: Jose LugoPedja’s Lab Meeting

October 12, 2011

Learning on Graphs• Application domains

– Bioinformatics, Cheminformatics, WWW link, Social networks

• Motivation: Study relationships between structured objects (graphs)

G G’Graph Comparison Problem

• Define kernels on pair of graphs

k(G, G’) = <Φ(G), Φ(G’)>

k(G, G’) – measure of similarity between G and G’

• Kernel Matrix K, where Kij = k(Gi, Gj) for 1 ≤ i,j ≤ n– Properties of K

I. Symmetric II. Positive semi-definite

Graph Kernels

k(G1, G2) = <Φ(G1), Φ(G2)> = 15

G1 G2

Φ(G) = (#(T), #(L))T :=

L :=

Φ(G1) = (1, 3) Φ(G2) = (0, 5)

Graph Kernel Example

Graph Kernels

Random WalkKernels

AlgebraicKernels

Rational Kernels1

1. Certain Rational Kernels when specialized to graphs reduced to Random Walk Graph Kernel (Vishwanathan et. al. 2010)

Gӓrtner et. al. (2003)Borgwardt et. al. (2005)

Vishwanathan et. al. (2006)

Cortes et. al. (2002,2003, 2004)

Tsuda et. al. (2002)Kashima et. al. (2003,2004)

Mahé et. al. (2004)

Kondor & Borgwardt (2008)

Graphlet Kernels

Borgwardt et. al. (2007)Shervashidze et. al. (2009)

Vacic et. al. (2010)

Graph Kernels Research Efforts

MarginalizedKernels

Other GraphKernels

Ramon and Gӓrtner (2003)Horváth et. al. (2004)

Ralaivola et. al. (2005)Frӧhlich et. al. (2005)

Menchetti et. al. (2005)Borgwardt et. al. (2005)Mahé and Vert (2008)

Reviewed onVishwanathan et. al. (2010)

Kondor & Lafferty (2002)

Image taken from “A Linear-time Graph Kernel” talk by Shohei Hido, IEEE ICDM2009, Miami, Florida, 12/09/2009

Question: How to scale up graph kernels to large, labeled graphs?

f: V → Σ = {A, R, N, D, C, E, Q, G, H, I, L, K, M, F, P, S, T, W, T, Y, V}

Graph Kernels on (Large) Labeled Graphs

f: V → Σ = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 20}

or

G = (V, E, f), |V| = n, |E| = m

Weisfeiler-Lehman Graphlet KernelShervashidze et. al. (2010)

• Weisfeiler-Lehman test of isomorphism (1968)

• Define Weisfeiler-Lehman graph kernels: – kWL(G, G’), kWLsubtree(G, G’) and kWLshortestpath(G, G’)

Observed that compressed labels li(v) correspond to subtree patterns of height i rooted at v

Example

Φ(G0) = (2, 1, 1, 1, 1)Φ(G0’) = (1, 2, 1, 1, 1)k(G0, G0’) = < Φ(G0) , Φ(G0’)> = 7

Φ(G1) = (2, 0, 1, 0, 1, 1, 0, 1)Φ(G1’) = (1, 1, 0, 1, 1, 0, 1, 1)k(G1, G1’) = < Φ(G1) , Φ(G1’)> = 4

Example

Neighborhood Hash Graphlet KernelHido and Kashima (2009)

• Bit-represented node label

• Logical operations

• Neighborhood hash over nodes

• Define neighborhood hash graph kernel, kNH(G, G’)

• Linear time complexity with # of edges

Bit-represented Node Label

Image taken from “A Linear-time Graph Kernel” talk by Shohei Hido, IEEE ICDM2009, Miami, Florida, 12/09/2009

Logical Operations on Bit Labels

• XOR (si, sj)– Exclusive OR– Order-independent

• ROTk– k-bit rotation– move left most k-bits to

the right

Neighborhood Hash over a Node, NH(v)

NH(v) uniquely represents the distribution of the node labels around v

Neighborhood Hash over a Graph, NH(G)

G0

Gi = NH(Gi-1)GiG1 …

ith-Hash graph1st-Hash graph

Image taken from “A Linear-time Graph Kernel” talk by Shohei Hido, IEEE ICDM2009, Miami, Florida, 12/09/2009

Gr+1 contains high-order relationships between the nodes with order r

Neighborhood Hash Graph Kernel K(i)

NH(Gi, Gi’)

Example

Image taken from “A Linear-time Graph Kernel” talk by Shohei Hido, IEEE ICDM2009, Miami, Florida, 12/09/2009

QUESTIONS?

Thank You!