Hypergraph Neural NetworksIntroduction Graph-based convolutional neural networks (Kipf and Welling...

Hypergraph Neural Networks

Yifan Feng†, Haoxuan You‡, Zizhao Zhang‡, Rongrong Ji†, Yue Gao‡∗‡KLISS, School of Software, Tsinghua University

‡Beijing National Research Center for Information Science and Technology† School of Information Science and Engineering, Xiamen University

evanfeng97, [email protected], [email protected], zz-z14,[email protected]

Abstract

In this paper, we present a hypergraph neural networks(HGNN) framework for data representation learning, whichcan encode high-order data correlation in a hypergraph struc-ture. Confronting the challenges of learning representationfor complex data in real practice, we propose to incorpo-rate such data structure in a hypergraph, which is more flexi-ble on data modeling, especially when dealing with complexdata. In this method, a hyperedge convolution operation isdesigned to handle the data correlation during representationlearning. In this way, traditional hypergraph learning proce-dure can be conducted using hyperedge convolution opera-tions efficiently. HGNN is able to learn the hidden layer rep-resentation considering the high-order data structure, whichis a general framework considering the complex data correla-tions. We have conducted experiments on citation networkclassification and visual object recognition tasks and com-pared HGNN with graph convolutional networks and othertraditional methods. Experimental results demonstrate thatthe proposed HGNN method outperforms recent state-of-the-art methods. We can also reveal from the results that the pro-posed HGNN is superior when dealing with multi-modal datacompared with existing methods.

IntroductionGraph-based convolutional neural networks (Kipf andWelling 2016), (Defferrard, Bresson, and Vandergheynst2016) have attracted much attention in recent years. Dif-ferent from traditional convolutional neural networks, graphconvolution is able to encode the graph structure of differentinput data using a neural network model and it can be usedin the semi-supervised learning procedure. Graph convolu-tional neural networks have shown superiority on represen-tation learning compared with traditional neural networksdue to its ability of using data graph structure.

In traditional graph convolutional neural network meth-ods, the pairwise connections among data are employed. Itis noted that the data structure in real practice could be be-yond pairwise connections and even far more complicated.Confronting the scenarios with multi-modal data, the situa-tion for data correlation modelling could be more complex.

∗Corresponding author.Copyright c© 2019, Association for the Advancement of ArtificialIntelligence (www.aaai.org). All rights reserved.

Visual connections

Textconnections

Socialconnections

Tweets/Microblogs

Figure 1: Examples of complex connections on social me-dia data. Each color point represents a tweet or microblog,and there could be visual connections, text connections andsocial connections among them.

Figure 1 provides examples of complex connections on so-cial media data. On one hand, the data correlation can bemore complex than pairwise relationship, which is difficultto be modeled by a graph structure. On the other hand, thedata representation tends to be multi-modal, such as the vi-sual connections, text connections and social connections inthis example. Under such circumstances, traditional graphstructure has the limitation to formulate the data correlation,which limits the application of graph convolutional neuralnetworks. Under such circumstance, it is important and ur-gent to further investigate better and more general data struc-ture model to learn representation.

To tackle this challenging issue, in this paper, we proposea hypergraph neural networks (HGNN) framework, whichuses the hypergraph structure for data modeling. Comparedwith simple graph, on which the degree for all edges ismandatory 2, a hypergraph can encode high-order data cor-relation (beyond pairwise connections) using its degree-freehyperedges, as shown in Figure 2. In Figure 2, the graph isrepresented using the adjacency matrix, in which each edgeconnects just two vertices. On the contrary, a hypergraphis easy to be expanded for multi-modal and heterogeneous

arX

iv:1

809.

0940

1v1

[cs

.LG

] 2

5 Se

p 20

18

n3

n4

n5

n6

n7

n8

n1

n2

0 00 0

0 00 0

0 00 0

0 00 0

1 01 1

0 01 0

0 00 0

0 11 0

1 10 1

0 00 0

0 10 0

0 11 0

0 00 0

0 00 0

0 00 0

0 00 0

n1n2

n3n4

n5n6

n7n8

n1 n2 n3 n4 n5n6 n7 n8W:

Graph: Hypergraph:

e1

e2

e3n3

n4

n5

n6

n7

n8

n1

n2

em-1

emn3

n4

n5

n6

n7

n8

n1

n2

Hyperedge group 1

Hyperedge group N

Data

type

1Da

ta ty

pe N

n1

n2

n3

n4

n5

n6

n7

n8

0 11 0

00

0 01 0

10

0 00 1

10

0 11 0

10

e1 e2 e3

Concat

H1:

H:

HN:1 01 00 10 11 01 00 10 1

em-1 em

n1

n2

n3

n4

n5

n6

n7

n8

0 11 0

00

0 01 0

10

0 00 1

10

0 11 0

10

e1 e2 e3

1 01 00 10 11 01 00 10 1

em-1 em

n1

n2

n3

n4

n5

n6

n7

n8

(H1) (HN)

Figure 2: The comparison between graph and hypergraph.

data representation using its flexible hyperedges. For ex-ample, a hypergraph can jointly employ multi-modal datafor hypergraph generation by combining the adjacency ma-trix, as illustrated in Figure 2. Therefore, hypergraph hasbeen employed in many computer vision tasks such as clas-sification and retrieval tasks (Gao et al. 2012). However,traditional hypergraph learning methods (Zhou, Huang, andScholkopf 2007) suffer from their high computation com-plexity and storage cost, which limits the wide applicationof hypergraph learning methods.

In this paper, we propose a hypergraph neural networksframework (HGNN) for data representation learning. In thismethod, the complex data correlation is formulated in a hy-pergraph structure, and we design a hyperedge convolutionoperation to better exploit the high-order data correlationfor representation learning. More specifically, HGNN is ageneral framework which can incorporate with multi-modaldata and complicated data correlations, and traditional graphconvolutional neural networks can be regarded as a specialcase of HGNN. To evaluate the performance of the pro-posed HGNN framework, we have conducted experimentson citation network classification and visual object recog-nition tasks. The experimental results on four datasets andcomparisons with graph convolutional network (GCN) andother traditional methods have shown better performanceof HGNN. These results indicate that the proposed HGNNmethod is more effective on learning data representation us-ing high-order and complex correlations.

The main contributions of this paper are two-fold:1. We propose a hypergraph neural networks framework,

i.e., HGNN, for representation learning using hypergraphstructure. HGNN is able to formulate complex and high-order data correlation through its hypergraph structureand can be also efficient using hyperedge convolutionoperations. It is effective on dealing with multi-modaldata/features. Moreover, GCN (Kipf and Welling 2016)can be regarded as a special case of HGNN, for which theedges in simple graph can be regarded as 2-order hyper-edges which connect just two vertices.

2. We have conducted extensive experiments on citation net-work classification and visual object classification tasks.Comparisons with state-of-the-art methods demonstratethe effectiveness of the proposed HGNN framework. Ex-periments also indicate the better performance of the pro-posed method when dealing with multi-modal data.

Related WorkIn this section, we briefly review existing works of hyper-graph learning and neural networks on graph.

Hypergraph learningIn many computer vision tasks, the hypergraph structure hasbeen employed to model high-order correlation among data.Hypergraph learning is first introduced in (Zhou, Huang, andScholkopf 2007), as a propagation process on hypergraphstructure. The transductive inference on hypergraph aims tominimize the label difference among vertices with strongerconnections on hypergraph. In (Huang, Liu, and Metaxas2009), hypergraph learning is further employed in video ob-ject segmentation. (Huang et al. 2010) used the hypergraphstructure to model image relationship and conduct transduc-tive inference process for image ranking. To further im-prove the hypergraph structure, research attention has beenattracted for leaning the weights of hyperedges, which canhave great influence on modeling the correlation of data. In(Gao et al. 2013), a l2 regularize on the weights is introducedto learn optimal hyperedge weights. In (Hwang et al. 2008),the correlation among hyperedges is further explored by aassumption that highly correlated hyperedges should havesimilar weights. Regrading multi-modal data, in (Gao etal. 2012), multi-hypergraph structure is introduced to assignweights for different sub-hypergraphs, which corresponds todifferent modalities.

Neural networks on graphSince many irregular data that do not own a grid-like struc-ture can only be represented in the form of graph, extending

Multimodal DataTraining Data Testing Data

Data

Typ

e 1

Data

Typ

e m

Hyp

ered

gegr

oup

1H

yper

edge

grou

p m

Hyperedge Generation

Hyp

ergr

aph

Con

v

Hyp

ergr

aph

Con

v

Hypergraph Neural Network

HConv 1 HConv n

Labels

Figure 3: The proposed HGNN framework.

neural networks to graph structure has attracted great atten-tion from researchers. In (Gori, Monfardini, and Scarselli2005) and (Scarselli et al. 2009), the neural network ongraph is first introduced to apply recurrent neural networksto deal with graphs. For generalizing convolution network tograph, the methods are divided into spectral and non-spectralapproaches.

For spectral approaches, the convolution operation is for-mulated in spectral domain of graph. (Bruna et al. 2013)introduces the first graph CNN, which uses the graph Lapla-cian eigenbasis as an analogy of the Fourier transform.In (Henaff, Bruna, and LeCun 2015), the spectral filterscan be parameterized with smooth coefficients to makethem spatial-localized. In (Defferrard, Bresson, and Van-dergheynst 2016), a Chebyshev expansion of the graphLaplacian is further uses to approximate the spectral filters.Then, in (Kipf and Welling 2016), the chebyshev polynomi-als are simplified into 1-order polynomials to form an effi-cient layer-wise propagation model.

For spatial approaches, the convolution operation is de-fined in groups of spatial close nodes. In (Atwood andTowsley 2016), the powers of a transition matrix is em-ployed to define the neighborhood of nodes. (Monti et al.2017) uses the local path operators in the form of Gaussianmixture models to generalize convolution in spatial domain.In (Velickovic et al. 2017), the attention mechanisms is in-troduced into the graph to build attention-based architectureto perform the node classification task on graph.

Hypergraph Neural NetworksIn this section, we introduce our proposed hypergraph neu-ral networks (HGNN). We first briefly introduce hypergraphlearning, and then the spectral convolution on hypergraphis provided. Following, we analyze the relations betweenHGNN and existing methods. In the last part of the section,some implementation details will be given.

Hypergraph learning statementWe first review the hypergraph analysis theory. Differentfrom simple graph, a hyperedge in a hypergraph an con-nect two or more vertices. A hypergraph is defined asG = (V, E ,W), which includes a vertex set V , a hyperedge

set E . Each hyperedge is assigned with a weight by W, adiagonal matrix of edge weights. The hypergraph G can bedenoted by a |V| × |E| incidence matrix H, with entries de-fined as

h(v, e) =

1, if v ∈ e0, if v 6∈ e (1)

For a vertex v ∈ V , its degree is defined as d(v) =∑e∈E ω(e)h(v, e). For an edge e ∈ E , its degree is defined

as δ(e) =∑v∈Vh(v,e). Further, De and Dv denote the di-

agonal matrices of the edge degrees and the vertex degrees,respectively.

Here let us consider the node(vertex) classification prob-lem on hypergraph, where the node labels should be smoothon the hypergraph structure. The task can be formulated asa regularization framework as introduced by (Zhou, Huang,and Scholkopf 2007):

arg minfRemp(f) + Ω(f), (2)

where Ω(f) is a regularize on hypergraph,Remp(f) denotesthe supervised empirical loss, f(·) is a classification func-tion. The regularize Ω(f) is defined as:

Ω(f) =1

2

∑e∈E

∑u,v∈V

w(e)h(u, e)h(v, e)

δ(e)( f(u)√d(u)

− f(v)√d(v)

)2 (3)

We let θ = D−1/2v HWD−e 1H>D

−1/2v and ∆ = I−Θ.

Then the normalized Ω(f) can be written as

Ω(f) = f>∆, (4)

where ∆ is positive semi-definite, and usually called the hy-pergraph Laplacian.

Spectral convolution on hypergraphGiven a hypergraph G = (V, E ,∆) with n vertices,since the hypergraph Laplacian ∆ is a n × n positivesemi-definite matrix, the eigen decomposition ∆ = ΦΛΦ>

can be employed to get the orthonormal eigen vectorsΛ = diag(λ1, . . . , φn) and a diagonal matrix Λ =

Node Feature Transform

Node Feature Node Feature Hyperedge Feature Node Feature

Edge Feature Gathering

Node Feature Aggregating N: Node Number

E: Hyperedge Number

N

C1

<latexit sha1_base64="iL6zRC2XmWpyDMX0fR/mBzocyR4=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GOxF09SwX5AE8pmu2mXbjZhdyKU0L/hxYMiXv0z3vw3btsctPpg4PHeDDPzwlQKg6775ZTW1jc2t8rblZ3dvf2D6uFRxySZZrzNEpnoXkgNl0LxNgqUvJdqTuNQ8m44ac797iPXRiTqAacpD2I6UiISjKKV/Dvio4i5Ic2BN6jW3Lq7APlLvILUoEBrUP30hwnLYq6QSWpM33NTDHKqUTDJZxU/MzylbEJHvG+ponZRkC9unpEzqwxJlGhbCslC/TmR09iYaRzazpji2Kx6c/E/r59hdB3kQqUZcsWWi6JMEkzIPAAyFJozlFNLKNPC3krYmGrK0MZUsSF4qy//JZ2LuufWvfvLWuOmiKMMJ3AK5+DBFTTgFlrQBgYpPMELvDqZ8+y8Oe/L1pJTzBzDLzgf37rlkNE=</latexit><latexit sha1_base64="iL6zRC2XmWpyDMX0fR/mBzocyR4=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GOxF09SwX5AE8pmu2mXbjZhdyKU0L/hxYMiXv0z3vw3btsctPpg4PHeDDPzwlQKg6775ZTW1jc2t8rblZ3dvf2D6uFRxySZZrzNEpnoXkgNl0LxNgqUvJdqTuNQ8m44ac797iPXRiTqAacpD2I6UiISjKKV/Dvio4i5Ic2BN6jW3Lq7APlLvILUoEBrUP30hwnLYq6QSWpM33NTDHKqUTDJZxU/MzylbEJHvG+ponZRkC9unpEzqwxJlGhbCslC/TmR09iYaRzazpji2Kx6c/E/r59hdB3kQqUZcsWWi6JMEkzIPAAyFJozlFNLKNPC3krYmGrK0MZUsSF4qy//JZ2LuufWvfvLWuOmiKMMJ3AK5+DBFTTgFlrQBgYpPMELvDqZ8+y8Oe/L1pJTzBzDLzgf37rlkNE=</latexit><latexit sha1_base64="iL6zRC2XmWpyDMX0fR/mBzocyR4=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GOxF09SwX5AE8pmu2mXbjZhdyKU0L/hxYMiXv0z3vw3btsctPpg4PHeDDPzwlQKg6775ZTW1jc2t8rblZ3dvf2D6uFRxySZZrzNEpnoXkgNl0LxNgqUvJdqTuNQ8m44ac797iPXRiTqAacpD2I6UiISjKKV/Dvio4i5Ic2BN6jW3Lq7APlLvILUoEBrUP30hwnLYq6QSWpM33NTDHKqUTDJZxU/MzylbEJHvG+ponZRkC9unpEzqwxJlGhbCslC/TmR09iYaRzazpji2Kx6c/E/r59hdB3kQqUZcsWWi6JMEkzIPAAyFJozlFNLKNPC3krYmGrK0MZUsSF4qy//JZ2LuufWvfvLWuOmiKMMJ3AK5+DBFTTgFlrQBgYpPMELvDqZ8+y8Oe/L1pJTzBzDLzgf37rlkNE=</latexit><latexit sha1_base64="iL6zRC2XmWpyDMX0fR/mBzocyR4=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GOxF09SwX5AE8pmu2mXbjZhdyKU0L/hxYMiXv0z3vw3btsctPpg4PHeDDPzwlQKg6775ZTW1jc2t8rblZ3dvf2D6uFRxySZZrzNEpnoXkgNl0LxNgqUvJdqTuNQ8m44ac797iPXRiTqAacpD2I6UiISjKKV/Dvio4i5Ic2BN6jW3Lq7APlLvILUoEBrUP30hwnLYq6QSWpM33NTDHKqUTDJZxU/MzylbEJHvG+ponZRkC9unpEzqwxJlGhbCslC/TmR09iYaRzazpji2Kx6c/E/r59hdB3kQqUZcsWWi6JMEkzIPAAyFJozlFNLKNPC3krYmGrK0MZUsSF4qy//JZ2LuufWvfvLWuOmiKMMJ3AK5+DBFTTgFlrQBgYpPMELvDqZ8+y8Oe/L1pJTzBzDLzgf37rlkNE=</latexit> N

C2

<latexit sha1_base64="3r+HEMsLu6mYRJ4k0vZFNb1c9Xo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiL56kgv2AJpTNdtMu3WzC7kQopX/DiwdFvPpnvPlv3LY5aOuDgcd7M8zMC1MpDLrut1PY2Nza3inulvb2Dw6PyscnbZNkmvEWS2SiuyE1XArFWyhQ8m6qOY1DyTvhuDH3O09cG5GoR5ykPIjpUIlIMIpW8u+JjyLmhjT6tX654lbdBcg68XJSgRzNfvnLHyQsi7lCJqkxPc9NMZhSjYJJPiv5meEpZWM65D1LFbWLguni5hm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+5/UyjG6CqVBphlyx5aIokwQTMg+ADITmDOXEEsq0sLcSNqKaMrQxlWwI3urL66Rdq3pu1Xu4qtRv8ziKcAbncAkeXEMd7qAJLWCQwjO8wpuTOS/Ou/OxbC04+cwp/IHz+QO8aZDS</latexit><latexit sha1_base64="3r+HEMsLu6mYRJ4k0vZFNb1c9Xo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiL56kgv2AJpTNdtMu3WzC7kQopX/DiwdFvPpnvPlv3LY5aOuDgcd7M8zMC1MpDLrut1PY2Nza3inulvb2Dw6PyscnbZNkmvEWS2SiuyE1XArFWyhQ8m6qOY1DyTvhuDH3O09cG5GoR5ykPIjpUIlIMIpW8u+JjyLmhjT6tX654lbdBcg68XJSgRzNfvnLHyQsi7lCJqkxPc9NMZhSjYJJPiv5meEpZWM65D1LFbWLguni5hm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+5/UyjG6CqVBphlyx5aIokwQTMg+ADITmDOXEEsq0sLcSNqKaMrQxlWwI3urL66Rdq3pu1Xu4qtRv8ziKcAbncAkeXEMd7qAJLWCQwjO8wpuTOS/Ou/OxbC04+cwp/IHz+QO8aZDS</latexit><latexit sha1_base64="3r+HEMsLu6mYRJ4k0vZFNb1c9Xo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiL56kgv2AJpTNdtMu3WzC7kQopX/DiwdFvPpnvPlv3LY5aOuDgcd7M8zMC1MpDLrut1PY2Nza3inulvb2Dw6PyscnbZNkmvEWS2SiuyE1XArFWyhQ8m6qOY1DyTvhuDH3O09cG5GoR5ykPIjpUIlIMIpW8u+JjyLmhjT6tX654lbdBcg68XJSgRzNfvnLHyQsi7lCJqkxPc9NMZhSjYJJPiv5meEpZWM65D1LFbWLguni5hm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+5/UyjG6CqVBphlyx5aIokwQTMg+ADITmDOXEEsq0sLcSNqKaMrQxlWwI3urL66Rdq3pu1Xu4qtRv8ziKcAbncAkeXEMd7qAJLWCQwjO8wpuTOS/Ou/OxbC04+cwp/IHz+QO8aZDS</latexit><latexit sha1_base64="3r+HEMsLu6mYRJ4k0vZFNb1c9Xo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiL56kgv2AJpTNdtMu3WzC7kQopX/DiwdFvPpnvPlv3LY5aOuDgcd7M8zMC1MpDLrut1PY2Nza3inulvb2Dw6PyscnbZNkmvEWS2SiuyE1XArFWyhQ8m6qOY1DyTvhuDH3O09cG5GoR5ykPIjpUIlIMIpW8u+JjyLmhjT6tX654lbdBcg68XJSgRzNfvnLHyQsi7lCJqkxPc9NMZhSjYJJPiv5meEpZWM65D1LFbWLguni5hm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+5/UyjG6CqVBphlyx5aIokwQTMg+ADITmDOXEEsq0sLcSNqKaMrQxlWwI3urL66Rdq3pu1Xu4qtRv8ziKcAbncAkeXEMd7qAJLWCQwjO8wpuTOS/Ou/OxbC04+cwp/IHz+QO8aZDS</latexit>

E

C2

<latexit sha1_base64="XsY80slebKt2ayXYT46tJsGAa2U=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiETxWsB/QhLLZbtqlm03YnQil9G948aCIV/+MN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dgobm1vbO8Xd0t7+weFR+fikbZJMM95iiUx0N6SGS6F4CwVK3k01p3EoeSccN+Z+54lrIxL1iJOUBzEdKhEJRtFK/h3xUcTckEa/1i9X3Kq7AFknXk4qkKPZL3/5g4RlMVfIJDWm57kpBlOqUTDJZyU/MzylbEyHvGeponZRMF3cPCMXVhmQKNG2FJKF+ntiSmNjJnFoO2OKI7PqzcX/vF6G0U0wFSrNkCu2XBRlkmBC5gGQgdCcoZxYQpkW9lbCRlRThjamkg3BW315nbRrVc+teg9XlfptHkcRzuAcLsGDa6jDPTShBQxSeIZXeHMy58V5dz6WrQUnnzmFP3A+fwCuYpDJ</latexit><latexit sha1_base64="XsY80slebKt2ayXYT46tJsGAa2U=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiETxWsB/QhLLZbtqlm03YnQil9G948aCIV/+MN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dgobm1vbO8Xd0t7+weFR+fikbZJMM95iiUx0N6SGS6F4CwVK3k01p3EoeSccN+Z+54lrIxL1iJOUBzEdKhEJRtFK/h3xUcTckEa/1i9X3Kq7AFknXk4qkKPZL3/5g4RlMVfIJDWm57kpBlOqUTDJZyU/MzylbEyHvGeponZRMF3cPCMXVhmQKNG2FJKF+ntiSmNjJnFoO2OKI7PqzcX/vF6G0U0wFSrNkCu2XBRlkmBC5gGQgdCcoZxYQpkW9lbCRlRThjamkg3BW315nbRrVc+teg9XlfptHkcRzuAcLsGDa6jDPTShBQxSeIZXeHMy58V5dz6WrQUnnzmFP3A+fwCuYpDJ</latexit><latexit sha1_base64="XsY80slebKt2ayXYT46tJsGAa2U=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiETxWsB/QhLLZbtqlm03YnQil9G948aCIV/+MN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dgobm1vbO8Xd0t7+weFR+fikbZJMM95iiUx0N6SGS6F4CwVK3k01p3EoeSccN+Z+54lrIxL1iJOUBzEdKhEJRtFK/h3xUcTckEa/1i9X3Kq7AFknXk4qkKPZL3/5g4RlMVfIJDWm57kpBlOqUTDJZyU/MzylbEyHvGeponZRMF3cPCMXVhmQKNG2FJKF+ntiSmNjJnFoO2OKI7PqzcX/vF6G0U0wFSrNkCu2XBRlkmBC5gGQgdCcoZxYQpkW9lbCRlRThjamkg3BW315nbRrVc+teg9XlfptHkcRzuAcLsGDa6jDPTShBQxSeIZXeHMy58V5dz6WrQUnnzmFP3A+fwCuYpDJ</latexit><latexit sha1_base64="XsY80slebKt2ayXYT46tJsGAa2U=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiETxWsB/QhLLZbtqlm03YnQil9G948aCIV/+MN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dgobm1vbO8Xd0t7+weFR+fikbZJMM95iiUx0N6SGS6F4CwVK3k01p3EoeSccN+Z+54lrIxL1iJOUBzEdKhEJRtFK/h3xUcTckEa/1i9X3Kq7AFknXk4qkKPZL3/5g4RlMVfIJDWm57kpBlOqUTDJZyU/MzylbEyHvGeponZRMF3cPCMXVhmQKNG2FJKF+ntiSmNjJnFoO2OKI7PqzcX/vF6G0U0wFSrNkCu2XBRlkmBC5gGQgdCcoZxYQpkW9lbCRlRThjamkg3BW315nbRrVc+teg9XlfptHkcRzuAcLsGDa6jDPTShBQxSeIZXeHMy58V5dz6WrQUnnzmFP3A+fwCuYpDJ</latexit>N

C2

<latexit sha1_base64="3r+HEMsLu6mYRJ4k0vZFNb1c9Xo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiL56kgv2AJpTNdtMu3WzC7kQopX/DiwdFvPpnvPlv3LY5aOuDgcd7M8zMC1MpDLrut1PY2Nza3inulvb2Dw6PyscnbZNkmvEWS2SiuyE1XArFWyhQ8m6qOY1DyTvhuDH3O09cG5GoR5ykPIjpUIlIMIpW8u+JjyLmhjT6tX654lbdBcg68XJSgRzNfvnLHyQsi7lCJqkxPc9NMZhSjYJJPiv5meEpZWM65D1LFbWLguni5hm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+5/UyjG6CqVBphlyx5aIokwQTMg+ADITmDOXEEsq0sLcSNqKaMrQxlWwI3urL66Rdq3pu1Xu4qtRv8ziKcAbncAkeXEMd7qAJLWCQwjO8wpuTOS/Ou/OxbC04+cwp/IHz+QO8aZDS</latexit><latexit sha1_base64="3r+HEMsLu6mYRJ4k0vZFNb1c9Xo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiL56kgv2AJpTNdtMu3WzC7kQopX/DiwdFvPpnvPlv3LY5aOuDgcd7M8zMC1MpDLrut1PY2Nza3inulvb2Dw6PyscnbZNkmvEWS2SiuyE1XArFWyhQ8m6qOY1DyTvhuDH3O09cG5GoR5ykPIjpUIlIMIpW8u+JjyLmhjT6tX654lbdBcg68XJSgRzNfvnLHyQsi7lCJqkxPc9NMZhSjYJJPiv5meEpZWM65D1LFbWLguni5hm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+5/UyjG6CqVBphlyx5aIokwQTMg+ADITmDOXEEsq0sLcSNqKaMrQxlWwI3urL66Rdq3pu1Xu4qtRv8ziKcAbncAkeXEMd7qAJLWCQwjO8wpuTOS/Ou/OxbC04+cwp/IHz+QO8aZDS</latexit><latexit sha1_base64="3r+HEMsLu6mYRJ4k0vZFNb1c9Xo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiL56kgv2AJpTNdtMu3WzC7kQopX/DiwdFvPpnvPlv3LY5aOuDgcd7M8zMC1MpDLrut1PY2Nza3inulvb2Dw6PyscnbZNkmvEWS2SiuyE1XArFWyhQ8m6qOY1DyTvhuDH3O09cG5GoR5ykPIjpUIlIMIpW8u+JjyLmhjT6tX654lbdBcg68XJSgRzNfvnLHyQsi7lCJqkxPc9NMZhSjYJJPiv5meEpZWM65D1LFbWLguni5hm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+5/UyjG6CqVBphlyx5aIokwQTMg+ADITmDOXEEsq0sLcSNqKaMrQxlWwI3urL66Rdq3pu1Xu4qtRv8ziKcAbncAkeXEMd7qAJLWCQwjO8wpuTOS/Ou/OxbC04+cwp/IHz+QO8aZDS</latexit><latexit sha1_base64="3r+HEMsLu6mYRJ4k0vZFNb1c9Xo=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMdiL56kgv2AJpTNdtMu3WzC7kQopX/DiwdFvPpnvPlv3LY5aOuDgcd7M8zMC1MpDLrut1PY2Nza3inulvb2Dw6PyscnbZNkmvEWS2SiuyE1XArFWyhQ8m6qOY1DyTvhuDH3O09cG5GoR5ykPIjpUIlIMIpW8u+JjyLmhjT6tX654lbdBcg68XJSgRzNfvnLHyQsi7lCJqkxPc9NMZhSjYJJPiv5meEpZWM65D1LFbWLguni5hm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+5/UyjG6CqVBphlyx5aIokwQTMg+ADITmDOXEEsq0sLcSNqKaMrQxlWwI3urL66Rdq3pu1Xu4qtRv8ziKcAbncAkeXEMd7qAJLWCQwjO8wpuTOS/Ou/OxbC04+cwp/IHz+QO8aZDS</latexit>

Node Feature Hyperedge Feature Node Feature

n1<latexit sha1_base64="TbOVm4a1bj7EWv2q2BG16ZUEIEA=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7fd6tezZuDrBK/IFUo0Oi7X71BwrIYpWGCat31vdQEOVWGM4HTSi/TmFI2pkPsWippjDrI56dOyZlVBiRKlC1pyFz9PZHTWOtJHNrOmJqRXvZm4n9eNzPRdZBzmWYGJVssijJBTEJmf5MBV8iMmFhCmeL2VsJGVFFmbDoVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOcF+fd+Vi0lpxi5hj+wPn8AfxljZY=</latexit><latexit sha1_base64="TbOVm4a1bj7EWv2q2BG16ZUEIEA=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7fd6tezZuDrBK/IFUo0Oi7X71BwrIYpWGCat31vdQEOVWGM4HTSi/TmFI2pkPsWippjDrI56dOyZlVBiRKlC1pyFz9PZHTWOtJHNrOmJqRXvZm4n9eNzPRdZBzmWYGJVssijJBTEJmf5MBV8iMmFhCmeL2VsJGVFFmbDoVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOcF+fd+Vi0lpxi5hj+wPn8AfxljZY=</latexit><latexit sha1_base64="TbOVm4a1bj7EWv2q2BG16ZUEIEA=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7fd6tezZuDrBK/IFUo0Oi7X71BwrIYpWGCat31vdQEOVWGM4HTSi/TmFI2pkPsWippjDrI56dOyZlVBiRKlC1pyFz9PZHTWOtJHNrOmJqRXvZm4n9eNzPRdZBzmWYGJVssijJBTEJmf5MBV8iMmFhCmeL2VsJGVFFmbDoVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOcF+fd+Vi0lpxi5hj+wPn8AfxljZY=</latexit><latexit sha1_base64="TbOVm4a1bj7EWv2q2BG16ZUEIEA=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7fd6tezZuDrBK/IFUo0Oi7X71BwrIYpWGCat31vdQEOVWGM4HTSi/TmFI2pkPsWippjDrI56dOyZlVBiRKlC1pyFz9PZHTWOtJHNrOmJqRXvZm4n9eNzPRdZBzmWYGJVssijJBTEJmf5MBV8iMmFhCmeL2VsJGVFFmbDoVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOcF+fd+Vi0lpxi5hj+wPn8AfxljZY=</latexit>

n2<latexit sha1_base64="bjIC+35MlRj3rmKoxI1XWIuyzrA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68Ttq1qudWvfurSuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPH/3pjZc=</latexit><latexit sha1_base64="bjIC+35MlRj3rmKoxI1XWIuyzrA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68Ttq1qudWvfurSuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPH/3pjZc=</latexit><latexit sha1_base64="bjIC+35MlRj3rmKoxI1XWIuyzrA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68Ttq1qudWvfurSuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPH/3pjZc=</latexit><latexit sha1_base64="bjIC+35MlRj3rmKoxI1XWIuyzrA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68Ttq1qudWvfurSuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPH/3pjZc=</latexit>

n3<latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit> n4

<latexit sha1_base64="qj1TO30AZnbRJEF2eK9SuYWxaeA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68TtpXVc+teve1SuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwEAjZk=</latexit><latexit sha1_base64="qj1TO30AZnbRJEF2eK9SuYWxaeA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68TtpXVc+teve1SuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwEAjZk=</latexit><latexit sha1_base64="qj1TO30AZnbRJEF2eK9SuYWxaeA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68TtpXVc+teve1SuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwEAjZk=</latexit><latexit sha1_base64="qj1TO30AZnbRJEF2eK9SuYWxaeA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68TtpXVc+teve1SuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwEAjZk=</latexit>

n5<latexit sha1_base64="p9y3cs0G0q7hp9YjBH6OpaJsVlg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwKEjZo=</latexit><latexit sha1_base64="p9y3cs0G0q7hp9YjBH6OpaJsVlg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwKEjZo=</latexit><latexit sha1_base64="p9y3cs0G0q7hp9YjBH6OpaJsVlg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwKEjZo=</latexit><latexit sha1_base64="p9y3cs0G0q7hp9YjBH6OpaJsVlg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwKEjZo=</latexit>

n6<latexit sha1_base64="tbNUust/TB+TBhNg9a465x4qcNI=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwQIjZs=</latexit><latexit sha1_base64="tbNUust/TB+TBhNg9a465x4qcNI=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwQIjZs=</latexit><latexit sha1_base64="tbNUust/TB+TBhNg9a465x4qcNI=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwQIjZs=</latexit><latexit sha1_base64="tbNUust/TB+TBhNg9a465x4qcNI=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwQIjZs=</latexit>

n7<latexit sha1_base64="DpErsjR/8ayk1yFvuxTfBiYrwsg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6KYd3PhEpS5IotF4WpJBiT+d9kKDRnKKeWUKaFvZWwMdWUoU2nZEPwVl9eJ+2rqudWvfvrSuMmj6MIZ3AOl+BBDRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwWMjZw=</latexit><latexit sha1_base64="DpErsjR/8ayk1yFvuxTfBiYrwsg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6KYd3PhEpS5IotF4WpJBiT+d9kKDRnKKeWUKaFvZWwMdWUoU2nZEPwVl9eJ+2rqudWvfvrSuMmj6MIZ3AOl+BBDRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwWMjZw=</latexit><latexit sha1_base64="DpErsjR/8ayk1yFvuxTfBiYrwsg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6KYd3PhEpS5IotF4WpJBiT+d9kKDRnKKeWUKaFvZWwMdWUoU2nZEPwVl9eJ+2rqudWvfvrSuMmj6MIZ3AOl+BBDRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwWMjZw=</latexit><latexit sha1_base64="DpErsjR/8ayk1yFvuxTfBiYrwsg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6KYd3PhEpS5IotF4WpJBiT+d9kKDRnKKeWUKaFvZWwMdWUoU2nZEPwVl9eJ+2rqudWvfvrSuMmj6MIZ3AOl+BBDRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwWMjZw=</latexit>

e1<latexit sha1_base64="6zA7gEs4vLWyBlUv9CAvgcAHwFg=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9YN/vu1Wv5s1BVolfkCoUaPTdr94gYVmM0jBBte76XmqCnCrDmcBppZdpTCkb0yF2LZU0Rh3k81On5MwqAxIlypY0ZK7+nshprPUkDm1nTM1IL3sz8T+vm5noOsi5TDODki0WRZkgJiGzv8mAK2RGTCyhTHF7K2EjqigzNp2KDcFffnmVtC5qvlfz7y+r9ZsijjKcwCmcgw9XUIc7aEATGAzhGV7hzRHOi/PufCxaS04xcwx/4Hz+AO6vjY0=</latexit><latexit sha1_base64="6zA7gEs4vLWyBlUv9CAvgcAHwFg=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9YN/vu1Wv5s1BVolfkCoUaPTdr94gYVmM0jBBte76XmqCnCrDmcBppZdpTCkb0yF2LZU0Rh3k81On5MwqAxIlypY0ZK7+nshprPUkDm1nTM1IL3sz8T+vm5noOsi5TDODki0WRZkgJiGzv8mAK2RGTCyhTHF7K2EjqigzNp2KDcFffnmVtC5qvlfz7y+r9ZsijjKcwCmcgw9XUIc7aEATGAzhGV7hzRHOi/PufCxaS04xcwx/4Hz+AO6vjY0=</latexit><latexit sha1_base64="6zA7gEs4vLWyBlUv9CAvgcAHwFg=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9YN/vu1Wv5s1BVolfkCoUaPTdr94gYVmM0jBBte76XmqCnCrDmcBppZdpTCkb0yF2LZU0Rh3k81On5MwqAxIlypY0ZK7+nshprPUkDm1nTM1IL3sz8T+vm5noOsi5TDODki0WRZkgJiGzv8mAK2RGTCyhTHF7K2EjqigzNp2KDcFffnmVtC5qvlfz7y+r9ZsijjKcwCmcgw9XUIc7aEATGAzhGV7hzRHOi/PufCxaS04xcwx/4Hz+AO6vjY0=</latexit><latexit sha1_base64="6zA7gEs4vLWyBlUv9CAvgcAHwFg=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9YN/vu1Wv5s1BVolfkCoUaPTdr94gYVmM0jBBte76XmqCnCrDmcBppZdpTCkb0yF2LZU0Rh3k81On5MwqAxIlypY0ZK7+nshprPUkDm1nTM1IL3sz8T+vm5noOsi5TDODki0WRZkgJiGzv8mAK2RGTCyhTHF7K2EjqigzNp2KDcFffnmVtC5qvlfz7y+r9ZsijjKcwCmcgw9XUIc7aEATGAzhGV7hzRHOi/PufCxaS04xcwx/4Hz+AO6vjY0=</latexit>

e2<latexit sha1_base64="mfO9ARkQyRLj4TbGagbpzJgRzY0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpAQe1QbniVt0FyDrxclKBHM1B+as/jFkaoTRMUK17npsYP6PKcCZwVuqnGhPKJnSEPUsljVD72eLUGbmwypCEsbIlDVmovycyGmk9jQLbGVEz1qveXPzP66UmvPYzLpPUoGTLRWEqiInJ/G8y5AqZEVNLKFPc3krYmCrKjE2nZEPwVl9eJ+1a1XOr3v1VpXGTx1GEMziHS/CgDg24gya0gMEInuEV3hzhvDjvzseyteDkM6fwB87nD/AzjY4=</latexit><latexit sha1_base64="mfO9ARkQyRLj4TbGagbpzJgRzY0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpAQe1QbniVt0FyDrxclKBHM1B+as/jFkaoTRMUK17npsYP6PKcCZwVuqnGhPKJnSEPUsljVD72eLUGbmwypCEsbIlDVmovycyGmk9jQLbGVEz1qveXPzP66UmvPYzLpPUoGTLRWEqiInJ/G8y5AqZEVNLKFPc3krYmCrKjE2nZEPwVl9eJ+1a1XOr3v1VpXGTx1GEMziHS/CgDg24gya0gMEInuEV3hzhvDjvzseyteDkM6fwB87nD/AzjY4=</latexit><latexit sha1_base64="mfO9ARkQyRLj4TbGagbpzJgRzY0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpAQe1QbniVt0FyDrxclKBHM1B+as/jFkaoTRMUK17npsYP6PKcCZwVuqnGhPKJnSEPUsljVD72eLUGbmwypCEsbIlDVmovycyGmk9jQLbGVEz1qveXPzP66UmvPYzLpPUoGTLRWEqiInJ/G8y5AqZEVNLKFPc3krYmCrKjE2nZEPwVl9eJ+1a1XOr3v1VpXGTx1GEMziHS/CgDg24gya0gMEInuEV3hzhvDjvzseyteDkM6fwB87nD/AzjY4=</latexit><latexit sha1_base64="mfO9ARkQyRLj4TbGagbpzJgRzY0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpAQe1QbniVt0FyDrxclKBHM1B+as/jFkaoTRMUK17npsYP6PKcCZwVuqnGhPKJnSEPUsljVD72eLUGbmwypCEsbIlDVmovycyGmk9jQLbGVEz1qveXPzP66UmvPYzLpPUoGTLRWEqiInJ/G8y5AqZEVNLKFPc3krYmCrKjE2nZEPwVl9eJ+1a1XOr3v1VpXGTx1GEMziHS/CgDg24gya0gMEInuEV3hzhvDjvzseyteDkM6fwB87nD/AzjY4=</latexit>

e3<latexit sha1_base64="Ixs6byDxI13PHmv3i/ziV98fHIw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0gP3LfrniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvftapX6Tx1GEEziFc/DgCupwBw1oAoMhPMMrvDnCeXHenY9Fa8HJZ47hD5zPH/G3jY8=</latexit><latexit sha1_base64="Ixs6byDxI13PHmv3i/ziV98fHIw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0gP3LfrniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvftapX6Tx1GEEziFc/DgCupwBw1oAoMhPMMrvDnCeXHenY9Fa8HJZ47hD5zPH/G3jY8=</latexit><latexit sha1_base64="Ixs6byDxI13PHmv3i/ziV98fHIw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0gP3LfrniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvftapX6Tx1GEEziFc/DgCupwBw1oAoMhPMMrvDnCeXHenY9Fa8HJZ47hD5zPH/G3jY8=</latexit><latexit sha1_base64="Ixs6byDxI13PHmv3i/ziV98fHIw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0gP3LfrniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvftapX6Tx1GEEziFc/DgCupwBw1oAoMhPMMrvDnCeXHenY9Fa8HJZ47hD5zPH/G3jY8=</latexit> em

<latexit sha1_base64="D3jROYEN/QbXF/wsb0gVOC1ZS38=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzQOSJcxOepMhM7PLzKwQQj7BiwdFvPpF3vwbJ8keNLGgoajqprsrSgU31ve/vcLa+sbmVnG7tLO7t39QPjxqmiTTDBssEYluR9Sg4AoblluB7VQjlZHAVjS6nfmtJ9SGJ+rRjlMMJR0oHnNGrZMesCd75Ypf9ecgqyTISQVy1Hvlr24/YZlEZZmgxnQCP7XhhGrLmcBpqZsZTCkb0QF2HFVUogkn81On5MwpfRIn2pWyZK7+nphQacxYRq5TUjs0y95M/M/rZDa+DidcpZlFxRaL4kwQm5DZ36TPNTIrxo5Qprm7lbAh1ZRZl07JhRAsv7xKmhfVwK8G95eV2k0eRxFO4BTOIYArqMEd1KEBDAbwDK/w5gnvxXv3PhatBS+fOYY/8D5/AEmujck=</latexit><latexit sha1_base64="D3jROYEN/QbXF/wsb0gVOC1ZS38=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzQOSJcxOepMhM7PLzKwQQj7BiwdFvPpF3vwbJ8keNLGgoajqprsrSgU31ve/vcLa+sbmVnG7tLO7t39QPjxqmiTTDBssEYluR9Sg4AoblluB7VQjlZHAVjS6nfmtJ9SGJ+rRjlMMJR0oHnNGrZMesCd75Ypf9ecgqyTISQVy1Hvlr24/YZlEZZmgxnQCP7XhhGrLmcBpqZsZTCkb0QF2HFVUogkn81On5MwpfRIn2pWyZK7+nphQacxYRq5TUjs0y95M/M/rZDa+DidcpZlFxRaL4kwQm5DZ36TPNTIrxo5Qprm7lbAh1ZRZl07JhRAsv7xKmhfVwK8G95eV2k0eRxFO4BTOIYArqMEd1KEBDAbwDK/w5gnvxXv3PhatBS+fOYY/8D5/AEmujck=</latexit><latexit sha1_base64="D3jROYEN/QbXF/wsb0gVOC1ZS38=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzQOSJcxOepMhM7PLzKwQQj7BiwdFvPpF3vwbJ8keNLGgoajqprsrSgU31ve/vcLa+sbmVnG7tLO7t39QPjxqmiTTDBssEYluR9Sg4AoblluB7VQjlZHAVjS6nfmtJ9SGJ+rRjlMMJR0oHnNGrZMesCd75Ypf9ecgqyTISQVy1Hvlr24/YZlEZZmgxnQCP7XhhGrLmcBpqZsZTCkb0QF2HFVUogkn81On5MwpfRIn2pWyZK7+nphQacxYRq5TUjs0y95M/M/rZDa+DidcpZlFxRaL4kwQm5DZ36TPNTIrxo5Qprm7lbAh1ZRZl07JhRAsv7xKmhfVwK8G95eV2k0eRxFO4BTOIYArqMEd1KEBDAbwDK/w5gnvxXv3PhatBS+fOYY/8D5/AEmujck=</latexit><latexit sha1_base64="D3jROYEN/QbXF/wsb0gVOC1ZS38=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzQOSJcxOepMhM7PLzKwQQj7BiwdFvPpF3vwbJ8keNLGgoajqprsrSgU31ve/vcLa+sbmVnG7tLO7t39QPjxqmiTTDBssEYluR9Sg4AoblluB7VQjlZHAVjS6nfmtJ9SGJ+rRjlMMJR0oHnNGrZMesCd75Ypf9ecgqyTISQVy1Hvlr24/YZlEZZmgxnQCP7XhhGrLmcBpqZsZTCkb0QF2HFVUogkn81On5MwpfRIn2pWyZK7+nphQacxYRq5TUjs0y95M/M/rZDa+DidcpZlFxRaL4kwQm5DZ36TPNTIrxo5Qprm7lbAh1ZRZl07JhRAsv7xKmhfVwK8G95eV2k0eRxFO4BTOIYArqMEd1KEBDAbwDK/w5gnvxXv3PhatBS+fOYY/8D5/AEmujck=</latexit> nn<latexit sha1_base64="viX2VKqEK4KOxQmuXZwEBoDvRBE=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7su1Wv5s1BVolfkCoUaPTdr94gYVmM0jBBte76XmqCnCrDmcBppZdpTCkb0yF2LZU0Rh3k81On5MwqAxIlypY0ZK7+nshprPUkDm1nTM1IL3sz8T+vm5noOsi5TDODki0WRZkgJiGzv8mAK2RGTCyhTHF7K2EjqigzNp2KDcFffnmVtC5qvlfz7y+r9ZsijjKcwCmcgw9XUIc7aEATGAzhGV7hzRHOi/PufCxaS04xcwx/4Hz+AFjojdM=</latexit><latexit sha1_base64="viX2VKqEK4KOxQmuXZwEBoDvRBE=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7su1Wv5s1BVolfkCoUaPTdr94gYVmM0jBBte76XmqCnCrDmcBppZdpTCkb0yF2LZU0Rh3k81On5MwqAxIlypY0ZK7+nshprPUkDm1nTM1IL3sz8T+vm5noOsi5TDODki0WRZkgJiGzv8mAK2RGTCyhTHF7K2EjqigzNp2KDcFffnmVtC5qvlfz7y+r9ZsijjKcwCmcgw9XUIc7aEATGAzhGV7hzRHOi/PufCxaS04xcwx/4Hz+AFjojdM=</latexit><latexit sha1_base64="viX2VKqEK4KOxQmuXZwEBoDvRBE=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7su1Wv5s1BVolfkCoUaPTdr94gYVmM0jBBte76XmqCnCrDmcBppZdpTCkb0yF2LZU0Rh3k81On5MwqAxIlypY0ZK7+nshprPUkDm1nTM1IL3sz8T+vm5noOsi5TDODki0WRZkgJiGzv8mAK2RGTCyhTHF7K2EjqigzNp2KDcFffnmVtC5qvlfz7y+r9ZsijjKcwCmcgw9XUIc7aEATGAzhGV7hzRHOi/PufCxaS04xcwx/4Hz+AFjojdM=</latexit><latexit sha1_base64="viX2VKqEK4KOxQmuXZwEBoDvRBE=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7su1Wv5s1BVolfkCoUaPTdr94gYVmM0jBBte76XmqCnCrDmcBppZdpTCkb0yF2LZU0Rh3k81On5MwqAxIlypY0ZK7+nshprPUkDm1nTM1IL3sz8T+vm5noOsi5TDODki0WRZkgJiGzv8mAK2RGTCyhTHF7K2EjqigzNp2KDcFffnmVtC5qvlfz7y+r9ZsijjKcwCmcgw9XUIc7aEATGAzhGV7hzRHOi/PufCxaS04xcwx/4Hz+AFjojdM=</latexit>

n8<latexit sha1_base64="zka5wiLNeuIzdnNzOY6ZgfZJctY=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeiF48V7Ae0oWy2m3bp7ibsToQS+he8eFDEq3/Im//GpM1BWx8MPN6bYWZeEEth0XW/ndLG5tb2Tnm3srd/cHhUPT7p2CgxjLdZJCPTC6jlUmjeRoGS92LDqQok7wbTu9zvPnFjRaQfcRZzX9GxFqFgFHNJDxuVYbXm1t0FyDrxClKDAq1h9WswiliiuEYmqbV9z43RT6lBwSSfVwaJ5TFlUzrm/Yxqqrj108Wtc3KRKSMSRiYrjWSh/p5IqbJ2poKsU1Gc2FUvF//z+gmGDT8VOk6Qa7ZcFCaSYETyx8lIGM5QzjJCmRHZrYRNqKEMs3jyELzVl9dJ56ruuXXv4brWvC3iKMMZnMMleHADTbiHFrSBwQSe4RXeHOW8OO/Ox7K15BQzp/AHzucPPCWNsQ==</latexit><latexit sha1_base64="zka5wiLNeuIzdnNzOY6ZgfZJctY=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeiF48V7Ae0oWy2m3bp7ibsToQS+he8eFDEq3/Im//GpM1BWx8MPN6bYWZeEEth0XW/ndLG5tb2Tnm3srd/cHhUPT7p2CgxjLdZJCPTC6jlUmjeRoGS92LDqQok7wbTu9zvPnFjRaQfcRZzX9GxFqFgFHNJDxuVYbXm1t0FyDrxClKDAq1h9WswiliiuEYmqbV9z43RT6lBwSSfVwaJ5TFlUzrm/Yxqqrj108Wtc3KRKSMSRiYrjWSh/p5IqbJ2poKsU1Gc2FUvF//z+gmGDT8VOk6Qa7ZcFCaSYETyx8lIGM5QzjJCmRHZrYRNqKEMs3jyELzVl9dJ56ruuXXv4brWvC3iKMMZnMMleHADTbiHFrSBwQSe4RXeHOW8OO/Ox7K15BQzp/AHzucPPCWNsQ==</latexit><latexit sha1_base64="zka5wiLNeuIzdnNzOY6ZgfZJctY=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeiF48V7Ae0oWy2m3bp7ibsToQS+he8eFDEq3/Im//GpM1BWx8MPN6bYWZeEEth0XW/ndLG5tb2Tnm3srd/cHhUPT7p2CgxjLdZJCPTC6jlUmjeRoGS92LDqQok7wbTu9zvPnFjRaQfcRZzX9GxFqFgFHNJDxuVYbXm1t0FyDrxClKDAq1h9WswiliiuEYmqbV9z43RT6lBwSSfVwaJ5TFlUzrm/Yxqqrj108Wtc3KRKSMSRiYrjWSh/p5IqbJ2poKsU1Gc2FUvF//z+gmGDT8VOk6Qa7ZcFCaSYETyx8lIGM5QzjJCmRHZrYRNqKEMs3jyELzVl9dJ56ruuXXv4brWvC3iKMMZnMMleHADTbiHFrSBwQSe4RXeHOW8OO/Ox7K15BQzp/AHzucPPCWNsQ==</latexit><latexit sha1_base64="zka5wiLNeuIzdnNzOY6ZgfZJctY=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeiF48V7Ae0oWy2m3bp7ibsToQS+he8eFDEq3/Im//GpM1BWx8MPN6bYWZeEEth0XW/ndLG5tb2Tnm3srd/cHhUPT7p2CgxjLdZJCPTC6jlUmjeRoGS92LDqQok7wbTu9zvPnFjRaQfcRZzX9GxFqFgFHNJDxuVYbXm1t0FyDrxClKDAq1h9WswiliiuEYmqbV9z43RT6lBwSSfVwaJ5TFlUzrm/Yxqqrj108Wtc3KRKSMSRiYrjWSh/p5IqbJ2poKsU1Gc2FUvF//z+gmGDT8VOk6Qa7ZcFCaSYETyx8lIGM5QzjJCmRHZrYRNqKEMs3jyELzVl9dJ56ruuXXv4brWvC3iKMMZnMMleHADTbiHFrSBwQSe4RXeHOW8OO/Ox7K15BQzp/AHzucPPCWNsQ==</latexit>

n8<latexit sha1_base64="zka5wiLNeuIzdnNzOY6ZgfZJctY=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeiF48V7Ae0oWy2m3bp7ibsToQS+he8eFDEq3/Im//GpM1BWx8MPN6bYWZeEEth0XW/ndLG5tb2Tnm3srd/cHhUPT7p2CgxjLdZJCPTC6jlUmjeRoGS92LDqQok7wbTu9zvPnFjRaQfcRZzX9GxFqFgFHNJDxuVYbXm1t0FyDrxClKDAq1h9WswiliiuEYmqbV9z43RT6lBwSSfVwaJ5TFlUzrm/Yxqqrj108Wtc3KRKSMSRiYrjWSh/p5IqbJ2poKsU1Gc2FUvF//z+gmGDT8VOk6Qa7ZcFCaSYETyx8lIGM5QzjJCmRHZrYRNqKEMs3jyELzVl9dJ56ruuXXv4brWvC3iKMMZnMMleHADTbiHFrSBwQSe4RXeHOW8OO/Ox7K15BQzp/AHzucPPCWNsQ==</latexit><latexit sha1_base64="zka5wiLNeuIzdnNzOY6ZgfZJctY=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeiF48V7Ae0oWy2m3bp7ibsToQS+he8eFDEq3/Im//GpM1BWx8MPN6bYWZeEEth0XW/ndLG5tb2Tnm3srd/cHhUPT7p2CgxjLdZJCPTC6jlUmjeRoGS92LDqQok7wbTu9zvPnFjRaQfcRZzX9GxFqFgFHNJDxuVYbXm1t0FyDrxClKDAq1h9WswiliiuEYmqbV9z43RT6lBwSSfVwaJ5TFlUzrm/Yxqqrj108Wtc3KRKSMSRiYrjWSh/p5IqbJ2poKsU1Gc2FUvF//z+gmGDT8VOk6Qa7ZcFCaSYETyx8lIGM5QzjJCmRHZrYRNqKEMs3jyELzVl9dJ56ruuXXv4brWvC3iKMMZnMMleHADTbiHFrSBwQSe4RXeHOW8OO/Ox7K15BQzp/AHzucPPCWNsQ==</latexit><latexit sha1_base64="zka5wiLNeuIzdnNzOY6ZgfZJctY=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeiF48V7Ae0oWy2m3bp7ibsToQS+he8eFDEq3/Im//GpM1BWx8MPN6bYWZeEEth0XW/ndLG5tb2Tnm3srd/cHhUPT7p2CgxjLdZJCPTC6jlUmjeRoGS92LDqQok7wbTu9zvPnFjRaQfcRZzX9GxFqFgFHNJDxuVYbXm1t0FyDrxClKDAq1h9WswiliiuEYmqbV9z43RT6lBwSSfVwaJ5TFlUzrm/Yxqqrj108Wtc3KRKSMSRiYrjWSh/p5IqbJ2poKsU1Gc2FUvF//z+gmGDT8VOk6Qa7ZcFCaSYETyx8lIGM5QzjJCmRHZrYRNqKEMs3jyELzVl9dJ56ruuXXv4brWvC3iKMMZnMMleHADTbiHFrSBwQSe4RXeHOW8OO/Ox7K15BQzp/AHzucPPCWNsQ==</latexit><latexit sha1_base64="zka5wiLNeuIzdnNzOY6ZgfZJctY=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeiF48V7Ae0oWy2m3bp7ibsToQS+he8eFDEq3/Im//GpM1BWx8MPN6bYWZeEEth0XW/ndLG5tb2Tnm3srd/cHhUPT7p2CgxjLdZJCPTC6jlUmjeRoGS92LDqQok7wbTu9zvPnFjRaQfcRZzX9GxFqFgFHNJDxuVYbXm1t0FyDrxClKDAq1h9WswiliiuEYmqbV9z43RT6lBwSSfVwaJ5TFlUzrm/Yxqqrj108Wtc3KRKSMSRiYrjWSh/p5IqbJ2poKsU1Gc2FUvF//z+gmGDT8VOk6Qa7ZcFCaSYETyx8lIGM5QzjJCmRHZrYRNqKEMs3jyELzVl9dJ56ruuXXv4brWvC3iKMMZnMMleHADTbiHFrSBwQSe4RXeHOW8OO/Ox7K15BQzp/AHzucPPCWNsQ==</latexit>


e3<latexit sha1_base64="Ixs6byDxI13PHmv3i/ziV98fHIw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0gP3LfrniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvftapX6Tx1GEEziFc/DgCupwBw1oAoMhPMMrvDnCeXHenY9Fa8HJZ47hD5zPH/G3jY8=</latexit><latexit sha1_base64="Ixs6byDxI13PHmv3i/ziV98fHIw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0gP3LfrniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvftapX6Tx1GEEziFc/DgCupwBw1oAoMhPMMrvDnCeXHenY9Fa8HJZ47hD5zPH/G3jY8=</latexit><latexit sha1_base64="Ixs6byDxI13PHmv3i/ziV98fHIw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0gP3LfrniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvftapX6Tx1GEEziFc/DgCupwBw1oAoMhPMMrvDnCeXHenY9Fa8HJZ47hD5zPH/G3jY8=</latexit><latexit sha1_base64="Ixs6byDxI13PHmv3i/ziV98fHIw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0gP3LfrniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvftapX6Tx1GEEziFc/DgCupwBw1oAoMhPMMrvDnCeXHenY9Fa8HJZ47hD5zPH/G3jY8=</latexit>

n4<latexit sha1_base64="qj1TO30AZnbRJEF2eK9SuYWxaeA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68TtpXVc+teve1SuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwEAjZk=</latexit><latexit sha1_base64="qj1TO30AZnbRJEF2eK9SuYWxaeA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68TtpXVc+teve1SuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwEAjZk=</latexit><latexit sha1_base64="qj1TO30AZnbRJEF2eK9SuYWxaeA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68TtpXVc+teve1SuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwEAjZk=</latexit><latexit sha1_base64="qj1TO30AZnbRJEF2eK9SuYWxaeA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD2pQG5QrbtVdgKwTLycVyNEclL/6w5ilEVfIJDWm57kJ+hnVKJjks1I/NTyhbEJHvGepohE3frY4dUYurDIkYaxtKSQL9fdERiNjplFgOyOKY7PqzcX/vF6K4bWfCZWkyBVbLgpTSTAm87/JUGjOUE4toUwLeythY6opQ5tOyYbgrb68TtpXVc+teve1SuMmj6MIZ3AOl+BBHRpwB01oAYMRPMMrvDnSeXHenY9la8HJZ07hD5zPHwEAjZk=</latexit>


n3<latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit>

n5<latexit sha1_base64="p9y3cs0G0q7hp9YjBH6OpaJsVlg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwKEjZo=</latexit><latexit sha1_base64="p9y3cs0G0q7hp9YjBH6OpaJsVlg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwKEjZo=</latexit><latexit sha1_base64="p9y3cs0G0q7hp9YjBH6OpaJsVlg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwKEjZo=</latexit><latexit sha1_base64="p9y3cs0G0q7hp9YjBH6OpaJsVlg=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPu1frniVt05yCrxclKBHI1++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXUsljVD72fzUKTmzyoCEsbIlDZmrvycyGmk9iQLbGVEz0sveTPzP66YmvPYzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqudWvfvLSv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifPwKEjZo=</latexit>










n1<latexit sha1_base64="TbOVm4a1bj7EWv2q2BG16ZUEIEA=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7fd6tezZuDrBK/IFUo0Oi7X71BwrIYpWGCat31vdQEOVWGM4HTSi/TmFI2pkPsWippjDrI56dOyZlVBiRKlC1pyFz9PZHTWOtJHNrOmJqRXvZm4n9eNzPRdZBzmWYGJVssijJBTEJmf5MBV8iMmFhCmeL2VsJGVFFmbDoVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOcF+fd+Vi0lpxi5hj+wPn8AfxljZY=</latexit><latexit sha1_base64="TbOVm4a1bj7EWv2q2BG16ZUEIEA=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7fd6tezZuDrBK/IFUo0Oi7X71BwrIYpWGCat31vdQEOVWGM4HTSi/TmFI2pkPsWippjDrI56dOyZlVBiRKlC1pyFz9PZHTWOtJHNrOmJqRXvZm4n9eNzPRdZBzmWYGJVssijJBTEJmf5MBV8iMmFhCmeL2VsJGVFFmbDoVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOcF+fd+Vi0lpxi5hj+wPn8AfxljZY=</latexit><latexit sha1_base64="TbOVm4a1bj7EWv2q2BG16ZUEIEA=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7fd6tezZuDrBK/IFUo0Oi7X71BwrIYpWGCat31vdQEOVWGM4HTSi/TmFI2pkPsWippjDrI56dOyZlVBiRKlC1pyFz9PZHTWOtJHNrOmJqRXvZm4n9eNzPRdZBzmWYGJVssijJBTEJmf5MBV8iMmFhCmeL2VsJGVFFmbDoVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOcF+fd+Vi0lpxi5hj+wPn8AfxljZY=</latexit><latexit sha1_base64="TbOVm4a1bj7EWv2q2BG16ZUEIEA=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9yL7fd6tezZuDrBK/IFUo0Oi7X71BwrIYpWGCat31vdQEOVWGM4HTSi/TmFI2pkPsWippjDrI56dOyZlVBiRKlC1pyFz9PZHTWOtJHNrOmJqRXvZm4n9eNzPRdZBzmWYGJVssijJBTEJmf5MBV8iMmFhCmeL2VsJGVFFmbDoVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOcF+fd+Vi0lpxi5hj+wPn8AfxljZY=</latexit>


n3<latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit><latexit sha1_base64="yXoDxELpx3VhIJk9qVncqGfvwS0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2k3bpZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHstHM0nQj+hQ8pAzaqz0IPuX/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NeO1nXCapQckWi8JUEBOT2d9kwBUyIyaWUKa4vZWwEVWUGZtOyYbgLb+8SloXVc+teve1Sv0mj6MIJ3AK5+DBFdThDhrQBAZDeIZXeHOE8+K8Ox+L1oKTzxzDHzifP/9tjZg=</latexit>

Figure 4: The illustration of the hyperedge convolution layer.

diag(λ1, . . . , λn) containing corresponding non-negativeeigenvalues. Then, the Fourier transform for a signal x =(x1, . . . ,xn) in hypergraph is defined as x = Φ>x, wherethe eigen vectors are regarded as the Fourier bases and theeigenvalues are interpreted as frequencies. The spectral con-volution of signal x and filter g can be denoted as

g ? x = Φ((Φ>g) (Φ>x)) = Φg(Λ)Φ>x, (5)

where denotes the element-wise Hadamard product andg(Λ) = diag(g(λ1), . . . ,g(λn)) is a function of the Fouriercoefficients. However, the computation cost in forward andinverse Fourier transform is O(n2). To solve the prob-lem, we can follow (Defferrard, Bresson, and Vandergheynst2016) to parametrize g(Λ) with K order polynomials. Fur-thermore, we use the truncated Chebyshev expansion as onesuch polynomial. Chebyshv polynomials Tk(x) is recur-sively computed by Tk(x) = 2xTk−1(x) − Tk−2(x), withT0(x) = 1 and T1(x) = x. Thus, the g(Λ) can be parame-tried as

g ? x ≈K∑k=0

θkTk(∆)x, (6)

where Tk(∆) is the Chebyshev polynomial of order kwith scaled Laplacian ∆ = 2

λmax∆ − I. In Equation 6,

the expansive computation of Laplacian Eigen vectors isexcluded and only matrix powers, additions and multipli-cations are included, which brings further improvement incomputation complexity. We can further let K = 1 to limitthe order of convolution operation due to that the Laplacianin hypergraph can already well represent the high-order cor-relation between nodes. It is also suggested in (Kipf andWelling 2016) that λmax ≈ 2 because of the scale adapt-ability of neural networks. Then, the convolution operationcan be further simplified to

g ? x ≈ θ0x− θ1D−1/2HWD−1e H>D−1/2v x, (7)

where there are θ0 and θ1 as parameters of filters over allthe nodes. We further use a single parameter θ to avoid the

overfitting problem, which is defined asθ1 = − 1

2θ

θ0 = 12θD

−1/2v HD−1e H>D

−1/2v

(8)

Then the convolution operation can be simplified to thefollowing expression

g ? x ≈ 1

2θD−1/2v H(W + I)D−1e H>D−1/2v x

≈ θD−1/2v HWD−1e H>D−1/2v x,

(9)

where (W+I) can be regarded as the weight of the hyper-edges. W is initialized as an identity matrix, which meansequal weights for all hyperedges.

When we have a hypergraph signal X ∈ Rn×C1 with nnodes and C1 dimensional features, our hyperedge convolu-tion can be formulated by

Y = D−1/2v HWE−1e H>D−1/2v XΘ, (10)

where W = diag(w1, . . . ,wn). Θ ∈ RC1×C2 is the param-eter to be learned during the training process. The filter Θis applied over the nodes in hypergraph to extract features.After convolution, we can obtain Y ∈ Rn×C2 , which can beused for classification.

Hypergraph neural networks analysisFigure 3 illustrates the details of the hypergraph neural net-works. Multi-modality datasets are divided into training dataand testing data, and each data contains several nodes withfeatures. Then multiple hyperedge structure groups are con-structed from the complex correlation of the multi-modalitydatasets. We concatenate the hyperedge groups to generatethe hypergraph adjacent matrix H . The hypergraph adjacentmatrix H and the node feature are fed into the HGNN to getthe node output labels. As introduced in the above section,we can build a hyperedge convolutional layer f(X,W,Θ)in the following formulation

X(l+1) = σ(D−1/2v HWD−1e H>D−1/2v )X(l)Θ(l), (11)

where X(1) ∈ RN×C is the signal of hypergraph at l layer,X(0) = X and σ denotes the nonlinear activation function.

The HGNN model is based on the spectral convolution onthe hypergraph. Here, we further investigate HGNN in theproperty of exploiting high-order correlation among data.As is shown in Figure 4, the HGNN layer can perform node-edge-node transform, which can better refine the featuresusing the hypergraph structure. More specifically, at first,the initial node feature X(1) is processed by learnable fil-ter matrix Θ(1) to extract C2-dimensional feature. Then thenode feature is gathered according to the hyperedge to formthe hyperedge feature RE×C2 , which is implemented by themultiplication of H> ∈ RE×N. Finally the output nodefeature is obtained by aggregating their related hyperedgefeature, which is achieved by multiplying matrix H. Denotethat Dv and De play a role of normalization in Equation 11.Thus, the HGNN layer can efficiently extract the high-ordercorrelation on hypergraph by the node-edge-node transform.

Relations to existing methods When the hyperedges onlyconnect two vertices, the hypergraph is simplified into a sim-ple graph and the Laplacian Delta is also coincident withthe Laplacian of simple graph up to a factor of 1

2 . Comparedwith the existing graph convolution methods, our HGNN cannaturally model high-order relationship among data, whichis effectively exploited and encoded in form of feature ex-traction. Compared with the traditional hypergraph method,our model is highly efficient in computation without the in-verse operation of Laplacian ∆. It should also be noted thatour HGNN has great expansibility toward multi-modal fea-ture with the flexibility of hyperedge generation.

ImplementationHypergraph construction In our visual object classifica-tion task, the features of N visual object data can be rep-resented as X = [x1, . . . ,xn]

>. We build the hypergraphaccording to the distance between two feature. More specif-ically, Euclidean distance is used to calculate d(xi,xj). Inthe construction, each vertex represents one visual object,and each hyperedge is formed by connecting one vertex andits K nearest neighbors, which brings N hyperedges thatlinks K + 1 vertices. And thus, we get the incidence ma-trix H ∈ RN×N with N × (K + 1) entries equaling to 1while others equaling to 0. In the citation network classifi-cation, where the data are organized in graph structure, eachhyperedge is built by linking one vertex and their neighborsaccording to the adjacency relation on graph. So we also getN hyperedges and H ∈ RN×N.

Model for node classification In the problem of nodeclassification, we build the HGNN model as in Figure 3.The dataset is divided into training data and test data. Thenhypergraph is constructed as the section above, which gen-erates the incidence matrix H and corresponding De. Webuild a two-layer HGNN model to employ the powerful ca-pacity of HGNN layer. And the softmax function is used togenerate predicted labels. During training, the cross-entropy

loss for the training data is back-propagated to update the pa-rameters Θ and in testing, the labels of test data is predictedfor evaluating the performance. When there are multi-modalinformation incorporate them by the construction of hyper-edge groups and then various hyperedges are fused togetherto model the complex relationship on data.

ExperimentsIn this section, we evaluate our proposed hypergraph neuralnetworks on two task: citation network classification andvisual object recognition. We also compare the proposedmethod with graph convolutional networks and other state-of-the-art methods.

Dataset Cora Pumbed

Nodes 2708 19717Edges 5429 44338

Feature 1433 500Training node 140 60

Validation node 500 500Testing node 1000 1000

Classes 7 3

Table 1: Summary of the citation classification datasets.

Citation network classificationDatasets In this experiment, the task is to classify citationdata. Here, two widely used citation network datasets, i.e.,Cora and Pubmed (Sen et al. 2008) are employed. The ex-perimental setup follows the settings in (Yang, Cohen, andSalakhutdinov 2016). In both of those two datasets, the fea-ture for each data is the bag-of-words representation of doc-uments. The data connection, i.e., the graph structure, in-dicates the citations among those data. To generate the hy-pergraph structure for HGNN, each time one vertex in thegraph is selected as the centroid and its connected verticesare used to generate one hyperedge including the centroiditself. Through this we can obtain the same size incidencematrix compared with the original graph. It is noted that asthere are no more information for data relationship, the gen-erated hypergraph constructure is quite similar to the graph.The Cora dataset contains 2708 data and 5% are used as la-beled data for training. The Pubmed dataset contains 19717data, and only 0.3% are used for training. The detailed de-scription for the two datasets listed in Table 1.

Experimental settings In this experiment, a two-layerHGNN is applied. The feature dimension of the hidden layeris set to be 16 and the dropout (Srivastava et al. 2014) isemployed to avoid overfitting with drop rate p = 0.5. Wechoose the ReLU as the nonlinear activation function. Dur-ing the training process, we use Adam optimizer (Kingmaand Ba 2014) to minimize our cross-entropy loss functionwith a learning rate of 0.001. We have also compared theproposed HGNN with recent methods in these experiments.

Results and discussion The results of the experimental re-sults and comparisons on the citation network dataset areshown in Table 2. For our HGNN model, we report the aver-age classification accuracy of 100 runs on Core and Pumbed,which is 81.6% and 80.1%. As shown in the results, the pro-posed HGNN model can achieve the best or comparable per-formance compared with the state-of-the-art methods. Com-pared with GCN, the proposed HGNN method can achieve aslight improvement on the Cora dataset and 1.1% improve-ment on the Pubmed dataset. We note that the generatedhypergraph structure is quite similar to the graph structureas there is neither extra nor more complex information inthese data. Therefore, the gain obtained by HGNN is notvery significant.

Method Cora Pubmed

DeepWalk (Perozzi, Al-Rfou,and Skiena 2014)

67.2% 65.3%

ICA (Lu and Getoor 2003) 75.1% 73.9%Planetoid (Yang, Cohen, andSalakhutdinov 2016)

75.7% 77.2%

Chebyshev (Defferrard, Bres-son, and Vandergheynst 2016)

81.2% 74.4%

GCN (Kipf and Welling 2016) 81.5% 79.0%

HGNN 81.6% 80.1%

Table 2: Classification results on the Cora and PubMeddatasets.

Visual object classificationDatasets and experimental settings In this experiment,the task is to classify visual objects. Two public bench-marks are employed here, including the Princeton Model-Net40 dataset (Wu et al. 2015) and the National TaiwanUniversity (NTU) 3D model dataset (Chen et al. 2003), asshown in Table 3. The ModelNet40 dataset consists of12,311 objects from 40 popular categories, and the sametraining/testing split is applied as introduced in (Wu et al.2015), where 9,843 objects are used for training and 2,468objects are used for testing. The NTU dataset is composedof 2,012 3D shapes from 67 categories, including car, chair,chess, chip, clock, cup, door, frame, pen, plant leaf and soon. In the NTU dataset, 80% data are used for training andthe other 20% data are used for testing. In this experiment,each 3D object is represented by the extracted feature. Here,two recent state-of-the-art shape representation methods areemployed, including Multi-view Convolutional Neural Net-work (MVCNN) (Su et al. 2015) and Group-View Convolu-tional Neural Network (GVCNN) (Feng et al. 2018). Thesetwo methods are selected as they have shown satisfactoryperformance on 3D object representation. We follow theexperimental settings of MVCNN and GVCNN to generatemultiple views of each 3D object. Here, 12 virtual camerasare employed to capture views with a interval angle of 30 de-gree, and then both the MVCNN and the GVCNN featuresare extracted accordingly.

To compared with GCN method, it is noted that there is noavailable graph structure in the ModelNet40 dataset and theNTU dataset. Therefore, we construct a probability graphbased on the distance of nodes. Given the features of data,the affinity matrix A is generated to represent the relation-ship among different vertices, and Aij can be calculated by:

Aij = exp(−2Dij2

∆) (12)

where Dij indicates the Euclidean distance between nodei and node j. ∆ is the average pairwise distance betweennodes. For the GCN experiment with two features con-structed simple graphs, we simply average the two modal-ity adjacency matrices to get the fused graph structure forcomparison.

Dataset ModelNet40 NTU

Objects 12311 2012MVCNN Feature 4096 4096GVCNN Feature 2048 2048

Training node 9843 1639Testing node 2468 373

Classes 40 67

Table 3: The detailed information of the ModelNet40 andthe NTU datasets.

n2

CenteroidNeighborhood

n1

n7

n5

n8

n4

n6

n3

n9

n1

n2

n3

n4

n5

n6

n7

n8

n9

1ei

1

1

1

0

0

00

0

Hyperedge Generation

H=

Incidence Matrix Generation

GVCNN-basedhyperedgese1 e2 e3 em-1em

n1

n2

n3

n4

n5

n6

n7

n8

MVCNN-basedhyperedgese1 e2 e3 em-1em

n1

n2

n3

n4

n5

n6

n7

n8

Concat

Figure 5: An example of hyperedge generation in the vi-sual object classification task. Left: for each node we aggre-gate its N neighbor nodes by Euclidean distance to generatea hyperedge. Right: to generate the multi-modality hyper-graph adjacent matrix we concatenate adjacent matrix of twomodality.

Hypergraph structure construction on visual datasetsIn experiments on ModelNet40 and NTU datasets, two hy-pergraph construction methods are employed. The first oneis based on single modality feature and the other one is basedon multi-modality feature. In the first case, only one featureis used. Each time one object in the dataset is selected asthe centroid, and its 10 nearest neighbors in the selected fea-ture space are used to generate one hyperedge including thecentroid itself, as shown in Figure 5. Then, a hypergraphH with N hyperedges can be constructed. In the secondcase, multiple features are used to generate a hypergraph Hmodeling complex multi-modality correlation. Here, for the

FeatureFeatures for Structure

GVCNN MVCNN GVCNN+MVCNNGCN HGNN GCN HGNN GCN HGNN

GVCNN (Feng et al. 2018) 91.8% 92.6% 91.5% 91.8% 92.8% 96.6%MVCNN (Su et al. 2015) 92.5% 92.9% 86.7% 91.0% 92.3% 96.6%

GVCNN+MVCNN - - - - 94.4% 96.7%

Table 4: Comparison between GCN and HGNN on the ModelNet40 dataset.

FeatureFeatures for Structure

GVCNN MVCNN GVCNN+MVCNNGCN HGNN GCN HGNN GCN HGNN

GVCNN ((Feng et al. 2018)) 72.9% 73.7% 74.0% 76.0% 76.7% 86.6%MVCNN ((Su et al. 2015)) 72.3% 73.6% 73.8% 75.4% 75.6% 86.7%

GVCNN+MVCNN − − − − 76.4% 86.8%

Table 5: Comparison between GCN and HGNN on the NTU dataset.

Method ClassificationAccuracy

PointNet (Qi et al. 2017a) 89.2%PointNet++ (Qi et al. 2017b) 90.7%

PointCNN (Li et al. 2018) 91.8%SO-Net (Li, Chen, and Lee 2018) 93.4%

HGNN 96.6%

Table 6: Experimental comparison among recent classifica-tion methods on ModelNet40 dataset.

ith modality data, a hypergraph adjacent matrix Hi is con-structed accordingly. After all the hypergraphs from differ-ent features have been generated, these adjacent matricesHi

can be concatenated to build the multi-modality hypergraphadjacent matrixH . In this way, the hypergraphs using singlemodal feature and multi-modal features can be constructed.

Results and discussions Experiments and comparisonson the visual object recognition task are shown in Table 4and Table 5, respectively. For the ModelNet40 dataset, wehave compared the proposed method using two features withrecent state-of-the-are methods in Table 6. As shown in theresults, we can have the following observations:

1. The proposed HGNN method outperforms the state-of-the-art object recognition methods in the ModelNet40dataset. More specifically, compared with PointCNN andSO-Net, the proposed HGNN method can achieve gainsof 4.8% and 3.2%, respectively. These results demon-strate the superior performance of the proposed HGNNmethod on visual object recognition.

2. Compared with GCN, the proposed method achieves bet-ter performance in all experiments. As shown in Ta-ble 4 and Table 5, when only one feature is used forgraph/hypergraph structure generation, HGNN can ob-tain slightly improvement. For example, when GVCNN

is used as the object feature and MVCNN is used forgraph/hypergraph structure generation, HGNN achievesgains of 0.3% and 2.0% compared with GCN on theModelNet40 and the NTU datasets, respectively. Whenmore features, i.e., both GVCNN and MVCNN, areused for graph/hypergraph structure generation, HGNNachieves much better performance compared with GCN.For example, HGNN achieves gains of 9.9%, 11.1% and10.4% compared with GCN when GVCNN, MVCNN andGVCNN+MVCNN are used as the object features on theNTU dataset, respectively.The better performance can be dedicated to the employed

hypergraph structure. The hypergraph structure is able toconvey complex and high-order correlations among data,which can be better represent the underneath data relation-ship compared with graph structure or the methods withoutgraph structure. Moreover, when multi-modal data/featuresare available, HGNN has the advantage of combining suchmulti-modal information in the same structure by its flexiblehyperedges. Compared with traditional hypergraph learn-ing methods, which may suffer from the high computationalcomplexity and storage cost, the proposed HGNN frame-work is much more efficient through the hyperedge convo-lution operation.

ConclusionIn this paper, we propose a neural networks on hypergraph,i.e., hypergraph neural networks (HGNN). In this method,HGNN generalizes the convolution operation to the hyper-graph learning process. The convolution on spectral domainis conducted with hypergraph Laplacian and further approx-imated by truncated chebyshev polynomials. HGNN is amore general framework which is able to handle the complexand high-order correlations through the hypergraph structurefor representation learning. We have conducted experimentson citation network classification and visual object recog-nition tasks to evaluate the performance of the proposedHGNN method. Experimental results and comparisons with

the state-of-the-art methods demonstrate the better perfor-mance of the proposed HGNN model.

References[Atwood and Towsley 2016] Atwood, J., and Towsley, D.2016. Diffusion-convolutional neural networks. In Advancesin Neural Information Processing Systems, 1993–2001.

[Bruna et al. 2013] Bruna, J.; Zaremba, W.; Szlam, A.; andLeCun, Y. 2013. Spectral networks and locally connectednetworks on graphs. arXiv preprint arXiv:1312.6203.

[Chen et al. 2003] Chen, D.-Y.; Tian, X.-P.; Shen, Y.-T.; andOuhyoung, M. 2003. On visual similarity based 3d modelretrieval. In Computer graphics forum, volume 22, 223–232.Wiley Online Library.

[Defferrard, Bresson, and Vandergheynst 2016] Defferrard,M.; Bresson, X.; and Vandergheynst, P. 2016. Convolu-tional neural networks on graphs with fast localized spectralfiltering. In Advances in Neural Information ProcessingSystems, 3844–3852.

[Feng et al. 2018] Feng, Y.; Zhang, Z.; Zhao, X.; Ji, R.; andGao, Y. 2018. Gvcnn: Group-view convolutional neuralnetworks for 3d shape recognition,. In Proceedings of theIEEE conference on computer vision and pattern recogni-tion, 2048–2057.

[Gao et al. 2012] Gao, Y.; Wang, M.; Tao, D.; Ji, R.; andDai, Q. 2012. 3-d object retrieval and recognition with hy-pergraph analysis. IEEE Transactions on Image Processing21(9):4290–4303.

[Gao et al. 2013] Gao, Y.; Wang, M.; Zha, Z.-J.; Shen, J.; Li,X.; and Wu, X. 2013. Visual-textual joint relevance learningfor tag-based social image search. IEEE Transactions onImage Processing 22(1):363–376.

[Gori, Monfardini, and Scarselli 2005] Gori, M.; Monfar-dini, G.; and Scarselli, F. 2005. A new model for learningin graph domains. In Neural Networks, 2005. IJCNN’05.Proceedings. 2005 IEEE International Joint Conference on,volume 2, 729–734. IEEE.

[Henaff, Bruna, and LeCun 2015] Henaff, M.; Bruna, J.; andLeCun, Y. 2015. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163.

[Huang et al. 2010] Huang, Y.; Liu, Q.; Zhang, S.; andMetaxas, D. N. 2010. Image retrieval via probabilistic hy-pergraph ranking. In Computer Vision and Pattern Recogni-tion (CVPR), 2010 IEEE Conference on, 3376–3383. IEEE.

[Huang, Liu, and Metaxas 2009] Huang, Y.; Liu, Q.; andMetaxas, D. 2009. ] video object segmentation by hyper-graph cut. In Computer Vision and Pattern Recognition,2009. CVPR 2009. IEEE Conference on, 1738–1745. IEEE.

[Hwang et al. 2008] Hwang, T.; Tian, Z.; Kuangy, R.; andKocher, J.-P. 2008. Learning on weighted hypergraphs tointegrate protein interactions and gene expressions for can-cer outcome prediction. In Data Mining, 2008. ICDM’08.Eighth IEEE International Conference on, 293–302. IEEE.

[Kingma and Ba 2014] Kingma, D. P., and Ba, J. 2014.Adam: A method for stochastic optimization. arXiv preprintarXiv:1412.6980.

[Kipf and Welling 2016] Kipf, T. N., and Welling, M. 2016.Semi-supervised classification with graph convolutional net-works. arXiv preprint arXiv:1609.02907.

[Li et al. 2018] Li, Y.; Bu, R.; Sun, M.; and Chen, B. 2018.Pointcnn. arXiv preprint arXiv:1801.07791.

[Li, Chen, and Lee 2018] Li, J.; Chen, B. M.; and Lee, G. H.2018. So-net: Self-organizing network for point cloud anal-ysis. arXiv preprint arXiv:1803.04249.

[Lu and Getoor 2003] Lu, Q., and Getoor, L. 2003. Link-based classification. In Proceedings of the 20th Interna-tional Conference on Machine Learning (ICML-03), 496–503.

[Monti et al. 2017] Monti, F.; Boscaini, D.; Masci, J.;Rodola, E.; Svoboda, J.; and Bronstein, M. M. 2017. Geo-metric deep learning on graphs and manifolds using mixturemodel cnns. In Proc. CVPR, volume 1, 3.

[Perozzi, Al-Rfou, and Skiena 2014] Perozzi, B.; Al-Rfou,R.; and Skiena, S. 2014. Deepwalk: Online learning ofsocial representations. In Proceedings of the 20th ACMSIGKDD international conference on Knowledge discoveryand data mining, 701–710. ACM.

[Qi et al. 2017a] Qi, C. R.; Su, H.; Mo, K.; and Guibas, L. J.2017a. Pointnet: Deep learning on point sets for 3d classifi-cation and segmentation. Proc. Computer Vision and PatternRecognition (CVPR), IEEE 1(2):4.

[Qi et al. 2017b] Qi, C. R.; Yi, L.; Su, H.; and Guibas, L. J.2017b. Pointnet++: Deep hierarchical feature learning onpoint sets in a metric space. In Advances in Neural Informa-tion Processing Systems, 5105–5114.

[Scarselli et al. 2009] Scarselli, F.; Gori, M.; Tsoi, A. C.; Ha-genbuchner, M.; and Monfardini, G. 2009. The graph neu-ral network model. IEEE Transactions on Neural Networks20(1):61–80.

[Sen et al. 2008] Sen, P.; Namata, G.; Bilgic, M.; Getoor, L.;Galligher, B.; and Eliassi-Rad, T. 2008. Collective classifi-cation in network data. AI magazine 29(3):93.

[Srivastava et al. 2014] Srivastava, N.; Hinton, G.;Krizhevsky, A.; Sutskever, I.; and Salakhutdinov, R.2014. Dropout: A simple way to prevent neural net-works from overfitting. The Journal of Machine LearningResearch 15(1):1929–1958.

[Su et al. 2015] Su, H.; Maji, S.; Kalogerakis, E.; andLearned-Miller, E. 2015. Multi-view convolutional neu-ral networks for 3d shape recognition. In Proceedings ofthe IEEE international conference on computer vision, 945–953.

[Velickovic et al. 2017] Velickovic, P.; Cucurull, G.;Casanova, A.; Romero, A.; Lio, P.; and Bengio, Y. 2017.Graph attention networks. stat 1050:20.

[Wu et al. 2015] Wu, Z.; Song, S.; Khosla, A.; Yu, F.; Zhang,L.; Tang, X.; and Xiao, J. 2015. 3d shapenets: A deeprepresentation for volumetric shapes. In Proceedings of theIEEE conference on computer vision and pattern recogni-tion, 1912–1920.

[Yang, Cohen, and Salakhutdinov 2016] Yang, Z.; Cohen,W. W.; and Salakhutdinov, R. 2016. Revisiting semi-

supervised learning with graph embeddings. arXiv preprintarXiv:1603.08861.

[Zhou, Huang, and Scholkopf 2007] Zhou, D.; Huang, J.;and Scholkopf, B. 2007. Learning with hypergraphs: Clus-tering, classification, and embedding. In Advances in neuralinformation processing systems, 1601–1608.

Hypergraph Neural NetworksIntroduction Graph-based convolutional neural networks (Kipf and Welling...

Documents

Transcript of Hypergraph Neural NetworksIntroduction Graph-based convolutional neural networks (Kipf and Welling...