Interactive Attention Transfer Network for Cross-domainSentiment Classification

Kai Zhang†, Hefu Zhang†, Qi Liu†§∗, Hongke Zhao†, Hengshu Zhu‡, Enhong Chen†§†Anhui Province Key Laboratory of Big Data Analysis and Application, University of Science and Technology of China

§School of Data Science, University of Science and Technology of China‡Baidu Talent Intelligence Center, Baidu Inc

{sa517494, zhf2011, zhhk}@mail.ustc.edu.cn, {qiliuql, cheneh}@ustc.edu.cn, zhuhengshu@gmail.com

Abstract

Cross-domain sentiment classification refers to utilizing use-ful knowledge in the source domain to help sentiment clas-sification in the target domain which has few or no labeleddata. Most existing methods mainly concentrate on extract-ing common features between domains. Unfortunately, theycannot fully consider the effects of the aspect (e.g., the bat-tery life in reviewing an electronic product) information of thesentences. In order to better solve this problem, we proposean Interactive Attention Transfer Network (IATN) for cross-domain sentiment classification. IATN provides an interactiveattention transfer mechanism, which can better transfer sen-timent across domains by incorporating information of bothsentences and aspects. Specifically, IATN comprises two at-tention networks, one of them is to identify the common fea-tures between domains through domain classification, and theother aims to extract information from the aspects by usingthe common features as a bridge. Then, we conduct interac-tive attention learning for those two networks so that boththe sentences and the aspects can influence the final senti-ment representation. Extensive experiments on the Amazonreviews dataset and crowdfunding reviews dataset not onlydemonstrate the effectiveness and universality of our method,but also give an interpretable way to track the attention infor-mation for sentiment.

IntroductionSentiment analysis, which aims to identify the overall emo-tional label (i.e., positive or negative) of the sentences, hasattracted more and more research attention in recent years.Traditional sentiment classification methods usually per-form well on label-rich data (Wang et al. 2014; Tripathy,Agrawal, and Rath 2016). However, in practice, there stillexists a huge amount of insufficiently labeled data, where thetraditional methods are hard to be utilized. For example, thereviews on the prevalent e-commerce platforms (e.g., Ama-zon) often contain score option, which can reflect the sen-timent labels of the reviews. But in some specialized appli-cations such as crowdfunding platforms (Zhao et al. 2017a),(e.g., Indiegogo.com), nearly all the project reviews do nothave sentiment labels.

∗Corresponding Author.Copyright c© 2019, Association for the Advancement of ArtificialIntelligence (www.aaai.org). All rights reserved.

The appearance of the PC looks good, but the battery life is too short.

The appearance of this dress looks nice, and the fabric is not bad.

RAMCPUPrice…

Figure 1: Aspects in different domains. For computer, thereare “appearance” and “battery life” aspects; for cloth, thereare “appearance” and “fabric” aspects.

To learn the emotion of the sentences from unlabeleddata, cross-domain sentiment classification has been pro-posed as a promising direction. It uses effectual informa-tion in the source domain (with sufficient labeled data) tohelp sentiment classification in the target domain (with fewor no labeled data). As it is crucial for reducing the re-liance on the massive amount of labeled data and signifi-cant for domains which are lack of labels, much researchattention has been attracted from both academia and indus-try. In the literature, many methods have been proposedto solve the cross-domain sentiment classification prob-lem (Blitzer, McDonald, and Pereira 2006; Pan et al. 2010;Chen et al. 2012), especially various solutions for learn-ing shared features have been designed. The shared featureswere usually assumed as words with high co-occurrence inboth domains which are regarded as the good predictors ofsource domain labels.

Recently, with the rapid development of deep learn-ing techniques, researchers proposed many neural networkbased methods to automatically capture the shared sentimentfeatures across domains (Glorot, Bordes, and Bengio 2011;Chen et al. 2012; Li et al. 2017; 2018b). However, most ofthe previous efforts ignore the characteristics which do notexpress the sentiment directly, such as the individual model-ing of the aspect. As Figure 1 shows, there are two reviews“The appearance of the PC looks good, but the battery lifeis too short” and “The appearance of this dress looks nice,

and the fabric is not bad”. The aspect “appearance” ap-pears in both reviews but has different importance in the twodomains. Specifically, on electronics (e.g., PC) domain, theother aspect “battery life” affects the sentiment of the reviewmuch greater than “appearance”. Although the first review ispositive on the aspect of appearance, it is still negative over-all because the battery life is not satisfying. From this exam-ple, we can conclude two main characters of aspects. First,different aspects in the same domain may have different im-portance. Aspect with a heavier weight affects the sentimentof sentences greater. Second, different domains may sharethe same aspects. These shared aspects can help us extractmore common knowledge between the source and the targetdomain, and further help to determine the focus in the trans-fer process. Thus, it is necessary to specifically exploit theaspects of cross-domain sentiment classification.

With the above analysis, we propose an Interactive At-tention Transfer Network (IATN) model based on Long-Short Term Memory networks (Hochreiter and Schmidhu-ber 1997; Huang, Xu, and Yu 2015). IATN models sen-tences and aspects independently and conducts an interac-tive learning to them. First, IATN makes use of the inter-active information from sentences to supervising modelingaspects, which is helpful to judge the across-domain senti-ment. Second, IATN utilizes the attention mechanism asso-ciated with the aspects to get important information fromthe sentences and compute ultimate representation for senti-ment classification. Finally, by concatenating both the sen-tence representation and the aspect representation, we canimprove the effectiveness of cross-domain sentiment classi-fication. In summary, the main contributions of our work canbe summarized as follows.

• For the first time, we propose to make cross-domain senti-ment classification by integrating the sentence and aspectrepresentation interactively.

• We propose a novel IATN1 method which associates withaspects. It utilizes the interactive attention mechanism toget important information from both the sentence and as-pect for the sentiment classification task.

• We conduct extensive experiments on two real-worlddatasets. The experimental results clearly validate that ourmethod outperforms other state-of-the-art methods.

Related WorkDomain Adaptation. Domain adaptation such as cross-domain sentiment classification is a hot topic in natural lan-guage processing, which has been well studied over thedecades. Among them, Blitzer et al. (Blitzer, McDonald,and Pereira 2006) proposed a representative study calledStructural Correspondence Learning (SCL), which mainlyutilized multiple shared features to predict tasks. Pan etal. (Pan et al. 2010) proposed Spectral Feature Alignment(SFA) algorithms to solve the feature mismatch problem byaligning domain-specific words with the help of domain-independent words. Glorot et al. (Glorot, Bordes, and Ben-gio 2011; Chen et al. 2012) proposed a Marginal Stacking

1https://github.com/1146976048qq/IATN

Denoising Autoencoder (mSDA) model which aimed to im-prove the speed and scalability on high-dimensional data.However, all the methods mentioned above need to manuallyselect some information such as shared or unshared featuresbetween the source domain and the target domain.

In recent years, many researchers have studied the neu-ral network-based domain adaptation solutions. For exam-ple, Yu and Jiang (Yu and Jiang 2016) proposed two aux-iliary tasks to learn sentence embedding based on Convo-lutional Neural Network (Kim 2014). Ganin et al. (Ganinet al. 2016) added adversarial mechanism into the trainingof deep neural networks, which called Domain-AdversarialNeural Network (DANN). Li et al. (Li et al. 2017) pro-posed an Adversarial Memory Network (AMN) that auto-matically identified common features by applying attentionmechanisms and adversarial training. Hierarchical AttentionTransfer Network (HATN) was proposed by Li et al. (Li etal. 2018b) which paid attention to word-level and sentence-level sentiment at the same time. The above works have rec-ognized the importance of shared features and developedvarious methods to improve transfer efficiency through twomain tasks like domain classification and sentiment classi-fication. Unfortunately, they have not paid enough attentionto the effects of “aspects” (Wang et al. 2016) in the cross-domain sentiment classification task.Aspect Extraction. Aspect extraction is one of the key tasksin sentiment analysis and has been widely studied in recentyears. It aims to extract entity aspects on which opinionshave been expressed (Hu and Liu 2004). Currently, thereare many outstanding works in the area of aspect extraction,but it has not been applied to improve the effect of cross-domain sentiment classification yet. Li et al. (Li et al. 2018a)proposed an aspect extraction framework by exploiting theopinion summary and the aspect detection history. We applytheir model to process our data and extract all the aspects inthe reviews.Attention Mechanism. In the conventional neural net-works, encoding the whole input into one feature vectorwithout considering special words usually leads to unsatis-factory classification. In order to solve this problem, atten-tion mechanism has been successfully exploited in variousnatural language processing tasks, such as machine transla-tion (Tu et al. 2017), sentiment analysis (Ma et al. 2017; Ma,Peng, and Cambria 2018) and question prediction (Huang etal. 2017). Besides, the interactive attention mechanism hasbeen proved to be better than the traditional mechanism be-cause of its effectiveness in extracting powerful features atrelated domains (Zhang et al. 2017).

Interactive Attention Transfer NetworkIn this section, we first present the problem of cross-domainsentiment classification, followed by an overview of themodel. Then we introduce the technical details of IATN.

Problem DefinitionIn this paper, we focus on cross-domain sentiment classi-fication. We assume that there are two domains, Ds is thesource domain and Dt is the target domain. We further as-

Training

Targetdomain reviews

Sourcedomain

Review

AspectAspect

ReviewExtract

Targetdomain

Review

AspectAspect

ReviewExtract

Domain classifier

Sentiment classifier

IATN

Figure 2: The flowchart overview of our work.

sume that we are given a set of labeled training data Xls =

{xis, yis}N l

si=1 and unlabeled training data Xu

s = {xjs}Ns

j=N ls+1

from the source domain, where N ls and Nu

s are the num-ber of labeled data and all data, respectively. Also, we havea set of unlabeled data from the target domain, denoted byXt = {xjt}

Ntj=1, where Nt is the number of unlabeled data.

Note that each item (e.g., review) at both domains con-sists of n words marked as s = {w1

s , w2s , w

3s ...w

ns } and

their aspect sequence contains m words marked as a ={w1

a, w2a, w

3a...w

ma }. The goal of cross-domain sentiment

classification is to train a robust model based on labeled andunlabeled data in the source domain and adopt it to predictthe unlabeled data in the target domain.

An Overview of IATNAs shown in Figure 2, our approach is modeled with infor-mation from sentences (i.e., reviews) and aspects simulta-neously. In this study, we apply the existing work (Li et al.2018a) to extract all aspects of the sentence. After obtain-ing the aspects, we utilize all data in the source domain andthe target domain for training domain classifier. Meanwhile,we apply source labeled data for training sentiment classi-fier. Finally, with the shared features from both domains, wepredict the sentiment label for the target domain data.

Components of IATNIn this subsection, we will introduce the framework of IATNin technical details. As shown in Figure 3, IATN mainly con-tains two parts, i.e., the sentence network referred as S-netwhich is mainly for domain classification and the aspect net-work referred as A-net which is mainly for aspect learning.S-net and A-net have similar structures for word embedding,hidden state learning and pooling operation. While, after thepooling layer, S-net and A-net adopt an interactive attentionmechanism jointly. Finally, the output of S-net is sent to thedomain classifier and both of the outputs are sent to the sen-timent classifier. In the following, we introduce the compo-nents of IATN successively.Word Embedding. In order to represent sentences, we needto map each word into a low-dimensional real-value vector.Word embedding (Bengio et al. 2003) is a kind of requiredmapping methods, which can be regarded as a functionalpart of a neural network or a language model pre-trainedfrom proper corpus. Here we choose the pre-training methodand take each word as input to get the sentences embeddingvectors es = {e1s, e2s, e3s...ens }. Similarly, the aspects are alsoembedded as vectors ea = {e1a, e2a, e3a...ema }.

Hidden State Learning. After word embedding, we adoptLSTM to learn hidden states because it performs well inlearning long-term dependencies and can effectively solvegradient vanishing and expansion problems. Formally, giventhe word embedding of sentences es = {e1s, e2s, e3s...ens }as the input, LSTM updates the cell vector sequence c ={c1, c2, c3...cn} and hidden state h = {h1, h2, h3...hn}from t = 1 to n. After the initialization, at t-th interactionstep, the hidden state ht of each interaction is updated by theprevious hidden state ht−1 and the current sentences embed-ding vector ets as:

it = δ(Weiets + Whiht−1 + b̂i) ,

ft = δ(Wefets + Whfht−1 + b̂f ) ,

ct = ft · ct−1 + it · τ(Wecets + Whcht−1 + b̂c) ,

ot = δ(Weoets + Whoht−1 + b̂o) ,

ht = ot · tanh(ct) ,

(1)

where it, ft and ot are the input, forget and output gates att-th step respectively. ets is the embedding sentence vector.ct is the cell memory and ht is the output. δ(·) is non-linearactivation function which is stated as sigmoid in this pa-per. Dot · denotes the element-wise multiplication betweenvectors. W∗ denotes weight matrices, b̂∗ is the bias vectors.They are all optimized in the training by the network.

After this layer, the sentence representation is transformedfrom word embedding vectors to the semantic sentence hid-den states (i.e., hs = {h1s, h2s, h3s...hns }), which is the fi-nal word representation for sentences. Similarly, we usethe same method to obtain the aspect hidden states (i.e.,ha = {h1a, h2a, h3a...hma }) for all aspects of the sentences.Pooling Operation. After getting the hidden state represen-tations of sentences and aspects, we need to make them in-teractive. We adopt the pooling method, which is a form ofnon-linear down-sampling to reduce the spatial size of therepresentation and retain important features. Thus, we canprepare for the interaction of sentence hidden features andaspect hidden features. There are several non-linear func-tions to implement pooling, among which mean poolingworks better in practice. Here, we adopt it to calculate thesentence pooling vector (i.e., hps) and the aspect pooling vec-tor (i.e., hpa) through the following equations:

hps =

n∑i=1

his / n , hpa =

m∑i=1

hia / m . (2)

Interactive Word Attention. As we discussed before, eachword has a different influence on the representation of thesentences. As shown in Figure 1, compared to the electronicproduct domain, “appearance” may play a more importantrole in dress domain. Even in the same review of PC, “ap-pearance” and “battery life” have different effects on the ul-timate sentiment classification goal. Therefore, it is neces-sary to qualify the contributions of each word and learn thespecial representation for it. Fortunately, attention mecha-nisms can highlight different parts of the input by assigningweights to encoding vectors in each step of text represen-tation. As we mentioned above, we consider the impact of

The appearance of the PC looks good, but the battery

life is short.

……

Word Embedding

+

appearance, battery life

Word Embedding

Sentence Attention Aspect Attention

GRL

Domain label Sentiment label

……

S-net A-net

Pooling Pooling

e1s

<latexit sha1_base64="h7v/q/vr/0E8tRta+NTASLsud+A=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYCJ4Cru56DGQi8cI5gHJGmYnvcmQ2Z1lZlYIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7gkRwbVz329nY3Nre2S3sFfcPDo+OSyenbS1TxbDFpJCqG1CNgsfYMtwI7CYKaRQI7ASTxszvPKLSXMb3ZpqgH9FRzEPOqLFSr4KDTOcPmZdXBqWyW3XnIOvEW5IyLNEclL76Q8nSCGPDBNW657mJ8TOqDGcC82I/1ZhQNqEj7Fka0wi1n81PzsmlVYYklMpWbMhc/T2R0UjraRTYzoiasV71ZuJ/Xi814Y2f8ThJDcZssShMBTGSzP4nQ66QGTG1hDLF7a2EjamizNiUijYEb/XlddKuVT236t3VyvXG0yKOApzDBVyBB9dQh1toQgsYSHiGV3hzjPPivDsfi9YNZxnhGfyB8/kD6N6RZQ==</latexit><latexit sha1_base64="h7v/q/vr/0E8tRta+NTASLsud+A=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYCJ4Cru56DGQi8cI5gHJGmYnvcmQ2Z1lZlYIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7gkRwbVz329nY3Nre2S3sFfcPDo+OSyenbS1TxbDFpJCqG1CNgsfYMtwI7CYKaRQI7ASTxszvPKLSXMb3ZpqgH9FRzEPOqLFSr4KDTOcPmZdXBqWyW3XnIOvEW5IyLNEclL76Q8nSCGPDBNW657mJ8TOqDGcC82I/1ZhQNqEj7Fka0wi1n81PzsmlVYYklMpWbMhc/T2R0UjraRTYzoiasV71ZuJ/Xi814Y2f8ThJDcZssShMBTGSzP4nQ66QGTG1hDLF7a2EjamizNiUijYEb/XlddKuVT236t3VyvXG0yKOApzDBVyBB9dQh1toQgsYSHiGV3hzjPPivDsfi9YNZxnhGfyB8/kD6N6RZQ==</latexit><latexit sha1_base64="h7v/q/vr/0E8tRta+NTASLsud+A=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYCJ4Cru56DGQi8cI5gHJGmYnvcmQ2Z1lZlYIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7gkRwbVz329nY3Nre2S3sFfcPDo+OSyenbS1TxbDFpJCqG1CNgsfYMtwI7CYKaRQI7ASTxszvPKLSXMb3ZpqgH9FRzEPOqLFSr4KDTOcPmZdXBqWyW3XnIOvEW5IyLNEclL76Q8nSCGPDBNW657mJ8TOqDGcC82I/1ZhQNqEj7Fka0wi1n81PzsmlVYYklMpWbMhc/T2R0UjraRTYzoiasV71ZuJ/Xi814Y2f8ThJDcZssShMBTGSzP4nQ66QGTG1hDLF7a2EjamizNiUijYEb/XlddKuVT236t3VyvXG0yKOApzDBVyBB9dQh1toQgsYSHiGV3hzjPPivDsfi9YNZxnhGfyB8/kD6N6RZQ==</latexit><latexit sha1_base64="h7v/q/vr/0E8tRta+NTASLsud+A=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYCJ4Cru56DGQi8cI5gHJGmYnvcmQ2Z1lZlYIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7gkRwbVz329nY3Nre2S3sFfcPDo+OSyenbS1TxbDFpJCqG1CNgsfYMtwI7CYKaRQI7ASTxszvPKLSXMb3ZpqgH9FRzEPOqLFSr4KDTOcPmZdXBqWyW3XnIOvEW5IyLNEclL76Q8nSCGPDBNW657mJ8TOqDGcC82I/1ZhQNqEj7Fka0wi1n81PzsmlVYYklMpWbMhc/T2R0UjraRTYzoiasV71ZuJ/Xi814Y2f8ThJDcZssShMBTGSzP4nQ66QGTG1hDLF7a2EjamizNiUijYEb/XlddKuVT236t3VyvXG0yKOApzDBVyBB9dQh1toQgsYSHiGV3hzjPPivDsfi9YNZxnhGfyB8/kD6N6RZQ==</latexit>

e2s

<latexit sha1_base64="EMHCUkTkUXSdbMwzkvGbbPmgMBA=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYns8mQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGppqxFlVC6GxLDBJeshRwF6yaakTgUrBNOGjO/88i04Ure4zRhQUxGkkecErRSr8IGmckfslpeGZTKXtWbw10n/pKUYYnmoPTVHyqaxkwiFcSYnu8lGGREI6eC5cV+alhC6ISMWM9SSWJmgmx+cu5eWmXoRkrbkujO1d8TGYmNmcah7YwJjs2qNxP/83opRjdBxmWSIpN0sShKhYvKnf3vDrlmFMXUEkI1t7e6dEw0oWhTKtoQ/NWX10m7VvW9qn9XK9cbT4s4CnAOF3AFPlxDHW6hCS2goOAZXuHNQefFeXc+Fq0bzjLCM/gD5/MH6mSRZg==</latexit><latexit sha1_base64="EMHCUkTkUXSdbMwzkvGbbPmgMBA=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYns8mQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGppqxFlVC6GxLDBJeshRwF6yaakTgUrBNOGjO/88i04Ure4zRhQUxGkkecErRSr8IGmckfslpeGZTKXtWbw10n/pKUYYnmoPTVHyqaxkwiFcSYnu8lGGREI6eC5cV+alhC6ISMWM9SSWJmgmx+cu5eWmXoRkrbkujO1d8TGYmNmcah7YwJjs2qNxP/83opRjdBxmWSIpN0sShKhYvKnf3vDrlmFMXUEkI1t7e6dEw0oWhTKtoQ/NWX10m7VvW9qn9XK9cbT4s4CnAOF3AFPlxDHW6hCS2goOAZXuHNQefFeXc+Fq0bzjLCM/gD5/MH6mSRZg==</latexit><latexit sha1_base64="EMHCUkTkUXSdbMwzkvGbbPmgMBA=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYns8mQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGppqxFlVC6GxLDBJeshRwF6yaakTgUrBNOGjO/88i04Ure4zRhQUxGkkecErRSr8IGmckfslpeGZTKXtWbw10n/pKUYYnmoPTVHyqaxkwiFcSYnu8lGGREI6eC5cV+alhC6ISMWM9SSWJmgmx+cu5eWmXoRkrbkujO1d8TGYmNmcah7YwJjs2qNxP/83opRjdBxmWSIpN0sShKhYvKnf3vDrlmFMXUEkI1t7e6dEw0oWhTKtoQ/NWX10m7VvW9qn9XK9cbT4s4CnAOF3AFPlxDHW6hCS2goOAZXuHNQefFeXc+Fq0bzjLCM/gD5/MH6mSRZg==</latexit><latexit sha1_base64="EMHCUkTkUXSdbMwzkvGbbPmgMBA=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYns8mQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGppqxFlVC6GxLDBJeshRwF6yaakTgUrBNOGjO/88i04Ure4zRhQUxGkkecErRSr8IGmckfslpeGZTKXtWbw10n/pKUYYnmoPTVHyqaxkwiFcSYnu8lGGREI6eC5cV+alhC6ISMWM9SSWJmgmx+cu5eWmXoRkrbkujO1d8TGYmNmcah7YwJjs2qNxP/83opRjdBxmWSIpN0sShKhYvKnf3vDrlmFMXUEkI1t7e6dEw0oWhTKtoQ/NWX10m7VvW9qn9XK9cbT4s4CnAOF3AFPlxDHW6hCS2goOAZXuHNQefFeXc+Fq0bzjLCM/gD5/MH6mSRZg==</latexit>

ens

<latexit sha1_base64="qMdbMzklkGQJHxmjhsbn73a280s=">AAAB8nicbVDLSgNBEJz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYnvcmQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGpZtBiSijdDakBwSW0kKOAbqKBxqGATjhpzPzOI2jDlbzHaQJBTEeSR5xRtFKvAoPM5A+ZzCuDUtmrenO468RfkjJZojkoffWHiqUxSGSCGtPzvQSDjGrkTEBe7KcGEsomdAQ9SyWNwQTZ/OTcvbTK0I2UtiXRnau/JzIaGzONQ9sZUxybVW8m/uf1UoxugozLJEWQbLEoSoWLyp397w65BoZiagllmttbXTammjK0KRVtCP7qy+ukXav6XtW/q5XrjadFHAVyTi7IFfHJNamTW9IkLcKIIs/klbw56Lw4787HonXDWUZ4Rv7A+fwBRduRog==</latexit><latexit sha1_base64="qMdbMzklkGQJHxmjhsbn73a280s=">AAAB8nicbVDLSgNBEJz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYnvcmQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGpZtBiSijdDakBwSW0kKOAbqKBxqGATjhpzPzOI2jDlbzHaQJBTEeSR5xRtFKvAoPM5A+ZzCuDUtmrenO468RfkjJZojkoffWHiqUxSGSCGtPzvQSDjGrkTEBe7KcGEsomdAQ9SyWNwQTZ/OTcvbTK0I2UtiXRnau/JzIaGzONQ9sZUxybVW8m/uf1UoxugozLJEWQbLEoSoWLyp397w65BoZiagllmttbXTammjK0KRVtCP7qy+ukXav6XtW/q5XrjadFHAVyTi7IFfHJNamTW9IkLcKIIs/klbw56Lw4787HonXDWUZ4Rv7A+fwBRduRog==</latexit><latexit sha1_base64="qMdbMzklkGQJHxmjhsbn73a280s=">AAAB8nicbVDLSgNBEJz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYnvcmQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGpZtBiSijdDakBwSW0kKOAbqKBxqGATjhpzPzOI2jDlbzHaQJBTEeSR5xRtFKvAoPM5A+ZzCuDUtmrenO468RfkjJZojkoffWHiqUxSGSCGtPzvQSDjGrkTEBe7KcGEsomdAQ9SyWNwQTZ/OTcvbTK0I2UtiXRnau/JzIaGzONQ9sZUxybVW8m/uf1UoxugozLJEWQbLEoSoWLyp397w65BoZiagllmttbXTammjK0KRVtCP7qy+ukXav6XtW/q5XrjadFHAVyTi7IFfHJNamTW9IkLcKIIs/klbw56Lw4787HonXDWUZ4Rv7A+fwBRduRog==</latexit><latexit sha1_base64="qMdbMzklkGQJHxmjhsbn73a280s=">AAAB8nicbVDLSgNBEJz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYnvcmQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGpZtBiSijdDakBwSW0kKOAbqKBxqGATjhpzPzOI2jDlbzHaQJBTEeSR5xRtFKvAoPM5A+ZzCuDUtmrenO468RfkjJZojkoffWHiqUxSGSCGtPzvQSDjGrkTEBe7KcGEsomdAQ9SyWNwQTZ/OTcvbTK0I2UtiXRnau/JzIaGzONQ9sZUxybVW8m/uf1UoxugozLJEWQbLEoSoWLyp397w65BoZiagllmttbXTammjK0KRVtCP7qy+ukXav6XtW/q5XrjadFHAVyTi7IFfHJNamTW9IkLcKIIs/klbw56Lw4787HonXDWUZ4Rv7A+fwBRduRog==</latexit>

h01<latexit sha1_base64="NI+Fi0+IR/l/SgAaixWebmjaQCI=">AAACC3icbZA7T8MwFIUdniW8AowsUZsKpiphgRGJhREkWpDaNHLcm9bCeci+QVRRJEYW/goLAwix8gfY+De4jwFajmTp0znXsu8JM8EVuu63sbC4tLyyWlkz1zc2t7atnd2WSnPJoMlSkcqbkCoQPIEmchRwk0mgcSjgOrw9G+XXdyAVT5MrHGbgx7Sf8IgzitoKrGq9g3CPYVQ4EBSq7BZe6ZSm6QwCTd3ioHQCq+Y23LHsefCmUCNTXQTWV6eXsjyGBJmgSrU9N0O/oBI5E1CanVxBRtkt7UNbY0JjUH4x3qW069rp2VEq9UnQHru/bxQ0VmoYh3oypjhQs9nI/C9r5xid+AVPshwhYZOHolzYmNqjYuwel8BQDDVQJrn+q80GVFKGuj5Tl+DNrjwPraOG5za8y6Pa6dnDpI4K2SdVckg8ckxOyTm5IE3CyCN5Jq/kzXgyXox342MyumBMK9wjf2R8/gBE1JpL</latexit><latexit sha1_base64="NI+Fi0+IR/l/SgAaixWebmjaQCI=">AAACC3icbZA7T8MwFIUdniW8AowsUZsKpiphgRGJhREkWpDaNHLcm9bCeci+QVRRJEYW/goLAwix8gfY+De4jwFajmTp0znXsu8JM8EVuu63sbC4tLyyWlkz1zc2t7atnd2WSnPJoMlSkcqbkCoQPIEmchRwk0mgcSjgOrw9G+XXdyAVT5MrHGbgx7Sf8IgzitoKrGq9g3CPYVQ4EBSq7BZe6ZSm6QwCTd3ioHQCq+Y23LHsefCmUCNTXQTWV6eXsjyGBJmgSrU9N0O/oBI5E1CanVxBRtkt7UNbY0JjUH4x3qW069rp2VEq9UnQHru/bxQ0VmoYh3oypjhQs9nI/C9r5xid+AVPshwhYZOHolzYmNqjYuwel8BQDDVQJrn+q80GVFKGuj5Tl+DNrjwPraOG5za8y6Pa6dnDpI4K2SdVckg8ckxOyTm5IE3CyCN5Jq/kzXgyXox342MyumBMK9wjf2R8/gBE1JpL</latexit><latexit sha1_base64="NI+Fi0+IR/l/SgAaixWebmjaQCI=">AAACC3icbZA7T8MwFIUdniW8AowsUZsKpiphgRGJhREkWpDaNHLcm9bCeci+QVRRJEYW/goLAwix8gfY+De4jwFajmTp0znXsu8JM8EVuu63sbC4tLyyWlkz1zc2t7atnd2WSnPJoMlSkcqbkCoQPIEmchRwk0mgcSjgOrw9G+XXdyAVT5MrHGbgx7Sf8IgzitoKrGq9g3CPYVQ4EBSq7BZe6ZSm6QwCTd3ioHQCq+Y23LHsefCmUCNTXQTWV6eXsjyGBJmgSrU9N0O/oBI5E1CanVxBRtkt7UNbY0JjUH4x3qW069rp2VEq9UnQHru/bxQ0VmoYh3oypjhQs9nI/C9r5xid+AVPshwhYZOHolzYmNqjYuwel8BQDDVQJrn+q80GVFKGuj5Tl+DNrjwPraOG5za8y6Pa6dnDpI4K2SdVckg8ckxOyTm5IE3CyCN5Jq/kzXgyXox342MyumBMK9wjf2R8/gBE1JpL</latexit><latexit sha1_base64="NI+Fi0+IR/l/SgAaixWebmjaQCI=">AAACC3icbZA7T8MwFIUdniW8AowsUZsKpiphgRGJhREkWpDaNHLcm9bCeci+QVRRJEYW/goLAwix8gfY+De4jwFajmTp0znXsu8JM8EVuu63sbC4tLyyWlkz1zc2t7atnd2WSnPJoMlSkcqbkCoQPIEmchRwk0mgcSjgOrw9G+XXdyAVT5MrHGbgx7Sf8IgzitoKrGq9g3CPYVQ4EBSq7BZe6ZSm6QwCTd3ioHQCq+Y23LHsefCmUCNTXQTWV6eXsjyGBJmgSrU9N0O/oBI5E1CanVxBRtkt7UNbY0JjUH4x3qW069rp2VEq9UnQHru/bxQ0VmoYh3oypjhQs9nI/C9r5xid+AVPshwhYZOHolzYmNqjYuwel8BQDDVQJrn+q80GVFKGuj5Tl+DNrjwPraOG5za8y6Pa6dnDpI4K2SdVckg8ckxOyTm5IE3CyCN5Jq/kzXgyXox342MyumBMK9wjf2R8/gBE1JpL</latexit>

h02<latexit sha1_base64="9SXDgpqQDGC3B+GNyYQTSt21IWA=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dD7J6/pBd5tVBueLW3DnQOvGWpAJLtAblr/5QkjSmwhCOtfY9NzFBhpVhhNO81E81TTCZ4BH1LRU4pjrI5ifn6MIqQxRJZUsYNFd/T2Q41noah7YzxmasV72Z+J/npya6CTImktRQQRaLopQjI9HsfzRkihLDp5Zgopi9FZExVpgYm1LJhuCtvrxOOvWa59a8u3ql0XxaxFGEMziHK/DgGhpwCy1oAwEJz/AKb45xXpx352PRWnCWEZ7CHzifP3o/kR0=</latexit><latexit sha1_base64="9SXDgpqQDGC3B+GNyYQTSt21IWA=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dD7J6/pBd5tVBueLW3DnQOvGWpAJLtAblr/5QkjSmwhCOtfY9NzFBhpVhhNO81E81TTCZ4BH1LRU4pjrI5ifn6MIqQxRJZUsYNFd/T2Q41noah7YzxmasV72Z+J/npya6CTImktRQQRaLopQjI9HsfzRkihLDp5Zgopi9FZExVpgYm1LJhuCtvrxOOvWa59a8u3ql0XxaxFGEMziHK/DgGhpwCy1oAwEJz/AKb45xXpx352PRWnCWEZ7CHzifP3o/kR0=</latexit><latexit sha1_base64="9SXDgpqQDGC3B+GNyYQTSt21IWA=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dD7J6/pBd5tVBueLW3DnQOvGWpAJLtAblr/5QkjSmwhCOtfY9NzFBhpVhhNO81E81TTCZ4BH1LRU4pjrI5ifn6MIqQxRJZUsYNFd/T2Q41noah7YzxmasV72Z+J/npya6CTImktRQQRaLopQjI9HsfzRkihLDp5Zgopi9FZExVpgYm1LJhuCtvrxOOvWa59a8u3ql0XxaxFGEMziHK/DgGhpwCy1oAwEJz/AKb45xXpx352PRWnCWEZ7CHzifP3o/kR0=</latexit><latexit sha1_base64="9SXDgpqQDGC3B+GNyYQTSt21IWA=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dD7J6/pBd5tVBueLW3DnQOvGWpAJLtAblr/5QkjSmwhCOtfY9NzFBhpVhhNO81E81TTCZ4BH1LRU4pjrI5ifn6MIqQxRJZUsYNFd/T2Q41noah7YzxmasV72Z+J/npya6CTImktRQQRaLopQjI9HsfzRkihLDp5Zgopi9FZExVpgYm1LJhuCtvrxOOvWa59a8u3ql0XxaxFGEMziHK/DgGhpwCy1oAwEJz/AKb45xXpx352PRWnCWEZ7CHzifP3o/kR0=</latexit>

h0i

<latexit sha1_base64="swTA+BkkJWD94LW7vBNInCQOV8Y=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dDzKWP2SXeXVQrrg1dw60TrwlqcASrUH5qz+UJI2pMIRjrX3PTUyQYWUY4TQv9VNNE0wmeER9SwWOqQ6y+ck5urDKEEVS2RIGzdXfExmOtZ7Goe2MsRnrVW8m/uf5qYlugoyJJDVUkMWiKOXISDT7Hw2ZosTwqSWYKGZvRWSMFSbGplSyIXirL6+TTr3muTXvrl5pNJ8WcRThDM7hCjy4hgbcQgvaQEDCM7zCm2OcF+fd+Vi0FpxlhKfwB87nD87lkVQ=</latexit><latexit sha1_base64="swTA+BkkJWD94LW7vBNInCQOV8Y=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dDzKWP2SXeXVQrrg1dw60TrwlqcASrUH5qz+UJI2pMIRjrX3PTUyQYWUY4TQv9VNNE0wmeER9SwWOqQ6y+ck5urDKEEVS2RIGzdXfExmOtZ7Goe2MsRnrVW8m/uf5qYlugoyJJDVUkMWiKOXISDT7Hw2ZosTwqSWYKGZvRWSMFSbGplSyIXirL6+TTr3muTXvrl5pNJ8WcRThDM7hCjy4hgbcQgvaQEDCM7zCm2OcF+fd+Vi0FpxlhKfwB87nD87lkVQ=</latexit><latexit sha1_base64="swTA+BkkJWD94LW7vBNInCQOV8Y=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dDzKWP2SXeXVQrrg1dw60TrwlqcASrUH5qz+UJI2pMIRjrX3PTUyQYWUY4TQv9VNNE0wmeER9SwWOqQ6y+ck5urDKEEVS2RIGzdXfExmOtZ7Goe2MsRnrVW8m/uf5qYlugoyJJDVUkMWiKOXISDT7Hw2ZosTwqSWYKGZvRWSMFSbGplSyIXirL6+TTr3muTXvrl5pNJ8WcRThDM7hCjy4hgbcQgvaQEDCM7zCm2OcF+fd+Vi0FpxlhKfwB87nD87lkVQ=</latexit><latexit sha1_base64="swTA+BkkJWD94LW7vBNInCQOV8Y=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dDzKWP2SXeXVQrrg1dw60TrwlqcASrUH5qz+UJI2pMIRjrX3PTUyQYWUY4TQv9VNNE0wmeER9SwWOqQ6y+ck5urDKEEVS2RIGzdXfExmOtZ7Goe2MsRnrVW8m/uf5qYlugoyJJDVUkMWiKOXISDT7Hw2ZosTwqSWYKGZvRWSMFSbGplSyIXirL6+TTr3muTXvrl5pNJ8WcRThDM7hCjy4hgbcQgvaQEDCM7zCm2OcF+fd+Vi0FpxlhKfwB87nD87lkVQ=</latexit>

hi<latexit sha1_base64="6bKcd7dW+CXNJp71hPejv21TUz0=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubvWTJ3t6xOyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0ikMOi6305ha3tnd6+4Xzo4PDo+KZ+edUycasbbLJax7gXUcCkUb6NAyXuJ5jQKJO8G0+bC7z5ybUSsHnCWcD+iYyVCwShaqVudDDMxrw7LFbfmLkE2ibciFVihNSx/DUYxSyOukElqTN9zE/QzqlEwyeelQWp4QtmUjnnfUkUjbvxsee6cXFllRMJY21JIlurviYxGxsyiwHZGFCdm3VuI/3n9FMNbPxMqSZErli8KU0kwJovfyUhozlDOLKFMC3srYROqKUObUMmG4K2/vEk69Zrn1rz7eqXRfMrjKMIFXMI1eHADDbiDFrSBwRSe4RXenMR5cd6dj7y14KwiPIc/cD5/AOyUj68=</latexit><latexit sha1_base64="6bKcd7dW+CXNJp71hPejv21TUz0=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubvWTJ3t6xOyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0ikMOi6305ha3tnd6+4Xzo4PDo+KZ+edUycasbbLJax7gXUcCkUb6NAyXuJ5jQKJO8G0+bC7z5ybUSsHnCWcD+iYyVCwShaqVudDDMxrw7LFbfmLkE2ibciFVihNSx/DUYxSyOukElqTN9zE/QzqlEwyeelQWp4QtmUjnnfUkUjbvxsee6cXFllRMJY21JIlurviYxGxsyiwHZGFCdm3VuI/3n9FMNbPxMqSZErli8KU0kwJovfyUhozlDOLKFMC3srYROqKUObUMmG4K2/vEk69Zrn1rz7eqXRfMrjKMIFXMI1eHADDbiDFrSBwRSe4RXenMR5cd6dj7y14KwiPIc/cD5/AOyUj68=</latexit><latexit sha1_base64="6bKcd7dW+CXNJp71hPejv21TUz0=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubvWTJ3t6xOyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0ikMOi6305ha3tnd6+4Xzo4PDo+KZ+edUycasbbLJax7gXUcCkUb6NAyXuJ5jQKJO8G0+bC7z5ybUSsHnCWcD+iYyVCwShaqVudDDMxrw7LFbfmLkE2ibciFVihNSx/DUYxSyOukElqTN9zE/QzqlEwyeelQWp4QtmUjnnfUkUjbvxsee6cXFllRMJY21JIlurviYxGxsyiwHZGFCdm3VuI/3n9FMNbPxMqSZErli8KU0kwJovfyUhozlDOLKFMC3srYROqKUObUMmG4K2/vEk69Zrn1rz7eqXRfMrjKMIFXMI1eHADDbiDFrSBwRSe4RXenMR5cd6dj7y14KwiPIc/cD5/AOyUj68=</latexit><latexit sha1_base64="6bKcd7dW+CXNJp71hPejv21TUz0=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubvWTJ3t6xOyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0ikMOi6305ha3tnd6+4Xzo4PDo+KZ+edUycasbbLJax7gXUcCkUb6NAyXuJ5jQKJO8G0+bC7z5ybUSsHnCWcD+iYyVCwShaqVudDDMxrw7LFbfmLkE2ibciFVihNSx/DUYxSyOukElqTN9zE/QzqlEwyeelQWp4QtmUjnnfUkUjbvxsee6cXFllRMJY21JIlurviYxGxsyiwHZGFCdm3VuI/3n9FMNbPxMqSZErli8KU0kwJovfyUhozlDOLKFMC3srYROqKUObUMmG4K2/vEk69Zrn1rz7eqXRfMrjKMIFXMI1eHADDbiDFrSBwRSe4RXenMR5cd6dj7y14KwiPIc/cD5/AOyUj68=</latexit>

h2<latexit sha1_base64="aN1QsZLTvYV/zny52DFGqu6rB+s=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubuWTJ3t6xuyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0gE18Z1v53C1vbO7l5xv3RweHR8Uj496+g4VQzbLBax6gVUo+AS24Ybgb1EIY0Cgd1g2lz43UdUmsfywcwS9CM6ljzkjBordauTYVafV4fliltzlyCbxFuRCqzQGpa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tz52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asJbP+MySQ1Kli8KU0FMTBa/kxFXyIyYWUKZ4vZWwiZUUWZsQiUbgrf+8ibp1GueW/Pu65VG8ymPowgXcAnX4MENNOAOWtAGBlN4hld4cxLnxXl3PvLWgrOK8Bz+wPn8AZjKj3g=</latexit><latexit sha1_base64="aN1QsZLTvYV/zny52DFGqu6rB+s=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubuWTJ3t6xuyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0gE18Z1v53C1vbO7l5xv3RweHR8Uj496+g4VQzbLBax6gVUo+AS24Ybgb1EIY0Cgd1g2lz43UdUmsfywcwS9CM6ljzkjBordauTYVafV4fliltzlyCbxFuRCqzQGpa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tz52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asJbP+MySQ1Kli8KU0FMTBa/kxFXyIyYWUKZ4vZWwiZUUWZsQiUbgrf+8ibp1GueW/Pu65VG8ymPowgXcAnX4MENNOAOWtAGBlN4hld4cxLnxXl3PvLWgrOK8Bz+wPn8AZjKj3g=</latexit><latexit sha1_base64="aN1QsZLTvYV/zny52DFGqu6rB+s=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubuWTJ3t6xuyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0gE18Z1v53C1vbO7l5xv3RweHR8Uj496+g4VQzbLBax6gVUo+AS24Ybgb1EIY0Cgd1g2lz43UdUmsfywcwS9CM6ljzkjBordauTYVafV4fliltzlyCbxFuRCqzQGpa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tz52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asJbP+MySQ1Kli8KU0FMTBa/kxFXyIyYWUKZ4vZWwiZUUWZsQiUbgrf+8ibp1GueW/Pu65VG8ymPowgXcAnX4MENNOAOWtAGBlN4hld4cxLnxXl3PvLWgrOK8Bz+wPn8AZjKj3g=</latexit><latexit sha1_base64="aN1QsZLTvYV/zny52DFGqu6rB+s=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubuWTJ3t6xuyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0gE18Z1v53C1vbO7l5xv3RweHR8Uj496+g4VQzbLBax6gVUo+AS24Ybgb1EIY0Cgd1g2lz43UdUmsfywcwS9CM6ljzkjBordauTYVafV4fliltzlyCbxFuRCqzQGpa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tz52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asJbP+MySQ1Kli8KU0FMTBa/kxFXyIyYWUKZ4vZWwiZUUWZsQiUbgrf+8ibp1GueW/Pu65VG8ymPowgXcAnX4MENNOAOWtAGBlN4hld4cxLnxXl3PvLWgrOK8Bz+wPn8AZjKj3g=</latexit>

h1<latexit sha1_base64="+i932c6L49tAO+o/TkAdsL1b+CA=">AAAB7HicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKClwSSI+xt9pIle3vH7pwQjoD/wMZCEVt/kJ3/xs1HoYkPBh7vzTAzL0ylMOi6305ha3tnd6+4Xzo4PDo+KZ+etU2SacZ9lshEd0NquBSK+yhQ8m6qOY1DyTvhpDn3O49cG5GoB5ymPIjpSIlIMIpW8qvjgVcdlCtuzV2AbBJvRSqwQmtQ/uoPE5bFXCGT1Jie56YY5FSjYJLPSv3M8JSyCR3xnqWKxtwE+eLYGbmyypBEibalkCzU3xM5jY2ZxqHtjCmOzbo3F//zehlGt0EuVJohV2y5KMokwYTMPydDoTlDObWEMi3srYSNqaYMbT4lG4K3/vImaddrnlvz7uuVRvNpGUcRLuASrsGDG2jAHbTABwYCnuEV3hzlvDjvzseyteCsIjyHP3A+fwDRd45r</latexit><latexit sha1_base64="+i932c6L49tAO+o/TkAdsL1b+CA=">AAAB7HicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKClwSSI+xt9pIle3vH7pwQjoD/wMZCEVt/kJ3/xs1HoYkPBh7vzTAzL0ylMOi6305ha3tnd6+4Xzo4PDo+KZ+etU2SacZ9lshEd0NquBSK+yhQ8m6qOY1DyTvhpDn3O49cG5GoB5ymPIjpSIlIMIpW8qvjgVcdlCtuzV2AbBJvRSqwQmtQ/uoPE5bFXCGT1Jie56YY5FSjYJLPSv3M8JSyCR3xnqWKxtwE+eLYGbmyypBEibalkCzU3xM5jY2ZxqHtjCmOzbo3F//zehlGt0EuVJohV2y5KMokwYTMPydDoTlDObWEMi3srYSNqaYMbT4lG4K3/vImaddrnlvz7uuVRvNpGUcRLuASrsGDG2jAHbTABwYCnuEV3hzlvDjvzseyteCsIjyHP3A+fwDRd45r</latexit><latexit sha1_base64="+i932c6L49tAO+o/TkAdsL1b+CA=">AAAB7HicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKClwSSI+xt9pIle3vH7pwQjoD/wMZCEVt/kJ3/xs1HoYkPBh7vzTAzL0ylMOi6305ha3tnd6+4Xzo4PDo+KZ+etU2SacZ9lshEd0NquBSK+yhQ8m6qOY1DyTvhpDn3O49cG5GoB5ymPIjpSIlIMIpW8qvjgVcdlCtuzV2AbBJvRSqwQmtQ/uoPE5bFXCGT1Jie56YY5FSjYJLPSv3M8JSyCR3xnqWKxtwE+eLYGbmyypBEibalkCzU3xM5jY2ZxqHtjCmOzbo3F//zehlGt0EuVJohV2y5KMokwYTMPydDoTlDObWEMi3srYSNqaYMbT4lG4K3/vImaddrnlvz7uuVRvNpGUcRLuASrsGDG2jAHbTABwYCnuEV3hzlvDjvzseyteCsIjyHP3A+fwDRd45r</latexit><latexit sha1_base64="+i932c6L49tAO+o/TkAdsL1b+CA=">AAAB7HicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKClwSSI+xt9pIle3vH7pwQjoD/wMZCEVt/kJ3/xs1HoYkPBh7vzTAzL0ylMOi6305ha3tnd6+4Xzo4PDo+KZ+etU2SacZ9lshEd0NquBSK+yhQ8m6qOY1DyTvhpDn3O49cG5GoB5ymPIjpSIlIMIpW8qvjgVcdlCtuzV2AbBJvRSqwQmtQ/uoPE5bFXCGT1Jie56YY5FSjYJLPSv3M8JSyCR3xnqWKxtwE+eLYGbmyypBEibalkCzU3xM5jY2ZxqHtjCmOzbo3F//zehlGt0EuVJohV2y5KMokwYTMPydDoTlDObWEMi3srYSNqaYMbT4lG4K3/vImaddrnlvz7uuVRvNpGUcRLuASrsGDG2jAHbTABwYCnuEV3hzlvDjvzseyteCsIjyHP3A+fwDRd45r</latexit>

h01<latexit sha1_base64="NI+Fi0+IR/l/SgAaixWebmjaQCI=">AAACC3icbZA7T8MwFIUdniW8AowsUZsKpiphgRGJhREkWpDaNHLcm9bCeci+QVRRJEYW/goLAwix8gfY+De4jwFajmTp0znXsu8JM8EVuu63sbC4tLyyWlkz1zc2t7atnd2WSnPJoMlSkcqbkCoQPIEmchRwk0mgcSjgOrw9G+XXdyAVT5MrHGbgx7Sf8IgzitoKrGq9g3CPYVQ4EBSq7BZe6ZSm6QwCTd3ioHQCq+Y23LHsefCmUCNTXQTWV6eXsjyGBJmgSrU9N0O/oBI5E1CanVxBRtkt7UNbY0JjUH4x3qW069rp2VEq9UnQHru/bxQ0VmoYh3oypjhQs9nI/C9r5xid+AVPshwhYZOHolzYmNqjYuwel8BQDDVQJrn+q80GVFKGuj5Tl+DNrjwPraOG5za8y6Pa6dnDpI4K2SdVckg8ckxOyTm5IE3CyCN5Jq/kzXgyXox342MyumBMK9wjf2R8/gBE1JpL</latexit><latexit sha1_base64="NI+Fi0+IR/l/SgAaixWebmjaQCI=">AAACC3icbZA7T8MwFIUdniW8AowsUZsKpiphgRGJhREkWpDaNHLcm9bCeci+QVRRJEYW/goLAwix8gfY+De4jwFajmTp0znXsu8JM8EVuu63sbC4tLyyWlkz1zc2t7atnd2WSnPJoMlSkcqbkCoQPIEmchRwk0mgcSjgOrw9G+XXdyAVT5MrHGbgx7Sf8IgzitoKrGq9g3CPYVQ4EBSq7BZe6ZSm6QwCTd3ioHQCq+Y23LHsefCmUCNTXQTWV6eXsjyGBJmgSrU9N0O/oBI5E1CanVxBRtkt7UNbY0JjUH4x3qW069rp2VEq9UnQHru/bxQ0VmoYh3oypjhQs9nI/C9r5xid+AVPshwhYZOHolzYmNqjYuwel8BQDDVQJrn+q80GVFKGuj5Tl+DNrjwPraOG5za8y6Pa6dnDpI4K2SdVckg8ckxOyTm5IE3CyCN5Jq/kzXgyXox342MyumBMK9wjf2R8/gBE1JpL</latexit><latexit sha1_base64="NI+Fi0+IR/l/SgAaixWebmjaQCI=">AAACC3icbZA7T8MwFIUdniW8AowsUZsKpiphgRGJhREkWpDaNHLcm9bCeci+QVRRJEYW/goLAwix8gfY+De4jwFajmTp0znXsu8JM8EVuu63sbC4tLyyWlkz1zc2t7atnd2WSnPJoMlSkcqbkCoQPIEmchRwk0mgcSjgOrw9G+XXdyAVT5MrHGbgx7Sf8IgzitoKrGq9g3CPYVQ4EBSq7BZe6ZSm6QwCTd3ioHQCq+Y23LHsefCmUCNTXQTWV6eXsjyGBJmgSrU9N0O/oBI5E1CanVxBRtkt7UNbY0JjUH4x3qW069rp2VEq9UnQHru/bxQ0VmoYh3oypjhQs9nI/C9r5xid+AVPshwhYZOHolzYmNqjYuwel8BQDDVQJrn+q80GVFKGuj5Tl+DNrjwPraOG5za8y6Pa6dnDpI4K2SdVckg8ckxOyTm5IE3CyCN5Jq/kzXgyXox342MyumBMK9wjf2R8/gBE1JpL</latexit><latexit sha1_base64="NI+Fi0+IR/l/SgAaixWebmjaQCI=">AAACC3icbZA7T8MwFIUdniW8AowsUZsKpiphgRGJhREkWpDaNHLcm9bCeci+QVRRJEYW/goLAwix8gfY+De4jwFajmTp0znXsu8JM8EVuu63sbC4tLyyWlkz1zc2t7atnd2WSnPJoMlSkcqbkCoQPIEmchRwk0mgcSjgOrw9G+XXdyAVT5MrHGbgx7Sf8IgzitoKrGq9g3CPYVQ4EBSq7BZe6ZSm6QwCTd3ioHQCq+Y23LHsefCmUCNTXQTWV6eXsjyGBJmgSrU9N0O/oBI5E1CanVxBRtkt7UNbY0JjUH4x3qW069rp2VEq9UnQHru/bxQ0VmoYh3oypjhQs9nI/C9r5xid+AVPshwhYZOHolzYmNqjYuwel8BQDDVQJrn+q80GVFKGuj5Tl+DNrjwPraOG5za8y6Pa6dnDpI4K2SdVckg8ckxOyTm5IE3CyCN5Jq/kzXgyXox342MyumBMK9wjf2R8/gBE1JpL</latexit>

h02<latexit sha1_base64="9SXDgpqQDGC3B+GNyYQTSt21IWA=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dD7J6/pBd5tVBueLW3DnQOvGWpAJLtAblr/5QkjSmwhCOtfY9NzFBhpVhhNO81E81TTCZ4BH1LRU4pjrI5ifn6MIqQxRJZUsYNFd/T2Q41noah7YzxmasV72Z+J/npya6CTImktRQQRaLopQjI9HsfzRkihLDp5Zgopi9FZExVpgYm1LJhuCtvrxOOvWa59a8u3ql0XxaxFGEMziHK/DgGhpwCy1oAwEJz/AKb45xXpx352PRWnCWEZ7CHzifP3o/kR0=</latexit><latexit sha1_base64="9SXDgpqQDGC3B+GNyYQTSt21IWA=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dD7J6/pBd5tVBueLW3DnQOvGWpAJLtAblr/5QkjSmwhCOtfY9NzFBhpVhhNO81E81TTCZ4BH1LRU4pjrI5ifn6MIqQxRJZUsYNFd/T2Q41noah7YzxmasV72Z+J/npya6CTImktRQQRaLopQjI9HsfzRkihLDp5Zgopi9FZExVpgYm1LJhuCtvrxOOvWa59a8u3ql0XxaxFGEMziHK/DgGhpwCy1oAwEJz/AKb45xXpx352PRWnCWEZ7CHzifP3o/kR0=</latexit><latexit sha1_base64="9SXDgpqQDGC3B+GNyYQTSt21IWA=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dD7J6/pBd5tVBueLW3DnQOvGWpAJLtAblr/5QkjSmwhCOtfY9NzFBhpVhhNO81E81TTCZ4BH1LRU4pjrI5ifn6MIqQxRJZUsYNFd/T2Q41noah7YzxmasV72Z+J/npya6CTImktRQQRaLopQjI9HsfzRkihLDp5Zgopi9FZExVpgYm1LJhuCtvrxOOvWa59a8u3ql0XxaxFGEMziHK/DgGhpwCy1oAwEJz/AKb45xXpx352PRWnCWEZ7CHzifP3o/kR0=</latexit><latexit sha1_base64="9SXDgpqQDGC3B+GNyYQTSt21IWA=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dD7J6/pBd5tVBueLW3DnQOvGWpAJLtAblr/5QkjSmwhCOtfY9NzFBhpVhhNO81E81TTCZ4BH1LRU4pjrI5ifn6MIqQxRJZUsYNFd/T2Q41noah7YzxmasV72Z+J/npya6CTImktRQQRaLopQjI9HsfzRkihLDp5Zgopi9FZExVpgYm1LJhuCtvrxOOvWa59a8u3ql0XxaxFGEMziHK/DgGhpwCy1oAwEJz/AKb45xXpx352PRWnCWEZ7CHzifP3o/kR0=</latexit>

h0i

<latexit sha1_base64="swTA+BkkJWD94LW7vBNInCQOV8Y=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dDzKWP2SXeXVQrrg1dw60TrwlqcASrUH5qz+UJI2pMIRjrX3PTUyQYWUY4TQv9VNNE0wmeER9SwWOqQ6y+ck5urDKEEVS2RIGzdXfExmOtZ7Goe2MsRnrVW8m/uf5qYlugoyJJDVUkMWiKOXISDT7Hw2ZosTwqSWYKGZvRWSMFSbGplSyIXirL6+TTr3muTXvrl5pNJ8WcRThDM7hCjy4hgbcQgvaQEDCM7zCm2OcF+fd+Vi0FpxlhKfwB87nD87lkVQ=</latexit><latexit sha1_base64="swTA+BkkJWD94LW7vBNInCQOV8Y=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dDzKWP2SXeXVQrrg1dw60TrwlqcASrUH5qz+UJI2pMIRjrX3PTUyQYWUY4TQv9VNNE0wmeER9SwWOqQ6y+ck5urDKEEVS2RIGzdXfExmOtZ7Goe2MsRnrVW8m/uf5qYlugoyJJDVUkMWiKOXISDT7Hw2ZosTwqSWYKGZvRWSMFSbGplSyIXirL6+TTr3muTXvrl5pNJ8WcRThDM7hCjy4hgbcQgvaQEDCM7zCm2OcF+fd+Vi0FpxlhKfwB87nD87lkVQ=</latexit><latexit sha1_base64="swTA+BkkJWD94LW7vBNInCQOV8Y=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dDzKWP2SXeXVQrrg1dw60TrwlqcASrUH5qz+UJI2pMIRjrX3PTUyQYWUY4TQv9VNNE0wmeER9SwWOqQ6y+ck5urDKEEVS2RIGzdXfExmOtZ7Goe2MsRnrVW8m/uf5qYlugoyJJDVUkMWiKOXISDT7Hw2ZosTwqSWYKGZvRWSMFSbGplSyIXirL6+TTr3muTXvrl5pNJ8WcRThDM7hCjy4hgbcQgvaQEDCM7zCm2OcF+fd+Vi0FpxlhKfwB87nD87lkVQ=</latexit><latexit sha1_base64="swTA+BkkJWD94LW7vBNInCQOV8Y=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBhPRU9jNRY+BXDxGMA/YrGF2MpsMmZ1ZZmaFsCz4E148KOLVr/Hm3zh5HDSxoKGo6qa7K0w408Z1v53CxubW9k5xt7S3f3B4VD4+6WiZKkLbRHKpeiHWlDNB24YZTnuJojgOOe2Gk+bM7z5SpZkU92aa0CDGI8EiRrCxkl8dDzKWP2SXeXVQrrg1dw60TrwlqcASrUH5qz+UJI2pMIRjrX3PTUyQYWUY4TQv9VNNE0wmeER9SwWOqQ6y+ck5urDKEEVS2RIGzdXfExmOtZ7Goe2MsRnrVW8m/uf5qYlugoyJJDVUkMWiKOXISDT7Hw2ZosTwqSWYKGZvRWSMFSbGplSyIXirL6+TTr3muTXvrl5pNJ8WcRThDM7hCjy4hgbcQgvaQEDCM7zCm2OcF+fd+Vi0FpxlhKfwB87nD87lkVQ=</latexit>

hi<latexit sha1_base64="6bKcd7dW+CXNJp71hPejv21TUz0=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubvWTJ3t6xOyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0ikMOi6305ha3tnd6+4Xzo4PDo+KZ+edUycasbbLJax7gXUcCkUb6NAyXuJ5jQKJO8G0+bC7z5ybUSsHnCWcD+iYyVCwShaqVudDDMxrw7LFbfmLkE2ibciFVihNSx/DUYxSyOukElqTN9zE/QzqlEwyeelQWp4QtmUjnnfUkUjbvxsee6cXFllRMJY21JIlurviYxGxsyiwHZGFCdm3VuI/3n9FMNbPxMqSZErli8KU0kwJovfyUhozlDOLKFMC3srYROqKUObUMmG4K2/vEk69Zrn1rz7eqXRfMrjKMIFXMI1eHADDbiDFrSBwRSe4RXenMR5cd6dj7y14KwiPIc/cD5/AOyUj68=</latexit><latexit sha1_base64="6bKcd7dW+CXNJp71hPejv21TUz0=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubvWTJ3t6xOyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0ikMOi6305ha3tnd6+4Xzo4PDo+KZ+edUycasbbLJax7gXUcCkUb6NAyXuJ5jQKJO8G0+bC7z5ybUSsHnCWcD+iYyVCwShaqVudDDMxrw7LFbfmLkE2ibciFVihNSx/DUYxSyOukElqTN9zE/QzqlEwyeelQWp4QtmUjnnfUkUjbvxsee6cXFllRMJY21JIlurviYxGxsyiwHZGFCdm3VuI/3n9FMNbPxMqSZErli8KU0kwJovfyUhozlDOLKFMC3srYROqKUObUMmG4K2/vEk69Zrn1rz7eqXRfMrjKMIFXMI1eHADDbiDFrSBwRSe4RXenMR5cd6dj7y14KwiPIc/cD5/AOyUj68=</latexit><latexit sha1_base64="6bKcd7dW+CXNJp71hPejv21TUz0=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubvWTJ3t6xOyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0ikMOi6305ha3tnd6+4Xzo4PDo+KZ+edUycasbbLJax7gXUcCkUb6NAyXuJ5jQKJO8G0+bC7z5ybUSsHnCWcD+iYyVCwShaqVudDDMxrw7LFbfmLkE2ibciFVihNSx/DUYxSyOukElqTN9zE/QzqlEwyeelQWp4QtmUjnnfUkUjbvxsee6cXFllRMJY21JIlurviYxGxsyiwHZGFCdm3VuI/3n9FMNbPxMqSZErli8KU0kwJovfyUhozlDOLKFMC3srYROqKUObUMmG4K2/vEk69Zrn1rz7eqXRfMrjKMIFXMI1eHADDbiDFrSBwRSe4RXenMR5cd6dj7y14KwiPIc/cD5/AOyUj68=</latexit><latexit sha1_base64="6bKcd7dW+CXNJp71hPejv21TUz0=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubvWTJ3t6xOyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0ikMOi6305ha3tnd6+4Xzo4PDo+KZ+edUycasbbLJax7gXUcCkUb6NAyXuJ5jQKJO8G0+bC7z5ybUSsHnCWcD+iYyVCwShaqVudDDMxrw7LFbfmLkE2ibciFVihNSx/DUYxSyOukElqTN9zE/QzqlEwyeelQWp4QtmUjnnfUkUjbvxsee6cXFllRMJY21JIlurviYxGxsyiwHZGFCdm3VuI/3n9FMNbPxMqSZErli8KU0kwJovfyUhozlDOLKFMC3srYROqKUObUMmG4K2/vEk69Zrn1rz7eqXRfMrjKMIFXMI1eHADDbiDFrSBwRSe4RXenMR5cd6dj7y14KwiPIc/cD5/AOyUj68=</latexit>

h2<latexit sha1_base64="aN1QsZLTvYV/zny52DFGqu6rB+s=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubuWTJ3t6xuyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0gE18Z1v53C1vbO7l5xv3RweHR8Uj496+g4VQzbLBax6gVUo+AS24Ybgb1EIY0Cgd1g2lz43UdUmsfywcwS9CM6ljzkjBordauTYVafV4fliltzlyCbxFuRCqzQGpa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tz52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asJbP+MySQ1Kli8KU0FMTBa/kxFXyIyYWUKZ4vZWwiZUUWZsQiUbgrf+8ibp1GueW/Pu65VG8ymPowgXcAnX4MENNOAOWtAGBlN4hld4cxLnxXl3PvLWgrOK8Bz+wPn8AZjKj3g=</latexit><latexit sha1_base64="aN1QsZLTvYV/zny52DFGqu6rB+s=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubuWTJ3t6xuyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0gE18Z1v53C1vbO7l5xv3RweHR8Uj496+g4VQzbLBax6gVUo+AS24Ybgb1EIY0Cgd1g2lz43UdUmsfywcwS9CM6ljzkjBordauTYVafV4fliltzlyCbxFuRCqzQGpa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tz52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asJbP+MySQ1Kli8KU0FMTBa/kxFXyIyYWUKZ4vZWwiZUUWZsQiUbgrf+8ibp1GueW/Pu65VG8ymPowgXcAnX4MENNOAOWtAGBlN4hld4cxLnxXl3PvLWgrOK8Bz+wPn8AZjKj3g=</latexit><latexit sha1_base64="aN1QsZLTvYV/zny52DFGqu6rB+s=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubuWTJ3t6xuyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0gE18Z1v53C1vbO7l5xv3RweHR8Uj496+g4VQzbLBax6gVUo+AS24Ybgb1EIY0Cgd1g2lz43UdUmsfywcwS9CM6ljzkjBordauTYVafV4fliltzlyCbxFuRCqzQGpa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tz52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asJbP+MySQ1Kli8KU0FMTBa/kxFXyIyYWUKZ4vZWwiZUUWZsQiUbgrf+8ibp1GueW/Pu65VG8ymPowgXcAnX4MENNOAOWtAGBlN4hld4cxLnxXl3PvLWgrOK8Bz+wPn8AZjKj3g=</latexit><latexit sha1_base64="aN1QsZLTvYV/zny52DFGqu6rB+s=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKC+YDkCHubuWTJ3t6xuyeEI+BfsLFQxNbfY+e/cZNLoYkPBh7vzTAzL0gE18Z1v53C1vbO7l5xv3RweHR8Uj496+g4VQzbLBax6gVUo+AS24Ybgb1EIY0Cgd1g2lz43UdUmsfywcwS9CM6ljzkjBordauTYVafV4fliltzlyCbxFuRCqzQGpa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tz52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asJbP+MySQ1Kli8KU0FMTBa/kxFXyIyYWUKZ4vZWwiZUUWZsQiUbgrf+8ibp1GueW/Pu65VG8ymPowgXcAnX4MENNOAOWtAGBlN4hld4cxLnxXl3PvLWgrOK8Bz+wPn8AZjKj3g=</latexit>

h1<latexit sha1_base64="+i932c6L49tAO+o/TkAdsL1b+CA=">AAAB7HicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKClwSSI+xt9pIle3vH7pwQjoD/wMZCEVt/kJ3/xs1HoYkPBh7vzTAzL0ylMOi6305ha3tnd6+4Xzo4PDo+KZ+etU2SacZ9lshEd0NquBSK+yhQ8m6qOY1DyTvhpDn3O49cG5GoB5ymPIjpSIlIMIpW8qvjgVcdlCtuzV2AbBJvRSqwQmtQ/uoPE5bFXCGT1Jie56YY5FSjYJLPSv3M8JSyCR3xnqWKxtwE+eLYGbmyypBEibalkCzU3xM5jY2ZxqHtjCmOzbo3F//zehlGt0EuVJohV2y5KMokwYTMPydDoTlDObWEMi3srYSNqaYMbT4lG4K3/vImaddrnlvz7uuVRvNpGUcRLuASrsGDG2jAHbTABwYCnuEV3hzlvDjvzseyteCsIjyHP3A+fwDRd45r</latexit><latexit sha1_base64="+i932c6L49tAO+o/TkAdsL1b+CA=">AAAB7HicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKClwSSI+xt9pIle3vH7pwQjoD/wMZCEVt/kJ3/xs1HoYkPBh7vzTAzL0ylMOi6305ha3tnd6+4Xzo4PDo+KZ+etU2SacZ9lshEd0NquBSK+yhQ8m6qOY1DyTvhpDn3O49cG5GoB5ymPIjpSIlIMIpW8qvjgVcdlCtuzV2AbBJvRSqwQmtQ/uoPE5bFXCGT1Jie56YY5FSjYJLPSv3M8JSyCR3xnqWKxtwE+eLYGbmyypBEibalkCzU3xM5jY2ZxqHtjCmOzbo3F//zehlGt0EuVJohV2y5KMokwYTMPydDoTlDObWEMi3srYSNqaYMbT4lG4K3/vImaddrnlvz7uuVRvNpGUcRLuASrsGDG2jAHbTABwYCnuEV3hzlvDjvzseyteCsIjyHP3A+fwDRd45r</latexit><latexit sha1_base64="+i932c6L49tAO+o/TkAdsL1b+CA=">AAAB7HicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKClwSSI+xt9pIle3vH7pwQjoD/wMZCEVt/kJ3/xs1HoYkPBh7vzTAzL0ylMOi6305ha3tnd6+4Xzo4PDo+KZ+etU2SacZ9lshEd0NquBSK+yhQ8m6qOY1DyTvhpDn3O49cG5GoB5ymPIjpSIlIMIpW8qvjgVcdlCtuzV2AbBJvRSqwQmtQ/uoPE5bFXCGT1Jie56YY5FSjYJLPSv3M8JSyCR3xnqWKxtwE+eLYGbmyypBEibalkCzU3xM5jY2ZxqHtjCmOzbo3F//zehlGt0EuVJohV2y5KMokwYTMPydDoTlDObWEMi3srYSNqaYMbT4lG4K3/vImaddrnlvz7uuVRvNpGUcRLuASrsGDG2jAHbTABwYCnuEV3hzlvDjvzseyteCsIjyHP3A+fwDRd45r</latexit><latexit sha1_base64="+i932c6L49tAO+o/TkAdsL1b+CA=">AAAB7HicbVA9SwNBEJ2LXzF+RS1tFhPBKtyl0TKQxjKClwSSI+xt9pIle3vH7pwQjoD/wMZCEVt/kJ3/xs1HoYkPBh7vzTAzL0ylMOi6305ha3tnd6+4Xzo4PDo+KZ+etU2SacZ9lshEd0NquBSK+yhQ8m6qOY1DyTvhpDn3O49cG5GoB5ymPIjpSIlIMIpW8qvjgVcdlCtuzV2AbBJvRSqwQmtQ/uoPE5bFXCGT1Jie56YY5FSjYJLPSv3M8JSyCR3xnqWKxtwE+eLYGbmyypBEibalkCzU3xM5jY2ZxqHtjCmOzbo3F//zehlGt0EuVJohV2y5KMokwYTMPydDoTlDObWEMi3srYSNqaYMbT4lG4K3/vImaddrnlvz7uuVRvNpGUcRLuASrsGDG2jAHbTABwYCnuEV3hzlvDjvzseyteCsIjyHP3A+fwDRd45r</latexit>

ema

<latexit sha1_base64="KTL1qK7QCQ3Rn1McPoYrM+uCxDc=">AAAB8nicbVDJSgNBEO1xjXGLevTSmAiewkwuegzk4jGCWSAZQ0+nJ2nSy9BdI4RhwJ/w4kERr36NN//GznLQxAcFj/eqqKoXJYJb8P1vb2Nza3tnt7BX3D84PDounZy2rU4NZS2qhTbdiFgmuGIt4CBYNzGMyEiwTjRpzPzOIzOWa3UP04SFkowUjzkl4KRehQ0ykj9kMq8MSmW/6s+B10mwJGW0RHNQ+uoPNU0lU0AFsbYX+AmEGTHAqWB5sZ9alhA6ISPWc1QRyWyYzU/O8aVThjjWxpUCPFd/T2REWjuVkeuUBMZ21ZuJ/3m9FOKbMOMqSYEpulgUpwKDxrP/8ZAbRkFMHSHUcHcrpmNiCAWXUtGFEKy+vE7atWrgV4O7WrneeFrEUUDn6AJdoQBdozq6RU3UQhRp9Ixe0ZsH3ov37n0sWje8ZYRn6A+8zx8ooZGP</latexit><latexit sha1_base64="KTL1qK7QCQ3Rn1McPoYrM+uCxDc=">AAAB8nicbVDJSgNBEO1xjXGLevTSmAiewkwuegzk4jGCWSAZQ0+nJ2nSy9BdI4RhwJ/w4kERr36NN//GznLQxAcFj/eqqKoXJYJb8P1vb2Nza3tnt7BX3D84PDounZy2rU4NZS2qhTbdiFgmuGIt4CBYNzGMyEiwTjRpzPzOIzOWa3UP04SFkowUjzkl4KRehQ0ykj9kMq8MSmW/6s+B10mwJGW0RHNQ+uoPNU0lU0AFsbYX+AmEGTHAqWB5sZ9alhA6ISPWc1QRyWyYzU/O8aVThjjWxpUCPFd/T2REWjuVkeuUBMZ21ZuJ/3m9FOKbMOMqSYEpulgUpwKDxrP/8ZAbRkFMHSHUcHcrpmNiCAWXUtGFEKy+vE7atWrgV4O7WrneeFrEUUDn6AJdoQBdozq6RU3UQhRp9Ixe0ZsH3ov37n0sWje8ZYRn6A+8zx8ooZGP</latexit><latexit sha1_base64="KTL1qK7QCQ3Rn1McPoYrM+uCxDc=">AAAB8nicbVDJSgNBEO1xjXGLevTSmAiewkwuegzk4jGCWSAZQ0+nJ2nSy9BdI4RhwJ/w4kERr36NN//GznLQxAcFj/eqqKoXJYJb8P1vb2Nza3tnt7BX3D84PDounZy2rU4NZS2qhTbdiFgmuGIt4CBYNzGMyEiwTjRpzPzOIzOWa3UP04SFkowUjzkl4KRehQ0ykj9kMq8MSmW/6s+B10mwJGW0RHNQ+uoPNU0lU0AFsbYX+AmEGTHAqWB5sZ9alhA6ISPWc1QRyWyYzU/O8aVThjjWxpUCPFd/T2REWjuVkeuUBMZ21ZuJ/3m9FOKbMOMqSYEpulgUpwKDxrP/8ZAbRkFMHSHUcHcrpmNiCAWXUtGFEKy+vE7atWrgV4O7WrneeFrEUUDn6AJdoQBdozq6RU3UQhRp9Ixe0ZsH3ov37n0sWje8ZYRn6A+8zx8ooZGP</latexit><latexit sha1_base64="KTL1qK7QCQ3Rn1McPoYrM+uCxDc=">AAAB8nicbVDJSgNBEO1xjXGLevTSmAiewkwuegzk4jGCWSAZQ0+nJ2nSy9BdI4RhwJ/w4kERr36NN//GznLQxAcFj/eqqKoXJYJb8P1vb2Nza3tnt7BX3D84PDounZy2rU4NZS2qhTbdiFgmuGIt4CBYNzGMyEiwTjRpzPzOIzOWa3UP04SFkowUjzkl4KRehQ0ykj9kMq8MSmW/6s+B10mwJGW0RHNQ+uoPNU0lU0AFsbYX+AmEGTHAqWB5sZ9alhA6ISPWc1QRyWyYzU/O8aVThjjWxpUCPFd/T2REWjuVkeuUBMZ21ZuJ/3m9FOKbMOMqSYEpulgUpwKDxrP/8ZAbRkFMHSHUcHcrpmNiCAWXUtGFEKy+vE7atWrgV4O7WrneeFrEUUDn6AJdoQBdozq6RU3UQhRp9Ixe0ZsH3ov37n0sWje8ZYRn6A+8zx8ooZGP</latexit>

e2a

<latexit sha1_base64="7kktAu7b4FeVGgSA2BER+/IbhOk=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYns8mQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGppqxFlVC6GxLDBJeshRwF6yaakTgUrBNOGjO/88i04Ure4zRhQUxGkkecErRSr8IGGckfslpeGZTKXtWbw10n/pKUYYnmoPTVHyqaxkwiFcSYnu8lGGREI6eC5cV+alhC6ISMWM9SSWJmgmx+cu5eWmXoRkrbkujO1d8TGYmNmcah7YwJjs2qNxP/83opRjdBxmWSIpN0sShKhYvKnf3vDrlmFMXUEkI1t7e6dEw0oWhTKtoQ/NWX10m7VvW9qn9XK9cbT4s4CnAOF3AFPlxDHW6hCS2goOAZXuHNQefFeXc+Fq0bzjLCM/gD5/MHzrCRVA==</latexit><latexit sha1_base64="7kktAu7b4FeVGgSA2BER+/IbhOk=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYns8mQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGppqxFlVC6GxLDBJeshRwF6yaakTgUrBNOGjO/88i04Ure4zRhQUxGkkecErRSr8IGGckfslpeGZTKXtWbw10n/pKUYYnmoPTVHyqaxkwiFcSYnu8lGGREI6eC5cV+alhC6ISMWM9SSWJmgmx+cu5eWmXoRkrbkujO1d8TGYmNmcah7YwJjs2qNxP/83opRjdBxmWSIpN0sShKhYvKnf3vDrlmFMXUEkI1t7e6dEw0oWhTKtoQ/NWX10m7VvW9qn9XK9cbT4s4CnAOF3AFPlxDHW6hCS2goOAZXuHNQefFeXc+Fq0bzjLCM/gD5/MHzrCRVA==</latexit><latexit sha1_base64="7kktAu7b4FeVGgSA2BER+/IbhOk=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYns8mQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGppqxFlVC6GxLDBJeshRwF6yaakTgUrBNOGjO/88i04Ure4zRhQUxGkkecErRSr8IGGckfslpeGZTKXtWbw10n/pKUYYnmoPTVHyqaxkwiFcSYnu8lGGREI6eC5cV+alhC6ISMWM9SSWJmgmx+cu5eWmXoRkrbkujO1d8TGYmNmcah7YwJjs2qNxP/83opRjdBxmWSIpN0sShKhYvKnf3vDrlmFMXUEkI1t7e6dEw0oWhTKtoQ/NWX10m7VvW9qn9XK9cbT4s4CnAOF3AFPlxDHW6hCS2goOAZXuHNQefFeXc+Fq0bzjLCM/gD5/MHzrCRVA==</latexit><latexit sha1_base64="7kktAu7b4FeVGgSA2BER+/IbhOk=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYiJ4Cru56DGQi8cI5gHJGmYns8mQ2ZllplcIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7wkRwg5737Wxsbm3v7Bb2ivsHh0fHpZPTtlGppqxFlVC6GxLDBJeshRwF6yaakTgUrBNOGjO/88i04Ure4zRhQUxGkkecErRSr8IGGckfslpeGZTKXtWbw10n/pKUYYnmoPTVHyqaxkwiFcSYnu8lGGREI6eC5cV+alhC6ISMWM9SSWJmgmx+cu5eWmXoRkrbkujO1d8TGYmNmcah7YwJjs2qNxP/83opRjdBxmWSIpN0sShKhYvKnf3vDrlmFMXUEkI1t7e6dEw0oWhTKtoQ/NWX10m7VvW9qn9XK9cbT4s4CnAOF3AFPlxDHW6hCS2goOAZXuHNQefFeXc+Fq0bzjLCM/gD5/MHzrCRVA==</latexit>

e1a

<latexit sha1_base64="AovTdxOkpFtu8lbvbrYfNu6Yk5I=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYCJ4Cru56DGQi8cI5gHJGmYnvcmQ2Z1lZlYIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7gkRwbVz329nY3Nre2S3sFfcPDo+OSyenbS1TxbDFpJCqG1CNgsfYMtwI7CYKaRQI7ASTxszvPKLSXMb3ZpqgH9FRzEPOqLFSr4KDjOYPmZdXBqWyW3XnIOvEW5IyLNEclL76Q8nSCGPDBNW657mJ8TOqDGcC82I/1ZhQNqEj7Fka0wi1n81PzsmlVYYklMpWbMhc/T2R0UjraRTYzoiasV71ZuJ/Xi814Y2f8ThJDcZssShMBTGSzP4nQ66QGTG1hDLF7a2EjamizNiUijYEb/XlddKuVT236t3VyvXG0yKOApzDBVyBB9dQh1toQgsYSHiGV3hzjPPivDsfi9YNZxnhGfyB8/kDzSqRUw==</latexit><latexit sha1_base64="AovTdxOkpFtu8lbvbrYfNu6Yk5I=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYCJ4Cru56DGQi8cI5gHJGmYnvcmQ2Z1lZlYIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7gkRwbVz329nY3Nre2S3sFfcPDo+OSyenbS1TxbDFpJCqG1CNgsfYMtwI7CYKaRQI7ASTxszvPKLSXMb3ZpqgH9FRzEPOqLFSr4KDjOYPmZdXBqWyW3XnIOvEW5IyLNEclL76Q8nSCGPDBNW657mJ8TOqDGcC82I/1ZhQNqEj7Fka0wi1n81PzsmlVYYklMpWbMhc/T2R0UjraRTYzoiasV71ZuJ/Xi814Y2f8ThJDcZssShMBTGSzP4nQ66QGTG1hDLF7a2EjamizNiUijYEb/XlddKuVT236t3VyvXG0yKOApzDBVyBB9dQh1toQgsYSHiGV3hzjPPivDsfi9YNZxnhGfyB8/kDzSqRUw==</latexit><latexit sha1_base64="AovTdxOkpFtu8lbvbrYfNu6Yk5I=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYCJ4Cru56DGQi8cI5gHJGmYnvcmQ2Z1lZlYIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7gkRwbVz329nY3Nre2S3sFfcPDo+OSyenbS1TxbDFpJCqG1CNgsfYMtwI7CYKaRQI7ASTxszvPKLSXMb3ZpqgH9FRzEPOqLFSr4KDjOYPmZdXBqWyW3XnIOvEW5IyLNEclL76Q8nSCGPDBNW657mJ8TOqDGcC82I/1ZhQNqEj7Fka0wi1n81PzsmlVYYklMpWbMhc/T2R0UjraRTYzoiasV71ZuJ/Xi814Y2f8ThJDcZssShMBTGSzP4nQ66QGTG1hDLF7a2EjamizNiUijYEb/XlddKuVT236t3VyvXG0yKOApzDBVyBB9dQh1toQgsYSHiGV3hzjPPivDsfi9YNZxnhGfyB8/kDzSqRUw==</latexit><latexit sha1_base64="AovTdxOkpFtu8lbvbrYfNu6Yk5I=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYCJ4Cru56DGQi8cI5gHJGmYnvcmQ2Z1lZlYIy4I/4cWDIl79Gm/+jZPHQRMLGoqqbrq7gkRwbVz329nY3Nre2S3sFfcPDo+OSyenbS1TxbDFpJCqG1CNgsfYMtwI7CYKaRQI7ASTxszvPKLSXMb3ZpqgH9FRzEPOqLFSr4KDjOYPmZdXBqWyW3XnIOvEW5IyLNEclL76Q8nSCGPDBNW657mJ8TOqDGcC82I/1ZhQNqEj7Fka0wi1n81PzsmlVYYklMpWbMhc/T2R0UjraRTYzoiasV71ZuJ/Xi814Y2f8ThJDcZssShMBTGSzP4nQ66QGTG1hDLF7a2EjamizNiUijYEb/XlddKuVT236t3VyvXG0yKOApzDBVyBB9dQh1toQgsYSHiGV3hzjPPivDsfi9YNZxnhGfyB8/kDzSqRUw==</latexit>

Figure 3: The framework of IATN.

the aspects of the sentences, which can provide more infor-mation to represent the final sentiment features. Thus, wetake a pair of sentences (i.e., hs = {h1s, h2s, h3s...hns }) and itsaspects (i.e., hpa) to describe the attention process. With therepresentation of those two feature vectors, we can simplyconnect them to one vector (i.e., his = {h1s, h2s, h3s...hns , hpa},here we denote hn+1

s = hpa). Then with the following for-mula, we can generate the attention vector αi:

αi =exp(γ(his))∑n+1i=1 exp(γ(h

is))

, (3)

where γ is score function which is defined as:

γ(his) = tanh(his · Ws + b̂s) , (4)

Ws and b̂s are weight matrix and bias matrix respectively.tanh is a non-linear function. Particularly, the attentionweight αi greatly enhances the explanatory ability of IATN.It enables us to extract words with high sentiment scores,which is helpful for sentiment cross-domain transfer. In theexperiments, we will conduct a deep analysis on attentionresults to specific words.

After computing the word attention weights, we can getthe sentence representation with its auxiliary aspects. Thenthe final expression is:

Sr =

n+1∑i=1

αihis . (5)

Similarly, for the aspects, we use the same method to gener-ate the final representation with its auxiliary sentence hidden

pooling vector hps . The attention vector βi is calculated by:

βi =exp(γ(hia, h

ps))∑m

i=1 exp(γ(hia, h

ps))

, (6)

where γ is the score function which is different from Eq.(5).We adopt spot multiplication (·) rather than simple connec-tion. Because we assume that the pooling feature hps containsemotional features and domain shared features. In this way,we can develop sentiment tendencies of different aspects.The score function γ and ultimate aspects representation Ar

are formalized as follows:

γ(hia, hps) = tanh(hia · hps · Wa + b̂a) , (7)

Ar =

m∑i=1

βihia . (8)

Until now, we get the sentence representation and the as-pect representation by interacting with them. Then we willtake them into the following tasks.Domain Classifier. The domain classifier aims to learncross-domain sentiment feature representations, where theinputs are the source domain data and target domain data.In detail, we utilize all the data Xs and Xt to do domainclassification, which predicts the domain labels of the sam-ples. Meantime, we use the labeled data Xl

s in the sourcedomain to do sentiment classification. However, the goal ofthe common training process is to minimize the classifica-tion error, i.e., to distinguish the two domains as accuratelyas possible. Differently, our intention is to learn commonfeatures which the domain classifier cannot discriminate be-tween domains. To solve this problem, we add the Gra-dient Reversal Layer (GRL) (Ganin and Lempitsky 2014;Ganin et al. 2016) to reverse the gradient direction in thetraining process. Through the domain classifier, we can getinvariance features which are domain shared and sentimentsensitive.

Mathematically, we can formally treat the gradient re-versal layer as a “pseudo-function”, which is defined bytwo incompatible equations describing its forward and back-propagation behaviors:

G(x) = x ,∂G(x)

∂x= −λI . (9)

Then we deal with the sentence representation Sr throughthe GRL as G(Sr) = S̃r and then feed it to the softmaxlayer as domain classification:

y′d = softmax(WdS̃r + b̂d) . (10)

Sentiment Classifier. The sentiment classifier focuses onmining the information of aspects in sentences. Many ef-forts have proved that aspects are crucial for classificationtasks (Wang et al. 2016). We presume that the coordinationof aspects and sentences can enhance the performance ofsentiment classification. For example, as shown in Figure 1,not only the aspect “appearance” has specific effects onthese two reviews, but also “appearance” and “battery life”make unique contribution to the same sentence. Thus, we

54%

22% 20%

36%26%

45%46%

78% 80%

64%74%

55%

00.10.20.30.40.50.60.70.80.9

B D B E B K D E D K E K

Common aspect

Unique aspect

Figure 4: Top-100 aspects analysis between domains.

propose that the weights of aspects should be computed inorder to capture their sentiment tendency. Finally, we com-bine sentence representation and aspect representation forsentiment classification:

y′s = softmax(Ws · [Sr ⊕ Ar] + b̂s) . (11)

where the symbol ⊕ represent the connection of vectors.

Training StrategyDifferent from the traditional methods, IATN has two tasks,i.e., domain classification and sentiment classification. Thus,our training process comprises two parts.

• Individual Attention Learning. The cross-domain senti-ment classifier needs to learn the domain-shared featurerepresentations which contribute to sentiment classifica-tion. In order to achieve this objective, we design the twotasks, i.e., domain classification and sentiment classifica-tion. We introduce cross-entropy loss functions for train-ing these two classifiers respectively:

Lsen = −(yslny′s + (1− ys)ln(1− y

′s)) , (12)

Ldom = −(ydlny′d + (1− yd)ln(1− y

′d)) , (13)

where ys and yd denote the ground truth of sentiment labeland domain label. y

′s and y

′d are prediction for the senti-

ment and domain, respectively.

• Interactive Attention Learning. Based on the above indi-vidual attention learning results, we also conduct interac-tive attention learning for them to optimize the parame-ters of both tasks simultaneously. In order to avoid over-fitting, we add the squared regularization and combinethem into an entire objective function:

L = Lsen + Ldom + ρLreg , (14)

where Lreg is the regularization which can avoid overfit-ting, and ρ is the regularization parameter. The traininggoal is to minimize L with respect to the model param-eters except the GRL training part which will be maxi-mized. Additionally, all the parameters are optimized bythe standard back-propagation algorithm (LeCun, Bengio,and Hinton 2015).

Table 1: Statistics of datasets after pre-processing.

Domains Testing set percentage# Train # Test # Unlabel

Books 5,000 1,000 8,000DVD 5,000 1,000 8,000Electronics 5,000 1,000 8,000Kitchen 5,000 1,000 8,000

ExperimentsDataset PreparationFor the reliability and authority of experimental results, weuse the Amazon reviews dataset, which has been widelyused for cross-domain sentiment classification. Meanwhile,we make the necessary pre-processing as follows. First, wechoose the reviews data from four domains: Book (B), DVD(D), Electronics (E) and Kitchen appliances (K). Each ofthe domains contains 6,000 labeled data, in which there are3,000 positive reviews (higher than 3 stars) and 3,000 neg-ative reviews (lower than 3 stars). Additionally, the datasetalso contains lots of unlabeled data. Here we randomly se-lect 8,000 unlabeled reviews from each domain as trainingdata. Table 1 summarizes the statistics of dataset after pre-processing.

Hyperparameters SettingIn our experiments, all word embeddings from sentencesand aspects are initialized as 200-dimension vectors byword2vec (Goldberg and Levy 2014). The dimensions ofword embeddings, attention vectors and LSTM hidden statesare set to 200, 64 and 64 respectively. All weight ma-trices are randomly initialized by a uniform distributionU(−0.01, 0.01), and all biases are set to zeros. For the per-formance of IATN, we finally set the coefficient of l2 nor-malization, the learning rate and the dropout rate as 10−4,10−3 and 0.25.

The aspect words are not given in the training data di-rectly. Therefore, we need to extract every aspect word of thesentences so that we can make full use of the aspect infor-mation. For each sentence S (i.e., w1

s , w2s ...w

ns ), we extract

their aspect sequence with m words asA (i.e.,w1a, w

2a...w

ma ).

As shown in Figure 4, we also extract the top 100 most fre-quently occurring aspects in each domain and analyze thesimilar proportions of them between domains. After all thedata is processed, we conduct the cross-domain experimentsbetween every two domains, which means we have 12 clas-sification tasks: B→D, B→E, B→K, D→B, D→E, D→K,E→B, E→D, E→K, K→B, K→D, K→E. For example, thenotation “B→D” represents the task which transfers fromthe source domain B to the target domain D.

Benchmark Methods• Naive is a non-domain-adaptive method which is trained

in the source domain and predicts in target domain di-rectly. It is designed based on LSTM (Hochreiter andSchmidhuber 1997).

Table 2: Sentiment classification accuracy on the Amazon reviews dataset.(a) Book→

Benchmarks Tasks

B→D B→E B→K

Naive 0.786 0.752 0.737SCL 0.807 0.763 0.771SFA 0.813 0.776 0.785mSDA 0.819 0.783 0.789DANN 0.832 0.764 0.790CNN-a 0.843 0.810 0.813AMN 0.855 0.824 0.811HATN 0.858 0.853 0.849HATNh 0.861 0.857 0.852

IATNn 0.854 0.849 0.838IATN 0.868 0.865 0.859

(b) DVD→

Tasks

D→B D→E D→K

0.756 0.734 0.7670.782 0.754 0.7790.788 0.758 0.7860.783 0.770 0.7930.805 0.796 0.8140.829 0.803 0.8190.846 0.812 0.8270.858 0.849 0.8530.863 0.856 0.862

0.848 0.855 0.8390.870 0.869 0.858

(c) Electronics→

Tasks

E→B E→D E→K

0.696 0.722 0.7870.716 0.745 0.8170.724 0.754 0.8250.738 0.761 0.8370.735 0.786 0.8410.749 0.793 0.8430.766 0.827 0.8570.808 0.838 0.8680.810 0.840 0.879

0.768 0.825 0.8590.818 0.841 0.887

(d) Kitchen→

Tasks AvgK→B K→D K→E

0.686 0.723 0.807 0.7460.713 0.752 0.818 0.7680.724 0.758 0.825 0.7760.730 0.755 0.831 0.7820.752 0.776 0.843 0.7940.779 0.803 0.855 0.8110.805 0.812 0.867 0.8250.824 0.841 0.868 0.8470.833 0.845 0.870 0.851

0.828 0.835 0.864 0.8370.847 0.844 0.876 0.859

• SCL (Blitzer, McDonald, and Pereira 2006) is a linearmethod, which aims to solve feature mismatch problemby aligning domain common and unique features.

• SFA (Pan et al. 2010) is a method which aims to builda bridge between the source and the target domains byaligning common and unique features.

• mSDA (Chen et al. 2012) is proposed to automaticallylearn a unified feature representation for sentences from alarge amount of data in all the domains.

• DANN (Chen et al. 2012) is based on the adversarial train-ing. DANN performs domain adaptation with the repre-sentation encoded in a 5000-dimension feature vector.

• CNN-aux (Yu and Jiang 2016) is based on ConvolutionalNeural Network (Kim 2014) and makes use of two auxil-iary tasks to help inducing sentence embedding.

• AMN (Li et al. 2017) is a method which learns domain-shared representations based on memory networks andadversarial training.

• HATN&HATNh (Li et al. 2018b) are hierarchical atten-tion networks to focus on both the word level and the sen-tence level sentiment. The former one does not contain thehierarchical positional encoding and the latter one does.

Experimental ResultsTo demonstrate the effectiveness of our proposed model, wecompare IATN with other state-of-the-art methods on thecross-domain sentiment classification task. Meanwhile, weuse classification accuracy (Stehman 1997) to evaluate themodels because our training and testing data are balanced.The results of all methods on Amazon dataset are shownin Table 2. From the comprehensive views, IATN model hasachieved the best performances on most tasks of this dataset.

Specifically, the Naive method performs badly at everytask because it does not use the data of the target domainwhen training. The performance of traditional methods (i.e.,

Book-pos Example:ofvagueVivianasthefromrightmecaptivatedstory outset,This

anddetail, isshe a realisticallyvery .portrayedinsightfulanddescribedsensitivelydrawnarescenesHer in.memory

Book-neg Example:.bookthispossessedwhateversurenotam to buyI me ,Honestly

wasitbestthenot

friendquoteofwastecompletea free .time To a , itwas youofuse my entertainment .dollar If you are fana

writingpedestrianof , lack-luster plots and hackneyed characterdevelopment , this is your book .

Dvd-pos Example:,posterseeingamazing movie’sAn wasthis not too!film When i

,excited reallyawesomebut it realized how it .iswhen watchingonly it’s story wellis laid amount specialout , but the of ,

scencesgreat and good .actorsDvd-neg Example:

.paceacting adequateTheofjustThe

was the plotmovie content reeks the.was The

wasgood ,However

. intelligent thrillersyndrome.

.what Jody Foster’s character brings to the plotClive’s calling the perfectkept this .character robbery I’m

still don’t understand

Noteffects

predictable

Figure 5: Attention visualization of the aspect words for sen-timent classification in B→ D task.

SCL, SFA, mSDA) has reached to 78.2% on average, whichare still not accurate enough because they are all based onmanually selecting common features, which means the fea-tures learned by them are limited. The neural network-basedmethods have made great improvements compared with thetraditional ones, which come to 85.1%. IATN outperformsthe state-of-art methods by reaching 85.9% because we ex-tract and make full use of the aspect and sentence informa-tion. In order to demonstrate the effects of aspect more intu-itively, we also compare IATN with a variant without aspectsinformation, denoted by IATNn. The average accuracy ofIATNn is 83.7%, which is 2.2% lower than IATN. Throughthe comparison between IATNn and IATN, we can concludethat IATN with aspect information does improve the accu-racy of the sentiment classification. Specifically, from the

…

Predicting

Backers PerksOwner Others

+ Vi<latexit sha1_base64="gs/8WL55v5a3aFbCRMbB5zJLs3k=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRIRdFl002VF+4BaSjKd1qF5MZkopQj+gFv9NPEP9C+8M6agFtEJSc6ce8+Zuff6SSBS5TivBWthcWl5pbhaWlvf2Nwqb++00jiTjDdZHMSy43spD0TEm0qogHcSyb3QD3jbH5/rePuWy1TE0ZWaJLwXeqNIDAXzFFGXrb7olytO1THLngduDirIVyMuv+AaA8RgyBCCI4IiHMBDSk8XLhwkxPUwJU4SEibOcY8SaTPK4pThETum74h23ZyNaK89U6NmdEpArySljQPSxJQnCevTbBPPjLNmf/OeGk99twn9/dwrJFbhhti/dLPM/+p0LQpDnJoaBNWUGEZXx3KXzHRF39z+UpUih4Q4jQcUl4SZUc76bBtNamrXvfVM/M1kalbvWZ6b4V3fkgbs/hznPGgdVV2n6l4cV2pn+aiL2MM+DmmeJ6ihjgaa5D3CI57wbNWtyMqsu89Uq5BrdvFtWQ8fQgeQNw==</latexit><latexit sha1_base64="gs/8WL55v5a3aFbCRMbB5zJLs3k=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRIRdFl002VF+4BaSjKd1qF5MZkopQj+gFv9NPEP9C+8M6agFtEJSc6ce8+Zuff6SSBS5TivBWthcWl5pbhaWlvf2Nwqb++00jiTjDdZHMSy43spD0TEm0qogHcSyb3QD3jbH5/rePuWy1TE0ZWaJLwXeqNIDAXzFFGXrb7olytO1THLngduDirIVyMuv+AaA8RgyBCCI4IiHMBDSk8XLhwkxPUwJU4SEibOcY8SaTPK4pThETum74h23ZyNaK89U6NmdEpArySljQPSxJQnCevTbBPPjLNmf/OeGk99twn9/dwrJFbhhti/dLPM/+p0LQpDnJoaBNWUGEZXx3KXzHRF39z+UpUih4Q4jQcUl4SZUc76bBtNamrXvfVM/M1kalbvWZ6b4V3fkgbs/hznPGgdVV2n6l4cV2pn+aiL2MM+DmmeJ6ihjgaa5D3CI57wbNWtyMqsu89Uq5BrdvFtWQ8fQgeQNw==</latexit><latexit sha1_base64="gs/8WL55v5a3aFbCRMbB5zJLs3k=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRIRdFl002VF+4BaSjKd1qF5MZkopQj+gFv9NPEP9C+8M6agFtEJSc6ce8+Zuff6SSBS5TivBWthcWl5pbhaWlvf2Nwqb++00jiTjDdZHMSy43spD0TEm0qogHcSyb3QD3jbH5/rePuWy1TE0ZWaJLwXeqNIDAXzFFGXrb7olytO1THLngduDirIVyMuv+AaA8RgyBCCI4IiHMBDSk8XLhwkxPUwJU4SEibOcY8SaTPK4pThETum74h23ZyNaK89U6NmdEpArySljQPSxJQnCevTbBPPjLNmf/OeGk99twn9/dwrJFbhhti/dLPM/+p0LQpDnJoaBNWUGEZXx3KXzHRF39z+UpUih4Q4jQcUl4SZUc76bBtNamrXvfVM/M1kalbvWZ6b4V3fkgbs/hznPGgdVV2n6l4cV2pn+aiL2MM+DmmeJ6ihjgaa5D3CI57wbNWtyMqsu89Uq5BrdvFtWQ8fQgeQNw==</latexit><latexit sha1_base64="gs/8WL55v5a3aFbCRMbB5zJLs3k=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRIRdFl002VF+4BaSjKd1qF5MZkopQj+gFv9NPEP9C+8M6agFtEJSc6ce8+Zuff6SSBS5TivBWthcWl5pbhaWlvf2Nwqb++00jiTjDdZHMSy43spD0TEm0qogHcSyb3QD3jbH5/rePuWy1TE0ZWaJLwXeqNIDAXzFFGXrb7olytO1THLngduDirIVyMuv+AaA8RgyBCCI4IiHMBDSk8XLhwkxPUwJU4SEibOcY8SaTPK4pThETum74h23ZyNaK89U6NmdEpArySljQPSxJQnCevTbBPPjLNmf/OeGk99twn9/dwrJFbhhti/dLPM/+p0LQpDnJoaBNWUGEZXx3KXzHRF39z+UpUih4Q4jQcUl4SZUc76bBtNamrXvfVM/M1kalbvWZ6b4V3fkgbs/hznPGgdVV2n6l4cV2pn+aiL2MM+DmmeJ6ihjgaa5D3CI57wbNWtyMqsu89Uq5BrdvFtWQ8fQgeQNw==</latexit>

Xi<latexit sha1_base64="2Nth70l0u28LPpwmv9L/fRYS+xg=">AAAB6nicbVA9SwNBEJ2LXzF+RVPaLAbBKtzZaHlgYxnRfEByhL3NXLJkb+/Y3RPCEfAP2FgoYusvshD8K1ZuPgpNfDDweG+GmXlhKrg2rvvlFNbWNza3itulnd29/YPy4VFTJ5li2GCJSFQ7pBoFl9gw3AhspwppHApshaOrqd+6R6V5Iu/MOMUgpgPJI86osdJtu8d75apbc2cgq8RbkKpf+fx+AIB6r/zR7Scsi1EaJqjWHc9NTZBTZTgTOCl1M40pZSM6wI6lksaog3x26oScWqVPokTZkobM1N8TOY21Hseh7YypGeplbyr+53UyE10GOZdpZlCy+aIoE8QkZPo36XOFzIixJZQpbm8lbEgVZcamU7IheMsvr5Lmec1za96NTcOHOYpwDCdwBh5cgA/XUIcGMBjAIzzDiyOcJ+fVeZu3FpzFTAX+wHn/Ab0KkFU=</latexit><latexit sha1_base64="J+7qQ4TRRmGAFh3xsNTOtfk2Z5A=">AAAB6nicbVA9SwNBEJ2LXzF+RVPaLAbBQsKdjZYBG8uI5gOSI+xt5pIle3vH7p4QjjT2NhaK2PqLLAT/gb/Bys1HoYkPBh7vzTAzL0gE18Z1P53cyura+kZ+s7C1vbO7V9w/aOg4VQzrLBaxagVUo+AS64Ybga1EIY0Cgc1geDnxm3eoNI/lrRkl6Ee0L3nIGTVWuml1ebdYdivuFGSZeHNSrpY+vu+/xqe1bvG904tZGqE0TFCt256bGD+jynAmcFzopBoTyoa0j21LJY1Q+9n01DE5tkqPhLGyJQ2Zqr8nMhppPYoC2xlRM9CL3kT8z2unJrzwMy6T1KBks0VhKoiJyeRv0uMKmREjSyhT3N5K2IAqyoxNp2BD8BZfXiaNs4rnVrxrm0YVZsjDIRzBCXhwDlW4ghrUgUEfHuAJnh3hPDovzuusNefMZ0rwB87bD9hKkeg=</latexit><latexit sha1_base64="J+7qQ4TRRmGAFh3xsNTOtfk2Z5A=">AAAB6nicbVA9SwNBEJ2LXzF+RVPaLAbBQsKdjZYBG8uI5gOSI+xt5pIle3vH7p4QjjT2NhaK2PqLLAT/gb/Bys1HoYkPBh7vzTAzL0gE18Z1P53cyura+kZ+s7C1vbO7V9w/aOg4VQzrLBaxagVUo+AS64Ybga1EIY0Cgc1geDnxm3eoNI/lrRkl6Ee0L3nIGTVWuml1ebdYdivuFGSZeHNSrpY+vu+/xqe1bvG904tZGqE0TFCt256bGD+jynAmcFzopBoTyoa0j21LJY1Q+9n01DE5tkqPhLGyJQ2Zqr8nMhppPYoC2xlRM9CL3kT8z2unJrzwMy6T1KBks0VhKoiJyeRv0uMKmREjSyhT3N5K2IAqyoxNp2BD8BZfXiaNs4rnVrxrm0YVZsjDIRzBCXhwDlW4ghrUgUEfHuAJnh3hPDovzuusNefMZ0rwB87bD9hKkeg=</latexit><latexit sha1_base64="Tz8l/+hZSguCsbakHFPzZQnbxQo=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5stAzYWEY0H5AcYW8zlyzZ3Tt294Rw5CfYWChi6y+y89+4Sa7QxAcDj/dmmJkXpYIb6/vfXmljc2t7p7xb2ds/ODyqHp+0TZJphi2WiER3I2pQcIUty63AbqqRykhgJ5rczv3OE2rDE/VopymGko4Ujzmj1kkP3QEfVGt+3V+ArJOgIDUo0BxUv/rDhGUSlWWCGtML/NSGOdWWM4GzSj8zmFI2oSPsOaqoRBPmi1Nn5MIpQxIn2pWyZKH+nsipNGYqI9cpqR2bVW8u/uf1MhvfhDlXaWZRseWiOBPEJmT+NxlyjcyKqSOUae5uJWxMNWXWpVNxIQSrL6+T9lU98OvBvV9rNIo4ynAG53AJAVxDA+6gCS1gMIJneIU3T3gv3rv3sWwtecXMKfyB9/kDLfaNsg==</latexit>

X2<latexit sha1_base64="Iqq8+KHSQiw+o/4LtIS0j4ox5T4=">AAAB6nicbVA9SwNBEJ2LXzF+RVPaLAbBKtylMWXAxjKi+YDkCHubvWTJ3t6xOyeEI+AfsLFQxNZfZCH4V6zcfBSa+GDg8d4MM/OCRAqDrvvl5DY2t7Z38ruFvf2Dw6Pi8UnLxKlmvMliGetOQA2XQvEmCpS8k2hOo0DydjC+mvnte66NiNUdThLuR3SoRCgYRSvddvrVfrHsVtw5yDrxlqRcL31+PwBAo1/86A1ilkZcIZPUmK7nJuhnVKNgkk8LvdTwhLIxHfKupYpG3PjZ/NQpObfKgISxtqWQzNXfExmNjJlEge2MKI7MqjcT//O6KYY1PxMqSZErtlgUppJgTGZ/k4HQnKGcWEKZFvZWwkZUU4Y2nYINwVt9eZ20qhXPrXg3No06LJCHUziDC/DgEupwDQ1oAoMhPMIzvDjSeXJenbdFa85ZzpTgD5z3H2mukB4=</latexit><latexit sha1_base64="VvO7gwlxIOqNfHLvCplE2ri4gJQ=">AAAB6nicbVA9SwNBEJ3zM8avaEqbxSBYSLhLo2XAxjKi+YDkCHubuWTJ3t6xuyeEI429jYUitv4iC8F/4G+wcvNRaOKDgcd7M8zMCxLBtXHdT2dldW19YzO3ld/e2d3bLxwcNnScKoZ1FotYtQKqUXCJdcONwFaikEaBwGYwvJz4zTtUmsfy1owS9CPalzzkjBor3bS6lW6h5JbdKcgy8eakVC1+fN9/jc9q3cJ7pxezNEJpmKBatz03MX5GleFM4DjfSTUmlA1pH9uWShqh9rPpqWNyYpUeCWNlSxoyVX9PZDTSehQFtjOiZqAXvYn4n9dOTXjhZ1wmqUHJZovCVBATk8nfpMcVMiNGllCmuL2VsAFVlBmbTt6G4C2+vEwalbLnlr1rm0YVZsjBERzDKXhwDlW4ghrUgUEfHuAJnh3hPDovzuusdcWZzxThD5y3H4TukbE=</latexit><latexit sha1_base64="VvO7gwlxIOqNfHLvCplE2ri4gJQ=">AAAB6nicbVA9SwNBEJ3zM8avaEqbxSBYSLhLo2XAxjKi+YDkCHubuWTJ3t6xuyeEI429jYUitv4iC8F/4G+wcvNRaOKDgcd7M8zMCxLBtXHdT2dldW19YzO3ld/e2d3bLxwcNnScKoZ1FotYtQKqUXCJdcONwFaikEaBwGYwvJz4zTtUmsfy1owS9CPalzzkjBor3bS6lW6h5JbdKcgy8eakVC1+fN9/jc9q3cJ7pxezNEJpmKBatz03MX5GleFM4DjfSTUmlA1pH9uWShqh9rPpqWNyYpUeCWNlSxoyVX9PZDTSehQFtjOiZqAXvYn4n9dOTXjhZ1wmqUHJZovCVBATk8nfpMcVMiNGllCmuL2VsAFVlBmbTt6G4C2+vEwalbLnlr1rm0YVZsjBERzDKXhwDlW4ghrUgUEfHuAJnh3hPDovzuusdcWZzxThD5y3H4TukbE=</latexit><latexit sha1_base64="z5eVL9edz3UjLHA2P5+kwtkzBWQ=">AAAB6nicbVA9SwNBEJ3zM8avqKXNYhCswl0aLQM2lhHNByRH2NvMJUt2947dPSEc+Qk2ForY+ovs/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9or7x8cHh1XTk7bJsk0wxZLRKK7ETUouMKW5VZgN9VIZSSwE01u537nCbXhiXq00xRDSUeKx5xR66SH7qA+qFT9mr8AWSdBQapQoDmofPWHCcskKssENaYX+KkNc6otZwJn5X5mMKVsQkfYc1RRiSbMF6fOyKVThiROtCtlyUL9PZFTacxURq5TUjs2q95c/M/rZTa+CXOu0syiYstFcSaITcj8bzLkGpkVU0co09zdStiYasqsS6fsQghWX14n7Xot8GvBvV9tNIo4SnAOF3AFAVxDA+6gCS1gMIJneIU3T3gv3rv3sWzd8IqZM/gD7/MH2ouNew==</latexit>

X1<latexit sha1_base64="DIURhOzWaNQVrQFO8RofRwV52qU=">AAAB6nicbVA9SwNBEJ2LXzF+RVPaLAbBKtzZaHlgYxnRfEByhL3NXLJkb+/Y3RPCEfAP2FgoYusvshD8K1ZuPgpNfDDweG+GmXlhKrg2rvvlFNbWNza3itulnd29/YPy4VFTJ5li2GCJSFQ7pBoFl9gw3AhspwppHApshaOrqd+6R6V5Iu/MOMUgpgPJI86osdJtu+f1ylW35s5AVom3IFW/8vn9AAD1Xvmj209YFqM0TFCtO56bmiCnynAmcFLqZhpTykZ0gB1LJY1RB/ns1Ak5tUqfRImyJQ2Zqb8nchprPY5D2xlTM9TL3lT8z+tkJroMci7TzKBk80VRJohJyPRv0ucKmRFjSyhT3N5K2JAqyoxNp2RD8JZfXiXN85rn1rwbm4YPcxThGE7gDDy4AB+uoQ4NYDCAR3iGF0c4T86r8zZvLTiLmQr8gfP+A2gqkB0=</latexit><latexit sha1_base64="8exluLnHNA6v7wA8rFSEP0L/m+g=">AAAB6nicbVA9SwNBEJ2LXzF+RVPaLAbBQsKdjZYBG8uI5gOSI+xt9pIle3vH7pwQjjT2NhaK2PqLLAT/gb/Bys1HoYkPBh7vzTAzL0ikMOi6n05uZXVtfSO/Wdja3tndK+4fNEycasbrLJaxbgXUcCkUr6NAyVuJ5jQKJG8Gw8uJ37zj2ohY3eIo4X5E+0qEglG00k2r63WLZbfiTkGWiTcn5Wrp4/v+a3xa6xbfO72YpRFXyCQ1pu25CfoZ1SiY5ONCJzU8oWxI+7xtqaIRN342PXVMjq3SI2GsbSkkU/X3REYjY0ZRYDsjigOz6E3E/7x2iuGFnwmVpMgVmy0KU0kwJpO/SU9ozlCOLKFMC3srYQOqKUObTsGG4C2+vEwaZxXPrXjXNo0qzJCHQziCE/DgHKpwBTWoA4M+PMATPDvSeXRenNdZa86Zz5TgD5y3H4NqkbA=</latexit><latexit sha1_base64="8exluLnHNA6v7wA8rFSEP0L/m+g=">AAAB6nicbVA9SwNBEJ2LXzF+RVPaLAbBQsKdjZYBG8uI5gOSI+xt9pIle3vH7pwQjjT2NhaK2PqLLAT/gb/Bys1HoYkPBh7vzTAzL0ikMOi6n05uZXVtfSO/Wdja3tndK+4fNEycasbrLJaxbgXUcCkUr6NAyVuJ5jQKJG8Gw8uJ37zj2ohY3eIo4X5E+0qEglG00k2r63WLZbfiTkGWiTcn5Wrp4/v+a3xa6xbfO72YpRFXyCQ1pu25CfoZ1SiY5ONCJzU8oWxI+7xtqaIRN342PXVMjq3SI2GsbSkkU/X3REYjY0ZRYDsjigOz6E3E/7x2iuGFnwmVpMgVmy0KU0kwJpO/SU9ozlCOLKFMC3srYQOqKUObTsGG4C2+vEwaZxXPrXjXNo0qzJCHQziCE/DgHKpwBTWoA4M+PMATPDvSeXRenNdZa86Zz5TgD5y3H4NqkbA=</latexit><latexit sha1_base64="nd0OLIsnz53C89lOtBR1t8R9H68=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5stAzYWEY0H5AcYW8zlyzZ3Tt294Rw5CfYWChi6y+y89+4Sa7QxAcDj/dmmJkXpYIb6/vfXmljc2t7p7xb2ds/ODyqHp+0TZJphi2WiER3I2pQcIUty63AbqqRykhgJ5rczv3OE2rDE/VopymGko4Ujzmj1kkP3UEwqNb8ur8AWSdBQWpQoDmofvWHCcskKssENaYX+KkNc6otZwJnlX5mMKVsQkfYc1RRiSbMF6fOyIVThiROtCtlyUL9PZFTacxURq5TUjs2q95c/M/rZTa+CXOu0syiYstFcSaITcj8bzLkGpkVU0co09zdStiYasqsS6fiQghWX14n7at64NeDe7/WaBRxlOEMzuESAriGBtxBE1rAYATP8ApvnvBevHfvY9la8oqZU/gD7/MH2QeNeg==</latexit>

h01

<latexit sha1_base64="xDkIE1qJt5ZATMHRcAupM1AivaI=">AAAB8HicbVC7SgNBFL0bXzG+opY2Q4KYKuzaxDJoYxnBPCRZw+xkNhkyM7vMzAph2a+wsVDE1s+xEyz8FCePQhMPXDiccy/33hPEnGnjup9Obm19Y3Mrv13Y2d3bPygeHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsZXU7/9QJVmkbw1k5j6Ag8lCxnBxkp3o37qZffpWdYvlt2qOwNaJd6ClOulytc3ADT6xY/eICKJoNIQjrXuem5s/BQrwwinWaGXaBpjMsZD2rVUYkG1n84OztCpVQYojJQtadBM/T2RYqH1RAS2U2Az0sveVPzP6yYmvPBTJuPEUEnmi8KEIxOh6fdowBQlhk8swUQxeysiI6wwMTajgg3BW355lbTOq55b9W5sGpcwRx5OoAQV8KAGdbiGBjSBgIBHeIYXRzlPzqvzNm/NOYuZY/gD5/0HpqmSgw==</latexit><latexit sha1_base64="GA/hk6lomxZU/XdAX82jMXcQBSQ=">AAAB8HicbVDLSgNBEOyNrxhfUY9ehgQxp7DrJR6DXjxGMA9JYpidzCZDZmaXmVkhLPsVXgQV8eqfePUm+jFOHgdNLGgoqrrp7vIjzrRx3U8ns7K6tr6R3cxtbe/s7uX3Dxo6jBWhdRLyULV8rClnktYNM5y2IkWx8Dlt+qOLid+8o0qzUF6bcUS7Ag8kCxjBxko3w17ipbfJSdrLF92yOwVaJt6cFKuF0vdX5f2x1st/dPohiQWVhnCsddtzI9NNsDKMcJrmOrGmESYjPKBtSyUWVHeT6cEpOrZKHwWhsiUNmqq/JxIstB4L33YKbIZ60ZuI/3nt2ARn3YTJKDZUktmiIObIhGjyPeozRYnhY0swUczeisgQK0yMzShnQ/AWX14mjdOy55a9K5vGOcyQhSMoQAk8qEAVLqEGdSAg4B6e4NlRzoPz4rzOWjPOfOYQ/sB5+wHEvZQZ</latexit><latexit sha1_base64="GA/hk6lomxZU/XdAX82jMXcQBSQ=">AAAB8HicbVDLSgNBEOyNrxhfUY9ehgQxp7DrJR6DXjxGMA9JYpidzCZDZmaXmVkhLPsVXgQV8eqfePUm+jFOHgdNLGgoqrrp7vIjzrRx3U8ns7K6tr6R3cxtbe/s7uX3Dxo6jBWhdRLyULV8rClnktYNM5y2IkWx8Dlt+qOLid+8o0qzUF6bcUS7Ag8kCxjBxko3w17ipbfJSdrLF92yOwVaJt6cFKuF0vdX5f2x1st/dPohiQWVhnCsddtzI9NNsDKMcJrmOrGmESYjPKBtSyUWVHeT6cEpOrZKHwWhsiUNmqq/JxIstB4L33YKbIZ60ZuI/3nt2ARn3YTJKDZUktmiIObIhGjyPeozRYnhY0swUczeisgQK0yMzShnQ/AWX14mjdOy55a9K5vGOcyQhSMoQAk8qEAVLqEGdSAg4B6e4NlRzoPz4rzOWjPOfOYQ/sB5+wHEvZQZ</latexit><latexit sha1_base64="XlUTD+gMwdOF+nwmX06e/Ungt1g=">AAAB8HicbVC7TgMxENzjGcIrQEljESGoIh8NlBE0lEEiD5Qckc/xJVZs38n2IUWn+woaChCi5XPo+Buc5ApIGGml0cyudnfCRHBjMf72VlbX1jc2S1vl7Z3dvf3KwWHLxKmmrEljEetOSAwTXLGm5VawTqIZkaFg7XB8M/XbT0wbHqt7O0lYIMlQ8YhTYp30MOpnfv6YneX9ShXX8AxomfgFqUKBRr/y1RvENJVMWSqIMV0fJzbIiLacCpaXe6lhCaFjMmRdRxWRzATZ7OAcnTplgKJYu1IWzdTfExmRxkxk6DolsSOz6E3F/7xuaqOrIOMqSS1TdL4oSgWyMZp+jwZcM2rFxBFCNXe3IjoimlDrMiq7EPzFl5dJ66Lm45p/h6v16yKOEhzDCZyDD5dQh1toQBMoSHiGV3jztPfivXsf89YVr5g5gj/wPn8AldqQPQ==</latexit>

h02

<latexit sha1_base64="mr4dW27+TPHy+Qk6qDp6c301K9o=">AAAB8HicbVC7SgNBFL0bXzG+opY2Q4KYKuym0TJoYxnBPCRZw+xkNhkyM7vMzAph2a+wsVDE1s+xEyz8FCePQhMPXDiccy/33hPEnGnjup9Obm19Y3Mrv13Y2d3bPygeHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsZXU7/9QJVmkbw1k5j6Ag8lCxnBxkp3o35ay+7Ts6xfLLtVdwa0SrwFKddLla9vAGj0ix+9QUQSQaUhHGvd9dzY+ClWhhFOs0Iv0TTGZIyHtGupxIJqP50dnKFTqwxQGClb0qCZ+nsixULriQhsp8BmpJe9qfif101MeOGnTMaJoZLMF4UJRyZC0+/RgClKDJ9Ygoli9lZERlhhYmxGBRuCt/zyKmnVqp5b9W5sGpcwRx5OoAQV8OAc6nANDWgCAQGP8AwvjnKenFfnbd6acxYzx/AHzvsPqDKShA==</latexit><latexit sha1_base64="0kZ7MTYqPk1SYafmhIhjvjdAZ4A=">AAAB8HicbVDLSgNBEOz1GeMr6tHLkCDmFHZzicegF48RzEOSGGYns8mQmdllZlYIy36FF0FFvPonXr2JfoyTx0ETCxqKqm66u/yIM21c99NZWV1b39jMbGW3d3b39nMHhw0dxorQOgl5qFo+1pQzSeuGGU5bkaJY+Jw2/dHFxG/eUaVZKK/NOKJdgQeSBYxgY6WbYS8pp7fJadrLFdySOwVaJt6cFKr54vdX5f2x1st9dPohiQWVhnCsddtzI9NNsDKMcJpmO7GmESYjPKBtSyUWVHeT6cEpOrFKHwWhsiUNmqq/JxIstB4L33YKbIZ60ZuI/3nt2ARn3YTJKDZUktmiIObIhGjyPeozRYnhY0swUczeisgQK0yMzShrQ/AWX14mjXLJc0velU3jHGbIwDHkoQgeVKAKl1CDOhAQcA9P8Owo58F5cV5nrSvOfOYI/sB5+wHGRpQa</latexit><latexit sha1_base64="0kZ7MTYqPk1SYafmhIhjvjdAZ4A=">AAAB8HicbVDLSgNBEOz1GeMr6tHLkCDmFHZzicegF48RzEOSGGYns8mQmdllZlYIy36FF0FFvPonXr2JfoyTx0ETCxqKqm66u/yIM21c99NZWV1b39jMbGW3d3b39nMHhw0dxorQOgl5qFo+1pQzSeuGGU5bkaJY+Jw2/dHFxG/eUaVZKK/NOKJdgQeSBYxgY6WbYS8pp7fJadrLFdySOwVaJt6cFKr54vdX5f2x1st9dPohiQWVhnCsddtzI9NNsDKMcJpmO7GmESYjPKBtSyUWVHeT6cEpOrFKHwWhsiUNmqq/JxIstB4L33YKbIZ60ZuI/3nt2ARn3YTJKDZUktmiIObIhGjyPeozRYnhY0swUczeisgQK0yMzShrQ/AWX14mjXLJc0velU3jHGbIwDHkoQgeVKAKl1CDOhAQcA9P8Owo58F5cV5nrSvOfOYI/sB5+wHGRpQa</latexit><latexit sha1_base64="eLkH3cnVQVEWvHyuMbWLEdRxzeQ=">AAAB8HicbVA9T8MwEL2Ur1K+CowsFhWCqUq6wFjBwlgk+oHaUDmu01q1nch2kKoov4KFAYRY+Tls/BucNgO0POmkp/fudHcviDnTxnW/ndLa+sbmVnm7srO7t39QPTzq6ChRhLZJxCPVC7CmnEnaNsxw2osVxSLgtBtMb3K/+0SVZpG8N7OY+gKPJQsZwcZKD5Nh2sge0/NsWK25dXcOtEq8gtSgQGtY/RqMIpIIKg3hWOu+58bGT7EyjHCaVQaJpjEmUzymfUslFlT76fzgDJ1ZZYTCSNmSBs3V3xMpFlrPRGA7BTYTvezl4n9ePzHhlZ8yGSeGSrJYFCYcmQjl36MRU5QYPrMEE8XsrYhMsMLE2IwqNgRv+eVV0mnUPbfu3bm15nURRxlO4BQuwINLaMIttKANBAQ8wyu8Ocp5cd6dj0VrySlmjuEPnM8fl2OQPg==</latexit>

h0i

<latexit sha1_base64="JDy/E+OVM1pM8t5be7B9WLlD9cQ=">AAAB8HicbVC7SgNBFL0bXzG+opY2Q4KYKuzaxDJoYxnBPCRZw+xkNhkyM7vMzAph2a+wsVDE1s+xEyz8FCePQhMPXDiccy/33hPEnGnjup9Obm19Y3Mrv13Y2d3bPygeHrV0lChCmyTikeoEWFPOJG0aZjjtxIpiEXDaDsZXU7/9QJVmkbw1k5j6Ag8lCxnBxkp3o37Ksvv0LOsXy27VnQGtEm9ByvVS5esbABr94kdvEJFEUGkIx1p3PTc2foqVYYTTrNBLNI0xGeMh7VoqsaDaT2cHZ+jUKgMURsqWNGim/p5IsdB6IgLbKbAZ6WVvKv7ndRMTXvgpk3FiqCTzRWHCkYnQ9Hs0YIoSwyeWYKKYvRWREVaYGJtRwYbgLb+8SlrnVc+tejc2jUuYIw8nUIIKeFCDOlxDA5pAQMAjPMOLo5wn59V5m7fmnMXMMfyB8/4D/KGSuw==</latexit><latexit sha1_base64="3384ueNazDSryVX5TIJH+zru/cE=">AAAB8HicbVDLSgNBEOyNrxhfUY9ehgQxp7DrJR6DXjxGMA9JYpidzCZDZmaXmVkhLPsVXgQV8eqfePUm+jFOHgdNLGgoqrrp7vIjzrRx3U8ns7K6tr6R3cxtbe/s7uX3Dxo6jBWhdRLyULV8rClnktYNM5y2IkWx8Dlt+qOLid+8o0qzUF6bcUS7Ag8kCxjBxko3w17C0tvkJO3li27ZnQItE29OitVC6fur8v5Y6+U/Ov2QxIJKQzjWuu25kekmWBlGOE1znVjTCJMRHtC2pRILqrvJ9OAUHVulj4JQ2ZIGTdXfEwkWWo+FbzsFNkO96E3E/7x2bIKzbsJkFBsqyWxREHNkQjT5HvWZosTwsSWYKGZvRWSIFSbGZpSzIXiLLy+TxmnZc8velU3jHGbIwhEUoAQeVKAKl1CDOhAQcA9P8Owo58F5cV5nrRlnPnMIf+C8/QAaxJRR</latexit><latexit sha1_base64="3384ueNazDSryVX5TIJH+zru/cE=">AAAB8HicbVDLSgNBEOyNrxhfUY9ehgQxp7DrJR6DXjxGMA9JYpidzCZDZmaXmVkhLPsVXgQV8eqfePUm+jFOHgdNLGgoqrrp7vIjzrRx3U8ns7K6tr6R3cxtbe/s7uX3Dxo6jBWhdRLyULV8rClnktYNM5y2IkWx8Dlt+qOLid+8o0qzUF6bcUS7Ag8kCxjBxko3w17C0tvkJO3li27ZnQItE29OitVC6fur8v5Y6+U/Ov2QxIJKQzjWuu25kekmWBlGOE1znVjTCJMRHtC2pRILqrvJ9OAUHVulj4JQ2ZIGTdXfEwkWWo+FbzsFNkO96E3E/7x2bIKzbsJkFBsqyWxREHNkQjT5HvWZosTwsSWYKGZvRWSIFSbGZpSzIXiLLy+TxmnZc8velU3jHGbIwhEUoAQeVKAKl1CDOhAQcA9P8Owo58F5cV5nrRlnPnMIf+C8/QAaxJRR</latexit><latexit sha1_base64="N437KXQO4q/KveKGsz+Gva7yNcI=">AAAB8HicbVC7TgMxENzjGcIrQEljESGoIh8NlBE0lEEiD5Qckc/xJVZs38n2IUWn+woaChCi5XPo+Buc5ApIGGml0cyudnfCRHBjMf72VlbX1jc2S1vl7Z3dvf3KwWHLxKmmrEljEetOSAwTXLGm5VawTqIZkaFg7XB8M/XbT0wbHqt7O0lYIMlQ8YhTYp30MOpnPH/MzvJ+pYpreAa0TPyCVKFAo1/56g1imkqmLBXEmK6PExtkRFtOBcvLvdSwhNAxGbKuo4pIZoJsdnCOTp0yQFGsXSmLZurviYxIYyYydJ2S2JFZ9Kbif143tdFVkHGVpJYpOl8UpQLZGE2/RwOuGbVi4gihmrtbER0RTah1GZVdCP7iy8ukdVHzcc2/w9X6dRFHCY7hBM7Bh0uowy00oAkUJDzDK7x52nvx3r2PeeuKV8wcwR94nz/r0pB1</latexit>

…h1<latexit sha1_base64="dX40VFm5C91g3I8hxIvsRzyk5Do=">AAAB7HicbVA9SwNBEJ2LXzF+RQUbm8UgWIU7Gy2DNpYJeEkgOeLeZi9Zsrd37M4J4chvsLFQxNYfZGfjb3HzUWjig4HHezPMzAtTKQy67pdTWFvf2Nwqbpd2dvf2D8qHR02TZJpxnyUy0e2QGi6F4j4KlLydak7jUPJWOLqd+q1Hro1I1D2OUx7EdKBEJBhFK/nDXu5NeuWKW3VnIKvEW5BK7aTx/QAA9V75s9tPWBZzhUxSYzqem2KQU42CST4pdTPDU8pGdMA7lioacxPks2Mn5NwqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0y95U/M/rZBhdB7lQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+ZRsCN7yy6ukeVn13KrXsGncwBxFOIUzuAAPrqAGd1AHHxgIeIIXeHWU8+y8Oe/z1oKzmDmGP3A+fgBFLZB9</latexit><latexit sha1_base64="pMX5acFXv3cfPvV3Mcx/4mHFb78=">AAAB7HicbVC7SgNBFL0bXzG+ooKNzWAQrMKujZYhNpYJmAckS5id3E2GzM4uM7NCWPINNhaK2Nr5F36BnY3f4uRRaOKBC4dz7uXee4JEcG1c98vJra1vbG7ltws7u3v7B8XDo6aOU8WwwWIRq3ZANQousWG4EdhOFNIoENgKRjdTv3WPSvNY3plxgn5EB5KHnFFjpcawl3mTXrHklt0ZyCrxFqRUOal/8/fqR61X/Oz2Y5ZGKA0TVOuO5ybGz6gynAmcFLqpxoSyER1gx1JJI9R+Njt2Qs6t0idhrGxJQ2bq74mMRlqPo8B2RtQM9bI3Ff/zOqkJr/2MyyQ1KNl8UZgKYmIy/Zz0uUJmxNgSyhS3txI2pIoyY/Mp2BC85ZdXSfOy7Lllr27TqMIceTiFM7gAD66gArdQgwYw4PAAT/DsSOfReXFe5605ZzFzDH/gvP0Alm6SOQ==</latexit><latexit sha1_base64="pMX5acFXv3cfPvV3Mcx/4mHFb78=">AAAB7HicbVC7SgNBFL0bXzG+ooKNzWAQrMKujZYhNpYJmAckS5id3E2GzM4uM7NCWPINNhaK2Nr5F36BnY3f4uRRaOKBC4dz7uXee4JEcG1c98vJra1vbG7ltws7u3v7B8XDo6aOU8WwwWIRq3ZANQousWG4EdhOFNIoENgKRjdTv3WPSvNY3plxgn5EB5KHnFFjpcawl3mTXrHklt0ZyCrxFqRUOal/8/fqR61X/Oz2Y5ZGKA0TVOuO5ybGz6gynAmcFLqpxoSyER1gx1JJI9R+Njt2Qs6t0idhrGxJQ2bq74mMRlqPo8B2RtQM9bI3Ff/zOqkJr/2MyyQ1KNl8UZgKYmIy/Zz0uUJmxNgSyhS3txI2pIoyY/Mp2BC85ZdXSfOy7Lllr27TqMIceTiFM7gAD66gArdQgwYw4PAAT/DsSOfReXFe5605ZzFzDH/gvP0Alm6SOQ==</latexit><latexit sha1_base64="ExG92yfpXlC7gM/aji2aW3gFvzg=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0m86LHoxWMF0xbaUDbbSbt0swm7GyGE/gYvHhTx6g/y5r9x2+agrQ8GHu/NMDMvTAXXxnW/ncrG5tb2TnW3trd/cHhUPz7p6CRTDH2WiET1QqpRcIm+4UZgL1VI41BgN5zezf3uEyrNE/lo8hSDmI4ljzijxkr+ZFh4s2G94TbdBcg68UrSgBLtYf1rMEpYFqM0TFCt+56bmqCgynAmcFYbZBpTyqZ0jH1LJY1RB8Xi2Bm5sMqIRImyJQ1ZqL8nChprnceh7YypmehVby7+5/UzE90EBZdpZlCy5aIoE8QkZP45GXGFzIjcEsoUt7cSNqGKMmPzqdkQvNWX10nnqum5Te/BbbRuyziqcAbncAkeXEML7qENPjDg8Ayv8OZI58V5dz6WrRWnnDmFP3A+fwC2ho6Y</latexit>

h2<latexit sha1_base64="X78G3COXAanjqP+qk+U/E1fdiQs=">AAAB7HicbVA9SwNBEJ2LXzF+RQUbm8UgWIW7NFoGbSwT8BIhOeLeZi9Zsrd37M4J4chvsLFQxNYfZGfjb3HzUWjig4HHezPMzAtTKQy67pdTWFvf2Nwqbpd2dvf2D8qHRy2TZJpxnyUy0fchNVwKxX0UKPl9qjmNQ8nb4ehm6rcfuTYiUXc4TnkQ04ESkWAUreQPe3lt0itX3Ko7A1kl3oJU6ifN7wcAaPTKn91+wrKYK2SSGtPx3BSDnGoUTPJJqZsZnlI2ogPesVTRmJsgnx07IedW6ZMo0bYUkpn6eyKnsTHjOLSdMcWhWfam4n9eJ8PoKsiFSjPkis0XRZkkmJDp56QvNGcox5ZQpoW9lbAh1ZShzadkQ/CWX14lrVrVc6te06ZxDXMU4RTO4AI8uIQ63EIDfGAg4Ale4NVRzrPz5rzPWwvOYuYY/sD5+AFGspB+</latexit><latexit sha1_base64="vNHuY8lIWrKdqIHkkzh96jxFZds=">AAAB7HicbVA9SwNBEJ3zM8avqGBjsxgEq3CXRssQG8sEvCSQHGFvM0mW7O0du3tCOPIbbCwUsbXzX/gL7Gz8LW4+Ck18MPB4b4aZeWEiuDau++WsrW9sbm3ndvK7e/sHh4Wj44aOU8XQZ7GIVSukGgWX6BtuBLYShTQKBTbD0c3Ub96j0jyWd2acYBDRgeR9zqixkj/sZuVJt1B0S+4MZJV4C1KsnNa/+Xv1o9YtfHZ6MUsjlIYJqnXbcxMTZFQZzgRO8p1UY0LZiA6wbamkEeogmx07IRdW6ZF+rGxJQ2bq74mMRlqPo9B2RtQM9bI3Ff/z2qnpXwcZl0lqULL5on4qiInJ9HPS4wqZEWNLKFPc3krYkCrKjM0nb0Pwll9eJY1yyXNLXt2mUYU5cnAG53AJHlxBBW6hBj4w4PAAT/DsSOfReXFe561rzmLmBP7AefsBl/OSOg==</latexit><latexit sha1_base64="vNHuY8lIWrKdqIHkkzh96jxFZds=">AAAB7HicbVA9SwNBEJ3zM8avqGBjsxgEq3CXRssQG8sEvCSQHGFvM0mW7O0du3tCOPIbbCwUsbXzX/gL7Gz8LW4+Ck18MPB4b4aZeWEiuDau++WsrW9sbm3ndvK7e/sHh4Wj44aOU8XQZ7GIVSukGgWX6BtuBLYShTQKBTbD0c3Ub96j0jyWd2acYBDRgeR9zqixkj/sZuVJt1B0S+4MZJV4C1KsnNa/+Xv1o9YtfHZ6MUsjlIYJqnXbcxMTZFQZzgRO8p1UY0LZiA6wbamkEeogmx07IRdW6ZF+rGxJQ2bq74mMRlqPo9B2RtQM9bI3Ff/z2qnpXwcZl0lqULL5on4qiInJ9HPS4wqZEWNLKFPc3krYkCrKjM0nb0Pwll9eJY1yyXNLXt2mUYU5cnAG53AJHlxBBW6hBj4w4PAAT/DsSOfReXFe561rzmLmBP7AefsBl/OSOg==</latexit><latexit sha1_base64="eCyPV5LKHeCmjpUqZgG5Dfn4nFw=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRmLiibRc5Ej04hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5k2HemA+rNbfuLkE2iVeQGhRoD6tfg1HCshilYYJq3ffc1AQ5VYYzgfPKINOYUjalY+xbKmmMOsiXx87JlVVGJEqULWnIUv09kdNY61kc2s6Ymole9xbif14/M1EzyLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0GnXPrXsPbq11W8RRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QO4C46Z</latexit>

hi<latexit sha1_base64="yk/4uZXoj3/Kg0fWvcJBVFAcGLo=">AAAB6nicbVA9SwNBEJ2LXzF+RQUbm8UgWIU7m1gGbSwTNB+QHHFvs5cs2ds7dueEcOQn2FgoYusvsrPxt7j5KDTxwcDjvRlm5gWJFAZd98vJra1vbG7ltws7u3v7B8XDo6aJU814g8Uy1u2AGi6F4g0UKHk70ZxGgeStYHQz9VuPXBsRq3scJ9yP6ECJUDCKVrob9kSvWHLL7gxklXgLUqqe1L8fAKDWK352+zFLI66QSWpMx3MT9DOqUTDJJ4VuanhC2YgOeMdSRSNu/Gx26oScW6VPwljbUkhm6u+JjEbGjKPAdkYUh2bZm4r/eZ0Uwys/EypJkSs2XxSmkmBMpn+TvtCcoRxbQpkW9lbChlRThjadgg3BW355lTQvy55b9uo2jWuYIw+ncAYX4EEFqnALNWgAgwE8wQu8OtJ5dt6c93lrzlnMHMMfOB8/1YiPqQ==</latexit><latexit sha1_base64="+ejrLe9aEkqDt8OCbBZZ/+kEyys=">AAAB6nicbVA9SwNBEJ2LXzF+RQUbm8UgWIU7Gy1DbCwTNImQHGFvM5cs2ds7dveEcOQn2FgoYmvrv/AX2Nn4W9x8FJr4YODx3gwz84JEcG1c98vJrayurW/kNwtb2zu7e8X9g6aOU8WwwWIRq7uAahRcYsNwI/AuUUijQGArGF5N/NY9Ks1jeWtGCfoR7UseckaNlW4GXd4tltyyOwVZJt6clCpH9W/+Xv2odYufnV7M0gilYYJq3fbcxPgZVYYzgeNCJ9WYUDakfWxbKmmE2s+mp47JqVV6JIyVLWnIVP09kdFI61EU2M6ImoFe9Cbif147NeGln3GZpAYlmy0KU0FMTCZ/kx5XyIwYWUKZ4vZWwgZUUWZsOgUbgrf48jJpnpc9t+zVbRpVmCEPx3ACZ+DBBVTgGmrQAAZ9eIAneHaE8+i8OK+z1pwznzmEP3DefgAm2JFl</latexit><latexit sha1_base64="+ejrLe9aEkqDt8OCbBZZ/+kEyys=">AAAB6nicbVA9SwNBEJ2LXzF+RQUbm8UgWIU7Gy1DbCwTNImQHGFvM5cs2ds7dveEcOQn2FgoYmvrv/AX2Nn4W9x8FJr4YODx3gwz84JEcG1c98vJrayurW/kNwtb2zu7e8X9g6aOU8WwwWIRq7uAahRcYsNwI/AuUUijQGArGF5N/NY9Ks1jeWtGCfoR7UseckaNlW4GXd4tltyyOwVZJt6clCpH9W/+Xv2odYufnV7M0gilYYJq3fbcxPgZVYYzgeNCJ9WYUDakfWxbKmmE2s+mp47JqVV6JIyVLWnIVP09kdFI61EU2M6ImoFe9Cbif147NeGln3GZpAYlmy0KU0FMTCZ/kx5XyIwYWUKZ4vZWwgZUUWZsOgUbgrf48jJpnpc9t+zVbRpVmCEPx3ACZ+DBBVTgGmrQAAZ9eIAneHaE8+i8OK+z1pwznzmEP3DefgAm2JFl</latexit><latexit sha1_base64="apBVtzYObQUoeW39avd4HC7sXVo=">AAAB6nicbVA9TwJBEJ3DL8Qv1NJmIzGxInc2WhJtLDEKksCF7C17sGFv97I7Z0Iu/AQbC42x9RfZ+W9c4AoFXzLJy3szmZkXpVJY9P1vr7S2vrG5Vd6u7Ozu7R9UD4/aVmeG8RbTUptORC2XQvEWCpS8kxpOk0jyx2h8M/Mfn7ixQqsHnKQ8TOhQiVgwik66H/VFv1rz6/4cZJUEBalBgWa/+tUbaJYlXCGT1Npu4KcY5tSgYJJPK73M8pSyMR3yrqOKJtyG+fzUKTlzyoDE2rhSSObq74mcJtZOksh1JhRHdtmbif953QzjqzAXKs2QK7ZYFGeSoCazv8lAGM5QThyhzAh3K2EjaihDl07FhRAsv7xK2hf1wK8Hd36tcV3EUYYTOIVzCOASGnALTWgBgyE8wyu8edJ78d69j0VryStmjuEPvM8fRvCNxA==</latexit>

Figure 6: The prediction method of crowdfunding project.Vi is the review representation vector.

experimental results, we can also observe that the classi-fication accuracy between similar domains will be higherthan different domain. For example, “B↔D” task is moreaccurate than “B↔K” task because they have more similaraspects as shown in Figure 4. Finally, compared with thebenchmark methods, our model shows great improvementfor cross-domain sentiment classification.

Visualization of AttentionIn order to validate that our model is able to identify theimpact of aspect on sentiment representation, we visualizethe aspect attention layer in Figure 5. Here we choose fourreviews from the Book domain and DVD domain. Each do-main has a positive review and a negative one. The green-colored words in the sentences refer to the aspects. An as-pect colored in deeper green means that it gains the heavierweight through the aspect attention layer than the others.

Figure 5 shows that IATN pays higher word attention tothe domain-shared aspects, such as “story”, “scene”, “plot”and “character”. Specifically, in the Dvd-neg example, IATNhas assigned heavier weights to “character” and “plot” than“acting” and “pace”. Here we color the positive words inred and the negative words in blue by artificial judgment tohighlight the relationship of the aspect to the emotion of thewhole sentence. Although “acting” and “pace” both appearwith positive emotions in this review, but the more effectiveaspects “character” and “plot” have received negative emo-tions. Affected by these more powerful aspects, this reviewis finally labeled as negative. In conclusion, the visualizationof attention proves that IATN well models sentences and as-pects together, and the concatenated representation of themis helpful for the cross-domain sentiment classification.

Application Verification in CrowdfundingAs we mentioned above, the further mission of cross-domainsentiment classification aims to solve the problem of un-labeled domain. However, the training and testing data ofAmazon actually have labels so that the sentiment classifi-cation results can be evaluated. Thus, to further verify theeffectiveness of IATN, we design an application on an un-labeled dataset, i.e., the crowdfunding (Zhao et al. 2017b;

Table 3: Results of crowdfunding project prediction.

Benchmarks Metrics

Accu. Prec. Rec. F1-score.

SVM 0.7671 0.4567 0.6971 0.5536LSTMone 0.7862 0.4813 0.6732 0.5689LSTMhatn 0.7940 0.4843 0.6805 0.5674LSTMiatn 0.8182 0.4977 0.6733 0.5743

Liu et al. 2017) reviews dataset. Specifically, we use Ama-zon’s labeled reviews data as the source domain, and thecrowdfunding reviews data as the target domain to predictthe emotional tendency (Vi). To evaluate the sentiment clas-sification on unlabeled crowdfunding domain, the reviewsentiment is combined with multiple features to do the clas-sification of the final states (i.e., succeed or failed). In orderto achieve this goal, we combine review’s emotional featurerepresentation (e.g., V1, V2...Vi) with other represented fea-tures of projects as a new feature vector (e.g., X1, X2...Xi).Then we use this new feature vector as the input of LSTMto predict the final states of projects, the details are shownin Figure 6. For the benchmarks LSTMone, LSTMhatn andLSTMiatn, Vi is the one-hot vector, the output of HATN andthe output of IATN respectively.

In the crowdfunding dataset2, there are 12,328 projectswith almost 11,2560 reviews and 20 other kinds of fea-tures (e.g., goal, duration and information of owner). Notethat those project’s final states, which are unbalanced, con-sist of more failed but less successful projects. Therefore,we adopt various metrics to better evaluate the methods,i.e., accuracy, precision, recall and F1-score. As Table 3shows, the SVM method without review information per-forms the worst and the accuracy result is 5.11% lowerthan LSTMiatn. LSTMiatn improves the accuracy by 3.20%and 2.42% than LSTMone and LSTMhatn respectively. Fur-thermore, the comparisons between LSTMone, LSTMhatn

and LSTMiatn not only show the excellent performanceof LSTMiatn in the project success rate prediction, butalso prove that IATN has a good sentiment classification ofcrowdfunding reviews.

ConclusionsIn this paper, we studied the problem of cross-domain insentiment classification and proposed IATN model whichconsidered the information from both sentences and aspects.Specifically, we aimed to find common features across do-mains and then extracted information from the aspects withthe help of common features. Additionally, we adopted aninteractive attention learning mechanism for the sentimentclassification, which combined sentences and aspects to-gether. Experiments on Amazon review dataset and crowd-funding dataset clearly verified the effectiveness of IATN.Since we proposed to study the aspect information in thecross-domain sentiment classification for the first time, wehope this work could lead to more researches in the future.

2A specific website for crowdfunding platform, https://www.indiegogo.com/

AcknowledgementsThis research was partially supported by grants from the Na-tional Key Research and Development Program of China(No. 2016YFB1000904), and the National Natural ScienceFoundation of China (Grants No. 61672483, U1605251 and91546103). Qi Liu gratefully acknowledges the support ofthe Young Elite Scientist Sponsorship Program of CAST.

ReferencesBengio, Y.; Ducharme, R.; Vincent, P.; and Jauvin, C. 2003.A neural probabilistic language model. Journal of machinelearning research 3(Feb):1137–1155.Blitzer, J.; McDonald, R.; and Pereira, F. 2006. Domainadaptation with structural correspondence learning. In Pro-ceedings of the 2006 conference on empirical methods innatural language processing, 120–128. Association forComputational Linguistics.Chen, M.; Xu, Z.; Weinberger, K.; and Sha, F. 2012.Marginalized denoising autoencoders for domain adapta-tion. arXiv preprint arXiv:1206.4683.Ganin, Y., and Lempitsky, V. 2014. Unsuperviseddomain adaptation by backpropagation. arXiv preprintarXiv:1409.7495.Ganin, Y.; Ustinova, E.; Ajakan, H.; Germain, P.; Larochelle,H.; Laviolette, F.; Marchand, M.; and Lempitsky, V. 2016.Domain-adversarial training of neural networks. The Jour-nal of Machine Learning Research 17(1):2096–2030.Glorot, X.; Bordes, A.; and Bengio, Y. 2011. Domain adap-tation for large-scale sentiment classification: A deep learn-ing approach. In Proceedings of the 28th international con-ference on machine learning (ICML-11), 513–520.Goldberg, Y., and Levy, O. 2014. word2vec explained: de-riving mikolov et al.’s negative-sampling word-embeddingmethod. arXiv preprint arXiv:1402.3722.Hochreiter, S., and Schmidhuber, J. 1997. Long short-termmemory. Neural computation 9(8):1735–1780.Hu, M., and Liu, B. 2004. Mining and summarizing cus-tomer reviews. In Proceedings of the tenth ACM SIGKDDinternational conference on Knowledge discovery and datamining, 168–177. ACM.Huang, Z.; Liu, Q.; Chen, E.; Zhao, H.; Gao, M.; Wei, S.;Su, Y.; and Hu, G. 2017. Question difficulty prediction forreading problems in standard tests. In AAAI, 1352–1359.Huang, Z.; Xu, W.; and Yu, K. 2015. Bidirectionallstm-crf models for sequence tagging. arXiv preprintarXiv:1508.01991.Kim, Y. 2014. Convolutional neural networks for sentenceclassification. arXiv preprint arXiv:1408.5882.LeCun, Y.; Bengio, Y.; and Hinton, G. 2015. Deep learning.nature 521(7553):436.Li, Z.; Zhang, Y.; Wei, Y.; Wu, Y.; and Yang, Q. 2017. End-to-end adversarial memory network for cross-domain senti-ment classification. In Proceedings of the International JointConference on Artificial Intelligence (IJCAI 2017).

Li, X.; Bing, L.; Li, P.; Lam, W.; and Yang, Z. 2018a. Aspectterm extraction with history attention and selective transfor-mation. arXiv preprint arXiv:1805.00760.Li, Z.; Wei, Y.; Zhang, Y.; and Yang, Q. 2018b. Hierarchicalattention transfer network for cross-domain sentiment clas-sification. In Proceedings of the Thirty-Second AAAI Con-ference on Artificial Intelligence, AAAI 2018, New Orleans,Lousiana, USA, February 2–7, 2018.Liu, Q.; Wang, G.; Zhao, H.; Liu, C.; Xu, T.; and Chen,E. 2017. Enhancing campaign design in crowdfunding: aproduct supply optimization perspective. In Proceedings ofthe 26th International Joint Conference on Artificial Intelli-gence, 695–702. AAAI Press.Ma, D.; Li, S.; Zhang, X.; and Wang, H. 2017. Interactiveattention networks for aspect-level sentiment classification.arXiv preprint arXiv:1709.00893.Ma, Y.; Peng, H.; and Cambria, E. 2018. Targetedaspect-based sentiment analysis via embedding common-sense knowledge into an attentive lstm. In Proceedings ofAAAI.Pan, S. J.; Ni, X.; Sun, J.-T.; Yang, Q.; and Chen, Z. 2010.Cross-domain sentiment classification via spectral featurealignment. In Proceedings of the 19th international con-ference on World wide web, 751–760. ACM.Stehman, S. V. 1997. Selecting and interpreting measuresof thematic classification accuracy. Remote sensing of Envi-ronment 62(1):77–89.Tripathy, A.; Agrawal, A.; and Rath, S. K. 2016. Classifi-cation of sentiment reviews using n-gram machine learningapproach. Expert Systems with Applications 57:117–126.Tu, Z.; Liu, Y.; Shang, L.; Liu, X.; and Li, H. 2017. Neuralmachine translation with reconstruction. In AAAI, 3097–3103.Wang, G.; Sun, J.; Ma, J.; Xu, K.; and Gu, J. 2014. Senti-ment classification: The contribution of ensemble learning.Decision support systems 57:77–93.Wang, Y.; Huang, M.; Zhao, L.; et al. 2016. Attention-basedlstm for aspect-level sentiment classification. In Proceed-ings of the 2016 conference on empirical methods in naturallanguage processing, 606–615.Yu, J., and Jiang, J. 2016. Learning sentence embeddingswith auxiliary tasks for cross-domain sentiment classifica-tion. In Proceedings of the 2016 Conference on EmpiricalMethods in Natural Language Processing, 236–246.Zhang, X.; Li, S.; Sha, L.; and Wang, H. 2017. Attentive in-teractive neural networks for answer selection in communityquestion answering. In AAAI, 3525–3531.Zhao, H.; Ge, Y.; Liu, Q.; Wang, G.; Chen, E.; and Zhang,H. 2017a. P2p lending survey: platforms, recent advancesand prospects. ACM Transactions on Intelligent Systems andTechnology (TIST) 8(6):72.Zhao, H.; Zhang, H.; Ge, Y.; Liu, Q.; Chen, E.; Li, H.; andWu, L. 2017b. Tracking the dynamics in crowdfunding. InProceedings of the 23rd ACM SIGKDD International Con-ference on Knowledge Discovery and Data Mining, 625–634. ACM.