A Split-and-Recombine Approach for Follow-up Query Analysis

Qian Liu†∗ , Bei Chen§, Haoyan Liu♦∗, Jian-Guang Lou§, Lei Fang§,Bin Zhou†, Dongmei Zhang§

†State Key Laboratory of Virtual Reality Technology and Systems,School of Computer Science and Engineering, Beihang University, China

♦State Key Lab of Software Development Environment, Beihang University, China§Microsoft Research, Beijing, China

†♦{qian.liu, haoyan.liu, zhoubin}@buaa.edu.cn;§{beichen, jlou, leifa, dongmeiz}@microsoft.com

AbstractContext-dependent semantic parsing hasproven to be an important yet challengingtask. To leverage the advances in context-independent semantic parsing, we propose toperform follow-up query analysis, aiming torestate context-dependent natural languagequeries with contextual information. Toaccomplish the task, we propose STAR,a novel approach with a well-designedtwo-phase process. It is parser-independentand able to handle multifarious follow-upscenarios in different domains. Experimentson the FollowUp dataset show that STARoutperforms the state-of-the-art baseline by alarge margin of nearly 8%. The superiorityon parsing results verifies the feasibility offollow-up query analysis. We also explore theextensibility of STAR on the SQA dataset,which is very promising.

1 Introduction

Recently, Natural Language Interfaces to Data-bases (NLIDB) has received considerable atten-tion, as they allow users to query databases bydirectly using natural language. Current stud-ies mainly focus on context-independent seman-tic parsing, which translates a single natural lan-guage sentence into its corresponding executableform (e.g. Structured Query Language) and re-trieves the answer from databases regardless ofits context. However, context does matter in realworld applications. Users tend to issue queriesin a coherent way when communicating withNLIDB. For example, after the query “How muchmoney has Smith earned?” (Precedent Query),users may pose another query by simply asking“How about Bill Collins?” (Follow-up Query) in-stead of the complete “How much money has Bill

∗Work done during an internship at Microsoft Research.

Collins earned?” (Restated Query). Therefore,contextual information is essential for more accu-rate and robust semantic parsing, namely context-dependent semantic parsing.

Compared with context-independent semanticparsing, context-dependent semantic parsing hasreceived less attention. Several attempts includea statistical model with parser trees (Miller et al.,1996), a linear model with context-dependent log-ical forms (Zettlemoyer and Collins, 2009) and asequence-to-sequence model (Suhr et al., 2018).However, all these methods cannot apply to differ-ent domains, since the ATIS dataset (Dahl et al.,1994) they rely on is domain-specific. A search-based neural method DynSP∗ arises along with theSequentialQA (SQA) dataset (Iyyer et al., 2017),which takes the first step towards cross-domaincontext-dependent semantic parsing. Neverthe-less, DynSP∗ focuses on dealing with relativelysimple scenarios. All the aforementioned methodsdesign context-dependent semantic parser fromscratch. Instead, inspired by Liu et al. (2019),we propose to directly leverage the technical ad-vances in context-independent semantic parsing.We define follow-up query analysis as restatingthe follow-up queries using contextual informa-tion in natural language, then the restated queriescan be translated to the corresponding executableforms by existing context-independent parsers. Inthis way, we boost the performance of context-dependent semantic parsing.

In this paper, we focus on follow-up query anal-ysis and present a novel approach. The main ideais to decompose the task into two phases by in-troducing a learnable intermediate structure span:two queries first get split into several spans, andthen undergo the recombination process. As nointermediate annotation is involved, we design re-

arX

iv:1

909.

0890

5v1

[cs

.CL

] 1

9 Se

p 20

19

wards to jointly train the two phases by applyingreinforcement learning (RL) (Sutton and Barto,1998). Our major contributions are as follows:

• We propose a novel approach, named SpliT-And-Recombine (STAR)1, to restate follow-upqueries via two phases. It is parser-independentand can be seamlessly integrated with existingcontext-independent semantic parsers.

• We conduct experiments on the FollowUpdataset (Liu et al., 2019), which covers multifar-ious cross-domain follow-up scenarios. The re-sults demonstrate that our approach significantlyoutperforms the state-of-the-art baseline.

• We redesign the recombination process and ex-tend STAR to the SQA dataset, where the anno-tations are answers. Experiments show promis-ing results, that demonstrates the extensibility ofour approach.

2 Methodology

In this section, we first give an overview of ourproposed method with the idea of two-phase pro-cess, then introduce the two phases in turn.

2.1 Overview of Split-And-RecombineLet x = (x1, . . . , xn), y = (y1, . . . , ym) and z =(z1, . . . , zl) denote the precedent query, follow-up query and restated query respectively, each ofwhich is a natural language sentence. Our goal isto interpret the follow-up query y with its prece-dent query x as context, and generate the corre-sponding restated query z. The restated query hasthe same meaning with the follow-up query, butit is complete and unambiguous to facilitate bet-ter downstream parsing. Formally, given the pair(x,y), we aim to learn a model Pmodel(z|x,y)and maximize the objective:

L = E(x,y,z)∼D[logPmodel(z|x,y)], (1)

where D represents the set of training data. Asto Pmodel(z|x,y), since z always overlaps a greatwith x and y, it is intuitively more straightforwardto find a way to merge x and y. To this end, wedesign a two-phase process and present a novel ap-proach STAR to perform follow-up query analysiswith reinforcement learning.

A concrete example of the two-phase process isshown in Figure 1. Phase I is to Split input queries

1Code is available at http://github.com/microsoft/EMNLP2019-Split-And-Recombine.

Precedent Query Follow-up QueryHow much money has Smith earned How about Bill Collins

How much money has Smith earned How about Bill Collins

Restated Query

Phas

e I

Phas

e II

How much money has Smith earnedBill Collins

Conflict

Figure 1: The two-phase process of an example fromthe FollowUp dataset (More real cases of diversefollow-up scenarios can be found in Table 3).

into several spans. For example, the precedentquery is split into 3 spans: “How much moneyhas”, “Smith” and “earned”. Let q denote a kind ofway to split (x,y), then Phase I can be formulatedas Psplit(q|x,y). Phase II is to Recombine thespans by finding out the most probable conflict-ing way, and generating the final output by restate-ment, denoted as Prec(z|q). Two spans being con-flicting means they are semantically similar. Forexample, “Smith” conflicts with “Bill Collins”. Aconflicting way contains all conflicts between theprecedent and follow-up spans. Backed by thetwo-phase idea of splitting and recombination, theoverall likelihood of generating z given x,y is:

Pmodel(z|x,y) =∑q∈Q

Psplit(q|x,y)Prec(z|q), (2)

where Q represents the set of all possible ways tosplit (x,y). Due to the lack of annotations forsplitting and recombination, it is hard to directlyperform supervised learning. Inspired by Lianget al. (2017), we employ RL to optimize Pmodel.Denoting the predicted restated query by z, sim-plifying E(x,y,z)∼D as E, the goal of the RL train-ing is to maximize following objective:

Lrl=E[∑z∈Z

∑q∈Q

Psplit(q|x,y)Prec(z|q)r(z, z)], (3)

where Z is the space of all restated query can-didates and r represents the reward defined bycomparing z and the annotation z. However, theoverall candidate space Q×Z is vast, making itimpossible to exactly maximize Lrl. The moststraightforward usage of the REINFORCE algo-rithm (Williams, 1992), sampling both q and z,also poses challenges for learning. To alleviatethe problem, we propose to sample q and enumer-ate all candidate z after q is determined. It couldshrink the sampling space with an acceptable com-putational cost, which will be discussed in Sec-

LSTM

Softmax

Softmax

Recombine

Split/Retain

Reward

Split

Phase I Phase II

How much earned How Bill Collins… …

Retain Retain Split Retain

Figure 2: The overview of STAR with two phases.

tion 3.2.2. Thus the problem turns to design a re-ward function R(q, z) to evaluate q and guide thelearning. To achieve it, we reformulate Equation 3as:

Lrl=E[∑q∈Q

Psplit(q|x,y)∑z∈Z

Prec(z|q)r(z, z)], (4)

and set the R(q, z) as:∑z∈Z

Prec(z|q)r(z, z). (5)

The overview of STAR is summarized in Figure 2.Given x,y, during training of Phase I (in blue), wefix Prec to provide the reward R(q, z), then Psplit

can be learnt by the REINFORCE algorithm. Dur-ing training of Phase II (in red), we fix Psplit andutilize it to generate q, Prec is trained to maximizeEquation 5. In this way, Psplit and Prec can bejointly trained. The details are introduced below.

2.2 Phase I: SplitAs mentioned above, fixed Prec, Phase I updatesPsplit, the Split Neural Network (SplitNet). Tak-ing the precedent query and follow-up query as in-put, as shown in Figure 2, splitting spans can beviewed as a sequence labeling problem over in-put. For each word, SplitNet outputs a label Splitor Retain, indicating whether a split operation willbe performed after the corresponding word. A la-bel sequence uniquely identifies a way of splitting(x,y), mentioned as q in Section 2.1. Figure 3gives an example on the bottom. In the prece-dent query, two split operations are performed af-ter “has” and “Smith” , since their labels are Split.

2.2.1 Split Neural NetworkIntuitively, only after obtaining information fromboth the precedent query and follow-up query can

SplitNet get to know the reasonable way to splitspans. Inspired by BiDAF (Seo et al., 2017), weapply a bidirectional attention mechanism to cap-ture the interrelations between the two queries.

Embedding Layer We consider embedding inthree levels: character, word and sentence, re-spectively denoted as φc, φw and φs. Character-level embedding maps each word to a vector in ahigh-dimensional space using Convolutional Neu-ral Networks (Kim, 2014). Word-level embed-ding is initialized using GloVe (Pennington et al.,2014), and then it is updated along with other pa-rameters. Sentence-level embedding is a one-hotvector designed to distinguish between precedentand follow-up queries. Then, the overall embed-ding function is φ = [φc;φw;φs].

Context Layer On top of the embedding layer,Bidirectional Long Short-Term Memory Network(BiLSTM) (Hochreiter and Schmidhuber, 1997;Schuster and Paliwal, 1997) is applied to capturecontextual information within one query. For wordxi(i=1, . . . , n) in the precedent query x, the hid-den state hi = [

−→h i;←−h i] is computed, where the

forward hidden state is:

−→h i =

−−−−→LSTM

(φ(xi);

−→h i−1

). (6)

Similarly, a hidden state uj is computed for wordyj(j=1, . . . ,m). The BiLSTMs for x and y sharethe same parameters.

Attention Layer The interrelations between theprecedent and follow-up queries are captured viaattention layer. Let H=[h1,h2, . . . ,hn] and U=[u1,u2, . . . ,um] denote the hidden states of twoqueries respectively, the similarity matrix is:

A = cos(H>U), (7)

How much money has Smith earned How about Bill Collins

←( , )Enumerate (size: 13)

Retain Split Split

,

EMPTY

Retain

How much money has How about Smith Bill Collins←,

← How about Smith Bill Collins←,

How about Bill Collins,How much money has EMPTYY ←←

How about Bill Collins, EMPTYY ←←EMPTY

Smith How about earned Bill Collins,← d ←… …

72100647581

Retain Retain Retain Split

( )Restatement

… …

Figure 3: Illustration of reward computation in Phase II.

where A∈Rn×m and the entry Ai,j represents thesimilarity between words xi and yj . Then the soft-max function is used to obtain the precedent-to-follow (P2F) attention and the follow-to-precedent(F2P) attention. P2F attention represents yj usingthe similarities between yj and every word in x.Specifically, let fj = softmax(A:,j), where fj ∈Rn denotes the attention weights on x accordingto yj . Then yj can be represented by a precedent-aware vector uj =

∑nk=1 fj [k]·hk. Similarly, F2P

attention computes the attention weights on y ac-cording to xi, and represents xi as hi.

Output Layer Combining the outputs of thecontext layer and the attention layer, we design thefinal hidden state as follows:

cxi = [hi; hi◦hi; hi+1◦hi+1], (8)

cyj = [uj ; uj◦uj ; uj+1◦uj+1], (9)

where i∈{1,. . . ,n−1},j∈{1,. . . ,m−1} and ◦ denoteselement-wise multiplication (Lee et al., 2017). Letc = (ct)

Tt=1 = (cx1 , ..., cxn−1 , cy1 , ..., cym−1) de-

note the final hidden state sequence. At each po-sition t, the probability of Split is σ(W ∗ ct + b),where σ denotes the sigmoid function and {W, b}denotes the parameters.

2.2.2 TrainingIt is difficult to train RL model from scratch.Therefore, we propose to initialize SplitNet viapre-training, and then use reward to optimize it.

Pre-training We obtain the pre-training anno-tation a by finding the common substrings be-tween (x,y) and z. One a is a label sequence,each of which is Split or Retain. Given the pre-training data setDpre whose training instance is as(x,y,a), the objective function of pre-training is:

Lpre(θ) = E(x,y,a)∼Dpre[log pθ(a|x,y)], (10)

where θ is the parameter of SplitNet.

Policy Gradient After pre-training, we treat thelabel sequence as a variable a. The rewardR(a, z)(details in Section 2.3) is used to optimize the pa-rameter θ with policy gradient methods (Suttonet al., 1999). SplitNet is trained to maximize thefollowing objective function:

Lrl(θ)= E(x,y,z)∼D[Ea∼pθ(a|x,y)R(a, z)]. (11)

In practice, REINFORCE algorithm (Williams,1992) is applied to approximate Equation 11 viasampling a from pθ(a|x,y) for M times, whereM is a hyper-parameter representing the samplesize. Furthermore, subtracting a baseline (Weaverand Tao, 2001) on R(a, z) is also applied to re-duce variance. The final objective function is asfollows:

Lrl(θ)=E(x,y,z)∼D

[ M∑i=1

pθ(ai|x,y)(R(ai, z)−R

)],

where R=1

M

M∑i=1

R(ai, z). (12)

2.3 Phase II: RecombineHere we present Phase II with two questions: (1)Receiving the sampled label sequence a, how tocompute its rewardR(a, z); (2) How to do trainingand inference for Prec.

2.3.1 Reward ComputationReceiving the label sequence a, we first enumer-ate all conflicting way candidates. Following theexample in Figure 3, once we get a deterministica, the split of (x,y) is uniquely determined. Herex and y are split into 3 and 2 spans respectively.Treating spans as units, we enumerate all conflict-ing way candidates methodically. We act up to theone-to-one conflicting principle, which means a

span either has no conflict (denoted as EMPTY) orhas only one conflict with a span in another query.Let C denote the set of all conflicting way candi-dates, the size of which is 13 in Figure 3.

For each conflicting way, we deterministicallygenerate a restated query via the process namedRestatement. In general, we simply replace spansin the precedent query with their conflicting spansto generate the restated query. For example, inFigure 3, the first one in C is restated as “Howabout Bill Collins earned”. For spans in thefollow-up query, if they contain column names orcell values and do not have any conflict, they areappended to the tail of the precedent query. It isdesigned to remedy the sub-query situation wherethere is no conflict (e.g. “Which opponent re-ceived over 537 attendance” and “And which gotthe result won 5-4”). Specially, if a span in thefollow-up query contains a pronoun, we will in re-verse replace it with its conflicting span to obtainthe restated query.

Finally, the reward can be computed. Here weuse BLEU and SymAcc2 to build the reward func-tion, expanding r(z, z) in Equation 5 as:

r(z, z) = α·BLEU(z, z)+β ·SymAcc(z, z), (13)

where α, β > 0 and α + β = 1. The reward for acan be obtained using Equation 5.

2.3.2 Training and InferenceBesides the reward computation, the recombina-tion model Prec needs to be trained to maximizeEquation 5. To achieve this, we define a conflict-ing probability matrix F ∈ RNx×Ny , where Nx

and Ny denote the number of spans in x and y re-spectively. The entry Fu,v, the conflicting prob-ability between the u-th span in x and the v-thspan in y, is obtained by normalizing the cosinesimilarity between their representations. Here thespan representation is the subtraction representa-tion (Wang and Chang, 2016; Cross and Huang,2016), which means that span (xi, . . . , xk) is rep-resented by [

−→hk−−→hi;←−hi−←−hk] from the same BiL-

STM in the context layer in Section 2.2.1. Given aconflicting way denoted as c ∈ C, the probabilityof generating its corresponding z can be written asthe multiplication over g(u, v):

Prec(z|a) = P (c|F) =

Nx∏u=1

Ny∏v=1

g(u, v), (14)

2Their definitions along with the motivations of usingthem will be explained in Section 3.2.

where g(u, v) = Fu,v if the u-th span in x con-flicts with the v-th span in y; otherwise, g(u, v) =1 − Fu,v. With the above formulation, we canmaximize Equation 5 through automatic differ-entiation. To reduce the computation, we onlymaximize Prec(z

∗|a), the near-optimal solution toEquation 5, where z∗ = arg maxz∈Z(r(z, z)) de-notes the best predicted restated query so far.

Guided by the golden restated query z, in train-ing, we find out z∗ by computing the reward ofeach candidate. However in inference, where thereis no golden restate query, we can only obtain z∗

from F. Specially, for the v-th span in the follow-up query, we find u∗ = arg maxuFu,v. Thatmeans, compared to other spans in the precedentquery, the u∗-th span has the highest probabil-ity to conflict with the v-th span in the follow-upquery. Moreover, similar to Lee et al. (2017), ifFu∗,v < λ, then the v-th span in the follow-upquery has no conflict. The hyper-parameter λ > 0denotes the threshold.

2.4 Extension

So far, we have introduced the whole process ofSTAR. Next we explore its extensibility. As ob-served, when the annotations are restated queries,STAR is parser-independent and can be incor-porated into any context-independent semanticparser. But what if the annotations are answers tofollow-up queries? Assuming we have an ideal se-mantic parser, a predicted restated query z can beconverted into its corresponding answer w. For ex-ample, given z as “where are the players from”, wcould be “Las Vegas”. Therefore, revisiting Equa-tion 3, in theory STAR is able to be extended byredesigning r as r(w, w), where w denotes the an-swer annotation. We conduct an extension experi-ment to verify it, as discussed in Section 3.3.

3 Experiments

In this section, we demonstrate the effectivenessof STAR on the FollowUp dataset3 with restatedquery annotations, and its promising extensibilityon the SQA dataset4 with answer annotations.

3.1 Implementation details

We utilize PyTorch (Paszke et al., 2017) and Al-lenNLP (Gardner et al., 2018) for implementa-tion, and adopt Adam (Kingma and Ba, 2015) as

3http://github.com/SivilTaram/FollowUp4http://aka.ms/sqa

Model Dev Test

SymAcc (%) BLEU (%) SymAcc (%) BLEU (%) AnsAcc (%)

SEQ2SEQ† (Bahdanau et al., 2015) 0.63 ± 0.00 21.34 ± 1.14 0.50 ± 0.22 20.72 ± 1.31 –COPYNET† (Gu et al., 2016) 17.50 ± 0.87 43.36 ± 0.54 19.30 ± 0.93 43.34 ± 0.45 –COPY+BERT (Devlin et al., 2019) 18.63 ± 0.61 45.14 ± 0.68 22.00 ± 0.45 44.87 ± 0.52 –CONCAT† – – 22.00 ± – 52.02 ± – 25.24E2ECR† (Lee et al., 2017) – – 27.00 ± – 52.47 ± – 27.18FANDA† (Liu et al., 2019) 49.00 ± 1.28 60.14 ± 0.98 47.80 ± 1.14 59.02 ± 0.54 60.19

STAR 55.38 ± 1.21 67.62 ± 0.65 54.00 ± 1.09 67.05 ± 1.05 65.05

Table 1: SymAcc, BLEU and AnsAcc on the FollowUp dataset. Results marked † are from Liu et al. (2019).

the optimizer. The dimensions of word embed-ding and hidden state are both 100. Variationaldropout (Blum et al., 2015) is employed at embed-ding layer for better generalization ability (withprobability 0.5). The learning rate is set to be0.001 for pre-training, 0.0001 for RL training onFollowUp, and 0.0002 for SQA. In the implemen-tation of the REINFORCE algorithm, we set Mto be 20. Finally, for hyper-parameters, we setα = 0.5, β = 0.5 and λ = 0.6. All the results areaveraged over 5 runs with random initialization.

3.2 Results on FollowUp dataset

The FollowUp dataset contains 1000 natural lan-guage query triples (x,y, z). Each triple belongsto a single database table, and there are 120 ta-bles in several different domains. Following theprevious work, we split them into the sets of size640/160/200 for train/dev/test. We evaluate themethods using both answer level and query levelmetrics. AnsAcc is to check the answer accuracyof predicted queries manually. Concretely, 103golden restated queries can be successfully parsedby COARSE2FINE (Dong and Lapata, 2018). Weparse their corresponding predicted queries intoSQL using COARSE2FINE and manually checkthe answers. Although AnsAcc is most convinc-ing, it cannot cover the entire test set. Therefore,we apply two query level metrics: SymAcc detectswhether all the SQL-related words are correctlyinvolved in the predicted queries, for example col-umn names, cell values and so on. It reflects theapproximate upper bound of AnsAcc, as the cor-rectness of SQL-related words is a prerequisite ofcorrect execution in most cases; BLEU, referringto the cumulative 4-gram BLEU score, evaluateshow similar the predicted queries are to the goldenones (Papineni et al., 2002). SymAcc focuses onlimited keywords, so we introduce BLEU to eval-

uate quality of the entire predicted query.

3.2.1 Model ComparisonOur baselines fall into two categories. Gener-ation-based methods conform to the architec-ture of sequence-to-sequence (Sutskever et al.,2014) and generate restated queries by decodingeach word from scratch. SEQ2SEQ (Bahdanauet al., 2015) is the sequence-to-sequence modelwith attention, and COPYNET further incorporatesa copy mechanism. COPY+BERT incorporatesthe latest pre-trained BERT model (Devlin et al.,2019) as the encoder of COPYNET. Rewriting-based methods obtain restated queries by rewrit-ing precedent and follow-up queries. CONCAT di-rectly concatenates the two queries. E2ECR (Leeet al., 2017) obtain restated queries by perform-ing coreference resolution in follow-up queries.FANDA (Liu et al., 2019) utilizes a structure-aware model to merge the two queries. Ourmethod STAR also belongs to this category.

Answer Level Table 1 shows AnsAcc results ofcompetitive baselines on the test set. Comparedwith them, STAR achieves the highest, 65.05%,which demonstrates its superiority. Meanwhile, itverifies the feasibility of follow-up query analysisin cooperating with context-independent semanticparsing. Compared with CONCAT, our approachboosts over 39.81% on COARSE2FINE for the ca-pability of context-dependent semantic parsing.

Query Level Table 1 also shows SymAcc andBLEU of different methods on the dev and testsets. As observed, STAR significantly outper-forms all baselines, demonstrating its effective-ness. For example, STAR achieves an absoluteimprovement of 8.03% BLEU over the state-of-the-art baseline FANDA on testing. Moreover, therewriting-based baselines, even the simplest CON-CAT, perform better than the generation-based

Variant SymAcc (%) BLEU (%)

STAR 55.38 67.62

– Phase I 40.63 61.82– Phase II 23.12 48.65– RL 41.25 60.19+ Basic Reward 43.13 58.48+ Oracle Reward 45.20 63.04+ Uniform Reward 53.40 66.93

Table 2: Variant results on FollowUp dev set.

0 20 40 60Epoch

50

60

70

80

BLE

U (%

) STARBasic RewardOracle RewardUniform Reward

Figure 4: Learning curve on FollowUp train set.

ones. It suggests that the idea of rewriting ismore reasonable for the task, where precedent andfollow-up queries are of full utilization.

3.2.2 Variant AnalysisBesides baselines, we also conduct experimentswith several variants of STAR to further validatethe design of our model. As shown in Table 2,there are three variants with ablation: “– PhaseI” takes out SplitNet and performs Phase II onword level; “– Phase II” performs random guess inthe recombination process for testing; and “– RL”only contains pre-training. The SymAcc dropsfrom about 55% to 40% by ablating Phase I, andto 23% by ablating Phase II. Their poor perfor-mances indicate both of the two phases are in-dispensable. “– RL” also performs worse, whichagain demonstrates the rationality of applying RL.

Three more variants are presented with differ-ent designs of R(q, z) to prove the efficiency andeffectiveness of Equation 5 as a reward. “+ BasicReward” represents the most straightforward RE-INFORCE algorithm, which samples both q ∈ Qand z∈Z , then takes r(z, z) as R(q, z). “+ OracleReward” assumes the conflicts are always correctand rewrites R(q, z) as maxz∈Z(r(z, z)). “+ Uni-form Reward” assigns the same probability to allz and obtains R(q, z) as mean(

∑z∈Zr(z, z)). As

shown in Table 2 and Figure 4, STAR learns better

Afterthe week 6 , whic

hop

pone

nts

won the result

of the match

?

Precedent Query

howabout

beforeweek

10?Fo

llow

-up

Que

ry

Figure 5: An example of similarity matrix in SplitNet.

and faster than the variants due to the reasonablereward design. In fact, as mentioned in Section2.1, the vast action space of the most straightfor-ward REINFORCE algorithm leads to poor learn-ing. STAR shrinks the space from |Q|·|Z| down to|Q| by enumerating z. Meanwhile, statistics showthat STAR obtains a 15× speedup over “+ BasicReward” on the convergence time.

3.2.3 Case StudyFigure 5 shows a concrete example of the similar-ity matrix A on attention layer of SplitNet. Thespan “before week 10” is evidently more similarto “After the week 6” than to others, which meetsour expectations. Moreover, the results of threereal cases are shown in Table 3. The spans in blueare those have conflicts, and the histograms rep-resent the conflict probabilities to all the spans inprecedent queries. In Case 1, “glebe park”, “ham-pden park” and “balmoor” are all cell values in thedatabase table with similar meanings. STAR cor-rectly finds out the conflict between “compared toglebe park” and “compared to balmoor” with thehighest probability. Case 2 shows STAR can dis-cover the interrelation of words, where “the writerNancy miller” is learnt as a whole span to re-place “Nancy miller” in the precedent query. Asfor Case 3, STAR successfully performs corefer-ence resolution and interprets “those two films”as “greatest love and promised land”. Benefitingfrom two phases, STAR is able to deal with di-verse follow-up scenarios in different domains.

3.2.4 Error AnalysisOur approach works well in most cases exceptfor few ones, where SplitNet fails. For exam-ple, given the precedent query “what’s the biggestzone?” and the follow-up query “the smallestone”, STAR prefers to recognize “the biggestzone” and “the smallest one” as two spans, rather

Case Analysis

1PrecedentFollow-upSTAR

: [ compared to glebe park ] [ , does ] [ hampden park ] [ holds more attendances at capacity ? ]: [ how about ] [ compared to balmoor ]: compared to balmoor , does hampden park holds more attendances at capacity ?

2PrecedentFollow-upSTAR

: [ Is there any book which belongs to ] [ Nancy miller ]: [ I mean ] [ the writer Nancy miller ]: Is there any book which belongs to the writer Nancy miller

3PrecedentFollow-upSTAR

: [ show directors of ] [ greatest love and promised land ]: [ show air date of ] [ those two films ]: show air date of greatest love and promised land

No

Table 3: Case analysis of STAR on FollowUp dataset. Square brackets denote different spans.

Model Precedent Follow-up

DynSP (Iyyer et al., 2017) 70.9 35.8NP (Neelakantan et al., 2016) 58.9 35.9NP + STAR 58.9 38.1DynSP + STAR 70.9 39.5DynSP∗ (Iyyer et al., 2017) 70.4 41.1

Table 4: Answer accuracy on SQA test set.

than perform split operations inside them. TheSplitNet fails probably because the conflictingspans, “the biggest”↔ “the smallest” and “zone”↔ “one”, are adjacent, which makes it difficult toidentify span boundaries well.

3.3 Extension on SQA dataset

Finally, we demonstrate STAR’s extensibility inworking with different annotations. As mentionedin Section 2.4, by designing r(w, w), STAR cancooperate with the answer annotations. We con-duct experiments on the SQA dataset, which con-sists of 6066 query sequences (5042/1024 fortrain/test). Each sequence contains multiple nat-ural language queries and their answers, where weare only interested in the first query and the imme-diate follow-up one. As discussed in (Iyyer et al.,2017), every answer can be represented as a set ofcells in the tables, each of which is a multi-wordvalue, and the intentions of the follow-up queriesmainly fall into three categories. Column selec-tion means the follow-up answer is an entire col-umn; Subset selection means the follow-up answeris a subset of the precedent answer; and Row se-lection means the follow-up answer has the samerows with the precedent answer.

We employ two context-independent parsers,DynSP (Iyyer et al., 2017) and NP (Neelakan-tan et al., 2016), which are trained on the SQAdataset to provide relatively reliable answers for

reward computing. Unfortunately, they both per-form poorly for the restated queries, as the re-stated queries are quite different from the origi-nal queries in SQA. To address the problem, weredesign the recombination process. Instead ofgenerating the restated query, we recombine thepredicted precedent answer wx and the predictedfollow-up answer wy to produce the restated an-swer w. Therefore, the objective of Phase II is toassign an appropriate intention to each follow-upspan via an additional classifier. The goal of PhaseI turns to split out spans having obvious intentionssuch as “of those”. The way of recombining an-swer is determined by the voting from intentionson all spans. If the intention column selectionwins, then w = wy; for subset selection, we ob-tain the subset w by taking the rows of wy as theconstraint and applying it to wx; and for row se-lection, we take the rows of wx and the columns ofwy as the constraints, then apply them to the wholedatabase table to obtain the answer w retrieved bythe predicted SQL. The reward r(w, w) is com-puted based on Jaccard similarity between the goldanswer w and w as in (Iyyer et al., 2017), and theoverall training process remains unchanged.

Table 4 shows the answer accuracy of precedentand follow-up queries on test set. DynSP∗ (Iyyeret al., 2017) is designed for SQA by introducinga special action Subsequent to handle follow-upqueries based on DynSP. DynSP∗ is incapable ofbeing extended to work with the annotation of therestated queries. We attempt to apply DynSP∗

(trained on SQA) directly on FollowUp test set,which results in an extremely low AnsAcc. On thecontrary, STAR is extensible. “+STAR” meansour method STAR is incorporated into the context-independent parser and empowers them with theability to perform follow-up query analysis. Asobserved, integrating STAR consistently improves

performance for follow-up queries, which demon-strates the effectiveness of STAR in collaboratingwith different semantic parsers. The comparableresults of DynSP+STAR to DynSP∗ further veri-fies the promising extensibility of STAR.

4 Related Work

Our work is closely related to two lines ofwork: context-dependent sentence analysis andreinforcement learning. From the perspective ofcontext-dependent sentence analysis, our work isrelated to researches like reading comprehensionin dialogue (Reddy et al., 2019; Choi et al., 2018),dialogue state tracking (Williams et al., 2013),conversational question answering in knowledgebase (Saha et al., 2018; Guo et al., 2018),context-dependent logic forms (Long et al., 2016),and non-sentential utterance resolution in open-domain question answering (Raghu et al., 2015;Kumar and Joshi, 2017). The main difference isthat we focus on the context-dependent queriesin NLIDB which contain complex scenarios. Asfor the most related context-dependent semanticparsing, Zettlemoyer and Collins (2009) proposesa context-independent CCG parser and then con-duct context-dependent substitution, Iyyer et al.(2017) presents a search-based method for sequen-tial questions, and Suhr et al. (2018) presents asequence-to-sequence model to solve the problem.Compared to their methods, our work achievescontext-dependent semantic parsing via learnablerestated queries and existing context-independentsemantic parsers.

Moreover, the technique of reinforcement learn-ing has also been successfully applied to natu-ral language tasks in dialogue, such as hyper-parameters tuning for coreference resolution(Clark and Manning, 2016), sequential questionanswering (Iyyer et al., 2017) and coherent dia-logue responses generation (Li et al., 2016). Inthis paper, we employ reinforcement learning tocapture the structures of queries, which is similarto Zhang et al. (2018) for text classification.

5 Conclusion and Future Work

We present a novel method, named Split-And-Recombine (STAR), to perform follow-up queryanalysis. A two-phase process has been designed:one for splitting precedent and follow-up queriesinto spans, and the other for recombining them.Experiments on two different datasets demonstrate

the effectiveness and extensibility of our method.For future work, we may extend our method toother natural language tasks.

Acknowledgments

We thank all the anonymous reviewers for theirvaluable comments. This work was supported bythe National Natural Science Foundation of China(Grant Nos. U1736217 and 61932003).

ReferencesDzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben-

gio. 2015. Neural machine translation by jointlylearning to align and translate. In Proceedings ofthe 3rd International Conference on Learning Rep-resentations, ICLR 2015, San Diego, CA, USA, May7-9, 2015.

Avrim Blum, Nika Haghtalab, and Ariel D. Procaccia.2015. Variational dropout and the local reparam-eterization trick. In Advances in Neural Informa-tion Processing Systems 28: Annual Conference onNeural Information Processing Systems, NIPS 2015,Montreal, Quebec, Canada, December 7-12, 2015,pages 2575–2583.

Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettle-moyer. 2018. QuAC: Question answering in con-text. In Proceedings of the 2018 Conference onEmpirical Methods in Natural Language Process-ing, EMNLP 2018, Brussels, Belgium, October 31- November 4, 2018, pages 2174–2184.

Kevin Clark and Christopher D. Manning. 2016. Deepreinforcement learning for mention-ranking corefer-ence models. In Proceedings of the 2016 Confer-ence on Empirical Methods in Natural LanguageProcessing, EMNLP 2016, Austin, Texas, USA,November 1-4, 2016, pages 2256–2262.

James Cross and Liang Huang. 2016. Span-basedconstituency parsing with a structure-label systemand provably optimal dynamic oracles. In Proceed-ings of the 2016 Conference on Empirical Meth-ods in Natural Language Processing, EMNLP 2016,Austin, Texas, USA, November 1-4, 2016, pages 1–11.

Deborah A. Dahl, Madeleine Bates, Michael Brown,William M. Fisher, Kate Hunicke-Smith, David S.Pallett, Christine Pao, Alexander I. Rudnicky, andElizabeth Shriberg. 1994. Expanding the scope ofthe ATIS task: The ATIS-3 corpus. In Proceed-ings of the 1994 Conference of the North AmericanChapter of the Association for Computational Lin-guistics: Human Language Technologies, NAACL-HLT 1994, ,Plainsboro, New Jerey, USA, March 8-11, 1994.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2019. BERT: pre-training ofdeep bidirectional transformers for language under-standing. In Proceedings of the 2019 Conferenceof the North American Chapter of the Associationfor Computational Linguistics: Human LanguageTechnologies, NAACL-HLT 2019, Minneapolis, MN,USA, June 2-7, 2019, Volume 1:Long and Short Pa-pers, pages 4171–4186.

Li Dong and Mirella Lapata. 2018. Coarse-to-Fine de-coding for neural semantic parsing. In Proceedingsof the 56th Annual Meeting of the Association forComputational Linguistics, ACL 2018, Melbourne,Australia, July 15-20, 2018, Volume 1: Long Papers,pages 731–742.

Matt Gardner, Joel Grus, Mark Neumann, OyvindTafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Pe-ters, Michael Schmitz, and Luke Zettlemoyer. 2018.AllenNLP: A deep semantic natural language pro-cessing platform. In Proceedings of Workshop forNLP Open Source Software, NLP-OSS, of the 56thAnnual Meeting of the Association for Computa-tional Linguistics, ACL 2018, Melbourne, Australia,July 15-20, 2018, pages 1–6.

Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K.Li. 2016. Incorporating copying mechanism insequence-to-sequence learning. In Proceedings ofthe 54th Annual Meeting of the Association for Com-putational Linguistics, ACL 2016, Berlin, Germany,August 7-12, 2016, Volume 1: Long Papers.

Daya Guo, Duyu Tang, Nan Duan, Ming Zhou, andJian Yin. 2018. Dialog-to-action: Conversationalquestion answering over a large-scale knowledgebase. In Advances in Neural Information ProcessingSystems 31: Annual Conference on Neural Informa-tion Processing Systems, NeurIPS 2018, Montreal,Canada, December 3-8, 2018, pages 2946–2955.

Sepp Hochreiter and Jurgen Schmidhuber. 1997. Longshort-term memory. Neural Computation, Volume9, pages 1735–1780.

Mohit Iyyer, Wen-tau Yih, and Ming-Wei Chang. 2017.Search-based neural structured learning for sequen-tial question answering. In Proceedings of the 55thAnnual Meeting of the Association for Computa-tional Linguistics, ACL 2017, Vancouver, Canada,July 30 - August 4, Volume 1: Long Papers, pages1821–1831.

Yoon Kim. 2014. Convolutional neural networks forsentence classification. In Proceedings of the 2014Conference on Empirical Methods in Natural Lan-guage Processing, EMNLP 2014, Doha, Qatar, Oc-tober 25-29, 2014, pages 1746–1751.

Diederik P. Kingma and Jimmy Ba. 2015. Adam: Amethod for stochastic optimization. In Proceed-ings of the 3rd International Conference on Learn-ing Representations, ICLR 2015, San Diego, CA,USA, May 7-9, 2015.

Vineet Kumar and Sachindra Joshi. 2017. Incompletefollow-up question resolution using retrieval basedsequence to sequence learning. In Proceedings ofthe 40th International ACM Conference on Researchand Development in Information Retrieval, SIGIR2017, Shinjuku, Tokyo, Japan, August 7-11, 2017,pages 705–714.

Kenton Lee, Luheng He, Mike Lewis, and Luke Zettle-moyer. 2017. End-to-end neural coreference reso-lution. In Proceedings of the 2017 Conference onEmpirical Methods in Natural Language Process-ing, EMNLP 2017, Copenhagen, Denmark, Septem-ber 9-11, 2017, pages 188–197.

Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky,Michel Galley, and Jianfeng Gao. 2016. Deep rein-forcement learning for dialogue generation. In Pro-ceedings of the 2016 Conference on Empirical Meth-ods in Natural Language Processing, EMNLP 2016,Austin, Texas, USA, November 1-4, 2016, pages1192–1202.

Chen Liang, Jonathan Berant, Quoc V. Le, Kenneth D.Forbus, and Ni Lao. 2017. Neural symbolic ma-chines: Learning semantic parsers on freebase withweak supervision. In Proceedings of the 55th An-nual Meeting of the Association for ComputationalLinguistics, ACL 2017, Vancouver, Canada, July 30-August 4, 2017, Volume 1: Long Papers, pages 23–33.

Qian Liu, Bei Chen, Jian-Guang Lou, Ge Jin, andDongmei Zhang. 2019. FANDA: A novel approachto perform follow-up query analysis. In Proceed-ings of the 33rd AAAI Conference on Artificial Intel-ligence, AAAI 2019, Honolulu, Hawaii, USA, Jan-uary 27 - February 1, 2019, pages 6770–6777.

Reginald Long, Panupong Pasupat, and Percy Liang.2016. Simpler context-dependent logical forms viamodel projections. In Proceedings of the 54th An-nual Meeting of the Association for ComputationalLinguistics, ACL 2016, Berlin, Germany, August 7-12, 2016, Volume 1: Long Papers.

Scott Miller, David Stallard, Robert J. Bobrow, andRichard M. Schwartz. 1996. A fully statistical ap-proach to natural language interfaces. In Proceed-ings of the 34th Annual Meeting of the Associationfor Computational Linguistics, ACL 1996, Univer-sity of California, Santa Cruz, California, June 24-27, 1996, pages 55–61.

Arvind Neelakantan, Quoc V. Le, and Ilya Sutskever.2016. Neural programmer: Inducing latent pro-grams with gradient descent. In Proceedings of the4th International Conference on Learning Represen-tations, ICLR 2016, San Juan, Puerto Rico, May 2-4,2016.

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automaticevaluation of machine translation. In Proceedings of

the 40th Annual Meeting of the Association for Com-putational Linguistics, ACL 2002, Philadelphia, PA,USA., July 6-12, 2002, pages 311–318.

Adam Paszke, Sam Gross, Soumith Chintala, Gre-gory Chanan, Edward Yang, Zachary DeVito, Zem-ing Lin, Alban Desmaison, Luca Antiga, and AdamLerer. 2017. Automatic differentiation in PyTorch.In Advances in Neural Information Processing Sys-tems 30: Annual Conference on Neural InformationProcessing Systems, NIPS 2017, Long Beach, CA,USA, December 4-9, 2017.

Jeffrey Pennington, Richard Socher, and Christo-pher D. Manning. 2014. GloVe: Global vectors forword representation. In Proceedings of the 2014Conference on Empirical Methods in Natural Lan-guage Processing, EMNLP 2014, Doha, Qatar, Oc-tober 25-29, 2014, pages 1532–1543.

Dinesh Raghu, Sathish Indurthi, Jitendra Ajmera, andSachindra Joshi. 2015. A statistical approach fornon-sentential utterance resolution for interactiveQA system. In Proceedings of the 16th AnnualMeeting of the Special Interest Group on Discourseand Dialogue, SIGDIAL 2015, Prague, Czech Re-public, September 2-4, 2015, pages 335–343.

Siva Reddy, Danqi Chen, and Christopher D. Manning.2019. CoQA: A conversational question answeringchallenge. Transactions of the Association for Com-putational Linguistics, Volume 7, pages 249–266.

Amrita Saha, Vardaan Pahuja, Mitesh M. Khapra,Karthik Sankaranarayanan, and Sarath Chandar.2018. Complex sequential question answering: To-wards learning to converse over linked question an-swer pairs with a knowledge graph. In Proceedingsof the 32nd AAAI Conference on Artificial Intelli-gence, AAAI 2018, New Orleans, Louisiana, USA,February 2-7, 2018, pages 705–713.

Mike Schuster and Kuldip K. Paliwal. 1997. Bidirec-tional recurrent neural networks. IEEE Trans. Sig-nal Processing, Volume 45, pages 2673–2681.

Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, andHannaneh Hajishirzi. 2017. Bidirectional attentionflow for machine comprehension. In Proceedings ofthe 5th International Conference on Learning Rep-resentations, ICLR 2017, Toulon, France, April 24-26, 2017.

Alane Suhr, Srinivasan Iyer, and Yoav Artzi. 2018.Learning to map context-dependent sentences to ex-ecutable formal queries. In Proceedings of the 2018Conference of the North American Chapter of theAssociation for Computational Linguistics: HumanLanguage Technologies, NAACL-HLT 2018, NewOrleans, Louisiana, USA, June 1-6, 2018, Volume1:Long Papers, pages 2238–2249.

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014.Sequence to sequence learning with neural net-works. In Advances in Neural Information Pro-cessing Systems 27: Annual Conference on Neural

Information Processing Systems, NIPS 2014, Mon-treal, Quebec, Canada, December 8-13, 2014, pages3104–3112.

Richard S. Sutton and Andrew G. Barto. 1998. Rein-forcement learning: An introduction. IEEE Trans.Neural Networks, Volume 9, pages 1054–1054.

Richard S. Sutton, David A. McAllester, Satinder P.Singh, and Yishay Mansour. 1999. Policy gradi-ent methods for reinforcement learning with func-tion approximation. In Advances in Neural Informa-tion Processing Systems 12: Annual Conference onNeural Information Processing Systems, NIPS 1999,Denver Colorado, USA, November 29 - December 4,1999, pages 1057–1063.

Wenhui Wang and Baobao Chang. 2016. Graph-baseddependency parsing with bidirectional LSTM. InProceedings of the 54th Annual Meeting of the As-sociation for Computational Linguistics, ACL 2016,Berlin, Germany, August 7-12, 2016, Volume 1:Long Papers.

Lex Weaver and Nigel Tao. 2001. The optimal rewardbaseline for gradient-based reinforcement learning.In Proceedings of the 17th Conference in Uncer-tainty in Artificial Intelligence, UAI 2001, Seattle,Washington, USA, August 2-5, 2001, pages 538–545.

Jason D. Williams, Antoine Raux, Deepak Ramachan-dran, and Alan W. Black. 2013. The dialog statetracking challenge. In Proceedings of the 14th An-nual Meeting of the Special Interest Group on Dis-course and Dialogue, SIGDIAL 2013, Metz, France,August 22-24, 2013, pages 404–413.

Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforce-ment learning. Machine Learning, Volume 8, pages229–256.

Luke S. Zettlemoyer and Michael Collins. 2009.Learning context-dependent mappings from sen-tences to logical form. In Proceedings of the 47thAnnual Meeting of the Association for Computa-tional Linguistics, ACL 2009, and the 4th Interna-tional Joint Conference on Natural Language, IJC-NLP 2009, Singapore, August 2-7, 2009, pages 976–984.

Tianyang Zhang, Minlie Huang, and Li Zhao. 2018.Learning structured representation for text classifi-cation via reinforcement learning. In Proceedings ofthe 32nd AAAI Conference on Artificial Intelligence,AAAI-18, New Orleans, Louisiana, USA, February2-7, 2018, pages 6053–6060.