Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1535–1546Vancouver, Canada, July 30 - August 4, 2017. c©2017 Association for Computational Linguistics

https://doi.org/10.18653/v1/P17-1141

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1535–1546Vancouver, Canada, July 30 - August 4, 2017. c©2017 Association for Computational Linguistics

https://doi.org/10.18653/v1/P17-1141

Lexically Constrained Decoding for Sequence Generation Using GridBeam Search

Chris HokampADAPT Centre

Dublin City Universitychris.hokamp@computing.dcu.ie

Qun LiuADAPT Centre

Dublin City Universityqun.liu@dcu.ie

Abstract

We present Grid Beam Search (GBS), analgorithm which extends beam search toallow the inclusion of pre-specified lex-ical constraints. The algorithm can beused with any model that generates a se-quence y = {y0 . . . yT }, by maximizingp(y|x) =

∏tp(yt|x; {y0 . . . yt−1}). Lex-

ical constraints take the form of phrasesor words that must be present in the out-put sequence. This is a very general wayto incorporate additional knowledge intoa model’s output without requiring anymodification of the model parameters ortraining data. We demonstrate the feasibil-ity and flexibility of Lexically ConstrainedDecoding by conducting experiments onNeural Interactive-Predictive Translation,as well as Domain Adaptation for NeuralMachine Translation. Experiments showthat GBS can provide large improvementsin translation quality in interactive scenar-ios, and that, even without any user in-put, GBS can be used to achieve signifi-cant gains in performance in domain adap-tation scenarios.

1 Introduction

The output of many natural language processingmodels is a sequence of text. Examples includeautomatic summarization (Rush et al., 2015), ma-chine translation (Koehn, 2010; Bahdanau et al.,2014), caption generation (Xu et al., 2015), and di-alog generation (Serban et al., 2016), among oth-ers.

In some real-world scenarios, additional infor-mation that could inform the search for the opti-mal output sequence may be available at inference

time. Humans can provide corrections after view-ing a system’s initial output, or separate classifi-cation models may be able to predict parts of theoutput with high confidence. When the domain ofthe input is known, a domain terminology may beemployed to ensure specific phrases are present ina system’s predictions. Our goal in this work is tofind a way to force the output of a model to containsuch lexical constraints, while still taking advan-tage of the distribution learned from training data.

For Machine Translation (MT) usecases in par-ticular, final translations are often produced bycombining automatically translated output withuser inputs. Examples include Post-Editing(PE) (Koehn, 2009; Specia, 2011) and Interactive-Predictive MT (Foster, 2002; Barrachina et al.,2009; Green, 2014). These interactive scenarioscan be unified by considering user inputs to be lex-ical constraints which guide the search for the op-timal output sequence.

In this paper, we formalize the notion of lexi-cal constraints, and propose a decoding algorithmwhich allows the specification of subsequencesthat are required to be present in a model’s out-put. Individual constraints may be single tokens ormulti-word phrases, and any number of constraintsmay be specified simultaneously.

Although we focus upon interactive applica-tions for MT in our experiments, lexically con-strained decoding is relevant to any scenariowhere a model is asked to generate a sequencey = {y0 . . . yT } given both an input x, and aset {c0...cn}, where each ci is a sub-sequence{ci0 . . . cij}, that must appear somewhere in y.This makes our work applicable to a wide rangeof text generation scenarios, including image de-scription, dialog generation, abstractive summa-rization, and question answering.

The rest of this paper is organized as follows:Section 2 gives the necessary background for our

1535

Figure 1: A visualization of the decoding process for an actual example from our English-German MT experiments. The outputtoken at each timestep appears at the top of the figure, with lexical constraints enclosed in boxes. Generation is shown inblue, Starting new constraints in green, and Continuing constraints in red. The function used to create the hypothesis at eachtimestep is written at the bottom. Each box in the grid represents a beam; a colored strip inside a beam represents an individualhypothesis in the beam’s k-best stack. Hypotheses with circles inside them are closed, all other hypotheses are open. (Bestviewed in colour).

discussion of GBS, Section 3 discusses the lex-ically constrained decoding algorithm in detail,Section 4 presents our experiments, and Section 5gives an overview of closely related work.

2 Background: Beam Search forSequence Generation

Under a model parameterized by θ, let the bestoutput sequence y given input x be Eq. 1.

y = argmaxy∈{y[T]}

pθ(y|x), (1)

where we use {y[T]} to denote the set of all se-quences of length T . Because the number of pos-sible sequences for such a model is |v|T , where |v|is the number of output symbols, the search for ycan be made more tractable by factorizing pθ(y|x)into Eq. 2:

pθ(y|x) =T∏

t=0

pθ(yt|x; {y0 . . . yt−1}). (2)

The standard approach is thus to generate theoutput sequence from beginning to end, condition-ing the output at each timestep upon the input x,

and the already-generated symbols {y0 . . . yi−t}.However, greedy selection of the most probableoutput at each timestep, i.e.:

yt = argmaxyi∈{v}

p(yi|x; {y0 . . . yt−1}), (3)

risks making locally optimal decisions which areactually globally sub-optimal. On the other hand,an exhaustive exploration of the output spacewould require scoring |v|T sequences, which isintractable for most real-world models. Thus, asearch or decoding algorithm is often used as acompromise between these two extremes. A com-mon solution is to use a heuristic search to at-tempt to find the best output efficiently (Pearl,1984; Koehn, 2010; Rush et al., 2013). The keyidea is to discard bad options early, while tryingto avoid discarding candidates that may be locallyrisky, but could eventually result in the best overalloutput.

Beam search (Och and Ney, 2004) is probablythe most popular search algorithm for decoding se-quences. Beam search is simple to implement, andis flexible in the sense that the semantics of the

1536

Figure 2: Different structures for beam search. Boxes repre-sent beams which hold k-best lists of hypotheses. (A) ChartParsing using SCFG rules to cover spans in the input. (B)Source coverage as used in PB-SMT. (C) Sequence timesteps(as used in Neural Sequence Models), GBS is an extension of(C). In (A) and (B), hypotheses are finished once they reachthe final beam. In (C), a hypothesis is only complete if it hasgenerated an end-of-sequence (EOS) symbol.

graph of beams can be adapted to take advantageof additional structure that may be available forspecific tasks. For example, in Phrase-Based Sta-tistical MT (PB-SMT) (Koehn, 2010), beams areorganized by the number of source words that arecovered by the hypotheses in the beam – a hypoth-esis is “finished” when it has covered all sourcewords. In chart-based decoding algorithms such asCYK, beams are also tied to coverage of the input,but are organized as cells in a chart, which facili-tates search for the optimal latent structure of theoutput (Chiang, 2007). Figure 2 visualizes threecommon ways to structure search. (A) and (B) de-pend upon explicit structural information betweenthe input and output, (C) only assumes that theoutput is a sequence where later symbols dependupon earlier ones. Note also that (C) correspondsexactly to the bottom rows of Figures 1 and 3.

With the recent success of neural models fortext generation, beam search has become thede-facto choice for decoding optimal output se-quences (Sutskever et al., 2014). However,with neural sequence models, we cannot organizebeams by their explicit coverage of the input. Asimpler alternative is to organize beams by outputtimesteps from t0 · · · tN , where N is a hyperpa-rameter that can be set heuristically, for exampleby multiplying a factor with the length of the in-put to make an educated guess about the maximumlength of the output (Sutskever et al., 2014). Out-put sequences are generally considered completeonce a special “end-of-sentence”(EOS) token hasbeen generated. Beam size in these models is alsotypically kept small, and recent work has shown

Figure 3: Visualizing the lexically constrained decoder’scomplete search graph. Each rectangle represents a beamcontaining k hypotheses. Dashed (diagonal) edges indicatestarting or continuing constraints. Horizontal edges repre-sent generating from the model’s distribution. The horizontalaxis covers the timesteps in the output sequence, and the ver-tical axis covers the constraint tokens (one row for each tokenin each constraint). Beams on the top level of the grid containhypotheses which cover all constraints.

that the performance of some architectures can ac-tually degrade with larger beam size (Tu et al.,2016).

3 Grid Beam Search

Our goal is to organize decoding in such a way thatwe can constrain the search space to outputs whichcontain one or more pre-specified sub-sequences.We thus wish to use a model’s distribution both to“place” lexical constraints correctly, and to gener-ate the parts of the output which are not coveredby the constraints.

Algorithm 1 presents the pseudo-code for lex-ically constrained decoding, see Figures 1 and 3for visualizations of the search process. Beamsin the grid are indexed by t and c. The t vari-able tracks the timestep of the search, while thec variable indicates how many constraint tokensare covered by the hypotheses in the current beam.Note that each step of c covers a single constrainttoken. In other words, constraints is an array ofsequences, where individual tokens can be indexedas constraintsij , i.e. tokenj in constrainti. ThenumC parameter in Algorithm 1 represents the to-tal number of tokens in all constraints.

The hypotheses in a beam can be separatedinto two types (see lines 9-11 and 15-19 of Algo-rithm 1):

1. open hypotheses can either generate from themodel’s distribution, or start available con-straints,

2. closed hypotheses can only generate the next

1537

Algorithm 1 Pseudo-code for Grid Beam Search, note that t and c indices are 0-based1: procedure CONSTRAINEDSEARCH(model, input, constraints, maxLen, numC, k)2: startHyp⇐ model.getStartHyp(input, constraints)3: Grid⇐ initGrid(maxLen, numC, k) . initialize beams in grid4: Grid[0][0] = startHyp5: for t = 1, t++, t < maxLen do6: for c = max(0, (numC + t)−maxLen), c++, c ≤ min(t, numC) do7: n, s, g = ∅8: for each hyp ∈ Grid[t− 1][c] do9: if hyp.isOpen() then

10: g ⇐ g⋃

model.generate(hyp, input, constraints) . generate new open hyps11: end if12: end for13: if c > 0 then14: for each hyp ∈ Grid[t− 1][c− 1] do15: if hyp.isOpen() then16: n⇐ n

⋃model.start(hyp, input, constraints) . start new constrained hyps

17: else18: s⇐ s

⋃model.continue(hyp, input, constraints) . continue unfinished

19: end if20: end for21: end if22: Grid[t][c] = k-argmax

h∈n⋃s⋃g

model.score(h) . k-best scoring hypotheses stay on the beam

23: end for24: end for25: topLevelHyps⇐ Grid[:][numC] . get hyps in top-level beams26: finishedHyps⇐ hasEOS(topLevelHyps) . finished hyps have generated the EOS token27: bestHyp⇐ argmax

h∈finishedHypsmodel.score(h)

28: return bestHyp29: end procedure

token for in a currently unfinished constraint.

At each step of the search the beam atGrid[t][c] is filled with candidates which may becreated in three ways:

1. the open hypotheses in the beam to theleft (Grid[t − 1][c]) may generate con-tinuations from the model’s distributionpθ(yi|x, {y0 . . . yi−1}),

2. the open hypotheses in the beam to the leftand below (Grid[t−1][c−1]) may start newconstraints,

3. the closed hypotheses in the beam to the leftand below (Grid[t−1][c−1]) may continueconstraints.

Therefore, the model in Algorithm 1 imple-ments an interface with three functions: generate,

start, and continue, which build new hypothesesin each of the three ways. Note that the scoringfunction of the model does not need to be aware ofthe existence of constraints, but it may be, for ex-ample via a feature which indicates if a hypothesisis part of a constraint or not.

The beams at the top level of the grid (beamswhere c = numConstraints) contain hypothe-ses which cover all of the constraints. Once a hy-pothesis on the top level generates the EOS token,it can be added to the set of finished hypotheses.The highest scoring hypothesis in the set of fin-ished hypotheses is the best sequence which cov-ers all constraints.1

1Our implementation of GBS is available at https://github.com/chrishokamp/constrained_decoding

1538

3.1 Multi-token ConstraintsBy distinguishing between open and closed hy-potheses, we can allow for arbitrary multi-tokenphrases in the search. Thus, the set of constraintsfor a particular output may include both individ-ual tokens and phrases. Each hypothesis main-tains a coverage vector to ensure that constraintscannot be repeated in a search path – hypotheseswhich have already covered constrainti can onlygenerate, or start constraints that have not yetbeen covered.

Note also that discontinuous lexical constraints,such as phrasal verbs in English or German, areeasy to incorporate into GBS, by adding filters tothe search, which require that one or more con-ditions must be met before a constraint can beused. For example, adding the phrasal verb “ask〈someone〉 out” as a constraint would mean using“ask” as constraint0 and “out” as constraint1,with two filters: one requiring that constraint1cannot be used before constraint0, and anotherrequiring that there must be at least one generatedtoken between the constraints.

3.2 Subword UnitsBoth the computation of the score for a hypoth-esis, and the granularity of the tokens (character,subword, word, etc...) are left to the underlyingmodel. Because our decoder can handle arbitraryconstraints, there is a risk that constraints will con-tain tokens that were never observed in the trainingdata, and thus are unknown by the model. Espe-cially in domain adaptation scenarios, some user-specified constraints are very likely to contain un-seen tokens. Subword representations provide anelegant way to circumvent this problem, by break-ing unknown or rare tokens into character n-gramswhich are part of the model’s vocabulary (Sen-nrich et al., 2016; Wu et al., 2016). In the ex-periments in Section 4, we use this technique toensure that no input tokens are unknown, even ifa constraint contains words which never appearedin the training data.2

3.3 EfficiencyBecause the number of beams is multiplied by thenumber of constraints, the runtime complexity ofa naive implementation of GBS is O(ktc). Stan-dard time-based beam search is O(kt); therefore,

2If a character that was not observed in training data isobserved at prediction time, it will be unknown. However,we did not observe this in any of our experiments.

some consideration must be given to the efficiencyof this algorithm. Note that the beams in each col-umn c of Figure 3 are independent, meaning thatGBS can be parallelized to allow all beams at eachtimestep to be filled simultaneously. Also, we findthat the most time is spent computing the states forthe hypothesis candidates, so by keeping the beamsize small, we can make GBS significantly faster.

3.4 ModelsThe models used for our experiments are state-of-the-art Neural Machine Translation (NMT) sys-tems using our own implementation of NMT withattention over the source sequence (Bahdanauet al., 2014). We used Blocks and Fuel to im-plement our NMT models (van Merrinboer et al.,2015). To conduct the experiments in the fol-lowing section, we trained baseline translationmodels for English–German (EN-DE), English–French (EN-FR), and English–Portuguese (EN-PT). We created a shared subword representationfor each language pair by extracting a vocabularyof 80000 symbols from the concatenated sourceand target data. See the Appendix for more de-tails on our training data and hyperparameter con-figuration for each language pair. The beamSizeparameter is set to 10 for all experiments.

Because our experiments use NMT models, wecan now be more explicit about the implemen-tations of the generate, start, and continuefunctions for this GBS instantiation. For anNMT model at timestep t, generate(hypt−1) firstcomputes a vector of output probabilities ot =softmax(g(yt−1, si, ci))3 using the state infor-mation available from hypt−1. and returns the bestk continuations, i.e. Eq. 4:

gt = k-argmaxi

oti. (4)

The start and continue functions simply indexinto the softmax output of the model, selectingspecific tokens instead of doing a k-argmax overthe entire target language vocabulary. For exam-ple, to start constraint ci, we find the score of to-ken ci0 , i.e. otci0 .

4 Experiments

4.1 Pick-Revise for Interactive Post EditingPick-Revise is an interaction cycle for MT Post-Editing proposed by Cheng et al. (2016). Starting

3we use the notation for the g function from Bahdanauet al. (2014)

1539

ITERATION 0 1 2 3Strict ConstraintsEN-DE 18.44 27.64 (+9.20) 36.66 (+9.01) 43.92 (+7.26)EN-FR 28.07 36.71 (+8.64) 44.84 (+8.13) 45.48 +(0.63)EN-PT* 15.41 23.54 (+8.25) 31.14 (+7.60) 35.89 (+4.75)Relaxed ConstraintsEN-DE 18.44 26.43 (+7.98) 34.48 (+8.04) 41.82 (+7.34)EN-FR 28.07 33.8 (+5.72) 40.33 (+6.53) 47.0 (+6.67)EN-PT* 15.41 23.22 (+7.80) 33.82 (+10.6) 40.75 (+6.93)

Table 1: Results for four simulated editing cycles using WMT test data. EN-DE uses newstest2013, EN-FR uses newstest2014,and EN-PT uses the Autodesk corpus discussed in Section 4.2. Improvement in BLEU score over the previous cycle is shownin parentheses. * indicates use of our test corpus created from Autodesk post-editing data.

with the original translation hypothesis, a (sim-ulated) user first picks a part of the hypothesiswhich is incorrect, and then provides the correcttranslation for that portion of the output. The user-provided correction is then used as a constraint forthe next decoding cycle. The Pick-Revise processcan be repeated as many times as necessary, witha new constraint being added at each cycle.

We modify the experiments of Cheng et al.(2016) slightly, and assume that the user only pro-vides sequences of up to three words which aremissing from the hypothesis.4 To simulate userinteraction, at each iteration we chose a phraseof up to three tokens from the reference transla-tion which does not appear in the current MT hy-potheses. In the strict setting, the complete phrasemust be missing from the hypothesis. In the re-laxed setting, only the first word must be missing.Table 1 shows results for a simulated editing ses-sion with four cycles. When a three-token phrasecannot be found, we backoff to two-token phrases,then to single tokens as constraints. If a hypoth-esis already matches the reference, no constraintsare added. By specifying a new constraint of up tothree words at each cycle, an increase of over 20BLEU points is achieved in all language pairs.

4.2 Domain Adaptation via Terminology

The requirement for use of domain-specific termi-nologies is common in real-world applications ofMT (Crego et al., 2016). Existing approaches in-corporate placeholder tokens into NMT systems,which requires modifying the pre- and post- pro-cessing of the data, and training the system with

4NMT models do not use explicit alignment betweensource and target, so we cannot use alignment informationto map target phrases to source phrases

data that contains the same placeholders which oc-cur in the test data (Crego et al., 2016). The MTsystem also loses any possibility to model the to-kens in the terminology, since they are representedby abstract tokens such as “〈TERM 1〉”. An at-tractive alternative is to simply provide term map-pings as constraints, allowing any existing systemto adapt to the terminology used in a new test do-main.

For the target domain data, we use the AutodeskPost-Editing corpus (Zhechev, 2012), which is adataset collected from actual MT post-editing ses-sions. The corpus is focused upon software local-ization, a domain which is likely to be very dif-ferent from the WMT data used to train our gen-eral domain models. We divide the corpus into ap-proximately 100,000 training sentences, and 1000test segments, and automatically generate a termi-nology by computing the Pointwise Mutual Infor-mation (PMI) (Church and Hanks, 1990) betweensource and target n-grams in the training set. Weextract all n-grams from length 2-5 as terminologycandidates.

pmi(x;y) = logp(x, y)

p(x)p(y)(5)

npmi(x;y) =pmi(x;y)

h(x,y)(6)

Equations 5 and 6 show how we compute thenormalized PMI for a terminology candidate pair.The PMI score is normalized to the range [−1,+1]by dividing by the entropy h of the joint prob-ability p(x,y). We then filter the candidates toonly include pairs whose PMI is≥ 0.9, and whereboth the source and target phrases occur at leastfive times in the corpus. When source phrasesthat match the terminology are observed in the test

1540

data, the corresponding target phrase is added tothe constraints for that segment. Results are shownin Table 2.

As a sanity check that improvements in BLEUare not merely due to the presence of the termssomewhere in the output, i.e. that the placementof the terms by GBS is reasonable, we also eval-uate the results of randomly inserting terms intothe baseline output, and of prepending terms to thebaseline output.

This simple method of domain adaptation leadsto a significant improvement in the BLEU scorewithout any human intervention. Surprisingly,even an automatically created terminology com-bined with GBS yields performance improve-ments of approximately +2 BLEU points for En-De and En-Fr, and a gain of almost 14 pointsfor En-Pt. The large improvement for En-Pt isprobably due to the training data for this sys-tem being very different from the IT domain(see Appendix). Given the performance improve-ments from our automatically extracted terminol-ogy, manually created domain terminologies withgood coverage of the test domain are likely to leadto even greater gains. Using a terminology withGBS is likely to be beneficial in any setting wherethe test domain is significantly different from thedomain of the model’s original training data.

System BLEUEN-DEBaseline 26.17Random 25.18 (-0.99)Beginning 26.44 (+0.26)GBS 27.99 (+1.82)EN-FRBaseline 32.45Random 31.48 (-0.97)Beginning 34.51 (+2.05)GBS 35.05 (+2.59)EN-PTBaseline 15.41Random 18.26 (+2.85)Beginning 20.43 (+5.02)GBS 29.15 (+13.73)

Table 2: BLEU Results for EN-DE, EN-FR, and EN-PT ter-minology experiments using the Autodesk Post-Editing Cor-pus. ”Random’ indicates inserting terminology constraintsat random positions in the baseline translation. ”Beginning”indicates prepending constraints to baseline translations.

4.3 Analysis

Subjective analysis of decoder output shows thatphrases added as constraints are not only placedcorrectly within the output sequence, but also haveglobal effects upon translation quality. This is adesirable effect for user interaction, since it im-plies that users can bootstrap quality by adding themost critical constraints (i.e. those that are mostessential to the output), first. Table 3 shows severalexamples from the experiments in Table 1, wherethe addition of lexical constraints was able toguide our NMT systems away from initially quitelow-scoring hypotheses to outputs which perfectlymatch the reference translations.

5 Related Work

Most related work to date has presented modifica-tions of SMT systems for specific usecases whichconstrain MT output via auxilliary inputs. Thelargest body of work considers Interactive Ma-chine Translation (IMT): an MT system searchesfor the optimal target-language suffix given a com-plete source sentence and a desired prefix forthe target output (Foster, 2002; Barrachina et al.,2009; Green, 2014). IMT can be viewed as sub-case of constrained decoding, where there is onlyone constraint which is guaranteed to be placed atthe beginning of the output sequence. Wuebkeret al. (2016) introduce prefix-decoding, whichmodifies the SMT beam search to first ensure thatthe target prefix is covered, and only then contin-ues to build hypotheses for the suffix using beamsorganized by coverage of the remaining phrasesin the source segment. Wuebker et al. (2016) andKnowles and Koehn (2016) also present a simplemodification of NMT models for IMT, enablingmodels to predict suffixes for user-supplied pre-fixes.

Recently, some attention has also been given toSMT decoding with multiple lexical constraints.The Pick-Revise (PRIMT) (Cheng et al., 2016)framework for Interactive Post Editing introducesthe concept of edit cycles. Translators specify con-straints by editing a part of the MT output that isincorrect, and then asking the system for a newhypothesis, which must contain the user-providedcorrection. This process is repeated, maintain-ing constraints from previous iterations and addingnew ones as needed. Importantly, their approachrelies upon the phrase segmentation provided bythe SMT system. The decoding algorithm can

1541

EN-DESourceHe was also an anti- smoking activist and took part in several campaigns .Original HypothesisEs war auch ein Anti- Rauch- Aktiv- ist und nahmen an mehreren Kampagnen teil .Reference ConstraintsEbenso setzte er sich gegen das Rauchen ein und nahm an mehreren Kampagnen teil . (1) Ebenso setzte erConstrained Hypothesis (2) gegen das RauchenEbenso setzte er sich gegen das Rauchen ein und nahm an mehreren Kampagnen teil . (3) nahm

EN-FRSourceAt that point I was no longer afraid of him and I was able to love him .Original HypothesisJe n’avais plus peur de lui et j’etais capable de l’aimer .Reference ConstraintsLa je n’ai plus eu peur de lui et j’ai pu l’aimer . (1) La je n’aiConstrained Hypothesis (2) j’ai puLa je n’ai plus eu peur de lui et j’ai pu l’aimer . (3) eu

EN-PTSourceMo- dif- y drain- age features by selecting them individually .Original Hypothesis- Ja temos as caracterısticas de extraccao de idade , com eles individualmente .Reference ConstraintsModi- fique os recursos de drenagem ao selec- ion- a-los individualmente . (1) drenagem ao selec-Constrained Hypothesis (2) Modi- fique osModi- fique os recursos de drenagem ao selec- ion- a-los individualmente . (3) recursos

Table 3: Manual analysis of examples from lexically constrained decoding experiments. “-” followed by whitespace indicatesthe internal segmentation of the translation model (see Section 3.2)

only make use of constraints that match phraseboundaries, because constraints are implementedas “rules” enforcing that source phrases must betranslated as the aligned target phrases that havebeen selected as constraints. In contrast, our ap-proach decodes at the token level, and is not de-pendent upon any explicit structure in the underly-ing model.

Domingo et al. (2016) also consider an interac-tive scenario where users first choose portions ofan MT hypothesis to keep, then query for an up-dated translation which preserves these portions.The MT system decodes the source phrases whichare not aligned to the user-selected phrases un-til the source sentence is fully covered. This ap-proach is similar to the system of Cheng et al., anduses the “XML input” feature in Moses (Koehnet al., 2007).

Some recent work considers the inclusion ofsoft lexical constraints directly into deep modelsfor dialog generation, and special cases, such asrecipe generation from a list of ingredients (Wenet al., 2015; Kiddon et al., 2016). Such constraint-aware models are complementary to our work, andcould be used with GBS decoding without anychange to the underlying models.

To the best of our knowledge, ours is thefirst work which considers general lexically con-strained decoding for any model which outputssequences, without relying upon alignments be-tween input and output, and without using a search

organized by coverage of the input.

6 Conclusion

Lexically constrained decoding is a flexible wayto incorporate arbitrary subsequences into the out-put of any model that generates output sequencestoken-by-token. A wide spectrum of popular textgeneration models have this characteristic, andGBS should be straightforward to use with anymodel that already uses beam search.

In translation interfaces where translators canprovide corrections to an existing hypothesis,these user inputs can be used as constraints, gener-ating a new output each time a user fixes an error.By simulating this scenario, we have shown thatsuch a workflow can provide a large improvementin translation quality at each iteration.

By using a domain-specific terminology to gen-erate target-side constraints, we have shown thata general domain model can be adapted to a newdomain without any retraining. Surprisingly, thissimple method can lead to significant performancegains, even when the terminology is created auto-matically.

In future work, we hope to evaluate GBS withmodels outside of MT, such as automatic sum-marization, image captioning or dialog genera-tion. We also hope to introduce new constraint-aware models, for example via secondary attentionmechanisms over lexical constraints.

1542

Acknowledgments

This project has received funding from ScienceFoundation Ireland in the ADAPT Centre for Dig-ital Content Technology (www.adaptcentre.ie) atDublin City University funded under the SFI Re-search Centres Programme (Grant 13/RC/2106)co-funded under the European Regional Develop-ment Fund and the European Union Horizon 2020research and innovation programme under grantagreement 645452 (QT21). We thank the anony-mous reviewers, as well as Iacer Calixto, PeymanPassban, and Henry Elder for helpful feedback onearly versions of this work.

ReferencesDzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben-

gio. 2014. Neural machine translation by jointlylearning to align and translate. arXiv preprintarXiv:1409.0473 .

Sergio Barrachina, Oliver Bender, Francisco Casacu-berta, Jorge Civera, Elsa Cubel, Shahram Khadivi,Antonio Lagarda, Hermann Ney, Jesus Tomas, En-rique Vidal, and Juan-Miguel Vilar. 2009. Sta-tistical approaches to computer-assisted transla-tion. Computational Linguistics 35(1):3–28.https://doi.org/10.1162/coli.2008.07-055-R2-06-29.

Ondrej Bojar, Rajen Chatterjee, Christian Federmann,Barry Haddow, Matthias Huck, Chris Hokamp,Philipp Koehn, Varvara Logacheva, Christof Monz,Matteo Negri, Matt Post, Carolina Scarton, LuciaSpecia, and Marco Turchi. 2015. Findings of the2015 workshop on statistical machine translation.In Proceedings of the Tenth Workshop on Statisti-cal Machine Translation. Association for Compu-tational Linguistics, Lisbon, Portugal, pages 1–46.http://aclweb.org/anthology/W15-3001.

Shanbo Cheng, Shujian Huang, Huadong Chen, XinyuDai, and Jiajun Chen. 2016. PRIMT: A pick-revise framework for interactive machine trans-lation. In NAACL HLT 2016, The 2016 Con-ference of the North American Chapter of theAssociation for Computational Linguistics: Hu-man Language Technologies, San Diego Califor-nia, USA, June 12-17, 2016. pages 1240–1249.http://aclweb.org/anthology/N/N16/N16-1148.pdf.

David Chiang. 2007. Hierarchical phrase-basedtranslation. Comput. Linguist. 33(2):201–228.https://doi.org/10.1162/coli.2007.33.2.201.

Kyunghyun Cho, Bart van Merrienboer, CalarGulcehre, Dzmitry Bahdanau, Fethi Bougares, Hol-ger Schwenk, and Yoshua Bengio. 2014. Learningphrase representations using rnn encoder–decoderfor statistical machine translation. In Proceedings of

the 2014 Conference on Empirical Methods in Nat-ural Language Processing (EMNLP). Associationfor Computational Linguistics, Doha, Qatar, pages1724–1734. http://www.aclweb.org/anthology/D14-1179.

Kenneth Ward Church and Patrick Hanks. 1990.Word association norms, mutual information, andlexicography. Comput. Linguist. 16(1):22–29.http://dl.acm.org/citation.cfm?id=89086.89095.

Josep Maria Crego, Jungi Kim, Guillaume Klein, An-abel Rebollo, Kathy Yang, Jean Senellart, EgorAkhanov, Patrice Brunelle, Aurelien Coquard,Yongchao Deng, Satoshi Enoue, Chiyo Geiss,Joshua Johanson, Ardas Khalsa, Raoum Khiari,Byeongil Ko, Catherine Kobus, Jean Lorieux, Leid-iana Martins, Dang-Chuan Nguyen, Alexandra Pri-ori, Thomas Riccardi, Natalia Segal, Christophe Ser-van, Cyril Tiquet, Bo Wang, Jin Yang, Dakun Zhang,Jing Zhou, and Peter Zoldan. 2016. Systran’spure neural machine translation systems. CoRRabs/1610.05540. http://arxiv.org/abs/1610.05540.

Miguel Domingo, Alvaro Peris, and Francisco Casacu-berta. 2016. Interactive-predictive translation basedon multiple word-segments. Baltic J. Modern Com-puting 4(2):282–291.

George F. Foster. 2002. Text Prediction for Transla-tors. Ph.D. thesis, Montreal, P.Q., Canada, Canada.AAINQ72434.

Spence Green. 2014. Mixed-Initiative Natural Lan-guage Translation. Ph.D. thesis, Stanford, CA,United States.

Chloe Kiddon, Luke Zettlemoyer, and Yejin Choi.2016. Globally coherent text generation withneural checklist models. In Proceedings of the2016 Conference on Empirical Methods in Natu-ral Language Processing, EMNLP 2016, Austin,Texas, USA, November 1-4, 2016. pages 329–339.http://aclweb.org/anthology/D/D16/D16-1032.pdf.

Rebecca Knowles and Philipp Koehn. 2016. Neuralinteractive translation prediction. AMTA 2016, Vol.page 107.

Philipp Koehn. 2009. A process study of computer-aided translation. Machine Translation 23(4):241–263. https://doi.org/10.1007/s10590-010-9076-3.

Philipp Koehn. 2010. Statistical Machine Translation.Cambridge University Press, New York, NY, USA,1st edition.

Philipp Koehn, Hieu Hoang, Alexandra Birch, ChrisCallison-Burch, Marcello Federico, Nicola Bertoldi,Brooke Cowan, Wade Shen, Christine Moran,Richard Zens, Chris Dyer, Ondrej Bojar, AlexandraConstantin, and Evan Herbst. 2007. Moses: Opensource toolkit for statistical machine translation.In Proceedings of the 45th Annual Meeting of theACL on Interactive Poster and Demonstration Ses-sions. Association for Computational Linguistics,

1543

Stroudsburg, PA, USA, ACL ’07, pages 177–180.http://dl.acm.org/citation.cfm?id=1557769.1557821.

Franz Josef Och and Hermann Ney. 2004. Thealignment template approach to statistical machinetranslation. Comput. Linguist. 30(4):417–449.https://doi.org/10.1162/0891201042544884.

Judea Pearl. 1984. Heuristics: Intelligent SearchStrategies for Computer Problem Solving. Addison-Wesley Longman Publishing Co., Inc., Boston, MA,USA.

Alexander Rush, Yin-Wen Chang, and MichaelCollins. 2013. Optimal beam search for machinetranslation. In Proceedings of the 2013 Confer-ence on Empirical Methods in Natural LanguageProcessing. Association for Computational Linguis-tics, Seattle, Washington, USA, pages 210–221.http://www.aclweb.org/anthology/D13-1022.

Alexander M. Rush, Sumit Chopra, and Jason We-ston. 2015. A neural attention model for abstrac-tive sentence summarization. In Llus Mrquez, ChrisCallison-Burch, Jian Su, Daniele Pighin, and YuvalMarton, editors, EMNLP. The Association for Com-putational Linguistics, pages 379–389.

Rico Sennrich, Barry Haddow, and Alexandra Birch.2016. Neural machine translation of rare wordswith subword units. In Proceedings of the54th Annual Meeting of the Association for Com-putational Linguistics, ACL 2016, August 7-12,2016, Berlin, Germany, Volume 1: Long Papers.http://aclweb.org/anthology/P/P16/P16-1162.pdf.

Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio,Aaron Courville, and Joelle Pineau. 2016. Buildingend-to-end dialogue systems using generativehierarchical neural network models. In Proceedingsof the Thirtieth AAAI Conference on Artificial Intel-ligence. AAAI Press, AAAI’16, pages 3776–3783.http://dl.acm.org/citation.cfm?id=3016387.3016435.

Jason R. Smith, Herve Saint-amand, Chris Callison-burch, Magdalena Plamada, and Adam Lopez. 2013.Dirt cheap web-scale parallel text from the commoncrawl. In In Proceedings of the Conference of theAssociation for Computational Linguistics (ACL.

Lucia Specia. 2011. Exploiting objective annotationsfor measuring translation post-editing effort. In Pro-ceedings of the European Association for MachineTranslation. May.

Ralf Steinberger, Bruno Pouliquen, Anna Widiger,Camelia Ignat, Toma Erjavec, and Dan Tufi. 2006.The jrc-acquis: A multilingual aligned parallel cor-pus with 20+ languages. In In Proceedings of the 5thInternational Conference on Language Resourcesand Evaluation (LREC’2006. pages 2142–2147.

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le.2014. Sequence to sequence learning withneural networks. In Proceedings of the 27th

International Conference on Neural Informa-tion Processing Systems. MIT Press, Cam-bridge, MA, USA, NIPS’14, pages 3104–3112.http://dl.acm.org/citation.cfm?id=2969033.2969173.

Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu,and Hang Li. 2016. Neural machine translation withreconstruction. arXiv preprint arXiv:1611.01874 .

Bart van Merrinboer, Dzmitry Bahdanau, Vincent Du-moulin, Dmitriy Serdyuk, David Warde-Farley, JanChorowski, and Yoshua Bengio. 2015. Blocksand fuel: Frameworks for deep learning. CoRRabs/1506.00619.

Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. 2015.Semantically conditioned lstm-based natural lan-guage generation for spoken dialogue systems.In Proceedings of the 2015 Conference on Em-pirical Methods in Natural Language Processing(EMNLP). Association for Computational Linguis-tics.

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V.Le, Mohammad Norouzi, Wolfgang Macherey,Maxim Krikun, Yuan Cao, Qin Gao, KlausMacherey, Jeff Klingner, Apurva Shah, MelvinJohnson, Xiaobing Liu, ukasz Kaiser, StephanGouws, Yoshikiyo Kato, Taku Kudo, HidetoKazawa, Keith Stevens, George Kurian, NishantPatil, Wei Wang, Cliff Young, Jason Smith, JasonRiesa, Alex Rudnick, Oriol Vinyals, Greg Corrado,Macduff Hughes, and Jeffrey Dean. 2016. Google’sneural machine translation system: Bridging the gapbetween human and machine translation. CoRRabs/1609.08144. http://arxiv.org/abs/1609.08144.

Joern Wuebker, Spence Green, John DeNero, SasaHasan, and Minh-Thang Luong. 2016. Modelsand inference for prefix-constrained machine trans-lation. In Proceedings of the 54th Annual Meet-ing of the Association for Computational Linguistics(Volume 1: Long Papers). Association for Compu-tational Linguistics, Berlin, Germany, pages 66–75.http://www.aclweb.org/anthology/P16-1007.

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho,Aaron Courville, Ruslan Salakhudinov, Rich Zemel,and Yoshua Bengio. 2015. Show, attend and tell:Neural image caption generation with visual atten-tion. In David Blei and Francis Bach, editors,Proceedings of the 32nd International Conferenceon Machine Learning (ICML-15). JMLR Workshopand Conference Proceedings, pages 2048–2057.http://jmlr.org/proceedings/papers/v37/xuc15.pdf.

Matthew D. Zeiler. 2012. ADADELTA: an adap-tive learning rate method. CoRR abs/1212.5701.http://arxiv.org/abs/1212.5701.

Ventsislav Zhechev. 2012. Machine Translation Infras-tructure and Post-editing Performance at Autodesk.

1544

In AMTA 2012 Workshop on Post-Editing Technol-ogy and Practice (WPTP 2012). Association for Ma-chine Translation in the Americas (AMTA), SanDiego, USA, pages 87–96.

1545

A NMT System Configurations

We train all systems for 500000 iterations, withvalidation every 5000 steps. The best single modelfrom validation is used in all of the experiments fora language pair. We use `2 regularization on all pa-rameters with α = 1e−5. Dropout is used on theoutput layers with p(drop) = 0.5. We sort mini-batches by source sentence length, and reshuffletraining data after each epoch.

All systems use a bidirectional GRUs (Choet al., 2014) to create the source representationand GRUs for the decoder transition. We useAdaDelta (Zeiler, 2012) to update gradients, andclip large gradients to 1.0.

Training ConfigurationsEN-DEEmbedding Size 300Recurrent Layers Size 1000Source Vocab Size 80000Target Vocab Size 90000Batch Size 50EN-FREmbedding Size 300Recurrent Layers Size 1000Source Vocab Size 66000Target Vocab Size 74000Batch Size 40EN-PTEmbedding Size 200Recurrent Layers Size 800Source Vocab Size 60000Target Vocab Size 74000Batch Size 40

A.1 English-GermanOur English-German training corpus consists of4.4 Million segments from the Europarl (Bojaret al., 2015) and CommonCrawl (Smith et al.,2013) corpora.

A.2 English-FrenchOur English-French training corpus consists of 4.9Million segments from the Europarl and Com-monCrawl corpora.

A.3 English-PortugueseOur English-Portuguese training corpus consistsof 28.5 Million segments from the Europarl, JRC-

Aquis (Steinberger et al., 2006) and OpenSubti-tles5 corpora.

5http://www.opensubtitles.org/

1546