The SIGMORPHON 2016 shared task— morphological...
Transcript of The SIGMORPHON 2016 shared task— morphological...
![Page 1: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/1.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
The SIGMORPHON 2016 shared task—
morphological reinflection
Ryan Cotterell, Christo Kirov, John Sylak-Glassman,David Yarowsky, Jason Eisner, Mans Hulden
1
![Page 2: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/2.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Shared task
‣ SIGMORPHON’s first shared task!‣ First shared task on supervised learning of
(inflectional) morphology‣ featuring …
• 3 tasks• 3 “tracks”• 10 languages• 9 systems submitted
2
![Page 3: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/3.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Shared task
‣ Tasks [MH]‣ Language data [CK]‣ Systems overview & results [RC]
3
Overview
![Page 4: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/4.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Shared task
‣ 1 Inflection (synthesis/generation)‣ 2 Reinflection (analysis + synthesis)‣ 3 Unlabeled Reinflection
4
Tasks
![Page 5: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/5.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 1(inflection)
5
train
test
lemma MSD (feature/value pairs) word form
![Page 6: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/6.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection 6
train
run lemma MSD (feature/value pairs) word form
test
Task 1(inflection)
![Page 7: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/7.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection 7
train
run pos=V,mood=IND,tense=PST,per=3,num=SG lemma MSD (feature/value pairs) word form
test
Task 1(inflection)
![Page 8: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/8.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection 8
train
run pos=V,mood=IND,tense=PST,per=3,num=SG ran lemma MSD (feature/value pairs) word form
test
Task 1(inflection)
![Page 9: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/9.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection 9
train
test
run pos=V,mood=IND,tense=PST,per=3,num=SG ran love pos=V,tense=PRS loving eat pos=V,mood=IND,tense=PST,per=1,num=SG ate
…
lemma MSD (feature/value pairs) word form
Task 1(inflection)
![Page 10: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/10.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection 10
train
test
run pos=V,mood=IND,tense=PST,per=3,num=SG ran love pos=V,tense=PRS loving eat pos=V,mood=IND,tense=PST,per=1,num=SG ate
…
lemma MSD (feature/value pairs) word form
hate pos=V,tense=PRS ?
Task 1(inflection)
![Page 11: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/11.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection 11
train
test
run pos=V,mood=IND,tense=PST,per=3,num=SG ran love pos=V,tense=PRS loving eat pos=V,mood=IND,tense=PST,per=1,num=SG ate
…
lemma MSD (feature/value pairs) word form
hate pos=V,tense=PRS ? read pos=V,mood=IND,tense=PST,per=3,num=SG ?
Task 1(inflection)
![Page 12: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/12.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection 12
train
test
run pos=V,mood=IND,tense=PST,per=3,num=SG ran love pos=V,tense=PRS loving eat pos=V,mood=IND,tense=PST,per=1,num=SG ate
…
lemma MSD (feature/value pairs) word form
hate pos=V,tense=PRS hating read pos=V,mood=IND,tense=PST,per=3,num=SG read
Task 1(inflection)
![Page 13: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/13.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Training data
13
![Page 14: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/14.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Training data
14
![Page 15: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/15.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Training data
15
schreiben pos=V,mood={OPT/SBJV},tense=PRS,per=1,num=PL schreiben
![Page 16: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/16.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 2 (reinflection)
16
train
test
MSD1 form1 MSD2 form2
![Page 17: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/17.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 2 (reinflection)
17
train
test
pos=V,tense=PRS running pos=V,tense=PST ran
MSD1 form1 MSD2 form2
![Page 18: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/18.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 2 (reinflection)
18
train
test
pos=V,tense=PRS running pos=V,tense=PST ran
MSD1 form1 MSD2 form2
pos=V,tense=PST sought pos=V,tense=INF
…
![Page 19: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/19.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 2 (reinflection)
19
train
test
pos=V,tense=PRS running pos=V,tense=PST ran
MSD1 form1 MSD2 form2
pos=V,tense=PST sought pos=V,tense=INF ?
…
…
![Page 20: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/20.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 2 (reinflection)
20
train
test
pos=V,tense=PRS running pos=V,tense=PST ran
MSD1 form1 MSD2 form2
pos=V,tense=PST sought pos=V,tense=INF seek
…
…
![Page 21: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/21.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 3 (unlabeled reinflection)
21
train
test
pos=V,tense=PRS running pos=V,tense=PST ran
MSD1 form1 MSD2 form2
pos=V,tense=PST sought pos=V,tense=INF seek
…
…
![Page 22: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/22.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 3 (unlabeled reinflection)
22
train
test
pos=V,tense=PRS running pos=V,tense=PST ran
MSD1 form1 MSD2 form2
pos=V,tense=PST sought pos=V,tense=INF seek
…
…
![Page 23: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/23.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Task 3 (unlabeled reinflection)
23
train
test
running pos=V,tense=PST ran
form1 MSD2 form2
sought pos=V,tense=INF seek
…
…
![Page 24: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/24.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
24
![Page 25: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/25.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
25
auto
FinnishTask 1
Lemma > inflection
![Page 26: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/26.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
26
auto
FinnishTask 1
Lemma > inflection
![Page 27: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/27.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
27
auto
FinnishTask 2
inflection > inflection
![Page 28: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/28.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
28
auto
FinnishTask 2
inflection > inflection
![Page 29: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/29.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
29
auto
FinnishTask 3
autona?
unk > inflection
![Page 30: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/30.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
30
auto
FinnishTask 3
unk > inflection
autona?
![Page 31: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/31.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
31
auto
FinnishTask 3
unk > inflection
autona
![Page 32: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/32.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
32
auto
FinnishTask 2
(reduction)
inflection > inflection
![Page 33: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/33.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
33
auto
FinnishTask 2
(reduction)
inflection > inflection
![Page 34: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/34.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
34
auto
FinnishTask 2
(reduction)
inflection > inflection
![Page 35: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/35.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
35
auto
Finnish
Task 3(reduction)
unk > inflection
autona
![Page 36: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/36.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
36
auto
Finnish
Task 3(reduction)
unk > inflection
autona
![Page 37: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/37.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
37
auto
Finnish
Task 3(reduction)
unk > inflection
autona
![Page 38: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/38.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Summary of tasks
38
auto
Finnish
Task 3(reduction)
unk > inflection
autona
![Page 39: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/39.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Tracks
39
forms, the probability of encountering any sin-gle word form decreases, reducing the effective-ness of frequency-based techniques in performingtasks like word alignment and language modeling(Koehn, 2010; Duh and Kirchhoff, 2004). Tech-niques like lemmatization and stemming can ame-liorate data sparsity (Goldwater and McClosky,2005), but these rely on morphological knowl-edge, particularly the mapping from inflectedforms to lemmas and the list of morphs togetherwith their ordering. Developing systems that canaccurately learn and capture these mappings, overtaffixes, and the principles that govern how thoseaffixes combine is crucial to maximizing the cross-linguistic capabilities of most human languagetechnology.
The goal of the 2016 SIGMORPHON SharedTask2 was to spur the development of systemsthat could accurately generate morphologically in-flected words for a set of 10 languages based on arange of training parameters. These 10 languagesincluded low resource languages with diverse mor-phological characteristics, and the training param-eters reflected a significant expansion upon the tra-ditional task of predicting a full paradigm froma lemma. Of the systems submitted, the neu-ral network-based systems performed best, clearlydemonstrating the effectiveness of recurrent neu-ral networks (RNNs) for morphological genera-tion and analysis.
We are releasing the shared task data and evalu-ation scripts for use in future research.
2 Tasks, Tracks, and Evaluation
Up to the present, the task of morphological in-flection has been narrowly defined as the gener-ation of a complete inflectional paradigm from alemma, based on training from a corpus of com-plete paradigms.3 This task implicitly assumes theavailability of a traditional dictionary or gazetteer,does not require explicit morphological analysis,and, though it mimics a common task in secondlanguage (L2) pedagogy, it is not a realistic learn-ing setting for first language (L1) acquisition.
Systems developed for the 2016 Shared Taskhad to carry out reinflection of an already inflectedform. This involved analysis of an already in-
2Official website: http://ryancotterell.
github.io/sigmorphon2016/
3A paradigm is defined here as the set of inflected wordforms associated with a single lemma (or lexeme), for exam-ple, a noun declension or verb conjugation.
Task 1 Task 2 Task 3Lemma run — —Source tag — PAST —Source form — ran ran
Target tag PRESPART PRESPART PRESPARTTarget form running running running
Lemma decir — —Source tag — PRESENT1S —Source form — digo digo
Target tag FUTURE2S FUTURE2S FUTURE2STarget form dir
´
as dir
´
as dir
´
as
Table 1: Systems were required to generate the target form,given the information above the line. Two examples areshown for each task—one in English and one in Spanish.Task 1 is inflection; tasks 2–3 are reinflection.
Restricted Standard BonusTask 1 1 1 1, MTask 2 2 1, 2 1, 2, MTask 3 3 1, 2, 3 1, 2, 3, M
Table 2: Datasets that were permitted for each task undereach condition. Numbers indicate a dataset from that respec-tive task, e.g. ‘1’ is the dataset from Task 1, and ‘M’ indicatesbonus monolingual text from Wikipedia dumps.
flected word form, together with synthesis of a dif-ferent inflection of that form. The systems hadto learn from limited data: they were not givencomplete paradigms to train on, nor a dictionaryof lemmas.
Specifically, systems competed on the threetasks illustrated in Table 1, of increasing difficulty.Notice that each task can be regarded as mapping asource string to a target string, with other input ar-guments (such as the target tag) that specify whichversion of the mapping is desired.
For each language and each task, participantswere provided with supervised training data: acollection of input tuples, each paired with the cor-rect output string (target form).
Each system could compete on a task under anyof three tracks (Table 2). Under the restrictedtrack, only data for that task could be used, whilefor the standard track, data from that task and anyfrom a lower task could be used. The bonus trackwas the same as the standard track, but allowed theuse of monolingual data in the form of Wikipediadumps from 2 November 2015.4
Each system was required to produce, for ev-ery input given at test time, either a single stringor a ranked list of up to 20 predicted strings foreach task. Systems were compared on the follow-
4https://dumps.wikimedia.org/
backup-index.html
11
![Page 40: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/40.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Tracks
40
forms, the probability of encountering any sin-gle word form decreases, reducing the effective-ness of frequency-based techniques in performingtasks like word alignment and language modeling(Koehn, 2010; Duh and Kirchhoff, 2004). Tech-niques like lemmatization and stemming can ame-liorate data sparsity (Goldwater and McClosky,2005), but these rely on morphological knowl-edge, particularly the mapping from inflectedforms to lemmas and the list of morphs togetherwith their ordering. Developing systems that canaccurately learn and capture these mappings, overtaffixes, and the principles that govern how thoseaffixes combine is crucial to maximizing the cross-linguistic capabilities of most human languagetechnology.
The goal of the 2016 SIGMORPHON SharedTask2 was to spur the development of systemsthat could accurately generate morphologically in-flected words for a set of 10 languages based on arange of training parameters. These 10 languagesincluded low resource languages with diverse mor-phological characteristics, and the training param-eters reflected a significant expansion upon the tra-ditional task of predicting a full paradigm froma lemma. Of the systems submitted, the neu-ral network-based systems performed best, clearlydemonstrating the effectiveness of recurrent neu-ral networks (RNNs) for morphological genera-tion and analysis.
We are releasing the shared task data and evalu-ation scripts for use in future research.
2 Tasks, Tracks, and Evaluation
Up to the present, the task of morphological in-flection has been narrowly defined as the gener-ation of a complete inflectional paradigm from alemma, based on training from a corpus of com-plete paradigms.3 This task implicitly assumes theavailability of a traditional dictionary or gazetteer,does not require explicit morphological analysis,and, though it mimics a common task in secondlanguage (L2) pedagogy, it is not a realistic learn-ing setting for first language (L1) acquisition.
Systems developed for the 2016 Shared Taskhad to carry out reinflection of an already inflectedform. This involved analysis of an already in-
2Official website: http://ryancotterell.
github.io/sigmorphon2016/
3A paradigm is defined here as the set of inflected wordforms associated with a single lemma (or lexeme), for exam-ple, a noun declension or verb conjugation.
Task 1 Task 2 Task 3Lemma run — —Source tag — PAST —Source form — ran ran
Target tag PRESPART PRESPART PRESPARTTarget form running running running
Lemma decir — —Source tag — PRESENT1S —Source form — digo digo
Target tag FUTURE2S FUTURE2S FUTURE2STarget form dir
´
as dir
´
as dir
´
as
Table 1: Systems were required to generate the target form,given the information above the line. Two examples areshown for each task—one in English and one in Spanish.Task 1 is inflection; tasks 2–3 are reinflection.
Restricted Standard BonusTask 1 1 1 1, MTask 2 2 1, 2 1, 2, MTask 3 3 1, 2, 3 1, 2, 3, M
Table 2: Datasets that were permitted for each task undereach condition. Numbers indicate a dataset from that respec-tive task, e.g. ‘1’ is the dataset from Task 1, and ‘M’ indicatesbonus monolingual text from Wikipedia dumps.
flected word form, together with synthesis of a dif-ferent inflection of that form. The systems hadto learn from limited data: they were not givencomplete paradigms to train on, nor a dictionaryof lemmas.
Specifically, systems competed on the threetasks illustrated in Table 1, of increasing difficulty.Notice that each task can be regarded as mapping asource string to a target string, with other input ar-guments (such as the target tag) that specify whichversion of the mapping is desired.
For each language and each task, participantswere provided with supervised training data: acollection of input tuples, each paired with the cor-rect output string (target form).
Each system could compete on a task under anyof three tracks (Table 2). Under the restrictedtrack, only data for that task could be used, whilefor the standard track, data from that task and anyfrom a lower task could be used. The bonus trackwas the same as the standard track, but allowed theuse of monolingual data in the form of Wikipediadumps from 2 November 2015.4
Each system was required to produce, for ev-ery input given at test time, either a single stringor a ranked list of up to 20 predicted strings foreach task. Systems were compared on the follow-
4https://dumps.wikimedia.org/
backup-index.html
11
can reduce
![Page 41: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/41.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Tracks
41
forms, the probability of encountering any sin-gle word form decreases, reducing the effective-ness of frequency-based techniques in performingtasks like word alignment and language modeling(Koehn, 2010; Duh and Kirchhoff, 2004). Tech-niques like lemmatization and stemming can ame-liorate data sparsity (Goldwater and McClosky,2005), but these rely on morphological knowl-edge, particularly the mapping from inflectedforms to lemmas and the list of morphs togetherwith their ordering. Developing systems that canaccurately learn and capture these mappings, overtaffixes, and the principles that govern how thoseaffixes combine is crucial to maximizing the cross-linguistic capabilities of most human languagetechnology.
The goal of the 2016 SIGMORPHON SharedTask2 was to spur the development of systemsthat could accurately generate morphologically in-flected words for a set of 10 languages based on arange of training parameters. These 10 languagesincluded low resource languages with diverse mor-phological characteristics, and the training param-eters reflected a significant expansion upon the tra-ditional task of predicting a full paradigm froma lemma. Of the systems submitted, the neu-ral network-based systems performed best, clearlydemonstrating the effectiveness of recurrent neu-ral networks (RNNs) for morphological genera-tion and analysis.
We are releasing the shared task data and evalu-ation scripts for use in future research.
2 Tasks, Tracks, and Evaluation
Up to the present, the task of morphological in-flection has been narrowly defined as the gener-ation of a complete inflectional paradigm from alemma, based on training from a corpus of com-plete paradigms.3 This task implicitly assumes theavailability of a traditional dictionary or gazetteer,does not require explicit morphological analysis,and, though it mimics a common task in secondlanguage (L2) pedagogy, it is not a realistic learn-ing setting for first language (L1) acquisition.
Systems developed for the 2016 Shared Taskhad to carry out reinflection of an already inflectedform. This involved analysis of an already in-
2Official website: http://ryancotterell.
github.io/sigmorphon2016/
3A paradigm is defined here as the set of inflected wordforms associated with a single lemma (or lexeme), for exam-ple, a noun declension or verb conjugation.
Task 1 Task 2 Task 3Lemma run — —Source tag — PAST —Source form — ran ran
Target tag PRESPART PRESPART PRESPARTTarget form running running running
Lemma decir — —Source tag — PRESENT1S —Source form — digo digo
Target tag FUTURE2S FUTURE2S FUTURE2STarget form dir
´
as dir
´
as dir
´
as
Table 1: Systems were required to generate the target form,given the information above the line. Two examples areshown for each task—one in English and one in Spanish.Task 1 is inflection; tasks 2–3 are reinflection.
Restricted Standard BonusTask 1 1 1 1, MTask 2 2 1, 2 1, 2, MTask 3 3 1, 2, 3 1, 2, 3, M
Table 2: Datasets that were permitted for each task undereach condition. Numbers indicate a dataset from that respec-tive task, e.g. ‘1’ is the dataset from Task 1, and ‘M’ indicatesbonus monolingual text from Wikipedia dumps.
flected word form, together with synthesis of a dif-ferent inflection of that form. The systems hadto learn from limited data: they were not givencomplete paradigms to train on, nor a dictionaryof lemmas.
Specifically, systems competed on the threetasks illustrated in Table 1, of increasing difficulty.Notice that each task can be regarded as mapping asource string to a target string, with other input ar-guments (such as the target tag) that specify whichversion of the mapping is desired.
For each language and each task, participantswere provided with supervised training data: acollection of input tuples, each paired with the cor-rect output string (target form).
Each system could compete on a task under anyof three tracks (Table 2). Under the restrictedtrack, only data for that task could be used, whilefor the standard track, data from that task and anyfrom a lower task could be used. The bonus trackwas the same as the standard track, but allowed theuse of monolingual data in the form of Wikipediadumps from 2 November 2015.4
Each system was required to produce, for ev-ery input given at test time, either a single stringor a ranked list of up to 20 predicted strings foreach task. Systems were compared on the follow-
4https://dumps.wikimedia.org/
backup-index.html
11
can’t reduce can reduce
![Page 42: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/42.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Tracks
42
forms, the probability of encountering any sin-gle word form decreases, reducing the effective-ness of frequency-based techniques in performingtasks like word alignment and language modeling(Koehn, 2010; Duh and Kirchhoff, 2004). Tech-niques like lemmatization and stemming can ame-liorate data sparsity (Goldwater and McClosky,2005), but these rely on morphological knowl-edge, particularly the mapping from inflectedforms to lemmas and the list of morphs togetherwith their ordering. Developing systems that canaccurately learn and capture these mappings, overtaffixes, and the principles that govern how thoseaffixes combine is crucial to maximizing the cross-linguistic capabilities of most human languagetechnology.
The goal of the 2016 SIGMORPHON SharedTask2 was to spur the development of systemsthat could accurately generate morphologically in-flected words for a set of 10 languages based on arange of training parameters. These 10 languagesincluded low resource languages with diverse mor-phological characteristics, and the training param-eters reflected a significant expansion upon the tra-ditional task of predicting a full paradigm froma lemma. Of the systems submitted, the neu-ral network-based systems performed best, clearlydemonstrating the effectiveness of recurrent neu-ral networks (RNNs) for morphological genera-tion and analysis.
We are releasing the shared task data and evalu-ation scripts for use in future research.
2 Tasks, Tracks, and Evaluation
Up to the present, the task of morphological in-flection has been narrowly defined as the gener-ation of a complete inflectional paradigm from alemma, based on training from a corpus of com-plete paradigms.3 This task implicitly assumes theavailability of a traditional dictionary or gazetteer,does not require explicit morphological analysis,and, though it mimics a common task in secondlanguage (L2) pedagogy, it is not a realistic learn-ing setting for first language (L1) acquisition.
Systems developed for the 2016 Shared Taskhad to carry out reinflection of an already inflectedform. This involved analysis of an already in-
2Official website: http://ryancotterell.
github.io/sigmorphon2016/
3A paradigm is defined here as the set of inflected wordforms associated with a single lemma (or lexeme), for exam-ple, a noun declension or verb conjugation.
Task 1 Task 2 Task 3Lemma run — —Source tag — PAST —Source form — ran ran
Target tag PRESPART PRESPART PRESPARTTarget form running running running
Lemma decir — —Source tag — PRESENT1S —Source form — digo digo
Target tag FUTURE2S FUTURE2S FUTURE2STarget form dir
´
as dir
´
as dir
´
as
Table 1: Systems were required to generate the target form,given the information above the line. Two examples areshown for each task—one in English and one in Spanish.Task 1 is inflection; tasks 2–3 are reinflection.
Restricted Standard BonusTask 1 1 1 1, MTask 2 2 1, 2 1, 2, MTask 3 3 1, 2, 3 1, 2, 3, M
Table 2: Datasets that were permitted for each task undereach condition. Numbers indicate a dataset from that respec-tive task, e.g. ‘1’ is the dataset from Task 1, and ‘M’ indicatesbonus monolingual text from Wikipedia dumps.
flected word form, together with synthesis of a dif-ferent inflection of that form. The systems hadto learn from limited data: they were not givencomplete paradigms to train on, nor a dictionaryof lemmas.
Specifically, systems competed on the threetasks illustrated in Table 1, of increasing difficulty.Notice that each task can be regarded as mapping asource string to a target string, with other input ar-guments (such as the target tag) that specify whichversion of the mapping is desired.
For each language and each task, participantswere provided with supervised training data: acollection of input tuples, each paired with the cor-rect output string (target form).
Each system could compete on a task under anyof three tracks (Table 2). Under the restrictedtrack, only data for that task could be used, whilefor the standard track, data from that task and anyfrom a lower task could be used. The bonus trackwas the same as the standard track, but allowed theuse of monolingual data in the form of Wikipediadumps from 2 November 2015.4
Each system was required to produce, for ev-ery input given at test time, either a single stringor a ranked list of up to 20 predicted strings foreach task. Systems were compared on the follow-
4https://dumps.wikimedia.org/
backup-index.html
11
can’t reduce can reduce can reduce+ raw text dumps
![Page 43: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/43.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Evaluation
43
‣ Accuracy (0/1)‣ Levenshtein distance to gold form‣ Reciprocal rank (for multiple guesses)
- 1/ranki (ranki = position of gold form among guesses)
Three types, averaged over all inputs
![Page 44: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/44.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Baseline
44
‣ Simple discriminative string transduction (similar to recent work*)
‣ Classifier is averaged perceptron‣ Applies greedy labeling of input characters,
given target features + features of surrounding characters, previous decisions
*Durrett & DeNero (2013), Nicolai et al (2015)
![Page 45: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/45.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Baseline
45
r u n s
REPTr
input
classificationoutput
P
source = [pos=V,tense=PRES…] target = lemma
# #
![Page 46: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/46.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Baseline
46
r u n s
REPT REPTr u
input
classificationoutput
P
source = [pos=V,tense=PRES…] target = lemma
# #
![Page 47: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/47.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Baseline
47
r u n s
REPT REPT REPTr u n
input
classificationoutput
P
source = [pos=V,tense=PRES…] target = lemma
# #
![Page 48: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/48.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Baseline
48
r u n s
REPT REPT REPT DEL1r u n
input
classificationoutput
P
source = [pos=V,tense=PRES…] target = lemma
# #
![Page 49: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/49.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Baseline
49
r u n s
REPT REPT REPT DEL1r u n
input
classificationoutput
P
source = [pos=V,tense=PRES…] target = lemma
# #
![Page 50: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/50.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Data Overview
‣ N, V, ADJ paradigms from 10 languages‣ 8 Development Languages
- Arabic, Finnish, Georgian, German, Navajo, Russian, Spanish, Turkish
‣ 2 Surprise Languages- Hungarian, Maltese
50
![Page 51: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/51.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Morphological Processes‣ German, Russian, Spanish
-Fusional suffixing with stem changes (Sp. denostar → denuesto)‣ Finnish, Hungarian, Turkish
-Agglutinating suffixing with vowel harmony- (Tr. akbaba → akbabalar, başkent → başkentler)
‣ Navajo-Prefixing with sibilant consonant harmony(atseeʼ → sitseeʼ, áʼázhoozh → shíʼázhoozh)
‣ Georgian-Circumfixing (აბრუნებს abrunebs → ვაბრუნებთ vabrunebt)
‣ Arabic, Maltese-Templatic, non-concatenative morphology (Maltese also concatenating from Italian contact; Ar. kātaba →ʾukātib, Ma. irreaġixxa → irreaġejt)
51
![Page 52: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/52.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Data Sources
‣ 9 Languages except Maltese (Arabic, Spanish, German, Georgian, Russian, Turkish, Hungarian, Navajo, Finnish): Wiktionary (wiktionary.org)
52
![Page 53: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/53.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Wiktionary Collection
53
Lemma Inflection Features
achįʼ iichįʼ V;REAL;1;{DU/PL},{IPFV/PROG}achįʼ daʼiichįʼ V;REAL;1;PL,{IPFV/PROG}achįʼ ashchįʼ V;REAL;1;SG,{IPFV/PROG}
… … …
Navajo
![Page 54: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/54.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Wiktionary Collection
‣ Current full parse available at unimorph.org
‣ Extraction/verification described in (Kirov et al. 2016. Very large scale parsing and normalization of Wiktionary morphological paradigms. LREC.)
‣ UniMorph feature format described in (Sylak-Glassman et al. 2015 A language-independent feature schema for inflectional morphology. ACL.)
54
![Page 55: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/55.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Maltese ‣ Maltese: Ġabra Open Lexicon (Camilleri, 2013, http://
mlrs.research.um.edu.mt/resources/gabra/)-Used as-is except for features remapped to UniMorph
55
![Page 56: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/56.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Data Sampling and Presentation‣ Subset of all available data used for shared
task-Train/Dev/Test forms sampled according to λ-smoothed unigram distribution in Bonus Track corpus data (Wikipedia)
‣ All data presented using native orthography, except Arabic-Arabic used Wiktionary romanization (DIN 31635) -No phonological transcriptions provided
56
![Page 57: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/57.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Training Data Statistics
57
Reinflection Pairs Lemmas Tags Examples Per
Tag PairArabic 12616 2130 225 1.57Finnish 12764 9855 95 5.70
Georgian 12390 4246 90 14.02German 12689 6703 99 7.76
Hungarian 18206 1508 83 9.05Maltese 19125 1453 3607 1.00Navajo 10478 355 54 17.48
Russian 12663 7941 83 10.32Spanish 12725 5872 84 3.24Turkish 12645 2353 190 1.81
![Page 58: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/58.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Meet Our Competitors
‣ For convenience, we categorized the submitted systems into three camps
‣ Camp 1: Align and Transduce‣ Camp 2: Revenge of the RNN‣ Camp 3: Time for Some Linguistics
58
![Page 59: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/59.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Camp 1: Align and Transduce
‣ Drew inspiration from the work of Durrett and DeNero (2013)
‣ Heuristically extract a set of edit transformations
‣ Apply transformations with a semi-Markov model
59
![Page 60: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/60.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
EHU (Alegria and Etxeberria 2016)
‣ Argued that morphological reinfection is very similar to the grapheme-to-phoneme problem
‣ Extended the Phonetisaurus (Novak et al. 2012) toolkit, which is based on OpenFST (Allauzen et al. 2007)
60
![Page 61: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/61.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Alberta (Nicolai et al. 2016)
‣ First run M2M-aligner (Jiampojamarn et al., 2007) — allows many-to-many alignments
‣ Train discriminative transduction algorithm DirectTL+ model (Jiampojamarn et al., 2008).
‣ Add a discriminative reranker on top!
61
![Page 62: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/62.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Colorado (Liu and Mao 2016)
‣ Made use of baseline unsupervised alignment system
‣ Applied semi-CRF solution of Durrett and DeNero (2013)
‣ Unsupervised discovery of C/V segments for features
62
![Page 63: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/63.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
OSU (King 2016)
‣ Unsupervised alignments with Hirschberg’s algorithm (Hirschberg 1975)
‣ Applied a 1st order semi-CRF to apply the edits- Very expensive compared to the 0th order model of Durrett and Denero (2013)
63
![Page 64: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/64.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Camp 2: Revenge of the RNN
‣ Took inspiration from recent advances in neural MT
‣ Most frameworks based on the encoder-decoder model (Cho et al. 2014, inter alia)
‣ Rather than words, translate characters‣ Achieved the best results
64
![Page 65: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/65.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
LMU (Kann and Schütze 2016)
‣ Builds off of the encoder-decoder model for machine translation
‣ Input word with source and target tag are formatted as a single string and fed to the network
‣ Won the shared task!
65
![Page 66: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/66.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
BIU-MIT (Aharoni et al. 2016)
‣ Extension of the encoder-decoder architecture
‣ Include extensions for templatic morphology
‣ Second place team (on average)
66
![Page 67: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/67.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Helsinki (Östling 2016)
‣ Again, neural encoder-decoder architecture
‣ Added an additional convolutional layer over the characters
‣ Third place team!
67
![Page 68: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/68.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Camp 3: Time for Some Linguistics
‣ Relied heavily on linguistic-inspired methods
‣ Reduces the problem to multi-way classification
68
![Page 69: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/69.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Moscow State (Sorokin 2016)
‣ Uses longest common substring to compute an ‘abstract paradigm’
‣ In short, learn a joint set of rules for every slot in the paradigm (Ahlberg et al. 2015)
‣ Generated candidate set and used an SVM classifier
69
![Page 70: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/70.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Columbia/NYUAD (Taji et al. 2016)
‣ The input words are first segmented into prefixes, stems, and suffixes
‣ Stems are further processed‣ Sets of patterns are extracted and applied
to the stems
70
![Page 71: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/71.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Results
‣
71
![Page 72: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/72.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Results
‣
72
Neural Systems
![Page 73: The SIGMORPHON 2016 shared task— morphological …jason/papers/cotterell+al.sigmorphon16.slides.pdfSIGMORPHON 2016 Shared task - morphological reinflection The SIGMORPHON 2016 shared](https://reader035.fdocuments.us/reader035/viewer/2022070819/5f1b542179518a558a11d69a/html5/thumbnails/73.jpg)
SIGMORPHON 2016 Shared task - morphological reinflection
Thank you‣ Training/Dev/Test data available at
- http://sigmorphon.org/sharedtask
73
Questions? Suggestions? Comments?