Post on 22-Sep-2020
Semantic Role Labeling with Labeled Span Graph Networks
Luheng He Google A.I. Oct. 2018
1
Work done at the University of Washington
!2
Predicting Rich Semantic Structure with Simple Models
On January 13, 2018, a false ballistic missile alert was issued via the Emergency Alert System and Commercial Mobile Alert System over television, radio, and cellphones in the U.S. state of Hawaii. The alert stated that there was an incoming ballistic missile threat to Hawaii, advised residents to seek shelter, and concluded "This is not a drill". The message was sent at 8:07 a.m. local time.
?Input Document Semantic Structure
!3
Task: Semantic Role Labeling (SRL)
On January 13, 2018, a false ballistic missile alert was issued via
the Emergency Alert System and Commercial Mobile Alert System over
television, radio, and cellphones in the U.S. state of Hawaii.
The alert stated that there was an incoming ballistic missile threat to Hawaii,
advised residents to seek shelter, and concluded "This is not a drill".
The message was sent at 8:07 a.m. local time.
From Wikipedia: 2018 Hawaii false missile alert. Only part of the structures are visualized.
SRL
Patient Time
“Who did what to whom, when and where”
!4
SRLCoreferencePatient Time
Agent Patient
PatientTime
From Wikipedia: 2018 Hawaii false missile alert. Only part of the structures are visualized.
Adding Coreference Resolution
On January 13, 2018, a false ballistic missile alert was issued via
the Emergency Alert System and Commercial Mobile Alert System over
television, radio, and cellphones in the U.S. state of Hawaii.
The alert stated that there was an incoming ballistic missile threat to Hawaii,
advised residents to seek shelter, and concluded "This is not a drill".
The message was sent at 8:07 a.m. local time.
!5
Another example: Relation Extraction on Scientific Documents
SciERC (Entity, Relation, Coreference): Luan et al., 2018
Annotation with entities, relations, and coreference
!5
Another example: Relation Extraction on Scientific Documents
SciERC (Entity, Relation, Coreference): Luan et al., 2018
Annotation with entities, relations, and coreference Document-level KG
!6
Another example: Relation Extraction
SciERC (Entity, Relation, Coreference): Luan et al., 2018
Annotation with entities, relations, and coreference Document level KG
Goal: A unified model for all these tasks.
Challenge: Very different structures, task-specific pipelines/features/architectures …
This talk: 1) Build end-to-end models for SRL.
2) Generalizes such model to all tasks.
Contributions
7
Contributions
7
• End-to-end prediction of SRL structure, without relying on NLP pipeline.
Contributions
7
• End-to-end prediction of SRL structure, without relying on NLP pipeline.
• Almost 40% error reduction over best pre-neural model despite being much simpler.
Contributions
7
• End-to-end prediction of SRL structure, without relying on NLP pipeline.
• Almost 40% error reduction over best pre-neural model despite being much simpler.
• First end-to-end result for jointly predicting predicates and argument spans.
Contributions
7
• End-to-end prediction of SRL structure, without relying on NLP pipeline.
• Almost 40% error reduction over best pre-neural model despite being much simpler.
• First end-to-end result for jointly predicting predicates and argument spans.
• Joint modeling for a variety of span-based tasks, opens up opportunities for full-text understanding.
purposeAgent / visitor Patient / visited
visit
Semantic Role Labeling (SRL)
8
Many tourists Disney to meet their favorite cartoon characters
visit
Predicate Arguments
Who is the visitor: [Many tourists]What is visited: [Disney]What purpose: [to meet … characters]
ARG0ARG1
AM-PRPvisitorvisited
purpose
visit
Semantic Role Labeling (SRL)
9
Many tourists Disney to meet their favorite cartoon characters
visit
Predicate Arguments
ARG0: [Many tourists]ARG1: [Disney]AM-PRP: [to meet … characters]
The Proposition Bank: An Annotated Corpus of Semantic Roles, Palmer et al., 2005
Frame: visit.01role description
ARG0 visitorARG1 visited
ARG0ARG1
AM-PRPvisitorvisited
purpose
visit
Semantic Role Labeling (SRL)
9
Many tourists Disney to meet their favorite cartoon characters
visit
Predicate Arguments
ARG0: [Many tourists]ARG1: [Disney]AM-PRP: [to meet … characters]
The Proposition Bank: An Annotated Corpus of Semantic Roles, Palmer et al., 2005
Frame: visit.01role description
ARG0 visitorARG1 visited
Core arguments: Verb-specific roles (A0-A5) Adjuncts: Arg-modifier (AM-) roles shared across verbs
visit
SRL Task: Given (gold) predicate, predict arguments
10
Many tourists Disney to meet their favorite cartoon characters
ARG0 ARG1
ARG0: [Many tourists]ARG1: [their favorite cartoon characters]
visit
Predicate Arguments
ARG0: [Many tourists]ARG1: [Disney]AM-PRP: [to meet their favorite cartoon characters]
meet
meeter entity met
visit
SRL Task: Given (gold) predicate, predict arguments
10
Many tourists Disney to meet their favorite cartoon characters
ARG0 ARG1
ARG0: [Many tourists]ARG1: [their favorite cartoon characters]
visit
Predicate Arguments
ARG0: [Many tourists]ARG1: [Disney]AM-PRP: [to meet their favorite cartoon characters]
meet
Most Span-based SRL Tasks: given gold predicates, predict the argument spans and labels.
meeter entity met
11
Outline
Predicting SRL with Deep BiLSTMs— DeepSRL (He et al., 2017)
Towards Unified and Full-text Semantic Analysis— Multi-task learning with LSGN; ScienceIE (Luan et al., 2018)
An End-to-End, Span-based SRL Model— Labeled Span Graph Network (He et al., 2018)
SRL Systems: Pipelined vs. BIO-based
syntactic features
candidate argument spans
labeled arguments
prediction
labeling
ILP/DP
sentence, predicate
argument id.
Pipeline Systems
Punyakanok et al., 2008 Täckström et al., 2015 FitzGerald et al., 2015
12
SRL Systems: Pipelined vs. BIO-based
syntactic features
candidate argument spans
labeled arguments
prediction
labeling
ILP/DP
sentence, predicate
argument id.
Pipeline Systems
Punyakanok et al., 2008 Täckström et al., 2015 FitzGerald et al., 2015
12
Syntactic Parser
SRL Systems: Pipelined vs. BIO-based
syntactic features
candidate argument spans
labeled arguments
prediction
labeling
ILP/DP
sentence, predicate
argument id.
Pipeline Systems
Punyakanok et al., 2008 Täckström et al., 2015 FitzGerald et al., 2015
12
Syntactic Parser
Hand-engineered Rules
SRL Systems: Pipelined vs. BIO-based
syntactic features
candidate argument spans
labeled arguments
prediction
labeling
ILP/DP
sentence, predicate
argument id.
Pipeline Systems
Punyakanok et al., 2008 Täckström et al., 2015 FitzGerald et al., 2015
12
Syntactic Parser
Hand-engineered Rules
Hand-engineered Features
SRL Systems: Pipelined vs. BIO-based
syntactic features
candidate argument spans
labeled arguments
prediction
labeling
ILP/DP
sentence, predicate
argument id.
Pipeline Systems
Punyakanok et al., 2008 Täckström et al., 2015 FitzGerald et al., 2015
12
Syntactic Parser
Hand-engineered Rules
Hand-engineered Features
Post-Processing
SRL Systems: Pipelined vs. BIO-based
syntactic features
candidate argument spans
labeled arguments
prediction
labeling
ILP/DP
sentence, predicate
argument id.
Pipeline Systems
Punyakanok et al., 2008 Täckström et al., 2015 FitzGerald et al., 2015
sentence, predicate
BIO sequence
prediction
Deep BiLSTM + CRF layer
Viterbi
word-level features
BIO-based Systems
Collobert et al., 2011 Zhou and Xu, 2015 Wang et. al, 2015
12
SRL Systems: Pipelined vs. BIO-based
syntactic features
candidate argument spans
labeled arguments
prediction
labeling
ILP/DP
sentence, predicate
argument id.
Pipeline Systems
Deep BiLSTM
Hard constraints
BIO sequence
prediction
sentence, predicate
DeepSRL
Punyakanok et al., 2008 Täckström et al., 2015 FitzGerald et al., 2015
sentence, predicate
BIO sequence
prediction
Deep BiLSTM + CRF layer
Viterbi
word-level features
BIO-based Systems
Collobert et al., 2011 Zhou and Xu, 2015 Wang et. al, 2015
He et al., 2017
12
No global normalization
13
the cats love hats[V=0] [V=0] [V=1] [V=0]
B-ARG0 0.4I-ARG0 0.05B-ARG1 0.5I-ARG1 0.03
… …
B-ARG0 0.1I-ARG0 0.5B-ARG1 0.1I-ARG1 0.2
… …
B-ARG0 0.001I-ARG0 0.001B-ARG1 0.001
… …B-V 0.95
B-ARG0 0.1I-ARG0 0.1B-ARG1 0.7I-ARG1 0.2
… …
(1) Inputs words and target predicate
13
the cats love hats[V=0] [V=0] [V=1] [V=0]
B-ARG0 0.4I-ARG0 0.05B-ARG1 0.5I-ARG1 0.03
… …
B-ARG0 0.1I-ARG0 0.5B-ARG1 0.1I-ARG1 0.2
… …
B-ARG0 0.001I-ARG0 0.001B-ARG1 0.001
… …B-V 0.95
B-ARG0 0.1I-ARG0 0.1B-ARG1 0.7I-ARG1 0.2
… …
(2) Deep BiLSTM tagger
(1) Inputs words and target predicate
13
the cats love hats[V=0] [V=0] [V=1] [V=0]
B-ARG0 0.4I-ARG0 0.05B-ARG1 0.5I-ARG1 0.03
… …
B-ARG0 0.1I-ARG0 0.5B-ARG1 0.1I-ARG1 0.2
… …
B-ARG0 0.001I-ARG0 0.001B-ARG1 0.001
… …B-V 0.95
B-ARG0 0.1I-ARG0 0.1B-ARG1 0.7I-ARG1 0.2
… …
(2) Deep BiLSTM tagger
(3) Highway connections,
variational dropouts, etc.
(1) Inputs words and target predicate
13
the cats love hats[V=0] [V=0] [V=1] [V=0]
B-ARG0 0.4I-ARG0 0.05B-ARG1 0.5I-ARG1 0.03
… …
B-ARG0 0.1I-ARG0 0.5B-ARG1 0.1I-ARG1 0.2
… …
B-ARG0 0.001I-ARG0 0.001B-ARG1 0.001
… …B-V 0.95
B-ARG0 0.1I-ARG0 0.1B-ARG1 0.7I-ARG1 0.2
… …
(2) Deep BiLSTM tagger
(3) Highway connections,
variational dropouts, etc.
(4) Viterbi decoding with
hard constraints at test time
(1) Inputs words and target predicate
13
the cats love hats[V=0] [V=0] [V=1] [V=0]
B-ARG0 0.4I-ARG0 0.05B-ARG1 0.5I-ARG1 0.03
… …
B-ARG0 0.1I-ARG0 0.5B-ARG1 0.1I-ARG1 0.2
… …
B-ARG0 0.001I-ARG0 0.001B-ARG1 0.001
… …B-V 0.95
B-ARG0 0.1I-ARG0 0.1B-ARG1 0.7I-ARG1 0.2
… …
(2) Deep BiLSTM tagger
(3) Highway connections,
variational dropouts, etc.
(4) Viterbi decoding with
hard constraints at test time
(1) Inputs words and target predicate
Strengths: No syntactic preprocessing;
Easy to implement (can use off-the-shelf sequential tagger)
Limitations: Needs to re-process the same sentence multiple times,
if sentence has multiple predicates
CoNLL 2005 (Original PropBank) Results
14
F1
60
65
70
75
80
85
90
Toutanova05* Täckström15 FitzGerald15* Zhou15 DeepSRL DeepSRL*
WSJ Test Brown (out-domain) Test
Pipeline models Deep BIO Models
*:Ensemble models
CoNLL 2005 (Original PropBank) Results
14
F1
60
65
70
75
80
85
90
Toutanova05* Täckström15 FitzGerald15* Zhou15 DeepSRL DeepSRL*
WSJ Test Brown (out-domain) Test
84.683.182.8
80.379.980.3
Pipeline models Deep BIO Models
*:Ensemble models
CoNLL 2005 (Original PropBank) Results
14
F1
60
65
70
75
80
85
90
Toutanova05* Täckström15 FitzGerald15* Zhou15 DeepSRL DeepSRL*
WSJ Test Brown (out-domain) Test
73.672.1
69.472.271.3
68.8
84.683.182.8
80.379.980.3
Pipeline models Deep BIO Models
*:Ensemble models
CoNLL 2005 (Original PropBank) Results
14
F1
60
65
70
75
80
85
90
Toutanova05* Täckström15 FitzGerald15* Zhou15 DeepSRL DeepSRL*
WSJ Test Brown (out-domain) Test
73.672.1
69.472.271.3
68.8
84.683.182.8
80.379.980.3
Pipeline models Deep BIO Models
*:Ensemble models
• 20% Error reduction over pipelined systems!
• Still way to go for out-domain data!
CoNLL 2012 (OntoNotes) Results
15
F1
60
65
70
75
80
85
90
Pradhan12 Täckström15 FitzGerald15* Zhou15 DeepSRL DeepSRL*
83.481.781.580.279.4
77.5
CoNLL 2012 Test*:Ensemble models
Larger dataset with 6 domains, contains nominal predicates
CoNLL 2012 (OntoNotes) Results
15
F1
60
65
70
75
80
85
90
Pradhan12 Täckström15 FitzGerald15* Zhou15 DeepSRL DeepSRL*
83.481.781.580.279.4
77.5
CoNLL 2012 Test
Pipeline models
*:Ensemble models
Deep BIO models
Larger dataset with 6 domains, contains nominal predicates
CoNLL 2012 (OntoNotes) Results
15
F1
60
65
70
75
80
85
90
Pradhan12 Täckström15 FitzGerald15* Zhou15 DeepSRL DeepSRL*
83.481.781.580.279.4
77.5
CoNLL 2012 Test
Pipeline models
*:Ensemble models
Deep BIO models
Larger dataset with 6 domains, contains nominal predicates
However, the model still relies on gold predicates …
Real Scenario: No Gold Predicates!
16
Deep BiLSTM
Hard constraints
BIO sequence
prediction
sentence, predicate
End-to-end SRL:Given sentence, predict all predicates as well as their arguments.
?
Real Scenario: No Gold Predicates!
16
Deep BiLSTM
Hard constraints
BIO sequence
prediction
sentence, predicate
sentence
predicates
BiLSTM
for each predicate
Pipelined approach: Identify predicates first, then run the BIO tagger for each predicate.
(No way to recover from recall loss at predicate ID stage …)
End-to-end SRL:Given sentence, predict all predicates as well as their arguments.
Real Scenario: No Gold Predicates!
16
Deep BiLSTM
Hard constraints
BIO sequence
prediction
sentence, predicate
sentence
predicates
BiLSTM
for each predicate
Pipelined approach: Identify predicates first, then run the BIO tagger for each predicate.
(No way to recover from recall loss at predicate ID stage …)
End-to-end SRL:Given sentence, predict all predicates as well as their arguments.
Deep BiLSTM
Hard constraints
BIO sequence
prediction
sentence, predicate
Real Scenario: No Gold Predicates!
16
Deep BiLSTM
Hard constraints
BIO sequence
prediction
sentence, predicate
sentence
predicates
BiLSTM
for each predicate
Pipelined approach: Identify predicates first, then run the BIO tagger for each predicate.
(No way to recover from recall loss at predicate ID stage …)
End-to-end SRL:Given sentence, predict all predicates as well as their arguments.
Deep BiLSTM
Hard constraints
BIO sequence
prediction
sentence, predicate
Deep BiLSTM
Hard constraints
BIO sequence
prediction
sentence, predicate
17
F1
60
65
70
75
80
85
90
WSJ Brown OntoNotes
Gold PredicateEnd-to-End
83.4
73.6
84.6
End-to-End SRL Result
17
F1
60
65
70
75
80
85
90
WSJ Brown OntoNotes
Gold PredicateEnd-to-End
78.4
70.1
82.7 83.4
73.6
84.6
End-to-End SRL Result
17
F1
60
65
70
75
80
85
90
WSJ Brown OntoNotes
Gold PredicateEnd-to-End
78.4
70.1
82.7 83.4
73.6
84.6
End-to-End SRL Result
Larger performance drop on Brown (out-domain) and
OntoNotes (nominal predicates)
18
Outline
Predicting SRL with Deep BiLSTMs— DeepSRL
Towards Unified and Full-text Semantic Analysis— Multi-task learning with LSGN; ScienceIE (Luan et al., 2018)
An End-to-End, Span-based SRL Model— Labeled Span Graph Network (LSGN)
☑A︎ccurate ☑N︎o NLP pipeline ☐Joint predicate ID ☐Full-text Semantics
!19
Intuition: SRL as Span-Span Relations
ARG0 ARG1AM-PRP
visitMany tourists Disney to meet their favorite cartoon characters
ARG0 ARG1
!19
Intuition: SRL as Span-Span Relations
Challenges: 1. Span can nest within each other. 2. Too many possible edges (n2 argument spans & n predicates).
ARG0 ARG1AM-PRP
visitMany tourists Disney to meet their favorite cartoon characters
ARG0 ARG1
!20
Labeled Span Graph Network (LSGN)
LSG: A graph with nodes as spans and labeled edges. LSGN: An end-to-end network for predicting an LSG.
!20
Labeled Span Graph Network (LSGN)
LSG: A graph with nodes as spans and labeled edges. LSGN: An end-to-end network for predicting an LSG.
Many NLP structures can be considered as an LSG: • SRL (First end-to-end model!)• Coreference resolution (Lee et al. 2017) • Named entity recognition and relation extraction (Luan
et al., 2018)
DeepSRL Architecture (Revisit)
Input sentence
Word & Pred.Embeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters[0] [0] [1] [0] [0] [0] [0] [0] [0] [0]Target Predicate
21
DeepSRL Architecture (Revisit)
Input sentence
Word & Pred.Embeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters[0] [0] [1] [0] [0] [0] [0] [0] [0] [0]Target Predicate
TaggingSoftmax
B-A0 I-A0 B-V B-V … …Output Labels
21
LSGN Architecture: Overview
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
No predicate input!
22
LSGN Architecture: Overview
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
Many tourists
tourists visit Disney
Disney to meet their
their favorite cartoon
cartoon characters
No predicate input!
(1) Construct span representations for all n2 spans!
22
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
Node & Edge Scores
LabelingSoftmax
23
(2) Local classifier over labels (including NULL) for all possible (predicate, argument) pairs
(1) Construct span representations for all n2 spans!
…
LSGN Architecture: Overview
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
Node & Edge Scores
LabelingSoftmax
24
(2) Local classifier over labels (including NULL) for all possible (predicate, argument) pairs
(1) Construct span representations for all n2 spans!
…(3) Greedy beam pruning for spans
LSGN Architecture: Overview
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
25
their favorite cartoon
(1) Span Representations (2) Local Label Classifiers
(3) Span Pruning
(Same as Lee et al., 2017)
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
25
LSTM boundary points
their favorite cartoon
(1) Span Representations (2) Local Label Classifiers
(3) Span Pruning
[BiLstm(w1 : wn)Start,BiLstm(w1 : wn)End]
(Same as Lee et al., 2017)
left context right context
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
26
LSTM boundary points
Attention over words
their favorite cartoon
(1) Span Representations (2) Local Label Classifiers
(3) Span Pruning
[BiLstm(w1 : wn)Start,BiLstm(w1 : wn)End]
XEnd
i=StartSoftMax(aStart : aEnd)i wi
(Same as Lee et al., 2017)
Disney to meet favorite cartoon charactersInput sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit
Span Representation
Node & Edge Scores
LabelingSoftmax
27
(1) Span Representations (2) Local Label Classifiers (3) Span
Pruning
predicateargument
ARG0ARG1ARG2
…AM-TMP
…𝜖 (No Edge)
n<latexit sha1_base64="nvU6K7Xllec9J/cWdW16/i7BwDY=">AAACCHicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlw02UL9gFtKJPJTTt0Mgkzk2IJ/QHd6n+4E7f+hb/hFzhts7CtBwYO59zLPXP8hDOlHefbKmxt7+zuFfdLB4dHxyfl07O2ilNJsUVjHsuuTxRyJrClmebYTSSSyOfY8ccPc78zQalYLB71NEEvIkPBQkaJNlJTDMoVp+osYG8SNycVyNEYlH/6QUzTCIWmnCjVc51EexmRmlGOs1I/VZgQOiZD7BkqSITKyxZBZ/aVUQI7jKV5QtsL9e9GRiKlppFvJiOiR2rdm4v/eb1gwhKV33paHltNosN7L2MiSTUKugwSptzWsT1vxQ6YRKr51BBCJTN/semISEK16a5kSnLXK9kk7Zuq61Td5m2lVs/rKsIFXMI1uHAHNahDA1pAAeEFXuHNerberQ/rczlasPKdc1iB9fULDneauA==</latexit><latexit sha1_base64="nvU6K7Xllec9J/cWdW16/i7BwDY=">AAACCHicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlw02UL9gFtKJPJTTt0Mgkzk2IJ/QHd6n+4E7f+hb/hFzhts7CtBwYO59zLPXP8hDOlHefbKmxt7+zuFfdLB4dHxyfl07O2ilNJsUVjHsuuTxRyJrClmebYTSSSyOfY8ccPc78zQalYLB71NEEvIkPBQkaJNlJTDMoVp+osYG8SNycVyNEYlH/6QUzTCIWmnCjVc51EexmRmlGOs1I/VZgQOiZD7BkqSITKyxZBZ/aVUQI7jKV5QtsL9e9GRiKlppFvJiOiR2rdm4v/eb1gwhKV33paHltNosN7L2MiSTUKugwSptzWsT1vxQ6YRKr51BBCJTN/semISEK16a5kSnLXK9kk7Zuq61Td5m2lVs/rKsIFXMI1uHAHNahDA1pAAeEFXuHNerberQ/rczlasPKdc1iB9fULDneauA==</latexit><latexit sha1_base64="nvU6K7Xllec9J/cWdW16/i7BwDY=">AAACCHicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlw02UL9gFtKJPJTTt0Mgkzk2IJ/QHd6n+4E7f+hb/hFzhts7CtBwYO59zLPXP8hDOlHefbKmxt7+zuFfdLB4dHxyfl07O2ilNJsUVjHsuuTxRyJrClmebYTSSSyOfY8ccPc78zQalYLB71NEEvIkPBQkaJNlJTDMoVp+osYG8SNycVyNEYlH/6QUzTCIWmnCjVc51EexmRmlGOs1I/VZgQOiZD7BkqSITKyxZBZ/aVUQI7jKV5QtsL9e9GRiKlppFvJiOiR2rdm4v/eb1gwhKV33paHltNosN7L2MiSTUKugwSptzWsT1vxQ6YRKr51BBCJTN/semISEK16a5kSnLXK9kk7Zuq61Td5m2lVs/rKsIFXMI1uHAHNahDA1pAAeEFXuHNerberQ/rczlasPKdc1iB9fULDneauA==</latexit><latexit sha1_base64="nvU6K7Xllec9J/cWdW16/i7BwDY=">AAACCHicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFlw02UL9gFtKJPJTTt0Mgkzk2IJ/QHd6n+4E7f+hb/hFzhts7CtBwYO59zLPXP8hDOlHefbKmxt7+zuFfdLB4dHxyfl07O2ilNJsUVjHsuuTxRyJrClmebYTSSSyOfY8ccPc78zQalYLB71NEEvIkPBQkaJNlJTDMoVp+osYG8SNycVyNEYlH/6QUzTCIWmnCjVc51EexmRmlGOs1I/VZgQOiZD7BkqSITKyxZBZ/aVUQI7jKV5QtsL9e9GRiKlppFvJiOiR2rdm4v/eb1gwhKV33paHltNosN7L2MiSTUKugwSptzWsT1vxQ6YRKr51BBCJTN/semISEK16a5kSnLXK9kk7Zuq61Td5m2lVs/rKsIFXMI1uHAHNahDA1pAAeEFXuHNerberQ/rczlasPKdc1iB9fULDneauA==</latexit>
n2<latexit sha1_base64="Cl3SwRPAtZyFFQe08H+ihuTViBc=">AAACCnicbVDLSsNAFL2pr1pfVZduBovgqiRF0GXBTZcV7QPaWCaTSTt0Mgkzk2IJ/QPd6n+4E7f+hL/hFzhNs7CtBwYO59zLPXO8mDOlbfvbKmxsbm3vFHdLe/sHh0fl45O2ihJJaItEPJJdDyvKmaAtzTSn3VhSHHqcdrzx7dzvTKhULBIPehpTN8RDwQJGsDbSvXisDcoVu2pnQOvEyUkFcjQH5Z++H5EkpEITjpXqOXas3RRLzQins1I/UTTGZIyHtGeowCFVbppFnaELo/goiKR5QqNM/buR4lCpaeiZyRDrkVr15uJ/Xs+fsFjlt54Wx5aT6ODGTZmIE00FWQQJEo50hOa9IJ9JSjSfGoKJZOYviIywxESb9kqmJGe1knXSrlUdu+rcXVXqjbyuIpzBOVyCA9dQhwY0oQUEhvACr/BmPVvv1of1uRgtWPnOKSzB+voFRgqbXA==</latexit><latexit sha1_base64="Cl3SwRPAtZyFFQe08H+ihuTViBc=">AAACCnicbVDLSsNAFL2pr1pfVZduBovgqiRF0GXBTZcV7QPaWCaTSTt0Mgkzk2IJ/QPd6n+4E7f+hL/hFzhNs7CtBwYO59zLPXO8mDOlbfvbKmxsbm3vFHdLe/sHh0fl45O2ihJJaItEPJJdDyvKmaAtzTSn3VhSHHqcdrzx7dzvTKhULBIPehpTN8RDwQJGsDbSvXisDcoVu2pnQOvEyUkFcjQH5Z++H5EkpEITjpXqOXas3RRLzQins1I/UTTGZIyHtGeowCFVbppFnaELo/goiKR5QqNM/buR4lCpaeiZyRDrkVr15uJ/Xs+fsFjlt54Wx5aT6ODGTZmIE00FWQQJEo50hOa9IJ9JSjSfGoKJZOYviIywxESb9kqmJGe1knXSrlUdu+rcXVXqjbyuIpzBOVyCA9dQhwY0oQUEhvACr/BmPVvv1of1uRgtWPnOKSzB+voFRgqbXA==</latexit><latexit sha1_base64="Cl3SwRPAtZyFFQe08H+ihuTViBc=">AAACCnicbVDLSsNAFL2pr1pfVZduBovgqiRF0GXBTZcV7QPaWCaTSTt0Mgkzk2IJ/QPd6n+4E7f+hL/hFzhNs7CtBwYO59zLPXO8mDOlbfvbKmxsbm3vFHdLe/sHh0fl45O2ihJJaItEPJJdDyvKmaAtzTSn3VhSHHqcdrzx7dzvTKhULBIPehpTN8RDwQJGsDbSvXisDcoVu2pnQOvEyUkFcjQH5Z++H5EkpEITjpXqOXas3RRLzQins1I/UTTGZIyHtGeowCFVbppFnaELo/goiKR5QqNM/buR4lCpaeiZyRDrkVr15uJ/Xs+fsFjlt54Wx5aT6ODGTZmIE00FWQQJEo50hOa9IJ9JSjSfGoKJZOYviIywxESb9kqmJGe1knXSrlUdu+rcXVXqjbyuIpzBOVyCA9dQhwY0oQUEhvACr/BmPVvv1of1uRgtWPnOKSzB+voFRgqbXA==</latexit><latexit sha1_base64="Cl3SwRPAtZyFFQe08H+ihuTViBc=">AAACCnicbVDLSsNAFL2pr1pfVZduBovgqiRF0GXBTZcV7QPaWCaTSTt0Mgkzk2IJ/QPd6n+4E7f+hL/hFzhNs7CtBwYO59zLPXO8mDOlbfvbKmxsbm3vFHdLe/sHh0fl45O2ihJJaItEPJJdDyvKmaAtzTSn3VhSHHqcdrzx7dzvTKhULBIPehpTN8RDwQJGsDbSvXisDcoVu2pnQOvEyUkFcjQH5Z++H5EkpEITjpXqOXas3RRLzQins1I/UTTGZIyHtGeowCFVbppFnaELo/goiKR5QqNM/buR4lCpaeiZyRDrkVr15uJ/Xs+fsFjlt54Wx5aT6ODGTZmIE00FWQQJEo50hOa9IJ9JSjSfGoKJZOYviIywxESb9kqmJGe1knXSrlUdu+rcXVXqjbyuIpzBOVyCA9dQhwY0oQUEhvACr/BmPVvv1of1uRgtWPnOKSzB+voFRgqbXA==</latexit>
their
Disney to meet favorite cartoon charactersInput sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit
Span Representation
Node & Edge Scores
LabelingSoftmax
28
(1) Span Representations (2) Local Label Classifiers (3) Span
Pruning
their
meetmany tourists
ARG0ARG1ARG2
…AM-TMP
…𝜖 (No Edge)
Disney to meet favorite cartoon charactersInput sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit
Span Representation
Node & Edge Scores
LabelingSoftmax
29
(1) Span Representations (2) Local Label Classifiers (3) Span
Pruning
their
meetDisney
ARG0ARG1ARG2
…AM-TMP
…𝜖 (No Edge)
Disney to meet favorite cartoon charactersInput sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit
Span Representation
Node & Edge Scores
LabelingSoftmax
30
(1) Span Representations (2) Local Label Classifiers (3) Span
Pruning
P (ypred,arg = l | X) / exp(�(pred, arg, l))�(pred, arg, l)<latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit>
/ exp( )<latexit sha1_base64="yXNfyhLpCndh2SGQIpxOI+pXVcg=">AAACNHicdVDLSgMxFM34rPU16tLNYBEqQpkRQZcFNy4r2Ad0hpLJZNrQTBKTTGkZ+hH+jG71MwR34taFX2DazsK2eiDhcM693HtPKChR2nXfrJXVtfWNzcJWcXtnd2/fPjhsKJ5KhOuIUy5bIVSYEobrmmiKW0JimIQUN8P+zcRvDrBUhLN7PRI4SGCXkZggqI3Usc99IbnQ3MdDUfYfUhj9+5117JJbcadwlomXkxLIUevY337EUZpgphGFSrU9V+ggg1ITRPG46KcKC4j6sIvbhjKYYBVk06PGzqlRIifm0jymnan6uyODiVKjJDSVCdQ9tehNxL+8djQgQuWzhrNh85vo+DrICBOpxgzNFolT6mjuTBJ0IiIx0nRkCESSmFsc1IMSIm1yLpqQvMVIlknjouK5Fe/uslSt5nEVwDE4AWXggStQBbegBuoAgUfwDF7Aq/VkvVsf1uesdMXKe47AHKyvH36prTs=</latexit><latexit sha1_base64="yXNfyhLpCndh2SGQIpxOI+pXVcg=">AAACNHicdVDLSgMxFM34rPU16tLNYBEqQpkRQZcFNy4r2Ad0hpLJZNrQTBKTTGkZ+hH+jG71MwR34taFX2DazsK2eiDhcM693HtPKChR2nXfrJXVtfWNzcJWcXtnd2/fPjhsKJ5KhOuIUy5bIVSYEobrmmiKW0JimIQUN8P+zcRvDrBUhLN7PRI4SGCXkZggqI3Usc99IbnQ3MdDUfYfUhj9+5117JJbcadwlomXkxLIUevY337EUZpgphGFSrU9V+ggg1ITRPG46KcKC4j6sIvbhjKYYBVk06PGzqlRIifm0jymnan6uyODiVKjJDSVCdQ9tehNxL+8djQgQuWzhrNh85vo+DrICBOpxgzNFolT6mjuTBJ0IiIx0nRkCESSmFsc1IMSIm1yLpqQvMVIlknjouK5Fe/uslSt5nEVwDE4AWXggStQBbegBuoAgUfwDF7Aq/VkvVsf1uesdMXKe47AHKyvH36prTs=</latexit><latexit sha1_base64="yXNfyhLpCndh2SGQIpxOI+pXVcg=">AAACNHicdVDLSgMxFM34rPU16tLNYBEqQpkRQZcFNy4r2Ad0hpLJZNrQTBKTTGkZ+hH+jG71MwR34taFX2DazsK2eiDhcM693HtPKChR2nXfrJXVtfWNzcJWcXtnd2/fPjhsKJ5KhOuIUy5bIVSYEobrmmiKW0JimIQUN8P+zcRvDrBUhLN7PRI4SGCXkZggqI3Usc99IbnQ3MdDUfYfUhj9+5117JJbcadwlomXkxLIUevY337EUZpgphGFSrU9V+ggg1ITRPG46KcKC4j6sIvbhjKYYBVk06PGzqlRIifm0jymnan6uyODiVKjJDSVCdQ9tehNxL+8djQgQuWzhrNh85vo+DrICBOpxgzNFolT6mjuTBJ0IiIx0nRkCESSmFsc1IMSIm1yLpqQvMVIlknjouK5Fe/uslSt5nEVwDE4AWXggStQBbegBuoAgUfwDF7Aq/VkvVsf1uesdMXKe47AHKyvH36prTs=</latexit><latexit sha1_base64="yXNfyhLpCndh2SGQIpxOI+pXVcg=">AAACNHicdVDLSgMxFM34rPU16tLNYBEqQpkRQZcFNy4r2Ad0hpLJZNrQTBKTTGkZ+hH+jG71MwR34taFX2DazsK2eiDhcM693HtPKChR2nXfrJXVtfWNzcJWcXtnd2/fPjhsKJ5KhOuIUy5bIVSYEobrmmiKW0JimIQUN8P+zcRvDrBUhLN7PRI4SGCXkZggqI3Usc99IbnQ3MdDUfYfUhj9+5117JJbcadwlomXkxLIUevY337EUZpgphGFSrU9V+ggg1ITRPG46KcKC4j6sIvbhjKYYBVk06PGzqlRIifm0jymnan6uyODiVKjJDSVCdQ9tehNxL+8djQgQuWzhrNh85vo+DrICBOpxgzNFolT6mjuTBJ0IiIx0nRkCESSmFsc1IMSIm1yLpqQvMVIlknjouK5Fe/uslSt5nEVwDE4AWXggStQBbegBuoAgUfwDF7Aq/VkvVsf1uesdMXKe47AHKyvH36prTs=</latexit>
their
31
Many tourists meetSpan Representation
Pred./Arg. score
�a(“Many tourists”)<latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit>
�p(“meet”)
(1) Span Representations (2) Local Label Classifiers (3) Span
Pruning
�(pred, arg, l) = �a(arg) + �p(pred) + �(l)rel(arg, pred)
<latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit><latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit><latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit>
�(pred, arg, l)<latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit>
31
Many tourists meetSpan Representation
Edge score
Pred./Arg. score
�(ARG1)rel (“Many tourists”, “meet”)�(ARG0)
rel (“Many tourists”, “meet”)
�a(“Many tourists”)<latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit>
�p(“meet”)
(1) Span Representations (2) Local Label Classifiers (3) Span
Pruning
�(pred, arg, l) = �a(arg) + �p(pred) + �(l)rel(arg, pred)
<latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit><latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit><latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit>
�(pred, arg, l)<latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit><latexit sha1_base64="FDP4vzdLhw9DhfaiCwy1qR/1xlg=">AAACT3icbVFNSwMxEM3W7/pV9eglWAQFKbsi6LEoiBdRwarQLSWbnW6D2eySzIpl2X/kn9Gb6K/w4k1M2xX8Ggi8eTOZN3kJUikMuu6zU5mYnJqemZ2rzi8sLi3XVlavTJJpDi2eyETfBMyAFApaKFDCTaqBxYGE6+D2aFi/vgNtRKIucZBCJ2aREj3BGVqqWzv2077Yyv3RpPxUhEpEfTyUGRQ+wj3mdlpY7NBxwnRU2OSr/UwzFUEhi+1ure423FHQv8ArQZ2Ucd6tvflhwrMYFHLJjGl7boodK4CCSyiqfmYgZfyWRdC2ULEYTCcfyRZ00zIh7SXaHoV0xH6/kbPYmEEc2M6YYd/8rg3J/2rt8E6kptS6H4v93AR7B51cqDRDUHy8SC+TFBM6NJeGQgNHObCAcS3sWyjvM8042i+oWpO835b8BVe7Dc9teBd79WaztGuWrJMNskU8sk+a5ISckxbh5IE8kRfy6jw6785HpWytOCVYIz+iMvcJb4C3Hg==</latexit>
32
Many tourists meetSpan Representation
Combined score
Softmax
Edge score
Pred./Arg. score
�(ARG1)rel (“Many tourists”, “meet”)�(ARG0)
rel (“Many tourists”, “meet”)
�a(“Many tourists”)<latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit><latexit sha1_base64="Ss5Tl4FoKSKJRKvsg4oMl04Ft/0=">AAACSHicbVDLSsNAFJ3Ud31VXboZLKJuSiLiY1d040aoYFVoSp1MbtvBySTM3Igl5Hf8Gd0q+Bm6EXdO2yx8XRg4nHMfZ06QSGHQdV+d0sTk1PTM7Fx5fmFxabmysnpp4lRzaPJYxvo6YAakUNBEgRKuEw0sCiRcBbcnQ/3qDrQRsbrAQQLtiPWU6ArO0FKdSt1v9EUn8xHuMWN5vp35o6XZmQiV6PXxWKaQ07F+c3PG1ICiPW6dma2tPN/pVKpuzR0V/Qu8AlRJUY1O5d0PY55GoJBLZkzLcxNsZ0yj4BLysp8aSBi/ZT1oWahYBKadjUzldNMyIe3G2j6FdMR+n8hYZMwgCmxnxLBvfmtD8j+tFd6JxBS37sfHfjrB7mE7EypJERQfG+mm0mZBh7HSUGjgKAcWMK6F/QvlfaYZRxt+2Ybk/Y7kL2ju1o5q3vletb5fpDVL1skG2SYeOSB1ckoapEk4eSBP5Jm8OI/Om/PhfI5bS04xs0Z+VKn0Bf4TtQM=</latexit>
�p(“meet”)
�(“Many tourists”, “meet”,ARG0) �(“Many tourists”, “meet”,ARG1)
�(“Many tourists”, “meet”, ✏) = 0
ARG0
(1) Span Representations (2) Local Label Classifiers (3) Span
Pruning
�(pred, arg, l) = �a(arg) + �p(pred) + �(l)rel(arg, pred)
<latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit><latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit><latexit sha1_base64="F2SSBYlFAcaRsK5CIxHyBXBOdoE=">AAADGXichVLLattAFB2pr9R9Oc2ym6Gm4NBg5FBosiiYdpNNqQt1E7BcMxpdS0NGIzFzlcaI+ZN2m/xHV6Xbrvob/YKOZYXGTiAXBs59nDNnLhMVUhgMgj+ef+v2nbv3Nu63Hjx89PhJe/PpZ5OXmsOI5zLXRxEzIIWCEQqUcFRoYFkk4TA6frfoH56ANiJXn3BewCRjiRIzwRm60nTTa4dFKrpVWEtV70WsRJLiW1mCDRFOsXJysd2hy4TpxLrkYvyDZioBK+02fUNb4TAV06oZtPZ60ctC2/TlKqu4gVV7+U/7UnXXnFBnxV6oaZDW0htt7KyKT9udoBfUQa+CfgM6pInhtP03jHNeZqCQS2bMuB8UOHHSKLgE2wpLAwXjxyyBsYOKZWAmVe3J0heuEtNZrt1RSOvqZUbFMmPmWeQmM4apWe8titf1xvGJKExz1+nyslUnONubVEIVJYLiSyOzUlLM6eKf0Fho4CjnDjCuhXsL5SnTjKP7TS23pP76Sq6C0W5vvxd8fNUZDJptbZBn5Dnpkj55TQbkgAzJiHDvq/fdO/PO/W/+D/+n/2s56nsNZ4ushP/7H61XBJ4=</latexit>
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
Many tourists
tourists visit Disney
Disney to meet their
their favorite cartoon
cartoon characters
33
(1) Span Representations
(2) Local Label Classifiers (3) Span Pruning
O(n2) arguments, O(n) predicates, —> O(n3) edges!
Input sentence
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
Many tourists
tourists visit Disney
Disney to meet their
their favorite cartoon
cartoon characters
Only keep top O(n) spans using their unary scores
34
(1) Span Representations
(2) Local Label Classifiers (3) Span Pruning
�a(“many tourists”) = 2.5<latexit sha1_base64="6qwwvAYOfJnvm20og4XIEFylnCM=">AAACTXicbVHLSgMxFM3Ud31VXboJFlE3ZUYU3QhFXbgRKlgVOqVmMrdtMJMZkjtiGeaH/Bnd6mcI7kRM21n4uhA4nHNv7slJkEhh0HVfnNLE5NT0zOxceX5hcWm5srJ6ZeJUc2jyWMb6JmAGpFDQRIESbhINLAokXAd3J0P9+h60EbG6xEEC7Yj1lOgKztBSncqp3+iLTuYjPGDG8nw780eXZuciVKLXx2OZQk7H+u1txNSAol1unZmtrTzfOdqt7XcqVbfmjor+BV4BqqSoRqfy5ocxTyNQyCUzpuW5CbYzplFwCXnZTw0kjN+xHrQsVCwC085GxnK6aZmQdmNtj0I6Yr9PZCwyZhAFtjNi2De/tSH5n9YK70Viil0P42U/nWD3sJ0JlaQIio+NdFNp86DDaGkoNHCUAwsY18K+hfI+04yj/YCyDcn7HclfcLVb89yad7FXrdeLuGbJOtkg28QjB6ROzkiDNAknj+SZvJBX58l5dz6cz3FrySlm1siPKs18AUX9tlM=</latexit><latexit sha1_base64="6qwwvAYOfJnvm20og4XIEFylnCM=">AAACTXicbVHLSgMxFM3Ud31VXboJFlE3ZUYU3QhFXbgRKlgVOqVmMrdtMJMZkjtiGeaH/Bnd6mcI7kRM21n4uhA4nHNv7slJkEhh0HVfnNLE5NT0zOxceX5hcWm5srJ6ZeJUc2jyWMb6JmAGpFDQRIESbhINLAokXAd3J0P9+h60EbG6xEEC7Yj1lOgKztBSncqp3+iLTuYjPGDG8nw780eXZuciVKLXx2OZQk7H+u1txNSAol1unZmtrTzfOdqt7XcqVbfmjor+BV4BqqSoRqfy5ocxTyNQyCUzpuW5CbYzplFwCXnZTw0kjN+xHrQsVCwC085GxnK6aZmQdmNtj0I6Yr9PZCwyZhAFtjNi2De/tSH5n9YK70Viil0P42U/nWD3sJ0JlaQIio+NdFNp86DDaGkoNHCUAwsY18K+hfI+04yj/YCyDcn7HclfcLVb89yad7FXrdeLuGbJOtkg28QjB6ROzkiDNAknj+SZvJBX58l5dz6cz3FrySlm1siPKs18AUX9tlM=</latexit><latexit sha1_base64="6qwwvAYOfJnvm20og4XIEFylnCM=">AAACTXicbVHLSgMxFM3Ud31VXboJFlE3ZUYU3QhFXbgRKlgVOqVmMrdtMJMZkjtiGeaH/Bnd6mcI7kRM21n4uhA4nHNv7slJkEhh0HVfnNLE5NT0zOxceX5hcWm5srJ6ZeJUc2jyWMb6JmAGpFDQRIESbhINLAokXAd3J0P9+h60EbG6xEEC7Yj1lOgKztBSncqp3+iLTuYjPGDG8nw780eXZuciVKLXx2OZQk7H+u1txNSAol1unZmtrTzfOdqt7XcqVbfmjor+BV4BqqSoRqfy5ocxTyNQyCUzpuW5CbYzplFwCXnZTw0kjN+xHrQsVCwC085GxnK6aZmQdmNtj0I6Yr9PZCwyZhAFtjNi2De/tSH5n9YK70Viil0P42U/nWD3sJ0JlaQIio+NdFNp86DDaGkoNHCUAwsY18K+hfI+04yj/YCyDcn7HclfcLVb89yad7FXrdeLuGbJOtkg28QjB6ROzkiDNAknj+SZvJBX58l5dz6cz3FrySlm1siPKs18AUX9tlM=</latexit><latexit sha1_base64="6qwwvAYOfJnvm20og4XIEFylnCM=">AAACTXicbVHLSgMxFM3Ud31VXboJFlE3ZUYU3QhFXbgRKlgVOqVmMrdtMJMZkjtiGeaH/Bnd6mcI7kRM21n4uhA4nHNv7slJkEhh0HVfnNLE5NT0zOxceX5hcWm5srJ6ZeJUc2jyWMb6JmAGpFDQRIESbhINLAokXAd3J0P9+h60EbG6xEEC7Yj1lOgKztBSncqp3+iLTuYjPGDG8nw780eXZuciVKLXx2OZQk7H+u1txNSAol1unZmtrTzfOdqt7XcqVbfmjor+BV4BqqSoRqfy5ocxTyNQyCUzpuW5CbYzplFwCXnZTw0kjN+xHrQsVCwC085GxnK6aZmQdmNtj0I6Yr9PZCwyZhAFtjNi2De/tSH5n9YK70Viil0P42U/nWD3sJ0JlaQIio+NdFNp86DDaGkoNHCUAwsY18K+hfI+04yj/YCyDcn7HclfcLVb89yad7FXrdeLuGbJOtkg28QjB6ROzkiDNAknj+SZvJBX58l5dz6cz3FrySlm1siPKs18AUX9tlM=</latexit>
�a(“tourists visit Disney”) = �0.8<latexit sha1_base64="Hd3Kgz6SWV4jo9WWpZUelG17YmI=">AAACVnicbVHBattAEF2rTeI6baKkx16WmpD0UCOFQnMJmLSHXAIu1EnAMs5qNbYHr1Zid2RshL6qP9Nc2z/IB5SubR2apAMLj/dmdt6+jXOFloLgvuG9eLm1vdN81dp9/WZv3z84vLZZYST0ZaYycxsLCwo19AlJwW1uQKSxgpt49mWl38zBWMz0d1rmMEzFROMYpSBHjfyrqDfFURkRLKgUVXVSRutLyytMNE6mdKEKqPhGv7sjt9eZsnyOFol/RatheXxcVR/OPwads5HfDjrBuvhzENagzerqjfyHKMlkkYImqYS1gzDIaVgKQygVVK2osJALORMTGDioRQp2WK4dVvzIMQkfZ8YdTXzN/jtRitTaZRq7zlTQ1D7VVuT/tEEyx9zWuxabZY+d0PhsWKLOCwItN0bGheKU8VXGPEEDktTSASENurdwORVGSHI/0XIhhU8jeQ6uTzth0Am/fWp3u3VcTfaOvWcnLGSfWZddsh7rM8l+sHv2i/1u/Gz88ba8nU2r16hn3rJH5fl/AZtSt+M=</latexit><latexit sha1_base64="Hd3Kgz6SWV4jo9WWpZUelG17YmI=">AAACVnicbVHBattAEF2rTeI6baKkx16WmpD0UCOFQnMJmLSHXAIu1EnAMs5qNbYHr1Zid2RshL6qP9Nc2z/IB5SubR2apAMLj/dmdt6+jXOFloLgvuG9eLm1vdN81dp9/WZv3z84vLZZYST0ZaYycxsLCwo19AlJwW1uQKSxgpt49mWl38zBWMz0d1rmMEzFROMYpSBHjfyrqDfFURkRLKgUVXVSRutLyytMNE6mdKEKqPhGv7sjt9eZsnyOFol/RatheXxcVR/OPwads5HfDjrBuvhzENagzerqjfyHKMlkkYImqYS1gzDIaVgKQygVVK2osJALORMTGDioRQp2WK4dVvzIMQkfZ8YdTXzN/jtRitTaZRq7zlTQ1D7VVuT/tEEyx9zWuxabZY+d0PhsWKLOCwItN0bGheKU8VXGPEEDktTSASENurdwORVGSHI/0XIhhU8jeQ6uTzth0Am/fWp3u3VcTfaOvWcnLGSfWZddsh7rM8l+sHv2i/1u/Gz88ba8nU2r16hn3rJH5fl/AZtSt+M=</latexit><latexit sha1_base64="Hd3Kgz6SWV4jo9WWpZUelG17YmI=">AAACVnicbVHBattAEF2rTeI6baKkx16WmpD0UCOFQnMJmLSHXAIu1EnAMs5qNbYHr1Zid2RshL6qP9Nc2z/IB5SubR2apAMLj/dmdt6+jXOFloLgvuG9eLm1vdN81dp9/WZv3z84vLZZYST0ZaYycxsLCwo19AlJwW1uQKSxgpt49mWl38zBWMz0d1rmMEzFROMYpSBHjfyrqDfFURkRLKgUVXVSRutLyytMNE6mdKEKqPhGv7sjt9eZsnyOFol/RatheXxcVR/OPwads5HfDjrBuvhzENagzerqjfyHKMlkkYImqYS1gzDIaVgKQygVVK2osJALORMTGDioRQp2WK4dVvzIMQkfZ8YdTXzN/jtRitTaZRq7zlTQ1D7VVuT/tEEyx9zWuxabZY+d0PhsWKLOCwItN0bGheKU8VXGPEEDktTSASENurdwORVGSHI/0XIhhU8jeQ6uTzth0Am/fWp3u3VcTfaOvWcnLGSfWZddsh7rM8l+sHv2i/1u/Gz88ba8nU2r16hn3rJH5fl/AZtSt+M=</latexit><latexit sha1_base64="Hd3Kgz6SWV4jo9WWpZUelG17YmI=">AAACVnicbVHBattAEF2rTeI6baKkx16WmpD0UCOFQnMJmLSHXAIu1EnAMs5qNbYHr1Zid2RshL6qP9Nc2z/IB5SubR2apAMLj/dmdt6+jXOFloLgvuG9eLm1vdN81dp9/WZv3z84vLZZYST0ZaYycxsLCwo19AlJwW1uQKSxgpt49mWl38zBWMz0d1rmMEzFROMYpSBHjfyrqDfFURkRLKgUVXVSRutLyytMNE6mdKEKqPhGv7sjt9eZsnyOFol/RatheXxcVR/OPwads5HfDjrBuvhzENagzerqjfyHKMlkkYImqYS1gzDIaVgKQygVVK2osJALORMTGDioRQp2WK4dVvzIMQkfZ8YdTXzN/jtRitTaZRq7zlTQ1D7VVuT/tEEyx9zWuxabZY+d0PhsWKLOCwItN0bGheKU8VXGPEEDktTSASENurdwORVGSHI/0XIhhU8jeQ6uTzth0Am/fWp3u3VcTfaOvWcnLGSfWZddsh7rM8l+sHv2i/1u/Gz88ba8nU2r16hn3rJH5fl/AZtSt+M=</latexit>
...<latexit sha1_base64="7Iu17wX+vYl0vGV+JihJ1bF/x/o=">AAACEXicbVDLSsNAFJ3UV62vqks3wSK4CokIuiy4cVnBPrANZTK5bYdOJmHmprSEfoVu9T/ciVu/wN/wC5y2WdjWAwOHc+7lnjlBIrhG1/22ChubW9s7xd3S3v7B4VH5+KSh41QxqLNYxKoVUA2CS6gjRwGtRAGNAgHNYHg385sjUJrH8hEnCfgR7Uve44yikZ46CGPMHMeZdssV13HnsNeJl5MKyVHrln86YczSCCQyQbVue26CfkYVciZgWuqkGhLKhrQPbUMljUD72Tzx1L4wSmj3YmWeRHuu/t3IaKT1JArMZERxoFe9mfif1w5HPNH5rfHi2HIS7N36GZdJiiDZIkgvFTbG9qweO+QKGIqJIZQpbv5iswFVlKEpsWRK8lYrWSeNK8dzHe/hulKt5nUVyRk5J5fEIzekSu5JjdQJI5K8kFfyZj1b79aH9bkYLVj5zilZgvX1C658nj8=</latexit><latexit sha1_base64="7Iu17wX+vYl0vGV+JihJ1bF/x/o=">AAACEXicbVDLSsNAFJ3UV62vqks3wSK4CokIuiy4cVnBPrANZTK5bYdOJmHmprSEfoVu9T/ciVu/wN/wC5y2WdjWAwOHc+7lnjlBIrhG1/22ChubW9s7xd3S3v7B4VH5+KSh41QxqLNYxKoVUA2CS6gjRwGtRAGNAgHNYHg385sjUJrH8hEnCfgR7Uve44yikZ46CGPMHMeZdssV13HnsNeJl5MKyVHrln86YczSCCQyQbVue26CfkYVciZgWuqkGhLKhrQPbUMljUD72Tzx1L4wSmj3YmWeRHuu/t3IaKT1JArMZERxoFe9mfif1w5HPNH5rfHi2HIS7N36GZdJiiDZIkgvFTbG9qweO+QKGIqJIZQpbv5iswFVlKEpsWRK8lYrWSeNK8dzHe/hulKt5nUVyRk5J5fEIzekSu5JjdQJI5K8kFfyZj1b79aH9bkYLVj5zilZgvX1C658nj8=</latexit><latexit sha1_base64="7Iu17wX+vYl0vGV+JihJ1bF/x/o=">AAACEXicbVDLSsNAFJ3UV62vqks3wSK4CokIuiy4cVnBPrANZTK5bYdOJmHmprSEfoVu9T/ciVu/wN/wC5y2WdjWAwOHc+7lnjlBIrhG1/22ChubW9s7xd3S3v7B4VH5+KSh41QxqLNYxKoVUA2CS6gjRwGtRAGNAgHNYHg385sjUJrH8hEnCfgR7Uve44yikZ46CGPMHMeZdssV13HnsNeJl5MKyVHrln86YczSCCQyQbVue26CfkYVciZgWuqkGhLKhrQPbUMljUD72Tzx1L4wSmj3YmWeRHuu/t3IaKT1JArMZERxoFe9mfif1w5HPNH5rfHi2HIS7N36GZdJiiDZIkgvFTbG9qweO+QKGIqJIZQpbv5iswFVlKEpsWRK8lYrWSeNK8dzHe/hulKt5nUVyRk5J5fEIzekSu5JjdQJI5K8kFfyZj1b79aH9bkYLVj5zilZgvX1C658nj8=</latexit><latexit sha1_base64="7Iu17wX+vYl0vGV+JihJ1bF/x/o=">AAACEXicbVDLSsNAFJ3UV62vqks3wSK4CokIuiy4cVnBPrANZTK5bYdOJmHmprSEfoVu9T/ciVu/wN/wC5y2WdjWAwOHc+7lnjlBIrhG1/22ChubW9s7xd3S3v7B4VH5+KSh41QxqLNYxKoVUA2CS6gjRwGtRAGNAgHNYHg385sjUJrH8hEnCfgR7Uve44yikZ46CGPMHMeZdssV13HnsNeJl5MKyVHrln86YczSCCQyQbVue26CfkYVciZgWuqkGhLKhrQPbUMljUD72Tzx1L4wSmj3YmWeRHuu/t3IaKT1JArMZERxoFe9mfif1w5HPNH5rfHi2HIS7N36GZdJiiDZIkgvFTbG9qweO+QKGIqJIZQpbv5iswFVlKEpsWRK8lYrWSeNK8dzHe/hulKt5nUVyRk5J5fEIzekSu5JjdQJI5K8kFfyZj1b79aH9bkYLVj5zilZgvX1C658nj8=</latexit>
F1
60
65
70
75
80
85
90
CoNL05 WSJ Test CoNL05 Brown Test CoNLL2012 (OntoNotes)
79.8
70.8
82.5
76.8
68.5
81.2
DeepSRL LSGN DeepSRL(Ensemble) LSGN+ELMo
35
End-to-End SRL Results
BIO-based, pipelined predicate ID
F1
60
65
70
75
80
85
90
CoNL05 WSJ Test CoNL05 Brown Test CoNLL2012 (OntoNotes)
82.9
76.1
86
78.4
70.1
82.779.8
70.8
82.5
76.8
68.5
81.2
DeepSRL LSGN DeepSRL (Ensemble) LSGN+ELMo
36
End-to-End SRL Results
With ELMo, over 3 points improvement over SotA ensemble!
*ELMo: Deep Contextualized Word Representations, Peters et al., 2018
F1
60
65
70
75
80
85
90
CoNL05 WSJ Test CoNL05 Brown Test CoNLL2012 (OntoNotes)
82.9
76.1
86
78.4
70.1
82.779.8
70.8
82.5
76.8
68.5
81.2
DeepSRL LSGN DeepSRL (Ensemble) LSGN+ELMo
36
End-to-End SRL Results
With ELMo, over 3 points improvement over SotA ensemble!
*ELMo: Deep Contextualized Word Representations, Peters et al., 2018
• New SotA (Strubell et al., 2018) with syntax-informed transformer model.
F1
60
65
70
75
80
85
90
Täckström15 FitzGerald15* Zhou15 DeepSRL17* Tan18* LSGN+ELMo
80.4
74.873.6
69.472.271.3
87.486.184.6
82.880.379.9
WSJ Test Brown (out-domain) Test
Deep BIO models
*:Ensemble models
37
Gold Predicates: CoNLL 2005 SRL Results
Pipeline models
F1
60
65
70
75
80
85
90
Täckström15 FitzGerald15* Zhou15 DeepSRL17* Tan18* LSGN+ELMo
80.4
74.873.6
69.472.271.3
87.486.184.6
82.880.379.9
WSJ Test Brown (out-domain) Test
Deep BIO models
*:Ensemble models
37
Gold Predicates: CoNLL 2005 SRL Results
Pipeline models
Transformer-style BIO-tagger
F1
60
65
70
75
80
85
90
Täckström15 FitzGerald15* Zhou15 DeepSRL17* Tan18* LSGN+ELMo
80.4
74.873.6
69.472.271.3
87.486.184.6
82.880.379.9
WSJ Test Brown (out-domain) Test
Deep BIO models
*:Ensemble models
37
Gold Predicates: CoNLL 2005 SRL Results
Pipeline models
Transformer-style BIO-tagger
•In-domain: 40% error reduction over pre-neural models.
•Out-domain: Reaching 80% F1.
F1
60
65
70
75
80
85
90
Täckström15 FitzGerald15* Zhou15 DeepSRL17* Tan18* LSGN+ELMo
80.4
74.873.6
69.472.271.3
87.486.184.6
82.880.379.9
WSJ Test Brown (out-domain) Test
Deep BIO models
*:Ensemble models
38
Gold Predicates: CoNLL 2005 SRL Results
Pipeline models
• New results (Ouchi. et al, 2018) with span-selection model +ELMo.
!39
Related Work on Span-based Models
Nesting Spans Span Feature
BIO-Taggers (Collobert et al., 2010, Chiu and Nichols, 2016, DeepSRL) No No
Semi-Markov Models (Kong et al., 2016) No Yes
LSGN Yes Yes
40
Span-based vs. BIO
DeepSRL (BIO)
LSGN(Span-based)
Inputs (Sentence, Predicate) Sentence
Predicate Identification Pipelined Joint
Global Consistency
Long-range Dependency
40
Span-based vs. BIO
DeepSRL (BIO)
LSGN(Span-based)
Inputs (Sentence, Predicate) Sentence
Predicate Identification Pipelined Joint
Global Consistency
Long-range Dependency
Due to the strong independence assumption LSGN makes
40
Span-based vs. BIO
DeepSRL (BIO)
LSGN(Span-based)
Inputs (Sentence, Predicate) Sentence
Predicate Identification Pipelined Joint
Global Consistency
Long-range Dependency
Due to the strong independence assumption LSGN makes
By allowing direct interaction between predicates and arguments
41
Outline
Predicting SRL with Deep BiLSTMs— DeepSRL
Towards Unified and Full-text Semantic Analysis— Multi-task learning with LSGN; ScienceIE (Luan et al., 2018)
An End-to-End, Span-based SRL Model— Labeled Span Graph Network (LSGN)
☑A︎ccurate ☑N︎o NLP pipeline ☑J︎oint predicate ID ☐Full-text Semantics
!42
On January 13, 2018, a false ballistic missile alert was issued via
the Emergency Alert System and Commercial Mobile Alert System over
television, radio, and cellphones in the U.S. state of Hawaii.
The alert stated that there was an incoming ballistic missile threat to Hawaii,
advised residents to seek shelter, and concluded "This is not a drill".
The message was sent at 8:07 a.m. local time.
LOC LOC
SRLCoreference
NERARG1 AM-TMP
From Wikipedia: 2018 Hawaii false missile alert. Only part of the structures are visualized.
Document-level Understanding
ARG0 ARG1
ARG1AM-TMP
!42
On January 13, 2018, a false ballistic missile alert was issued via
the Emergency Alert System and Commercial Mobile Alert System over
television, radio, and cellphones in the U.S. state of Hawaii.
The alert stated that there was an incoming ballistic missile threat to Hawaii,
advised residents to seek shelter, and concluded "This is not a drill".
The message was sent at 8:07 a.m. local time.
LOC LOC
SRLCoreference
NERARG1 AM-TMP
From Wikipedia: 2018 Hawaii false missile alert. Only part of the structures are visualized.
Document-level Understanding
ARG0 ARG1
ARG1AM-TMP
How can we build one model to predict all of them?
LSGN: • Unified view on span-span relations• General purpose span representations:
Minimizes task-specific engineering!
Input Document
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
Multi-task LSGN Architecture
Node & Edge Scores
LabelingSoftmax
43
Input Document
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
Multi-task LSGN Architecture
Node & Edge Scores
LabelingSoftmax
43
Shared span representations
Input Document
Word & CharEmbeddings
HighwayBiLSTMs
Many tourists visit Disney to meet their favorite cartoon characters
Span Representation
Multi-task LSGN Architecture
Node & Edge Scores
LabelingSoftmax
43
Shared span representations
Lightweight, task-specific span classifiers
44
Coreference SRL/Relation Extraction NER
Many tourists Disney their Many tourists meet DisneySpan Rep.
Combined score
Softmax
Edge score
Node score
ARG0 ARG1 ORG PER
null null null
Task-specific Span Classifiers
(Lee et al., 2017)
…
… …
44
Coreference SRL/Relation Extraction NER
Many tourists Disney their Many tourists meet DisneySpan Rep.
Combined score
Softmax
Edge score
Node score
ARG0 ARG1 ORG PER
null null null
Task-specific Span Classifiers
(Lee et al., 2017)
Shared span representations! (Efficiency gain)
…
… …
44
Coreference SRL/Relation Extraction NER
Many tourists Disney their Many tourists meet DisneySpan Rep.
Combined score
Softmax
Edge score
Node score
ARG0 ARG1 ORG PER
null null null
Task-specific Span Classifiers
(Lee et al., 2017)
Shared span representations! (Efficiency gain)
Multi-task learning objective
…
… …
45
Two multi-tasking setups
1. “One model for everything”: Train one model to predict all the n tasks. Performance is tuned on the average of the n metrics …
45
Two multi-tasking setups
1. “One model for everything”: Train one model to predict all the n tasks. Performance is tuned on the average of the n metrics …
2. “Let the tasks help each other”:Train n models, each predicts a target task, and treat the rest n-1 tasks as auxiliary. (Swayamdipta et al., 2018 …)
F1
60
70
80
90
E2E SRL Coref NER
87.2
68.8
78.4
DeepSRL17* Lee17* Li17 LSGN (3 models) LSGN (1 model)
46
*:Ensemble models
OntoNotes: Is it possible to have one model to do them all?Previous systems
F1
60
70
80
90
E2E SRL Coref NER
89
69.7
79.8
87.2
68.8
78.4
DeepSRL17* Lee17* Li17 LSGN (3 models) LSGN (1 model)
47
*:Ensemble models
OntoNotes: Is it possible to have one model to do them all?
LSGN (3 models, no task sharing): Almost 1 point improvement over all previous SotA! (Coref. improvement
due to larger model capacity)
F1
60
70
80
90
E2E SRL Coref NER
88
69.1
79.2
89
69.7
79.8
87.2
68.8
78.4
DeepSRL17* Lee17* Li17 LSGN (3 models) LSGN (1 model)
48
*:Ensemble models
OntoNotes: Is it possible to have one model to do them all?
One model for all tasks, treating them as equally important.
(Only modest accuracy loss …)
49
Coreference Relation Extraction Entity Extraction
MORPA PCFG It MORPA PCFG MORPASpan Rep.
Combined score
Softmax
Edge score
Node score
Hyponym-of Used-for TASK METHOD
null null null
ScienceERC Data (Luan et al., 2018): Can the tasks help each other?
…
… …
F1
20
30
40
50
60
70
Relation Coref. Entity
46.2
61.2
31.4
Miwa16 Miwa16 +ELMo Lee17 +ELMo SciIE +ELMo +MTL
50
ScienceERC: Can the tasks help each other?
Previous SotA. Uses dependency features.
F1
20
30
40
50
60
70
Relation Coref. Entity
46.2
63.7
36.6
61.2
31.4
Miwa16 Miwa16 +ELMo Lee17 +ELMo SciIE +ELMo +MTL
51
ScienceERC: Can the tasks help each other?
ELMo is very helpful on this small datasets (500 documents)!
F1
20
30
40
50
60
70
Relation Coref. Entity
64.2
48.2
39.3
46.2
63.7
36.6
61.2
31.4
Miwa16 Miwa16 +ELMo Lee17 +ELMo SciIE +ELMo +MTL
52
ScienceERC: Can the tasks help each other?
3 models for 3 tasks:Each task is tuned individually, bringing large gains from MTL.
53
ScienceERC: Ablations
53
ScienceERC: Ablations
30
40
50
60
70
Entity Relation Coreference
Single-taskMulti-task
53
ScienceERC: Ablations
30
40
50
60
70
Entity Relation Coreference
Single-taskMulti-task
55.3
37.9
65.7
53
ScienceERC: Ablations
30
40
50
60
70
Entity Relation Coreference
Single-taskMulti-task
58
39.5
68.1
55.3
37.9
65.7
54
ScienceERC: Qualitative Analysis
With predicted coreference links, the system extracted
less generic terms and more specific ones!
Conclusion
55
• A general framework for a variety of tasks.
Conclusion
55
• Our recipe: 1. Contextualized span
representations 2. Local label classifiers 3. Greedy span pruning
• A general framework for a variety of tasks.
Conclusion
55
• Our recipe: 1. Contextualized span
representations 2. Local label classifiers 3. Greedy span pruning
• A general framework for a variety of tasks.
• Multi-task learning works (sometimes)
Future Work
56
1. LSGNs for more NLP tasks.
2. Improve global consistency of the LSGN outputs with joint inference (e.g. Singh et al., 2013).
3. Pre-train transferrable span embeddings.
Links to Code and Data
57
1. DeepSRL: https://github.com/luheng/deep_srl
2. LSGN:https://github.com/luheng/lsgn
3. SciIE/ScienceERC (by Yi Luan): http://nlp.cs.washington.edu/sciIE/
Many Thanks to my Collaborators!
58
in collaboration with Kenton Lee, Mike Lewis, Omer Levy, Yi Luan, and Luke Zettlemoyer