An Incremental Parser for Abstract Meaning...
Transcript of An Incremental Parser for Abstract Meaning...
![Page 1: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/1.jpg)
An Incremental Parser for Abstract MeaningRepresentation
1 Marco Damonte 1 Shay B. Cohen 2 Giorgio Satta
1 University of Edinburgh2 University of Padua
EACL 2017
1 / 25
![Page 2: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/2.jpg)
AMR
He described her as a genius
describe-01
he genius she
ARG0 ARG1ARG2
2 / 25
![Page 3: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/3.jpg)
Dependency trees
• Transition-based dependency parsing (Nivre, 2004)
My dog also likes eating
root
poss
nsubj
advmodxcomp
3 / 25
![Page 4: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/4.jpg)
Concept identification
The teacher
person
teach-01
ARG0-of
The proposal
thing
propose-01
ARG1-of
10 January 1989
date-entity
101
1989
daymonth
year
4 / 25
![Page 5: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/5.jpg)
Reentrancy
I beg you to excuse me
beg-01
iyou excuse-01
ARG0ARG1 ARG2
ARG0
ARG1
5 / 25
![Page 6: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/6.jpg)
Reentrancy
I beg you to excuse me
beg-01
iyou excuse-01
ARG0ARG1 ARG2
ARG0
ARG1
6 / 25
![Page 7: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/7.jpg)
Reentrancy
I beg you to excuse me
beg-01
iyou excuse-01
ARG0ARG1 ARG2
ARG0
ARG1
7 / 25
![Page 8: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/8.jpg)
Transition-based AMR Parser
8 / 25
![Page 9: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/9.jpg)
Transition system
The boy wants to believe the girl
STACK
boy
GRAPH
9 / 25
![Page 10: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/10.jpg)
Transition system
The boy wants to believe the girl
STACK
want-01
boy
GRAPH
want-01
boy
10 / 25
![Page 11: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/11.jpg)
Transition system
The boy wants to believe the girl
STACK
want-01
GRAPH
want-01
boy
11 / 25
![Page 12: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/12.jpg)
Transition system
The boy wants to believe the girl
STACK
want-01
believe-01
GRAPH
want-01
boy believe-01
12 / 25
![Page 13: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/13.jpg)
Transition system
The boy wants to believe the girl
STACK
want-01
believe-01
girl
GRAPH
want-01
boy believe-01
girl
13 / 25
![Page 14: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/14.jpg)
Transition system
The boy wants to believe the girl
STACK
want-01
believe-01
GRAPH
want-01
boy believe-01
girl
14 / 25
![Page 15: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/15.jpg)
Transition system
The boy wants to believe the girl
STACK
want-01
believe-01
GRAPH
want-01
boy believe-01
girl
15 / 25
![Page 16: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/16.jpg)
Transition system
The boy wants to believe the girl
STACK
want-01
GRAPH
want-01
boy believe-01
girl
16 / 25
![Page 17: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/17.jpg)
Transition system
The boy wants to believe the girl
STACK GRAPH
want-01
boy believe-01
girl
17 / 25
![Page 18: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/18.jpg)
Oracle
• Given the current configuration (σ, β,A) and the gold-standardgraph G = (Vg ,Ag ):
T (G , σ, β,A) =
LARC(`)RARC(`)RED-REENT(`)REDUCESHIFT
• (English, AMR) ⇒ Transitions to obtain AMR* from English
18 / 25
![Page 19: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/19.jpg)
Evaluation
19 / 25
![Page 20: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/20.jpg)
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
![Page 21: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/21.jpg)
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
![Page 22: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/22.jpg)
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
![Page 23: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/23.jpg)
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
![Page 24: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/24.jpg)
Fine-grained evaluation
• Smatch. Cai and Knight (2013)
• Unlabeled. Smatch score after removing edge labels
• No WSD. Smatch score while ignoring Propbank senses
• Reentrancy. Smatch computed on reentrant edges
• Semantic Role Labelling. Smatch computed on :ARG roles
20 / 25
![Page 25: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/25.jpg)
Fine-grained evaluation (cont’d)
• Concepts. F-score on the concept identification task
• Negations. F-score on :polarity roles
• Named Entities. F-score on :name roles
• Wikification. F-score on :wiki roles
21 / 25
![Page 26: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/26.jpg)
Fine-grained evaluation (cont’d)
• Concepts. F-score on the concept identification task
• Negations. F-score on :polarity roles
• Named Entities. F-score on :name roles
• Wikification. F-score on :wiki roles
21 / 25
![Page 27: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/27.jpg)
Fine-grained evaluation (cont’d)
• Concepts. F-score on the concept identification task
• Negations. F-score on :polarity roles
• Named Entities. F-score on :name roles
• Wikification. F-score on :wiki roles
21 / 25
![Page 28: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/28.jpg)
Fine-grained evaluation (cont’d)
• Concepts. F-score on the concept identification task
• Negations. F-score on :polarity roles
• Named Entities. F-score on :name roles
• Wikification. F-score on :wiki roles
21 / 25
![Page 29: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/29.jpg)
Experiments
Metric JAMR (’14) CAMR JAMR (’16) OursSmatch 58 63 67 64Unlabeled 61 69 69 69No WSD 58 64 68 65NP-only 47 54 58 55Reentrancy 38 41 42 41Concepts 79 80 83 83Named Ent. 75 75 79 83Wikification 0 0 75 64Negations 16 18 45 48SRL 55 60 60 56
JAMR: Flanigan et al. (2014)
CAMR: Wang et al. (2015)
22 / 25
![Page 30: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/30.jpg)
Software
• Online demo:http://cohort.inf.ed.ac.uk/amreager.html
• Source code for parser:https://github.com/mdtux89/amr-eager
• Source code for evaluation:https://github.com/mdtux89/amr-evaluation
23 / 25
![Page 31: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/31.jpg)
Demo
24 / 25
![Page 32: An Incremental Parser for Abstract Meaning Representationmdtux89.github.io/assets/eacl2017_slides.pdf · The boy wants to believe the girl STACK want-01 believe-01 GRAPH want-01 boy](https://reader033.fdocuments.us/reader033/viewer/2022053101/60618ea7525600319e3ccacd/html5/thumbnails/32.jpg)
Conclusions
• AMREager is a linear-time, left-to-right transition system
• AMR parsing akin to dependency parsing
• Fine-grained evaluation suite to assess AMR parsers
25 / 25