Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning*...
Transcript of Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning*...
![Page 1: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/1.jpg)
Dependency Grammar
Introduc)on
![Page 2: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/2.jpg)
Christopher Manning
Dependency syntax postulates that syntac)c structure consists of rela)ons between lexical items, normally binary asymmetric rela)ons (“arrows”) called dependencies
Dependency Grammar and Dependency Structure
submitted
Bills were
Brownback
Senator
immigration
by
and
ports
on
Republican
Kansas
of
![Page 3: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/3.jpg)
Christopher Manning
Dependency syntax postulates that syntac)c structure consists of rela)ons between lexical items, normally binary asymmetric rela)ons (“arrows”) called dependencies
The arrows are commonly typed with the name of gramma)cal rela)ons (subject, preposi)onal object, apposi)on, etc.)
Dependency Grammar and Dependency Structure
submitted
Bills were
Brownback
Senator
nsubjpass auxpass prep
nn
immigration
conj
by
cc
and
ports
pobj
prep
on
pobj
Republican
Kansas
pobj
prep
of
appos
![Page 4: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/4.jpg)
Christopher Manning
Dependency syntax postulates that syntac)c structure consists of rela)ons between lexical items, normally binary asymmetric rela)ons (“arrows”) called dependencies
The arrow connects a head (governor, superior, regent) with a dependent (modifier, inferior, subordinate)
Usually, dependencies form a tree (connected, acyclic, single-‐head)
Dependency Grammar and Dependency Structure
submitted
Bills were
Brownback
Senator
nsubjpass auxpass prep
nn
immigration
conj
by
cc
and
ports
pobj
prep
on
pobj
Republican
Kansas
pobj
prep
of
appos
![Page 5: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/5.jpg)
Christopher Manning
ROOT Discussion of the outstanding issues was completed .
Dependency Grammar and Dependency Structure
• Some people draw the arrows one way; some the other way! • Tesnière had them point from head to dependent…
• Usually add a fake ROOT so every word is a dependent of precisely 1 other node
![Page 6: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/6.jpg)
Christopher Manning
Dependency Grammar/Parsing History
• The idea of dependency structure goes back a long way • To Pāṇini’s grammar (c. 5th century BCE)
• Basic approach of 1st millennium Arabic grammarians
• Cons)tuency is a new-‐fangled inven)on • 20th century inven)on (R.S. Wells, 1947)
• Modern dependency work o^en linked to work of L. Tesnière (1959) • Was dominant approach in “East” (Russia, China, …) • Good for free-‐er word order languages
• Among the earliest kinds of parsers in NLP, even in the US: • David Hays, one of the founders of U.S. computa)onal linguis)cs, built early (first?) dependency parser (Hays 1962)
![Page 7: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/7.jpg)
Christopher Manning
• A dependency grammar has a no)on of a head. Officially, CFGs don’t. • But modern linguis)c theory and all modern sta)s)cal parsers (Charniak,
Collins, Stanford, …) do, via hand-‐wrihen phrasal “head rules”: • The head of a Noun Phrase is a noun/number/adj/… • The head of a Verb Phrase is a verb/modal/….
• The head rules can be used to extract a dependency parse from a CFG parse
• The closure of dependencies give cons)tuency from a dependency tree
• But the dependents of a word must be at the same level (i.e., “flat”) – there can be no VP!
Rela9on between phrase structure and dependency structure
![Page 8: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/8.jpg)
Christopher Manning
What are the sources of informa)on for dependency parsing? 1. Bilexical affini)es [issues à the] is plausible
2. Dependency distance mostly with nearby words 3. Intervening material
Dependencies rarely span intervening verbs or punctua)on
4. Valency of heads How many dependents on which side are usual for a head?
ROOT Discussion of the outstanding issues was completed .
Dependency Condi9oning Preferences
![Page 9: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/9.jpg)
Christopher Manning
Dependency Parsing
• A sentence is parsed by choosing for each word what other word (including ROOT) that it is a dependent of.
• Usually some constraints: • Only one word is a dependent of ROOT • Don’t want cycles A → B, B → A
• This makes the dependencies a tree • Final issue is whether arrows can cross (non-‐projec)ve) or not
9
I give a on bootstrapping talk tomorrow ROOT ’ll
![Page 10: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/10.jpg)
Christopher Manning
Methods of Dependency Parsing
1. Dynamic programming (like in the CKY algorithm) You can do it similarly to lexicalized PCFG parsing: an O(n5) algorithm Eisner (1996) gives a clever algorithm that reduces the complexity to O(n3), by producing parse items with heads at the ends rather than in the middle
2. Graph algorithms You create a Minimum Spanning Tree for a sentence McDonald et al.’s (2005) MSTParser scores dependencies independently using a ML classifier (he uses MIRA, for online learning, but it can be something else)
3. Constraint Sa)sfac)on Edges are eliminated that don’t sa)sfy hard constraints. Karlsson (1990), etc.
4. “Determinis)c parsing” Greedy choice of ahachments guided by good machine learning classifiers MaltParser (Nivre et al. 2008)
![Page 11: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/11.jpg)
DP for genera9ve dependency grammars
![Page 12: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/12.jpg)
Christopher Manning
Probabilistic dependency grammar: generative model
1. Start with left wall $ 2. Generate root w0
3. Generate left children w-1, w-2, ..., w-ℓ from the FSA λw0
4. Generate right children w1, w2, ..., wr from the FSA ρw0
5. Recurse on each wi for i in {-ℓ, ..., -1, 1, ..., r}, sampling αi
(steps 2-4) 6. Return αℓ...α-1w0α1...αr
w0
w-1
w-2
w-ℓ wr
w2
w1
... ...
w-ℓ.-1
$
λw-ℓ
λw0 ρw0
These 5 slides are based on slides by Jason Eisner
and Noah Smith
![Page 13: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/13.jpg)
Christopher Manning
Naïve Recognition/Parsing
It takes two to tango
It takes two to tango
to takes
takes
takes
O(n5) combinations
It
p
p c i j k
r 0 n
goal
goal
![Page 14: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/14.jpg)
Christopher Manning
Dependency Grammar Cubic Recognition/Parsing (Eisner & Satta, 1999)
• Triangles: span over words, where tall side of triangle is the head, other side is dependent, and no non-head words expecting more dependents
• Trapezoids: span over words, where larger side is head, smaller side is dependent, and smaller side is still looking for dependents on its side of the trapezoid
}
}
![Page 15: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/15.jpg)
Christopher Manning
Dependency Grammar Cubic Recognition/Parsing (Eisner & Satta, 1999)
It takes two to tango
goal
One trapezoid per
dependency.
A triangle is a head with some left (or
right) subtrees.
![Page 16: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/16.jpg)
Christopher Manning
Cubic Recognition/Parsing (Eisner & Satta, 1999)
i j k i j k
i j k i j k
O(n3) combinations
O(n3) combinations
0 i n
goal
Gives O(n3) dependency grammar parsing
O(n) combinations
![Page 17: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/17.jpg)
Graph Algorithms: MSTs
17
![Page 18: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/18.jpg)
Christopher Manning
McDonald et al. (2005 ACL) Online Large-‐Margin Training of Dependency Parsers
• One of two best-‐known recent dependency parsers • Score of a dependency tree = sum of scores of dependencies
• Scores are independent of other dependencies • If scores are available, parsing can be solved as a minimum
spanning tree problem • Chiu-‐Liu-‐Edmonds algorithm
• One then needs a score for dependencies
![Page 19: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/19.jpg)
Christopher Manning
McDonald et al. (2005 ACL): Online Large-‐Margin Training of Dependency Parsers
• Edge scoring is via a discrimina)ve classifier • Can condi)on on rich features in that context • Each dependency is a linear func)on of features )mes weights
• Feature weights were learned by MIRA, an online large-‐margin algorithm • But you could use an SVM, maxent, or a perceptron
• Features cover: • Head and dependent word and POS separately • Head and dependent word and POS bigram features
• Words between head and dependent
• Length and direc)on of dependency
![Page 20: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/20.jpg)
Greedy Transi9on-‐Based
Parsing
MaltParser
![Page 21: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/21.jpg)
Christopher Manning
MaltParser [Nivre et al. 2008]
• A simple form of greedy discrimina)ve dependency parser • The parser does a sequence of bohom up ac)ons
• Roughly like “shi^” or “reduce” in a shi^-‐reduce parser, but the “reduce” ac)ons are specialized to create dependencies with head on le^ or right
• The parser has: • a stack σ, wrihen with top to the right • which starts with the ROOT symbol
• a buffer β, wrihen with top to the le^ • which starts with the input sentence
• a set of dependency arcs A • which starts off empty
• a set of ac)ons
![Page 22: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/22.jpg)
Christopher Manning
Basic transi9on-‐based dependency parser
Start: σ = [ROOT], β = w1, …, wn , A = ∅ 1. Shi^ σ, wi|β, A è σ|wi, β, A
2. Le^-‐Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} 3. Right-‐Arcr σ|wi, wj|β, A è σ, wi|β, A∪{r(wi,wj)}
Finish: β = ∅
Notes: • Unlike the regular presenta)on of the CFG reduce step,
dependencies combine one thing from each of stack and buffer
![Page 23: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/23.jpg)
Christopher Manning
Ac9ons (“arc-‐eager” dependency parser)
Start: σ = [ROOT], β = w1, …, wn , A = ∅ 1. Le^-‐Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)}
Precondi)on: r’ (wk, wi) ∉ A, wi ≠ ROOT
2. Right-‐Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondi)on: r’ (wk, wi) ∈ A
4. Shi^ σ, wi|β, A è σ|wi, β, A Finish: β = ∅
This is the common “arc-‐eager” variant: a head can immediately take a right dependent, before its dependents are found
![Page 24: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/24.jpg)
Christopher Manning
Example
Happy children like to play with their friends .
[ROOT] [Happy, children, …] ∅
Shi^ [ROOT, Happy] [children, like, …] ∅ LAamod [ROOT] [children, like, …] {amod(children, happy)} = A1
Shi^ [ROOT, children] [like, to, …] A1
LAnsubj [ROOT] [like, to, …] A1 ∪ {nsubj(like, children)} = A2
RAroot [ROOT, like] [to, play, …] A2 ∪{root(ROOT, like) = A3
Shi^ [ROOT, like, to] [play, with, …] A3
LAaux [ROOT, like] [play, with, …] A3∪{aux(play, to) = A4
RAxcomp [ROOT, like, play] [with their, …] A4∪{xcomp(like, play) = A5
1. Le^-‐Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondi)on: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-‐Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondi)on: (wk, r’, wi) ∈ A 4. Shi^ σ, wi|β, A è σ|wi, β, A
![Page 25: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/25.jpg)
Christopher Manning
Example
Happy children like to play with their friends . RAxcomp [ROOT, like, play] [with their, …] A4∪{xcomp(like, play) = A5
RAprep [ROOT, like, play, with] [their, friends, …] A5∪{prep(play, with) = A6
Shi^ [ROOT, like, play, with, their] [friends, .] A6
LAposs [ROOT, like, play, with] [friends, .] A6∪{poss(friends, their) = A7
RApobj [ROOT, like, play, with, friends] [.] A7∪{pobj(with, friends) = A8
Reduce [ROOT, like, play, with] [.] A8
Reduce [ROOT, like, play] [.] A8
Reduce [ROOT, like] [.] A8
RApunc [ROOT, like, .] [] A8∪{punc(like, .) = A9
You terminate as soon as the buffer is empty. Dependencies = A9
1. Le^-‐Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondi)on: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-‐Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondi)on: (wk, r’, wi) ∈ A 4. Shi^ σ, wi|β, A è σ|wi, β, A
![Page 26: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/26.jpg)
Example
October 20, 2014 Dependency Parsing (Prashanth Mannem) 26
Red figures on the screen indicated falling stocks _ROOT_ S Q
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 27: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/27.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 27
Red figures on the screen indicated falling stocks _ROOT_ S Q
Shift
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 28: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/28.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 28
Red figures on the screen indicated falling stocks _ROOT_ S Q
Left-arc
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 29: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/29.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 29
Red figures on the screen indicated falling stocks _ROOT_ S Q
Shift
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 30: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/30.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 30
Red figures on the screen indicated falling stocks _ROOT_ S Q
Right-arc
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 31: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/31.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 31
Red figures on the screen indicated falling stocks _ROOT_ S Q
Shift
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 32: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/32.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 32
Red figures on the screen indicated falling stocks _ROOT_ S Q
Left-arc
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 33: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/33.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 33
Red figures on the screen indicated falling stocks _ROOT_ S Q
Right-arc
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 34: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/34.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 34
Red figures on the screen indicated falling stocks _ROOT_ S Q
Reduce
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 35: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/35.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 35
Red figures on the screen indicated falling stocks _ROOT_ S Q
Reduce
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 36: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/36.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 36
Red figures on the screen indicated falling stocks _ROOT_ S Q
Left-arc
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 37: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/37.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 37
Red figures on the screen indicated falling stocks _ROOT_ S Q
Right-arc
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 38: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/38.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 38
Red figures on the screen indicated falling stocks _ROOT_ S Q
Shift
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 39: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/39.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 39
Red figures on the screen indicated falling stocks _ROOT_ S Q
Left-arc
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 40: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/40.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 40
Red figures on the screen indicated falling stocks _ROOT_ S Q
Right-arc
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 41: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/41.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 41
Red figures on the screen indicated falling stocks _ROOT_ S Q
Reduce
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 42: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/42.jpg)
Example
October 20, 2014 Dependency Parsing (P. Mannem) 42
Red figures on the screen indicated falling stocks _ROOT_ S Q
Reduce
1. Left-Arcr σ|wi, wj|β, A è σ, wj|β, A∪{r(wj,wi)} Precondition: (wk, r’, wi) ∉ A, wi ≠ ROOT
2. Right-Arcr σ|wi, wj|β, A è σ|wi|wj, β, A∪{r(wi,wj)} 3. Reduce σ|wi, β, A è σ, β, A
Precondition: (wk, r’, wi) ∈ A 4. Shift σ, wi|β, A è σ|wi, β, A
![Page 43: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/43.jpg)
Christopher Manning
MaltParser [Nivre et al. 2008]
• We have le^ to explain how we choose the next ac)on • Each ac)on is predicted by a discrimina)ve classifier (o^en SVM,
can be perceptron, maxent classifier) over each legal move • Max of 4 untyped choices, max of |R| × 2 + 2 when typed • Features: top of stack word, POS; first in buffer word, POS; etc.
• There is NO search (in the simplest and usual form) • But you could do some kind of beam search if you wish
• It provides VERY fast linear )me parsing • The model’s accuracy is slightly below the best Lexicalized
PCFGs (evaluated on dependencies), but
• It provides close to state of the art parsing performance
![Page 44: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/44.jpg)
Christopher Manning
Evalua9on of Dependency Parsing: (labeled) dependency accuracy
ROOT She saw the video lecture 0 1 2 3 4 5
Gold 1 2 She nsubj 2 0 saw root 3 5 the det 4 5 video nn 5 2 lecture dobj
Parsed 1 2 She nsubj 2 0 saw root 3 4 the det 4 5 video nsubj 5 2 lecture ccomp
Acc = # correct deps # of deps
UAS = 4 / 5 = 80% LAS = 2 / 5 = 40%
![Page 45: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/45.jpg)
Christopher Manning
Representa9ve performance numbers
• The CoNLL-‐X (2006) shared task provides evalua)on numbers for various dependency parsing approaches over 13 languages • MALT: LAS scores from 65–92%, depending greatly on language/treebank
• Here we give a few UAS numbers for English to allow some comparison to cons)tuency parsing
Parser UAS%
Sagae and Lavie (2006) ensemble of dependency parsers 92.7
Charniak (2000) genera)ve, cons)tuency, as dependencies 92.2
Collins (1999) genera)ve, cons)tuency, as dependencies 91.7
McDonald and Pereira (2005) – MST graph-‐based dependency 91.5
Yamada and Matsumoto (2003) – transi)on-‐based dependency 90.4
![Page 46: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/46.jpg)
Christopher Manning
• Dependencies from a CFG tree using heads, must be projec)ve • There must not be any crossing dependency arcs when the words are laid out in their linear order, with all arcs above the words.
• But dependency theory normally does allow non-‐projec)ve structures to account for displaced cons)tuents • You can’t easily get the seman)cs of certain construc)ons right without these nonprojec)ve dependencies
Who did Bill buy the coffee from yesterday ?
Projec9vity
![Page 47: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/47.jpg)
Christopher Manning
Handling non-‐projec9vity
• The arc-‐eager algorithm we presented only builds projec)ve dependency trees
• Possible direc)ons to head: 1. Just declare defeat on nonprojec)ve arcs 2. Use a dependency formalism which only admits projec)ve
representa)ons (a CFG doesn’t represent such structures…)
3. Use a postprocessor to a projec)ve dependency parsing algorithm to iden)fy and resolve nonprojec)ve links
4. Add extra types of transi)ons that can model at least most non-‐projec)ve structures
5. Move to a parsing mechanism that does not use or require any constraints on projec)vity (e.g., the graph-‐based MSTParser)
![Page 48: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/48.jpg)
Dependencies encode rela9onal
structure
Rela)on Extrac)on with Stanford Dependencies
![Page 49: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/49.jpg)
Christopher Manning
Dependency paths iden9fy rela9ons like protein interac9on
[Erkan et al. EMNLP 07, Fundel et al. 2007] KaiC çnsubj interacts prep_withè SasA KaiC çnsubj interacts prep_withè SasA conj_andè KaiA KaiC çnsubj interacts prep_withè SasA conj_andè KaiB
demonstrated
results
KaiC
interacts
rythmically
nsubj
The
compl det
ccomp
that nsubj
KaiB KaiA
SasA conj_and conj_and
advmod prep_with
![Page 50: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/50.jpg)
Christopher Manning
Stanford Dependencies
[de Marneffe et al. LREC 2006] • The basic dependency representa)on is projec)ve
• It can be generated by postprocessing headed phrase structure parses (Penn Treebank syntax)
• It is generated directly by dependency parsers, such as MaltParser, or the Easy-‐First Parser
jumped!
boy! over!
the! the!little!
prep nsubj
det amod pobj
fence!det
![Page 51: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/51.jpg)
Christopher Manning
BioNLP 2009/2011 rela9on extrac9on shared tasks [Björne et al. 2009]
0
5
10
15
20
25
30
35
40
45
50
0 1 2 3 4 5 6 7 8 9 10 >10
Dependency distance
Linear distance
Many relationships become short distance!
![Page 52: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/52.jpg)
Christopher Manning
Graph modifica9on to facilitate seman9c analysis
Bell, based in LA, makes and distributes!
electronic and computer products.!
makes
and
nsubj dobj
products
computer
conj cc
and
electronic
amod
Bell
in
prep
partmod
based
pobj LA
cc
conj distributes
![Page 53: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/53.jpg)
Christopher Manning
Graph modifica9on to facilitate seman9c analysis
Bell, based in LA, makes and distributes!
electronic and computer products.!
makes
nsubj dobj
products
computer
conj_and
electronic
amod
Bell
prep_in
partmod
based
LA
conj_and distributes
amod
nsubj
![Page 54: Dependency( Grammar - Stanford University · 2014-11-12 · Christopher*Manning* Dependency*syntax*postulates*thatsyntac)c*structure*consists*of* relaons*between*lexical*items,*normally*binary*asymmetric*](https://reader033.fdocuments.us/reader033/viewer/2022041808/5e55f7cc916ac37b39325755/html5/thumbnails/54.jpg)
Christopher Manning
54