Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a...
Transcript of Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a...
![Page 1: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/1.jpg)
Recursive Descent Parsing and CYKANLP: Lecture 13
Shay Cohen
14 October 2019
1 / 1
![Page 2: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/2.jpg)
Last Class
Chomsky normal form grammars
English syntax
Agreement phenomena and the way to model them with CFGs
2 / 1
![Page 3: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/3.jpg)
Recap: Syntax
Two reasons to care about syntactic structure (parse tree):
I As a guide to the semantic interpretation of the sentence
I As a way to prove whether a sentence is grammatical or not
But having a grammar isn’t enough.
We also need a parsing algorithm to compute the parse tree for agiven input string and grammar.
3 / 1
![Page 4: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/4.jpg)
Parsing algorithms
Goal: compute the structure(s) for an input string given agrammar.I As usual, ambiguity is a huge problem.
I For correctness: need to find the right structure to get theright meaning.
I For efficiency: searching all possible structures can be veryslow; want to use parsing for large-scale language tasks (e.g.,used to create Google’s “infoboxes”).
4 / 1
![Page 5: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/5.jpg)
Global and local ambiguity
I We’ve already seen examples of global ambiguity: multipleanalyses for a full sentence, like I saw the man with thetelescope
I But local ambiguity is also a big problem: multiple analysesfor parts of sentence.
I the dog bit the child: first three words could be NP (butaren’t).
I Building useless partial structures wastes time.
I Avoiding useless computation is a major issue in parsing.
I Syntactic ambiguity is rampant; humans usually don’t evennotice because we are good at using context/semantics todisambiguate.
5 / 1
![Page 6: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/6.jpg)
Parser properties
All parsers have two fundamental properties:I Directionality: the sequence in which the structures are
constructed.I top-down: start with root category (S), choose expansions,
build down to words.I bottom-up: build subtrees over words, build up to S.I Mixed strategies also possible (e.g., left corner parsers)
I Search strategy: the order in which the search space ofpossible analyses is explored.
6 / 1
![Page 7: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/7.jpg)
Example: search space for top-down parser
I Start with S node.
I Choose one of many possible expansions.
I Each of which has children with manypossible expansions...
I etc
SS S
S S S S S
S
NP VP NP VPaux
S . . . . . . . . . . . . . . .
NP
NP
7 / 1
![Page 8: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/8.jpg)
Search strategies
I depth-first search: explore one branch of the search space at atime, as far as possible. If this branch is a dead-end, parserneeds to backtrack.
I breadth-first search: expand all possible branches in parallel(or simulated parallel). Requires storing many incompleteparses in memory at once.
I best-first search: score each partial parse and pursue thehighest-scoring options first. (Will get back to this whendiscussing statistical parsing.)
8 / 1
![Page 9: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/9.jpg)
Recursive Descent Parsing
I A recursive descent parser treats a grammar as a specificationof how to break down a top-level goal (find S) into subgoals(find NP VP).
I It is a top-down, depth-first parser:I Blindly expand nonterminals until reaching a terminal (word).
I If multiple options available, choose one but store current stateas a backtrack point (in a stack to ensure depth-first.)
I If terminal matches next input word, continue; else, backtrack.
9 / 1
![Page 10: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/10.jpg)
RD Parsing algorithm
Start with subgoal = S, then repeat until input/subgoals areempty:
I If first subgoal in list is a non-terminal A, then pick anexpansion A → B C from grammar and replace A in subgoallist with B C
I If first subgoal in list is a terminal w:I If input is empty, backtrack.I If next input word is different from w, backtrack.I If next input word is w, match! i.e., consume input word w and
subgoal w and move to next subgoal.
If we run out of backtrack points but not input, no parse ispossible.
10 / 1
![Page 11: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/11.jpg)
Recursive descent parsing pseudocodeIn the background: a CFG G , a sentence x1 · · · xnFunction RecursiveDescent(t, v , i) where
I t is a partially constructed treeI v is a node in tI i is a sentence position
I Let N be the nonterminal in vI For each rule with LHS N:
I If the rule is a lexical rule N → w , check whether xi = w , if soincrease i by 1 and call RecursiveDescent(t, u, i + 1) where uis the lowest point above v that has a nonterminal
I If the rule is a grammatical rule, Let t ′ be t with v expandedusing the rule N → A1 · · ·An. For each j ∈ {1 · · · n}, callRecursiveDescent(t ′, uj , i) where uj is the node fornonterminal Aj in t ′.
Start with: RecursiveDescent(S , topnode, 1)
Quick quiz: this algorithm has a bug. Where? What do we need toadd?
11 / 1
![Page 12: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/12.jpg)
Recursive descent parsing pseudocodeIn the background: a CFG G , a sentence x1 · · · xnFunction RecursiveDescent(t, v , i) where
I t is a partially constructed treeI v is a node in tI i is a sentence position
I Let N be the nonterminal in vI For each rule with LHS N:
I If the rule is a lexical rule N → w , check whether xi = w , if soincrease i by 1 and call RecursiveDescent(t, u, i + 1) where uis the lowest point above v that has a nonterminal
I If the rule is a grammatical rule, Let t ′ be t with v expandedusing the rule N → A1 · · ·An. For each j ∈ {1 · · · n}, callRecursiveDescent(t ′, uj , i) where uj is the node fornonterminal Aj in t ′.
Start with: RecursiveDescent(S , topnode, 1)
Quick quiz: this algorithm has a bug. Where? What do we need toadd? 11 / 1
![Page 13: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/13.jpg)
Recursive descent example
Consider a very simple example:
I Grammar contains only these rules:
S → NP VP VP → V NN → bit V → bit
NP → DT NN DT → the NN → dog V → dog
I The input sequence is the dog bit
12 / 1
![Page 14: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/14.jpg)
13 / 1
![Page 15: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/15.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
14 / 1
![Page 16: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/16.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP VP
15 / 1
![Page 17: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/17.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
Det N PP
16 / 1
![Page 18: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/18.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
N PPDet
the
17 / 1
![Page 19: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/19.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
N PPDet
the
18 / 1
![Page 20: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/20.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
PPDet
the
N
man
19 / 1
![Page 21: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/21.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
PPDet
the
N
park
20 / 1
![Page 22: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/22.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
PPDet
the
N
dog
21 / 1
![Page 23: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/23.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
Det
the
N
dog
PP
P NP
22 / 1
![Page 24: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/24.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
Det
the
N
dog
PP
NPP
in
23 / 1
![Page 25: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/25.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
NDet
the
24 / 1
![Page 26: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/26.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
VPNP
Det
the
N
dog
25 / 1
![Page 27: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/27.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
NP PPV
saw
26 / 1
![Page 28: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/28.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
PPV
saw
NP
N PPDet
a
27 / 1
![Page 29: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/29.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
PPV
saw
NP
PPDet
a
N
man
28 / 1
![Page 30: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/30.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
PPV
saw
NP
Det
a
N
man
PP
NPP
in
29 / 1
![Page 31: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/31.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
PPV
saw
NP
Det
a
N
man
PP
P
in
NP
Det N PP
30 / 1
![Page 32: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/32.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
PPV
saw
NP
Det
a
N
man
PP
P
in
NP
Det
the
N
park
PP
NPP
31 / 1
![Page 33: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/33.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
V
saw
PPNP
Det N
32 / 1
![Page 34: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/34.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
V
saw
PPNP
Det
a
N
man
33 / 1
![Page 35: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/35.jpg)
Recursive Descent Parsing
the dog saw a man in the park
S
NP
Det
the
N
dog
VP
V
saw
NP
Det
a
N
man
PP
P
in
NP
Det
the
N
park
34 / 1
![Page 36: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/36.jpg)
Left Recursion
Can recursive descent parsing handle left recursion?
Grammars for natural human languages should be revealing,left-recursive rules are needed in English.
NP → DET NNP → NPRDET → NP ’s
These rules generate NPs with possessive modifiers such as:
John’s sisterJohn’s mother’s sisterJohn’s mother’s uncle’s sisterJohn’s mother’s uncle’s sister’s niece
35 / 1
![Page 37: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/37.jpg)
Left Recursion
Can recursive descent parsing handle left recursion?Grammars for natural human languages should be revealing,left-recursive rules are needed in English.
NP → DET NNP → NPRDET → NP ’s
These rules generate NPs with possessive modifiers such as:
John’s sisterJohn’s mother’s sisterJohn’s mother’s uncle’s sisterJohn’s mother’s uncle’s sister’s niece
35 / 1
![Page 38: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/38.jpg)
Shift-Reduce Parsing
A Shift-Reduce parser tries to find sequences of words and phrasesthat correspond to the righthand side of a grammar productionand replace them with the lefthand side:
I Directionality = bottom-up: starts with the words of theinput and tries to build trees from the words up.
I Search strategy = breadth-first: starts with the words, thenapplies rules with matching right hand sides, and so on untilthe whole sentence is reduced to an S.
36 / 1
![Page 39: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/39.jpg)
Algorithm Sketch: Shift-Reduce Parsing
Until the words in the sentences are substituted with S:
I Scan through the input until we recognise something thatcorresponds to the RHS of one of the production rules (shift)
I Apply a production rule in reverse; i.e., replace the RHS of therule which appears in the sentential form with the LHS of therule (reduce)
A shift-reduce parser implemented using a stack:
1. start with an empty stack
2. a shift action pushes the current input symbol onto the stack
3. a reduce action replaces n items with a single item
37 / 1
![Page 40: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/40.jpg)
Shift-Reduce Parsing
Stack Remaining Text
my dog saw a man in the park with a statue
38 / 1
![Page 41: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/41.jpg)
Shift-Reduce Parsing
Stack Remaining Text
my
dog saw a man in the park with a statueDet
39 / 1
![Page 42: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/42.jpg)
Shift-Reduce Parsing
Stack Remaining Text
my dog
saw a man in the park with a statueDet N
40 / 1
![Page 43: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/43.jpg)
Shift-Reduce Parsing
Stack Remaining Text
my dog
saw a man in the park with a statue
Det N
NP
41 / 1
![Page 44: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/44.jpg)
Shift-Reduce Parsing
Stack Remaining Text
NP
Det
my
N
dog
V
saw
NP
Det
a
N
man
in the park with a statue
42 / 1
![Page 45: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/45.jpg)
Shift-Reduce Parsing
Stack Remaining Text
NP
Det
my
N
dog
V
saw
NP
Det
a
N
man
P
in
NP
Det
the
N
park
with a statuePP
43 / 1
![Page 46: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/46.jpg)
Shift-Reduce Parsing
Stack Remaining Text
NP
Det
my
N
dog
V
saw
NP
NP
Det
a
N
man
PP
P
in
NP
Det
the
N
park
with a statue
44 / 1
![Page 47: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/47.jpg)
Shift-Reduce Parsing
Stack Remaining Text
NP
Det
my
N
dog
V
saw
NP
NP
Det
a
N
man
PP
P
in
NP
Det
the
N
park
with a statueVP
45 / 1
![Page 48: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/48.jpg)
Shift-Reduce Parsing
Stack Remaining Text
NP
Det
my
N
dog
V
saw
NP
NP
Det
a
N
man
PP
P
in
NP
Det
the
N
park
with a statue
VP
S
46 / 1
![Page 49: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/49.jpg)
How many parses are there?
If our grammar is ambiguous (inherently, or by design) then howmany possible parses are there?
In general: an infinite number, if we allow unary recursion.
More specific: suppose that we have a grammar in Chomskynormal form. How many possible parses are there for a sentence ofn words? Imagine that every nonterminal can rewrite as every pairof nonterminals (A→BC) and every nonterminal (A→a)
1. n
2. n2
3. n log n
4. (2n)!(n+1)!n!
47 / 1
![Page 50: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/50.jpg)
How many parses are there?
If our grammar is ambiguous (inherently, or by design) then howmany possible parses are there?
In general: an infinite number, if we allow unary recursion.
More specific: suppose that we have a grammar in Chomskynormal form. How many possible parses are there for a sentence ofn words? Imagine that every nonterminal can rewrite as every pairof nonterminals (A→BC) and every nonterminal (A→a)
1. n
2. n2
3. n log n
4. (2n)!(n+1)!n!
47 / 1
![Page 51: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/51.jpg)
How many parses are there?
If our grammar is ambiguous (inherently, or by design) then howmany possible parses are there?
In general: an infinite number, if we allow unary recursion.
More specific: suppose that we have a grammar in Chomskynormal form. How many possible parses are there for a sentence ofn words? Imagine that every nonterminal can rewrite as every pairof nonterminals (A→BC) and every nonterminal (A→a)
1. n
2. n2
3. n log n
4. (2n)!(n+1)!n!
47 / 1
![Page 52: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/52.jpg)
How many parses are there?A
aa
A
A
aa
A
a
A
A
a
A
aa
A
A
A
aa
A
aa
A
A
A
aa
A
aa
A
A
A
a
A
aaa
A
a
A
A
a
A
aa
A
A
aa
A
aa
48 / 1
![Page 53: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/53.jpg)
How many parses are there?A
aa
A
A
aa
A
a
A
A
a
A
aa
A
A
A
aa
A
aa
A
A
A
aa
A
aa
A
A
A
a
A
aaa
A
a
A
A
a
A
aa
A
A
aa
A
aa
48 / 1
![Page 54: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/54.jpg)
How many parses are there?A
aa
A
A
aa
A
a
A
A
a
A
aa
A
A
A
aa
A
aa
A
A
A
aa
A
aa
A
A
A
a
A
aaa
A
a
A
A
a
A
aa
A
A
aa
A
aa
48 / 1
![Page 55: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/55.jpg)
How many parses are there?A
aa
A
A
aa
A
a
A
A
a
A
aa
A
A
A
aa
A
aa
A
A
A
aa
A
aa
A
A
A
a
A
aaa
A
a
A
A
a
A
aa
A
A
aa
A
aa
48 / 1
![Page 56: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/56.jpg)
How many parses are there?A
aa
A
A
aa
A
a
A
A
a
A
aa
A
A
A
aa
A
aa
A
A
A
aa
A
aa
A
A
A
a
A
aaa
A
a
A
A
a
A
aa
A
A
aa
A
aa
48 / 1
![Page 57: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/57.jpg)
How many parses are there?
Intution. Let C (n) be the number of binary trees over a sentenceof length n. The root of this tree has two subtrees: one over kwords (1 ≤ k < n), and one over n− k words. Hence, for all valuesof k, we can combine any subtree over k words with any subtreeover n − k words:
C (n) =n−1∑k=1
C (k)× C (n − k)
C (n) =(2n)!
(n + 1)!n!
These numbers are called the Catalan numbers. They’re bignumbers!
n 1 2 3 4 5 6 8 9 10 11 12C (n) 1 1 2 5 14 42 132 429 1430 4862 16796
49 / 1
![Page 58: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/58.jpg)
How many parses are there?
Intution. Let C (n) be the number of binary trees over a sentenceof length n. The root of this tree has two subtrees: one over kwords (1 ≤ k < n), and one over n− k words. Hence, for all valuesof k, we can combine any subtree over k words with any subtreeover n − k words:
C (n) =n−1∑k=1
C (k)× C (n − k)
C (n) =(2n)!
(n + 1)!n!
These numbers are called the Catalan numbers. They’re bignumbers!
n 1 2 3 4 5 6 8 9 10 11 12C (n) 1 1 2 5 14 42 132 429 1430 4862 16796
49 / 1
![Page 59: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/59.jpg)
Problems with Parsing as Search
1. A recursive descent parser (top-down) will do badly if thereare many different rules for the same LHS. Hopeless forrewriting parts of speech (preterminals) with words(terminals).
2. A shift-reduce parser (bottom-up) does a lot of uselesswork: many phrase structures will be locally possible, butglobally impossible. Also inefficient when there is much lexicalambiguity.
3. Both strategies do repeated work by re-analyzing the samesubstring many times.
We will see how chart parsing solves the re-parsing problem, andalso copes well with ambiguity.
50 / 1
![Page 60: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/60.jpg)
Dynamic Programming
With a CFG, a parser should be able to avoid re-analyzingsub-strings because the analysis of any sub-string is independent ofthe rest of the parse.
The dog saw a man in the park NP NP
NP
NP
The parser’s exploration of its search space can exploit thisindependence if the parser uses dynamic programming.
Dynamic programming is the basis for all chart parsing algorithms.
51 / 1
![Page 61: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/61.jpg)
Parsing as Dynamic Programming
I Given a problem, systematically fill a table of solutions tosub-problems: this is called memoization.
I Once solutions to all sub-problems have been accumulated,solve the overall problem by composing them.
I For parsing, the sub-problems are analyses of sub-strings andcorrespond to constituents that have been found.
I Sub-trees are stored in a chart (aka well-formed substringtable), which is a record of all the substructures that haveever been built during the parse.
Solves re-parsing problem: sub-trees are looked up, not re-parsed!Solves ambiguity problem: chart implicitly stores all parses!
52 / 1
![Page 62: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/62.jpg)
Depicting a Chart
A chart can be depicted as a matrix:
I Rows and columns of the matrix correspond to the start andend positions of a span (ie, starting right before the first word,ending right after the final one);
I A cell in the matrix corresponds to the sub-string that startsat the row index and ends at the column index.
I It can contain information about the type of constituent (orconstituents) that span(s) the substring, pointers to itssub-constituents, and/or predictions about what constituentsmight follow the substring.
53 / 1
![Page 63: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/63.jpg)
CYK Algorithm
CYK (Cocke, Younger, Kasami) is an algorithm for recognizing andrecording constituents in the chart.
I Assumes that the grammar is in Chomsky Normal Form: rulesall have form A→ BC or A→ w .
I Conversion to CNF can be done automatically.
NP → Det Nom NP → Det NomNom → N | OptAP Nom Nom → book | orange | AP Nom
OptAP → ε | OptAdv A AP → heavy | orange | Adv AA → heavy | orange A → heavy | orange
Det → a Det → aOptAdv → ε | very Adv → very
N → book | orange
54 / 1
![Page 64: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/64.jpg)
CYK: an example
Let’s look at a simple example before we explain the general case.
Grammar Rules in CNF
NP → Det NomNom → book | orange | AP Nom
AP → heavy | orange | Adv AA → heavy | orange
Det → aAdv → very
(N.B. Converting to CNF sometimes breeds duplication!)
Now let’s parse: a very heavy orange book
55 / 1
![Page 65: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/65.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a
Det NP NP
1 very
Adv AP Nom Nom
2 heavy
A,AP Nom Nom
3 orange
Nom,A,AP Nom
4 book
Nom
56 / 1
![Page 66: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/66.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det
NP NP
1 very
Adv AP Nom Nom
2 heavy
A,AP Nom Nom
3 orange
Nom,A,AP Nom
4 book
Nom
56 / 1
![Page 67: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/67.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det
NP NP
1 very Adv
AP Nom Nom
2 heavy
A,AP Nom Nom
3 orange
Nom,A,AP Nom
4 book
Nom
56 / 1
![Page 68: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/68.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det
NP NP
1 very Adv
AP Nom Nom
2 heavy A,AP
Nom Nom
3 orange
Nom,A,AP Nom
4 book
Nom
56 / 1
![Page 69: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/69.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det
NP NP
1 very Adv AP
Nom Nom
2 heavy A,AP
Nom Nom
3 orange
Nom,A,AP Nom
4 book
Nom
56 / 1
![Page 70: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/70.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det
NP NP
1 very Adv AP
Nom Nom
2 heavy A,AP
Nom Nom
3 orange Nom,A,AP
Nom
4 book
Nom
56 / 1
![Page 71: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/71.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det
NP NP
1 very Adv AP
Nom Nom
2 heavy A,AP Nom
Nom
3 orange Nom,A,AP
Nom
4 book
Nom
56 / 1
![Page 72: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/72.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det
NP NP
1 very Adv AP Nom
Nom
2 heavy A,AP Nom
Nom
3 orange Nom,A,AP
Nom
4 book
Nom
56 / 1
![Page 73: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/73.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det NP
NP
1 very Adv AP Nom
Nom
2 heavy A,AP Nom
Nom
3 orange Nom,A,AP
Nom
4 book
Nom
56 / 1
![Page 74: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/74.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det NP
NP
1 very Adv AP Nom
Nom
2 heavy A,AP Nom
Nom
3 orange Nom,A,AP
Nom
4 book Nom
56 / 1
![Page 75: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/75.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det NP
NP
1 very Adv AP Nom
Nom
2 heavy A,AP Nom
Nom
3 orange Nom,A,AP Nom
4 book Nom
56 / 1
![Page 76: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/76.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det NP
NP
1 very Adv AP Nom
Nom
2 heavy A,AP Nom Nom
3 orange Nom,A,AP Nom
4 book Nom
56 / 1
![Page 77: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/77.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det NP
NP
1 very Adv AP Nom Nom
2 heavy A,AP Nom Nom
3 orange Nom,A,AP Nom
4 book Nom
56 / 1
![Page 78: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/78.jpg)
Filling out the CYK chart
0 a 1 very 2 heavy 3 orange 4 book 5
1 2 3 4 5a very heavy orange book
0 a Det NP NP
1 very Adv AP Nom Nom
2 heavy A,AP Nom Nom
3 orange Nom,A,AP Nom
4 book Nom
56 / 1
![Page 79: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/79.jpg)
CYK: The general algorithm
function CKY-Parse(words, grammar) returns table for
j ←from 1 to Length(words) dotable[j − 1, j ]← {A | A→ words[j ] ∈ grammar}for i ←from j − 2 downto 0 do
for k ← i + 1 to j − 1 dotable[i , j ]← table[i , j ]∪
{A | A→ BC ∈ grammar ,B ∈ table[i , k]C ∈ table[k , j ]}
57 / 1
![Page 80: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/80.jpg)
CYK: The general algorithm
function CKY-Parse(words, grammar) returns table for
j ←from 1 to Length(words) do loop over the columns
table[j −1, j ]← {A | A→ words[j ] ∈ grammar} fill bottom cell
for i ←from j − 2 downto 0 do fill row i in column j
for k ← i + 1 to j − 1 do loop over split locationstable[i , j ]← table[i , j ]∪ between i and j
{A | A→ BC ∈ grammar , Check the grammarB ∈ table[i , k] for rules thatC ∈ table[k, j ]} link the constituents
in [i , k] with thosein [k, j ]. For eachrule found storeLHS in cell [i , j ].
58 / 1
![Page 81: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/81.jpg)
A succinct representation of CKY
We have a Boolean table called Chart, such that Chart[A, i , j ] istrue if there is a sub-phrase according the grammar that dominateswords i through words j
Build this chart recursively, similarly to the Viterbi algorithm:
For j > i + 1:
Chart[A, i , j ] =
j−1∨k=i+1
∨A→B C
Chart[B, i , k] ∧ Chart[C , k, j ]
Seed the chart, for i + 1 = j :Chart[A, i , i + 1] = True if there exists a rule A→ wi+1 wherewi+1 is the (i + 1)th word in the string
59 / 1
![Page 82: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/82.jpg)
From CYK Recognizer to CYK Parser
I So far, we just have a chart recognizer, a way of determiningwhether a string belongs to the given language.
I Changing this to a parser requires recording which existingconstituents were combined to make each new constituent.
I This requires another field to record the one or more ways inwhich a constituent spanning (i,j) can be made fromconstituents spanning (i,k) and (k,j). (More clearly displayedin graph representation, see next lecture.)
I In any case, for a fixed grammar, the CYK algorithm runs intime O(n3) on an input string of n tokens.
I The algorithm identifies all possible parses.
60 / 1
![Page 83: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/83.jpg)
CYK-style parse charts
Even without converting a grammar to CNF, we can drawCYK-style parse charts:
1 2 3 4 5a very heavy orange book
0 a Det NP NP
1 very OptAdv OptAP Nom Nom
2 heavy A,OptAP Nom Nom
3 orange N,Nom,A,AP Nom
4 book N,Nom
(We haven’t attempted to show ε-phrases here. Could in principleuse cells below the main diagonal for this . . . )However, CYK-style parsing will have run-time worse than O(n3) ife.g. the grammar has rules A→ BCD.
61 / 1
![Page 84: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/84.jpg)
Second example
Grammar Rules in CNFS → NP VP Nominal → book|flight|moneyS → X 1 VP Nominal → Nominal nounX 1→ Aux VP Nominal → Nominal PPS → book|include|prefer VP → book|include|preferS → Verb NP VPVerb → NPS → X 2 VP → X 2 PPS → Verb PP X 2→ Verb NPS → VP PP VP → Verb NPNP → TWA|Houston VP → VP PPNP → Det Nominal PP → Preposition NPVerb → book|include|prefer Noun→ book|flight|money
Let’s parse Book the flight through Houston!
62 / 1
![Page 85: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/85.jpg)
Second example
Grammar Rules in CNFS → NP VP Nominal → book|flight|moneyS → X 1 VP Nominal → Nominal nounX 1→ Aux VP Nominal → Nominal PPS → book|include|prefer VP → book|include|preferS → Verb NP VPVerb → NPS → X 2 VP → X 2 PPS → Verb PP X 2→ Verb NPS → VP PP VP → Verb NPNP → TWA|Houston VP → VP PPNP → Det Nominal PP → Preposition NPVerb → book|include|prefer Noun→ book|flight|money
Let’s parse Book the flight through Houston!
62 / 1
![Page 86: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/86.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1]
[0, 2] [0, 3] [0, 4] [0, 5]Det NP NP
[1, 2] [1, 3] [1, 4] [1, 5]Nominal, Nominal
Noun
[2, 3] [2, 4] [2, 5]Prep PP
[3, 4] [3, 5]NP, Proper-
Noun
[4, 5]
63 / 1
![Page 87: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/87.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1]
[0, 2] [0, 3] [0, 4] [0, 5]
Det
NP NP
[1, 2]
[1, 3] [1, 4] [1, 5]Nominal, Nominal
Noun
[2, 3] [2, 4] [2, 5]Prep PP
[3, 4] [3, 5]NP, Proper-
Noun
[4, 5]
63 / 1
![Page 88: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/88.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1]
[0, 2] [0, 3] [0, 4] [0, 5]
Det
NP NP
[1, 2]
[1, 3] [1, 4] [1, 5]
Nominal,
Nominal
Noun
[2, 3]
[2, 4] [2, 5]Prep PP
[3, 4] [3, 5]NP, Proper-
Noun
[4, 5]
63 / 1
![Page 89: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/89.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1]
[0, 2] [0, 3] [0, 4] [0, 5]
Det
NP NP
[1, 2]
[1, 3] [1, 4] [1, 5]
Nominal,
Nominal
Noun
[2, 3]
[2, 4] [2, 5]
Prep
PP
[3, 4]
[3, 5]NP, Proper-
Noun
[4, 5]
63 / 1
![Page 90: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/90.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1]
[0, 2] [0, 3] [0, 4] [0, 5]
Det
NP NP
[1, 2]
[1, 3] [1, 4] [1, 5]
Nominal,
Nominal
Noun
[2, 3]
[2, 4] [2, 5]
Prep
PP
[3, 4]
[3, 5]
NP, Proper-
Noun
[4, 5]
63 / 1
![Page 91: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/91.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2]
[0, 3] [0, 4] [0, 5]
Det
NP NP
[1, 2]
[1, 3] [1, 4] [1, 5]
Nominal,
Nominal
Noun
[2, 3]
[2, 4] [2, 5]
Prep
PP
[3, 4]
[3, 5]
NP, Proper-
Noun
[4, 5]
63 / 1
![Page 92: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/92.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2]
[0, 3] [0, 4] [0, 5]
Det NP
NP
[1, 2] [1, 3]
[1, 4] [1, 5]
Nominal,
Nominal
Noun
[2, 3]
[2, 4] [2, 5]
Prep
PP
[3, 4]
[3, 5]
NP, Proper-
Noun
[4, 5]
63 / 1
![Page 93: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/93.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2] [0, 3]
[0, 4] [0, 5]
Det NP
NP
[1, 2] [1, 3]
[1, 4] [1, 5]
Nominal,
Nominal
Noun
[2, 3]
[2, 4] [2, 5]
Prep
PP
[3, 4]
[3, 5]
NP, Proper-
Noun
[4, 5]
63 / 1
![Page 94: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/94.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2] [0, 3]
[0, 4] [0, 5]
Det NP
NP
[1, 2] [1, 3]
[1, 4] [1, 5]
Nominal,
Nominal
Noun
[2, 3] [2, 4]
[2, 5]
Prep
PP
[3, 4]
[3, 5]
NP, Proper-
Noun
[4, 5]
63 / 1
![Page 95: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/95.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2] [0, 3]
[0, 4] [0, 5]
Det NP
NP
[1, 2] [1, 3] [1, 4]
[1, 5]
Nominal,
Nominal
Noun
[2, 3] [2, 4]
[2, 5]
Prep
PP
[3, 4]
[3, 5]
NP, Proper-
Noun
[4, 5]
63 / 1
![Page 96: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/96.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2] [0, 3] [0, 4]
[0, 5]
Det NP
NP
[1, 2] [1, 3] [1, 4]
[1, 5]
Nominal,
Nominal
Noun
[2, 3] [2, 4]
[2, 5]
Prep
PP
[3, 4]
[3, 5]
NP, Proper-
Noun
[4, 5]
63 / 1
![Page 97: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/97.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2] [0, 3] [0, 4]
[0, 5]
Det NP
NP
[1, 2] [1, 3] [1, 4]
[1, 5]
Nominal,
Nominal
Noun
[2, 3] [2, 4]
[2, 5]
Prep PP
[3, 4] [3, 5]NP, Proper-
Noun
[4, 5]
63 / 1
![Page 98: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/98.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2] [0, 3] [0, 4]
[0, 5]
Det NP
NP
[1, 2] [1, 3] [1, 4]
[1, 5]
Nominal, Nominal
Noun
[2, 3] [2, 4] [2, 5]Prep PP
[3, 4] [3, 5]NP, Proper-
Noun
[4, 5]
63 / 1
![Page 99: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/99.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2] [0, 3] [0, 4]
[0, 5]
Det NP NP
[1, 2] [1, 3] [1, 4] [1, 5]Nominal, Nominal
Noun
[2, 3] [2, 4] [2, 5]Prep PP
[3, 4] [3, 5]NP, Proper-
Noun
[4, 5]
63 / 1
![Page 100: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/100.jpg)
Second example
Book the flight through Houston
S, VP, Verb,Nominal,Noun
S,VP,X2
S1, VP, X2,S2, VP,
S3
[0, 1] [0, 2] [0, 3] [0, 4] [0, 5]Det NP NP
[1, 2] [1, 3] [1, 4] [1, 5]Nominal, Nominal
Noun
[2, 3] [2, 4] [2, 5]Prep PP
[3, 4] [3, 5]NP, Proper-
Noun
[4, 5]
63 / 1
![Page 101: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/101.jpg)
Visualizing the Chart
64 / 1
![Page 102: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/102.jpg)
Visualizing the Chart
65 / 1
![Page 103: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/103.jpg)
Dynamic Programming as a problem-solving technique
I Given a problem, systematically fill a table of solutions tosub-problems: this is called memoization.
I Once solutions to all sub-problems have been accumulated,solve the overall problem by composing them.
I For parsing, the sub-problems are analyses of sub-strings andcorrespond to constituents that have been found.
I Sub-trees are stored in a chart (aka well-formed substringtable), which is a record of all the substructures that haveever been built during the parse.
Solves re-parsing problem: sub-trees are looked up, not re-parsed!Solves ambiguity problem: chart implicitly stores all parses!
66 / 1
![Page 104: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/104.jpg)
A Tribute to CKY (part 1/3)
You, my CKY algorithm,dictate every parser’s rhythm,if Cocke, Younger and Kasami hadn’t bothered,all of our parsing dreams would have been shattered.
You are so simple, yet so powerful,and with the proper semiring and time,you will be truthful,to return the best parse - anything less would be a crime.
With dynamic programming or memoization,you are one of a kind,I really don’t need to mention,if it weren’t for you, all syntax trees would be behind.
67 / 1
![Page 105: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/105.jpg)
A Tribute to CKY (part 2/3)
Failed attempts have been made to show there are better,for example, by using matrix multiplication,all of these impractical algorithms didn’t matter –you came out stronger, insisting on just using summation.
All parsing algorithms to you hail,at least those with backbones which are context-free,you will never become stale,as long as we need to have a syntax tree.
It doesn’t matter that the C is always in front,or that the K and Y can swap,you are still on the same hunt,maximizing and summing, nonstop.
68 / 1
![Page 106: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/106.jpg)
A Tribute to CKY (part 3/3)
Every Informatics student knows you intimately,they have seen your variants dozens of times,you have earned that respect legitimately,and you will follow them through their primes.
CKY, going backward and forward,inside and out,it is so straightforward -You are the best, there is no doubt.
69 / 1
![Page 107: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/107.jpg)
Questions to Ask Yourself
I How many spans are there for a given sequence (as a functionof the length of the sentence)?
I How long does it take to process each one of them (each“cell”) as a function of the length of the span and the size ofthe grammar?
I Does CYK perform any unnecessary calculation?
70 / 1
![Page 108: Recursive Descent Parsing and CYK · Recursive Descent Parsing I Arecursive descentparser treats a grammar as a speci cation of how to break down a top-level goal ( nd S) into subgoals](https://reader033.fdocuments.us/reader033/viewer/2022052014/602b40f39e5fe157290b3eec/html5/thumbnails/108.jpg)
Summary
I Parsing as search is inefficient (typically exponential time).
I Alternative: use dynamic programming and memoizesub-analysis in a chart to avoid duplicate work.
I The chart can be visualized as as a matrix.
I The CYK algorithm builds a chart in O(n3) time. The basicversion gives just a recognizer, but it can be made into aparser if more info is recorded in the chart.
71 / 1