Winter 2012-2013 Compiler Principles Syntax Analysis (Parsing) – Part 3
Compiler Components and their Generators - Traditional Parsing Algorithms
-
Upload
guido-wachsmuth -
Category
Education
-
view
1.992 -
download
0
description
Transcript of Compiler Components and their Generators - Traditional Parsing Algorithms
Challenge the future
DelftUniversity ofTechnology
Course IN4303Compiler Construction
Guido Wachsmuth
Compiler Components & GeneratorsTraditional Parsing Algorithms
lessons learned
LL Parsing
Recap: Lexical Analysis
What are the formalisms to describe regular languages?
• regular grammars
• regular expressions
• finite state automata
Why are these formalisms equivalent?
• constructive proofs
How can we generate compiler tools from that?
• implement DFAs
• generate transition tables
2
today’s lecture
LL Parsing
Overview
3
today’s lecture
LL Parsing
Overview
efficient parsing algorithms
• predictive parsing
• LR parsing
3
today’s lecture
LL Parsing
Overview
efficient parsing algorithms
• predictive parsing
• LR parsing
grammar classes
• LL(k) grammars
• LR(k) grammars
3
LL Parsing
predictive parsing
I
4
formal languages
LL Parsing 5
Recap: A Theory of Language
formal languages
LL Parsing
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
5
Recap: A Theory of Language
formal languages
LL Parsing
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
string over Σ
finite sequence of elements chosen from Σ
word, sentence, utterance
5
Recap: A Theory of Language
formal languages
LL Parsing
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
string over Σ
finite sequence of elements chosen from Σ
word, sentence, utterance
formal language λ
set of strings over a vocabulary Σ
λ ⊆ Σ*
5
Recap: A Theory of Language
formal grammars
LL Parsing 6
Recap: A Theory of Language
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
6
Recap: A Theory of Language
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
6
Recap: A Theory of Language
nonterminal symbol
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
6
Recap: A Theory of Language
context
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
6
Recap: A Theory of Language
replacement
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
grammar classes
type-0, unrestricted
type-1, context-sensitive: (a A c, a b c)
type-2, context-free: P ⊆ N × (N∪Σ)*
type-3, regular: (A, x) or (A, xB)
6
Recap: A Theory of Language
formal languages
LL Parsing 7
Recap: A Theory of Language
formal languages
LL Parsing
formal grammar G = (N, Σ, P, S)
7
Recap: A Theory of Language
formal languages
LL Parsing
formal grammar G = (N, Σ, P, S)
derivation relation ⇒G ⊆ (N∪Σ)* × (N∪Σ)*
w ⇒G w’ ⇔
∃(p, q)∈P: ∃u,v∈(N∪Σ)*:
w=u p v ∧ w’=u q v
7
Recap: A Theory of Language
formal languages
LL Parsing
formal grammar G = (N, Σ, P, S)
derivation relation ⇒G ⊆ (N∪Σ)* × (N∪Σ)*
w ⇒G w’ ⇔
∃(p, q)∈P: ∃u,v∈(N∪Σ)*:
w=u p v ∧ w’=u q v
formal language L(G) ⊆ Σ*
L(G) = {w∈Σ* | S ⇒G* w}
7
Recap: A Theory of Language
formal languages
LL Parsing
formal grammar G = (N, Σ, P, S)
derivation relation ⇒G ⊆ (N∪Σ)* × (N∪Σ)*
w ⇒G w’ ⇔
∃(p, q)∈P: ∃u,v∈(N∪Σ)*:
w=u p v ∧ w’=u q v
formal language L(G) ⊆ Σ*
L(G) = {w∈Σ* | S ⇒G* w}
classes of formal languages
7
Recap: A Theory of Language
recursive descent
LL Parsing 8
Exp → “while” Exp “do” Exp
Predictive parsing
public void parseExp() { consume(WHILE); parseExp(); consume(DO); parseExp();}
look ahead
LL Parsing 9
Exp → “while” Exp “do” ExpExp → “if” Exp “then” Exp “else” Exp
Predictive parsing
public void parseExp() {
switch current() { case WHILE: consume(WHILE); parseExp(); ...; break; case IF : consume(IF); parseExp(); ...; break; default : error(); }}
parse table
LL Parsing 10
rows
• nonterminal symbols N
• symbol to parse
columns
• terminal symbols Σk
• look ahead k
entries
• production rules P
• possible conflicts
Predictive parsing
T1 T2 T3 ...
N1 N1 →... N1 →...
N2 N2 →...
N3 N3 →... N3 →...
N4 N4 →...
N5 N5 →...
N6 N6 →... N6 →...
N7 N7 →...
N8 N8 →... N8 →... N8 →...
...
automaton
LL Parsing
Predictive parsing
11
… tn $t1
$S
input
parse tablestack
automaton
LL Parsing
Predictive parsing
12
x … $
$
x…
automaton
LL Parsing
Predictive parsing
12
… $
$…
automaton
LL Parsing
Predictive parsing
13
x … $
$
Xi
x
XiXi → Y1... Yk…
automaton
LL Parsing
Predictive parsing
13
x … $
$
x
XiXi → Y1... Yk…
Yk
…Y1
LL Parsing
LL parse tables
II
14
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
filling the table
LL Parsing 15
Predictive parsing
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
filling the table
LL Parsing 15
Predictive parsing
letters that w can start with
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
filling the table
LL Parsing 15
Predictive parsing
w ⇒G* ε
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
filling the table
LL Parsing 15
Predictive parsing
letters that can follow X
nullable
LL Parsing
nullable(X)
(X, ε) ∈ P ⇒ nullable(X)
(X0, X1 … Xk)∈P ∧ nullable(X1) ∧ … ∧ nullable(Xk) ⇒ nullable(X0)
nullable(w)
nullable(ε)
nullable(X1 … Xk) = nullable(X1) ∧ … ∧ nullable(Xk)
16
Predictive parsing
first sets
LL Parsing
FIRST(X)
X∈Σ : FIRST(X) = {X}
(X0, X1 … Xi … Xk)∈P ∧ nullable(X1 … Xi) ⇒ FIRST(X0) ⊇ FIRST(Xi+1)
FIRST(w)
FIRST(ε) = {}
¬nullable(X) ⇒ FIRST(Xw) = FIRST(X)
nullable(X) ⇒ FIRST(Xw) = FIRST(X) ∪ FIRST(w)
17
Predictive parsing
follow sets
LL Parsing
FOLLOW(X)
(X0, X1 … Xi … Xk)∈P ∧ nullable(Xi+1 … Xk) ⇒ FOLLOW(Xi) ⊇ FOLLOW(X0)
(X0, X1 … Xi … Xk)∈P ⇒ FOLLOW(Xi) ⊇ FIRST(Xi+1 … Xk)
18
Predictive parsing
LL Parsing 19
p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’p3: Exp’ →p4: Term → Fact Term’p5: Term’ → “*” Fact Term’p6: Term’ →p7: Fact → Num p8: Fact → “(” Exp “)”
Example
nullable FIRST FOLLOW
Exp
Exp’
Term
Term’
Fact
nullable
LL Parsing 20
p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’p3: Exp’ →p4: Term → Fact Term’p5: Term’ → “*” Fact Term’p6: Term’ →p7: Fact → Num p8: Fact → “(” Exp “)”
Example(X, ε) ∈ P ⇒ nullable(X)
(X0, X1 … Xk)∈P ∧
nullable(X1) ∧ … ∧ nullable(Xk) ⇒ nullable(X0)
nullable FIRST FOLLOW
Exp no
Exp’ yes
Term no
Term’ yes
Fact no
FIRST sets
LL Parsing 21
p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’p3: Exp’ →p4: Term → Fact Term’p5: Term’ → “*” Fact Term’p6: Term’ →p7: Fact → Num p8: Fact → “(” Exp “)”
Example
nullable FIRST FOLLOW
Exp no Num (
Exp’ yes +
Term no Num (
Term’ yes *
Fact no Num (
(X0, X1 … Xi … Xk)∈P ∧
nullable(X1 … Xi) ⇒ FIRST(X0) ⊇ FIRST(Xi+1)
FOLLOW sets
LL Parsing 22
p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’p3: Exp’ →p4: Term → Fact Term’p5: Term’ → “*” Fact Term’p6: Term’ →p7: Fact → Num p8: Fact → “(” Exp “)”
Example(X0, X1 … Xi … Xk)∈P ∧
nullable(Xi+1 … Xk) ⇒ FOLLOW(Xi) ⊇ FOLLOW(X0)
(X0, X1 … Xi … Xk)∈P ⇒ FOLLOW(Xi) ⊇ FIRST(Xi+1 … Xk)
nullable FIRST FOLLOW
Exp no Num ( )
Exp’ yes + )
Term no Num ( + )
Term’ yes * + )
Fact no Num ( * + )
LL parse table
LL Parsing 23
p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’p3: Exp’ →p4: Term → Fact Term’p5: Term’ → “*” Fact Term’p6: Term’ →p7: Fact → Num p8: Fact → “(” Exp “)”
Exampleentry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
+ * Num ( )
Exp p1 p1
Exp’ p2 p3
Term p4 p4
Term’ p6 p5 p6
Fact p7 p8
parsing
LL Parsing 24
Example
+ * Num ( )
Exp p1 p1
Exp’ p2 p3
Term p4 p4
Term’ p6 p5 p6
Fact p7 p8
p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’p3: Exp’ →p4: Term → Fact Term’p5: Term’ → “*” Fact Term’p6: Term’ →p7: Fact → Num p8: Fact → “(” Exp “)”
LL Parsing
Grammar classes
25
context-free grammars
LL Parsing
Grammar classes
25
context-free grammars
LL(0)
LL Parsing
Grammar classes
25
context-free grammars
LL(1)
LL(0)
LL Parsing
Grammar classes
25
context-free grammars
LL(k)
LL(1)
LL(0)
encoding precedence
LL Parsing 26
Exp → Num Exp → “(” Exp “)”Exp → Exp “*” Exp Exp → Exp “+” Exp
Predictive parsing
Fact → Num Fact → “(” Exp “)”Term → Term “*” FactTerm → Fact Exp → Exp “+” TermExp → Term
eliminating left recursion
LL Parsing 27
Term → Term “*” FactTerm → Fact Exp → Exp “+” TermExp → Term
Predictive parsing
Term’ → “*” Fact Term’Term’ →Term → Fact Term’Exp’ → “+” Term Exp’Exp’ →Exp → Term Exp’
left factoring
LL Parsing 28
Exp → “if” Exp “then” Exp “else” ExpExp → “if” Exp “then” Exp
Predictive parsing
Exp → “if” Exp “then” Exp ElseElse → “else” ExpElse →
LL Parsing
summary
III
29
lessons learned
LL Parsing
Summary
30
lessons learned
LL Parsing
Summary
How can we parse context-free languages effectively?
• predictive parsing algorithms
30
lessons learned
LL Parsing
Summary
How can we parse context-free languages effectively?
• predictive parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
30
lessons learned
LL Parsing
Summary
How can we parse context-free languages effectively?
• predictive parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
How can we generate compiler tools from that?
• implement automaton
• generate parse tables
30
lessons learned
LL Parsing
Summary
How can we parse context-free languages effectively?
• predictive parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
How can we generate compiler tools from that?
• implement automaton
• generate parse tables
30
learn more
LL Parsing
Literature
31
learn more
LL Parsing
Literature
formal languages
Noam Chomsky: Three models for the description of language. 1956
J. E. Hopcroft, R. Motwani, J. D. Ullman: Introduction to Automata Theory, Languages, and Computation. 2006
31
learn more
LL Parsing
Literature
formal languages
Noam Chomsky: Three models for the description of language. 1956
J. E. Hopcroft, R. Motwani, J. D. Ullman: Introduction to Automata Theory, Languages, and Computation. 2006
syntactical analysis
Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in Java, 2nd edition. 2002
Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers: Principles, Techniques, and Tools, 2nd edition. 2006
31
coming next
LL Parsing
Outlook
lectures
• last lecture: LR parsing
Question & Answer Jan 10
• 10 questions, submit & vote
Lab Dec 15
• translate expressions & statements
• challenge: stack limits
32
LL Parsing
copyrights
33
LL Parsing 34
copyrights
LL Parsing
Pictures
Slide 1: Book Scanner by Ben Woosley, some rights reserved
Slides 5-7: Noam Chomsky by Fellowsisters, some rights reserved
Slide 8, 9, 26-28: Tiger by Bernard Landgraf, some rights reserved
Slide 32: Ostsee by Mario Thiel, some rights reserved
35