04/19/23Evelyne Tropper 1

THEORY OF COMPUTATION

CSC 422

Evelyne Tropper 2

INTRODUCTION

• Are all problems programmable?

• What statement of a problem constitutes an implementable program?

• Do the specifications of a program always lead to a program?

• Is it always possible to find specifications of a problem that lead to a program?

Evelyne Tropper 3

Purpose of Theory of Computation

• Design a language for the mathematical specification of computer languages in general & for all computers– Describe the workings of a general computer in

the simplest and most basic terms possible– Find a mathematical language acceptable to the

above general, basic computer

SETS & FUNCTIONS• { } A collection

• {a,b,c} = A A set

• CxS = {(c1,s1),…,(cn,sn)} Cartesian product

• {0,1} Binary alphabet

• e & |e|=0 Empty set & its length

• wr Reverse of w (if w=wr ==> palindrome)

• Ā Complement of A (A even nos ==> Ā odd nos)

Evelyne Tropper 5

RELATIONS

• Reflexive if (a,a) R a S

• Symmetric if (a,b) R ==> (b,a) R

• Transitive if (a,b) R , (b,c) R ==> (a,c) R

• Equivalence relation is reflexive, symmetric & transitive

Evelyne Tropper 6

Examples of reflexivity

• Let relation R be: a <= b then a <= a & R is reflexive

• Let relation R be: a < bthen a is not < a & R is not reflexive

• Let set S be: {oaks}; let R be: oaks trees then a S => a trees, so (a,a) R

Evelyne Tropper 7

Examples of symmetry

• If R is (big,large) then a {big} => a {large} &

a {large} => a {big}

• If R is relation of synonyms then a synonym of b => b synonym of a

so R is symmetric & also reflexive

Evelyne Tropper 8

Examples of transitivity

• If R is “on top of” & (a,b) R; (b,c) Rthen (a,c) R & R is transitive

• If R is “<” & (a,b) R; (b,c) R then (a,c) R & R is transitive

• If R is “child of” & (Bob, Mary) R; (Mary, John) then R is not transitive

• If R is “successor of” & (Bob, Mary) R; (Mary, John) then R is transitive

Evelyne Tropper 9

Union, Intersection & Complement

A B

AB A

Ā

universe

A

A B

10

Laws of Set Operations

• Idempotency: AA = A, AA = A• Commutativity: AB=BA, AB=BA • Associativity: (AB)C= A(BC)

(AB)C=A(BC)• Distributivity: (AB)C=(AC)(BC)

(AB) C=(AC) (BC)

• Absorption: (AB)A=A, (AB)A=A• DeMorgan’s: A-(BC)=(A-B)(A-C)

A-(BC)=(A-B)(A-C)

Evelyne Tropper 11

Examples

• Apples apples = apples

• Apples oranges = oranges apples

Dad Mom = Mom Dad

…

Evelyne Tropper 12

FUNCTIONS

• R is a function if: x X ! y Y : (x,y) R

• There are two notations:

– f : X --> Y– f X x Y

• Alternate definition of a function: x X ! y Y f(x) = y

Evelyne Tropper 13

YX

xy

y1

For each x X there is a unique y Y

In other words, the dotted line going from x to y1 cannot exist

14

BIJECTIVE FUNCTION1-1 function

• If y Y ! x X then we have a Bijection or 1-1 function

• If f is bijective => f--1(y) = x for f(x) = y

YX

xy

y1

x1

FINITE AUTOMATA

• Are all problems as easily programmable?

• Consider a program to compute the integral of xn vs a sort program

• The first has constant memory requirements , the second can be of arbitrary length

• Can the implementation of a program be designed as a program?

programinput output

Evelyne Tropper 16

Deterministic Finite Automata

• A DFA is a program to design programs that use a constant amount of memory.

• A DFA can be thought of as a tape which reads one character at the time until the end of the tape. Each position on the tape puts DFA in a different state.

• It can elucidate errors that may show up in a programming problem.

Evelyne Tropper 17

Example 1

When a customer pays for goods with electronic money, that is with an ATM card, all possibilities must be accounted for :

• The customer may decide to pay, i.e. sends the money to the store• The customer may cancel & money is sent to bank to be deposited in customer’s account• The store may ship the goods to the customer• The store may redeem the money, i.e. the money is sent to the bank to be given to the store• The bank may send the money to the store

Can the store ship the goods without ever getting paid?

18

DFA for previous problem

For the store

For the bank

pay

cancel For the customer

ship ship ship

a

ge

fdb

c

Start pay redeem transfer

redeem transfer

3 41

2

start

Redeem transfer

cancel

19

Example 2Memory machine

States Outputs Input q0 q1 q0 q1 Time

0 (a) q0 q0 0 1 t

1 (b) q1 q1 0 1 t+1

q0 q1

0/0 1/1

1/0

0/1

20

Example 3Parity machine

States Outputs Input q0 q1 q0 q1 Time

0 q0 q1 0 1 t

1 q1 q0 1 0 t+1

q0 q1

0/0 0/1

1/1

1/0

21

Example 4Adding machine

1 1 1

101101 1110011100110

States are: ( i1 i2 carry)(000), (001), (010), (011), (100), (101), (110), (111)

000carry

10/1 10/0

11/0

00/1

01/0

11/100/0

01/1

Evelyne Tropper 22

Definition of a DFA

DFA A = (Q, , , s, F)Q - finite set of states - finite input alphabet - transition function from Q x --> Q s - initial state s Q F - favorable, or accepting, states F Q

Evelyne Tropper 23

Applications of a DFA

• Search engines on Web can use them• News analysts searching on-line for special topics• Stock analysts searching for stock names• “Shopping robots” searching for best price on-line• Searching for all books on Amazon containing a

certain phrase or word• grep, egrep, fgrep in Unix

Evelyne Tropper 24

Functioning of a DFA

•Automaton A in state q Q•reads a (letter in the alphabet)•enters state q1 = (q, a) which is determined completely by current state & which is the content of the current cell.

•Set of all accepted input words by A is called the language L(A) of the DFA.

Evelyne Tropper 25

Finite State Diagram

It is a directed graph representing a DFA.

q q1

aState q changing to state q1

after reading input a

Initial state is preceded by

A favorable state is doubly circled.

Evelyne Tropper 26

Programs & Finite State Diagrams

Program Astatement 1if ----

then exitelse ----

end

statement 2 statement 3

end

exit

end

s1

If-then

s0Statmt 1

If-else

s2end

s4

Statmt 3

s5end

Statmt 2

s3

27

Example 1

s rq

a

b a

a

b

Accepts language, L(anbm)

b

Ex: statement, statement,…, if-then, if-then, …, end

28

Example 2

s rq

b a

b

a

b a

Accepts any language with 2 consecutive a’sL(…, a, a, …)

Not acceptedTrapstays there

Ex: Any program with a nested pair of “while” loops

29

Example 3

s rq

b a

b

a

b a

It will not accept two consecutive a’s L(…, a, a, …) (Ex: no nested “for” loops)

It is the complement of the previous example

Dead, no further state

30

Example 4

b

f

qs

a

a

ab

b

• Initial “a” must be followed by “a” to be accepted

• Initial “b” can be followed by any number of a’s & b’s

Evelyne Tropper 31

Example 5

s fq

a

bb

a

p

a

a

b

b

L(a, bn, a, bm)

Ex: one “case” followed by multiple statements

Evelyne Tropper 32

Configurations

•In chess there are individual moves & standard patterns for the opening or end game.•Imagine that the players have reached the end game; the pieces left are BK, BQ, 1 BR, 1 BKt, 3 P; WK,WQ, 1 WB, 1 WKt, 2 P.•Imagine they are in a certain configuration, that is a certain pattern on the board.•Then the next moves can be figured out from that point on.

Definition of a configurationA configuration is a composite of state (pieces left), position (pattern on the board), input (next moves).

Ex: In reading input {aaaabba}, after state {aaa} has been reached, position q can be reached by reading input {abba}

(q,abba) is a configuration towards acceptance. w is accepted if it yields a favorable state.

If (q,w) -->(q1,w1) then σ : w = σ w1 & δ (q, σ ) = q1

then (q,w) yields (q1,w1) in one step

(q,w) yields (q1,w1) if a sequence of configurations:(q1,w1) … (qk,wk) : (q1,w1) = (q,w) , (qk,wk) = (q1,w1) &(qi,wi) yields (qi+1,wi+1) in one step.

All configurations yield unique configurations as it is deterministic.

Evelyne Tropper 34

Example 1

A vending machine for newspapers. The cover is released when $0.25 is reached. It does not return money if > $0.25 is put in.

= ( 5c, 10c, 25c )

10c 10c 10c 10c

s 20 15 10 5 a5c 5c 5c 5c 5c

25c

Evelyne Tropper 35

Examples of applications in Computer Science

• Programming sequences, branching, loops• Pattern matching (for WWW & AI)• Lexical analysis in compilers• Finite state machines in software specs & design• Word processors• Design of telecommunication protocols• Design of circuits for VLSI• Hardware design• Control mechanisms

36

Non-deterministic Finite Automata

• Union of 2 DFAs gives a NDFA as more than 1 arrow from any state with the same input.

• For example, in pattern matching of A B, we may match A or we may match B.

• NDFA A = ( Q,,,s,F )

where Q x ( {e}) x Q is a transition relation (q, a, p)

The empty string as input permits the automaton to jump from one state to the other. So after A is matched, whatever state the other branch is at, can jump

Evelyne Tropper 37

Example

•The ability to be in several different states at once, can be expressed as the ability to “guess” what will come next.

•When a system searches for a certain sequence of characters, such as a keyword, we can use a sequence of states to do nothing but jump from state to state until we find the keyword.

Evelyne Tropper 38

Example 1Find first occurrence of keywords

Searching for the keywords: “Web” or “Internet”Finding either gets us to a favorable state.

we b

in t e r n e t

Evelyne Tropper 39

Functioning of a NDFA vs DFA

• Automaton A in state q Q• reads a (letter in the alphabet)

• enters state q1 or q2 = (q, a) which is determined completely by current state & which is the content of the current cell.

q q1

a bState q changing to state q1 orstate q2 after reading input a bq2a b

Evelyne Tropper 40

Interpretation

• One input can send you to 2 different sub-programs (matching 2 Jones in a DB)

• This is useful when you build a program from multiple sub-programs. In industry, whole teams work on complicated programs, each member on separate sub-programs.

• In parallel programming, this represents different threads.

• NDFAs are easier to design & can be changed to DFAs

41

Examplesa

No jumping from q to pTherefore no need for trap state

(q,w) can yield many different configurations (q1, e)

b

qs r

a

b b Automaton that begins &ends with b

s

p

qe

e

b

Evelyne Tropper 42

Definition & theorem

• Definition: A & A1 that accept the same language are equivalent.

• Theorem: For all non-deterministic automaton A there exists a deterministic finite automaton A1 equivalent to A.

43

Example

q

s t

r

a b

e

ba e

b

This NDFA is equivalent tothe following deterministictransitions:

{s}---> {q,r,t}a

{s}---> {r,t}b

{s}---> {t}a

{t}---> {t}b

Accepting states are:{t}; {r,t}; {q,r,t}Trap state is {0}

Evelyne Tropper 44

Equivalent Deterministic Automaton

{r,t} {q,r,t}

{t}

{s}

b a

b a

Evelyne Tropper 45

Regular Expressions

Regular expressions are used to:

• Design a language for mathematical specification of languages acceptable by a finite automata (computers of all types)

• Do that by a general algorithmic procedure

• To convert FA back to its specs through an algorithmic procedure.

• These are used in turn to do simulations.

Evelyne Tropper 46

Reg. Exp. vs FA

• DFA & NDFA are machine-like descriptions

• Regular Expressions are algebraic-like descriptions.

Evelyne Tropper 47

Applications of Regular Expressions

• Lexical analyzers such as Lex & Flex which take source code and convert it into tokens

• Grep in Unix

• Alternative to FA notation for describing software components

• A declarative way to express which strings are acceptable

Evelyne Tropper 48

Language for regular expressions

Some of the complex programming languages can be obtained from simpler languages using , , , concatenation & Kleene stars.

Concatenation: of strings u, v is “uv” of languages L1, L2 is L1L2 = {uv | u L1, v L2}

Kleene Star: L* of L is the infinite union: {e} L L2 L3 …

L* = {w1w2 …wk} when wi L

Evelyne Tropper 49

Examples

L M = {001, 10, 111} {e, 001} = {e, 001, 10, 111}

L M = {001}

LM = {001, 10, 111, 001001, 10001, 111001}

L* for {0, 1} = all strings of 0’s & 1’s for {0, 11} = all strings of 0’s & 1’s such that 1’s come in pairs L0 = {e}; L1 = {0, 11}; L2 = {00, 011, 110, 1111}; L3 = {000, 0011, 0110, 01111, 1100, 11011, 11110, 111111}

…

Evelyne Tropper 50

Languages accepted by FA

Theorem: If languages L & M are acceptable by a finite automaton, so are L M, L M, * - L (complement), L – M, LM (conc), L*

(Kleene Star)

s

.

.

.

s

.

.

.

Suppose L accepted by

Suppose M accepted by

Evelyne Tropper 51

_ 2. Then L is accepted by a FA which is the DFA equivalent of the

NDFA and by flipping favorable & unfavorable states _______

_ __3. L M = (L M) and is accepted __4. L – M = L M and is accepted

s

.

.

.

s1

.

.

.

q

e

e

1. Then L M accepted by :

52

5. LM is accepted by:

6. L* is accepted by:Favorable state of L

Favorable state of L

Favorable state of L

s... s1

.

.

.

Favorable state of L

s1

.

.

.s e

e

eConnect back to original state

New startTo include {e}

53

Definition of a Regular Expression

It is a string over an alphabet: { (,), e, , , * }

It is mostly used in lexical analyzers.

1. , e, a ( the ground elements) are regular expressions.

2. If , are regular expressions then , , * are regular expressions by induction

3. No other string is a regular expression.

Evelyne Tropper 54

Mapping of regular expression to language L()

• L() = , L(e) = {e}, L(a) = {a} a

• If , are regular expressions thenL( ) = L() L()

L() = L()L( )

L(*) = (L())*

55

Examples

• L((ab*)a) = {w | w of form abna}• Identifier in C begins with a letter & may

be followed by a string of letters & digits. The identifiers can thus be expressed by the regular expression:([a-z] [A-Z]) ([a-z] [A-Z] [0-9])*

• Identifiers of languages with underscores can be expressed by: ([a-z] [A-Z]) ((-([a-z] [A-Z] [0-9]))* ([a-z] [A-Z] [0-9])*)*

56

Examples

• Strings with alternating 0’s & 1’s

(01)* + (10)* + 0(10)* + 1(01)*

Same as starting with (01)* and adding optional 1 at beginning and optional 0 at end:

(e+1)(01)*(e+0)

• Precedence of operators in Reg. Exp. is:

*, . , + or by grouping with parentheses

Evelyne Tropper 57

Regular Expression --> FA

For reg. exp.: ( (a ab)* ba )*• ground singletons become:

• doubletons become

• (a ab) becomes:

>a,b

>a b

>b a

a

a b

>e

e

Evelyne Tropper 58

(a ab)*

Add: bab

a

Add: ( ... )*

a

a be

e

e

e

e

>

e

59

FA --> Reg. Exp.

Start with FA

•Replace 2 favorable stateswith 1.•Label nodes 1 to n.

a

b ba

b>

ee

a

1

7

2 3 4

5 6

•Replace arrows from i to j & from j to k with an arrow labeled li,jlj,k

Label from 1 to 3 will be “ab”Label from 1 to 4 will be “abb”

• If there is an arrow from j to j, add label li,jlji*

a

b ba

b>

a

Labels b,a from 2 to 3 will be inserted as “ab(ab)*b”

•Different arrows from i to j will be replaced by l1 l2

Labels from 1 to 4 and 1 to 7 will be replaced by“ab(ab)*b ba”

Evelyne Tropper 60

Example

3

5 4 52 1 >a

a b

b a

b

1. Single favorable state2. Number nodes3. 3 2 4 becomes ab from 3 to 4; 1 2 4 becomes ab from 1 to 4; 4 2 4 becomes bb4. Multiple arrows from one node are replaced by a union

61

3

5 4 51 >ab a

bb

bab

After 1, 2, 3 babbb

5 4 51 >ab a

After 4

ab(bab bb)*a

Evelyne Tropper 62

Non-regular languages

• In the design of circuits on a chip, it is important to know whether two automata define the same language. This might permit to minimize the area & cost of the chip.

• FA & regular languages can describe programs using a fixed amount of memory regardless of input. This is not true for loops.

Evelyne Tropper 63

Example

• L(anbn) = {ab, a2b2, a3b3, … } is not a regular language because after accepting “an” the automaton is in a state “q” and that state cannot know what came before. So it does not know how many “b”s it should accept.

• A loop in a program can be represented by a regular language or a FA if we can insert or remove iterations without changing the nature of the program.

Evelyne Tropper 64

Pumping Lemma for Reg. Exp.

• For every Reg.lang. L there exists a constant n such that every string w in L, where |w| >= n, we can break w into 3 strings, w = xyz:

1. y e

2. |xy| <= n

3. For all k >=0 the string xykz is also in L

Evelyne Tropper 65

Steps in proof

• Assume it is regular.

• Find the defining property of language

• Define a y such that |xy| <= n and w=xyz

• Find a k such that xykz destroys the property of the language

• You have now found one example that is not in the language

• Therefore the language cannot be regular

Evelyne Tropper 66

Example 1

• L(anbn) ={w} is not a reg.lang.

Proof:

|w|= 2n

Let w = xyz, then if1. |xy| <= n, then y = am for some m

2. Therefore xykz = anamkbn and is not in the language

Evelyne Tropper 67

Example 2

• L(1n) = {w : |w| is a prime } is not a reg.lang.

Proof:1. If it were a reg.lang., there would exist a prime p >=

n, such that w = 1p, and |w| = p.

2. Then w = xyz, |xy| <= n

3. Let |y| = m. Then |xz| = p-m

4. Consider string xyp-mz which must then be in L.

5. |xyp-mz| = |xz| + (p-m)|y| = (p – m) + (p – m)m = (m + 1)(p – m)

6. Therefore |xyp-mz| is not a prime.

Evelyne Tropper 68

Example 3

• Language of palindromes is not regular.Proof:

1. If it were a regular language, then there would exist an n: every w L, where |w| >= n can be represented as w = xyz.

2. Consider anban. Since |xy| <= n, y is a substring of an on the left. Let y = am

3. Then w = an-myban and by the Pumping Lemma an-mykban L. But an-mykban = an-

mamkban and it is not a palindrome.

Evelyne Tropper 69

The closure properties of RL

• The union of 2 RLs is regular• The intersection of 2 RLs is regular• The complement of 2 RLs is regular• The difference of 2 RLs is regular• The reversal of a RL is regular• The Kleene star of a RL is regular• The concatenation of 2 RLs is regular• A homomorphism (substitution of strings for

symbols) of a RL is regular• The inverse homomorphism of a RL is regular

Evelyne Tropper 70

Equivalence & Minimization of Automata

• When are 2 descriptors of a RL equivalent?

• Consequently, when can we minimize a DFA? This would be a unique minimal DFA.

• We start by asking when 2 states can be replaced by a single state behaving like both.

Evelyne Tropper 71

Equivalent & distinguishable states

• p & q are equivalent if:

w: (p, w) is an accepting state iff (q,w) is an accepting state

• If 2 states are not equivalent, then they are distinguishable, and one is accepting while the other is not.

Evelyne Tropper 72

Writing efficient FA’s & RE’s

• There many various FA’s and RE’s that will satisfy a language, just as there are many different programs that will solve a particular problem.

• FA’s & RE’s are used to design VLSI chips.

• In all cases it is more efficient & cost effective to find the minimal FA or RE to represent the problem at hand.

Evelyne Tropper 73

State Minimization Problem

• The idea is to find the FA (or RE) with the minimal number of states equivalent to the one being used.

• Equivalent automata are indistinguishable by the time they get to an accepting state. That is, if they both get accepted with the same inputs, they are indistinguishable.

Evelyne Tropper 74

Example 1

1. State n is unreachable

2. States t & p are both reachable from input a and are indistinguishable

3. n will be eliminated and t & p will be merged

4. q & r are both accepted through a and reach a dead state through b. They are indistinguishable.

n

s

q

r p

t

m

ba

a

a

b

b

b

a

b

a

b

a

b

a

Evelyne Tropper 75

Equivalent automaton

s {q,r}

m

{t,p}

a

b

b

a

a,b

a,b

Evelyne Tropper

Example 2

A

HGFE

DCB0 1

0

0

1 0 1

111 1 0

1

0

1

0

1. A & E are both non-accepting, so input e does not apply.

2. On input 1 they both go to F.

3. On input 0, A goes to B & E goes to H. Then on input 1, B goes to C & H goes to C. So on input 01, both A & E go to C, a favorable state.

4. Therefore A & E are equivalent.

5. D & F both go to C on 0

Evelyne Tropper 77

Equivalent minimal FA

{A,E}

{B,H}

F

0

1

G

C

0

1

1

1

1

0

0

78

Definition of equivalent states

States p & q are equivalent if for all input strings w,

(p, w) is an accepting if and only if (q, w) is an accepting state.

Any pair of states that are not distinguishable as they proceed to be accepted are equivalent.

Evelyne Tropper 79

Finding equivalent states

• Any state that is not accepting cannot be equivalent to any accepting state.

• States that reach an accepting state with the same single input are equivalent.

• States that reach an accepting state with the same multiple input are equivalent.

Evelyne Tropper 80

Proof of equivalency

It is easier to show the states that are not equivalent than the ones that are equivalent on a very large FA. To prove they are equivalent all inputs (0, 1, 00, 01, 11, 000, …) must be considered.

1. The accepting state(s) is(are) not equivalent to any non-accepting state

2. Going back one step from the accepting state(s), the states getting there on different inputs are not equivalent.

3. Going back two steps from the accepting state(s), the states getting there on different inputs are not equivalent.

4. Etc.

Table of inequivalencesfor example 2

x

x

x

x

x

xxxx

xxxx

xx

xx

x

x

x

x

x

x

x

x

GFEDCBA

H

G

F

E

D

C

B

Evelyne Tropper 82

Minimization of FA through tables

• Eliminate any state that cannot be reached from the start state

• Partition remaining states into equivalent blocks so that no pair of states from different blocks are equivalent

Evelyne Tropper 83

Context-Free Grammars

• Some languages cannot be recognized by FA’s

• Can FA’s recognize legal English statements?

• Can FA’s recognize the syntactical correctness of statements in programs?

Evelyne Tropper 84

Uses of Context-Free Grammars

• Since the 1960’s CFGs have been used to turn out parsers automatically.

• They are used to describe document formats with Document Type Definitions, DTDs used in XML for creating Web pages.

• Grammars can define languages through the use of Parse Trees.

Evelyne Tropper 85

Parse Tree example 1

sentence noun-phrase, verb-phrasenoun-phrase (proper-noun | determiner),

common-nounverb-phrase verb | (verb, adverb)

…

A Parse Tree is a recursive definition of a language.

86

Palindromes example 2

The language of palindromes cannot be represented by RE’s, but it can be defined recursively as:

1. e, 0, 1 are palindromes2. If w is a palindrome, so are 0w0 and 1w1

As a Parse Tree:1. P e2. P 03. P 14. P 0P05. P 1P1

P

0 P 0

1 P 1

P

e

Evelyne Tropper 87

Definition of a CFG

• Set of symbols that form the strings of the grammar are called terminals,

• Set of variables, or strings, are called non-terminals, NT

• A start symbol, S NT• Set of productions or rules, R:

R NT x ( NT)* which serve to derive the terminals from the

non-terminals

88

Derivation mechanism

• Left part of arrow always has a non-terminal• Right part of arrow has a sentential form• A sentential form v is one-step derivable from sentential form u:u v if u = xAy and v = xzy and A z is in the rules R• v is derivable from u, u * v, if there exist a sequence, called a derivation, u0, … ,un : u=u0 and v=un and u0u1, u1u2,…,un-1un • The language L(G) generated or derivable in a grammar is defined as:

L(G) = {w | w *, S * w}

89

Example 3 - a pseudo-language

Id := Id + Id

Id := Id – Id

IF (Id [= < ><= >=] Id) Inst

ELSE Inst

END

Example of a pgm Inst Assign | Inst-phrase | END

Assign (Id) (:= )(Id) (op) (Id)

Id ([a-z]+ [A-Z]) ([0-9]* + [a-z]*+ [A-Z]*)

op + | -

Inst-phrase If-phrase | If-else-phrase

If-phrase (IF) ( condition)

condition (Id) (Cond-op)(Inst)

Cond-op = | < | > | <= | >=

If-else-phrase (If-phrase) (ELSE) (Inst)

90

Example 4

Let grammar G have non-terminalsNT = {S, Np, Vp, Ap, N, V, A}

and terminals = {big, stout, John, bought, white, car}

and rules, or productions

S NpVp

Np N | ApN | eAp ApA | eVp VNp

A big | stout | white N John | car V bought

Strings generated by G:•John bought car• John bought big car• big stout John bought big white car• big stout car bought big white car

91

Different derivations of a sentence

Given: S = John bought carwe can have derivation:

S NpVp NVp John Vp John bought Np John bought N John bought car

orS NpVp NpVNp NVNp NVN NVcar N bought car John bought car

Both derivations give the same result and can best be described in a parse tree. All derivations with same parse tree are equivalent.

We can always replace leftmost non-terminal, called leftmost derivations, or we can always replace the rightmost non-terminal, called rightmost derivations.CFG derivations are non-deterministic.

Parse tree for example 4

S

Np Vp

N

John

V Np

bought N

car

93

Parse trees for a+b*c

exp + exp

exp

exp * exp

exp

exp * exp

exp

exp +

They are not equivalent. That is why we need a precedence rule for algebraic expressions.Factors are parenthesized expressions that cannot be broken down.Terms are expressions that cannot be broken down by the + or - operators, such as a*b.A parser will accept only leftmost or only rightmost derivations.

Evelyne Tropper 94

Applications

1. Parsers were the first applications of CFG2. The YACC command in Unix is a CFG that creates either a tree or a piece of object code. It allows to state the precedence of operators in expressions.3. XML (Extensible Mark-up Language) was the precursor & is a superset of HTML (Hypertext Markup Language), a language with which Web pages are created, both require a DTD (Document Type Definition) which is a CFG describing the tags allowed.

95

Pushdown Automata

FA can easily be expressed as derivations, so that any FA can be expressed as a CFG:

(a,s) = s1 can be expressed as s as1

However, there are non-regular, context-free languages such as L(anbn).

If we add a stack to a FA such that every time it reads a b it pops an a, then it does not need to “remember” how many a’s there were.

Evelyne Tropper 96

Stack operation

Finite state

control

inputAccept/reject

stack

a b

ab

97

Example - language of palindromes

Take the language of palindromes of even length:Lww

r = {wwr | w is in (0,1)*}

• q0 is the state that represents a “guess” that we are not yet in the middle. In state q0 we read input symbols & push them onto the stack.• At any time we guess that we have seen the middle & go to state q1. Here the right part of w will be on top of the stack and the left part on the bottom.• In state q1 we compare input symbols with the symbol at the top of the stack. If they do not match, the guess was wrong & this branch dies.• If the input symbol matches the symbol on the top of the stack, we start popping until the stack is empty & enter an accepting state.

98

Example for w=1111

(q0,1111,a0)

(q0,111,1a0) (q1,1111,a0) (f,1111,a0)

(q0,11,11a0) (q1,111,1a0) (q1,11,a0)

(q0,1,111a0) (q1,11,11,a0) (f,11,a0)

(q0,e,1111a0) (q1,1,111a0) (q1,1,1a0)

(q1,e,1111a0) (q1,e,11a0) (q1,e,a0) (f,e,a0)

99

Definition of Pushdown Automata

P = (Q, , , s0, a0, , F)where:Q is a finite set of states is the input alphabet is the set of stack symbolss0 Q is the initial statea0 is the start symbol needed at bottom of stack to get to a

favorable state after stack has been emptied

(s,a,X | sQ, a{e}, X) is the transition relation with output (p,S | pQ, S is string of symbols

replacing X on top of stack) FQ is the set of favorable states.

100

PDA for language of palindromes

P = ( Q , , , s0, a0, , F )P = ({s0,s1,f}, {0,1}, {0,1,a0}, s0, a0, , {f})

• Originally in state s0 the stack contains a0 and we input 0 or 1 (s0,0,a0) = {(s0,0)} and (s0,1,a0) = {(s0,1)} • Reading another 0 or 1 we obtain transitions: (s0,0,0) = {(s0,00)}, (s0,0,1) = {(s0,01)}, (s0,1,0) = {(s0,10)} and (s0,1,1) = {(s0,11)} • We can go from state s0 to state s1 on input e: (s0,e,a0) = {(s1,a0)}; (s0,e,0) = {(s1,0)} and (s0,e,1) = {(s1,1)} • In state s1 we match input symbols to stack symbols: (s1,0,0) = {(s1,e)} and (s1,1,1) = {(s1,e)} • After emptying the stack of input we are left with a0 in it, and e takes us to a favorable state s2: (s1,e,a0) = {(f,a0)}

Evelyne Tropper 101

PDA for L={anbn)

Q = {s0,s1,f} , = {a,b}, = {a}, F = {f}

• (s0,e,e) = {(s1,a0)}• (s1,a,e) = {(s1,a)}• (s1,a,e) = {(s1,a)} • (s1,b,a) = {(s1,a0)}• (s1,e,a0) = {(f,e)}

Evelyne Tropper 102

PDA for L={a,b}*

Language with the same number of a’s and b’sQ = {s0,s1,f} , = {a,b}, = {e}, a0 , F = {f}

1. (s0,e,e) = {(s1,a0)}2. (s1,a,a0) = {(s1,aa0)}3. (s1,a,a) = {(s1,aa)}4. (s1,a,b) = {(s1,e)}5. (s1,b,a0) = {(s1,ba0)}6. (s1,b,b) = {(s1,bb)}7. (s1,b,a) = {(s1,a0)}8. (s1,e,a0) = {(f,e)}

103

Transitions accepting abbbaaaabb

State s0

s1 s1 s1

s1

s1

s1

s1

s1

s1

s1

s1

f

Input left abbbaaaabb abbbaaaabb bbbaaaabb bbaaaabb baaaabb aaaabb aaabb aabb abb bb b e e

Stack e a0

a a0

a0

b a0 bb a0

b a0

a0

a a0

aa a0

a a0

a0

e

Transition

1 2 7 5 6 4 4 2 3 7 7 8

Evelyne Tropper 104

Acceptance by empty stack

In the last example, we reached the end when the stack was empty. In the example, we did not reach a favorable state.This is called a PDA that accepts input strings by empty stack rather than by a favorable state, and it is described by:

P = (Q, , , s0, a0, )

Theorem:There is an algorithm that accepts a string by empty stack rather than by favorable state, and the two algorithms are equivalent.

105

CFG PDATheorem :Given any CFG, G, there exists an algorithm that constructs a PDA,

A, such that L(A) = L(G)Proof:1. Let A have 2 states, s0 and f.2. Push s0 onto the stack.3. If topmost symbol on stack is N, a non-terminal, A picks rule

Nw in G and replaces N on the top of the stack by w.4. If topmost symbol on stack is T, a terminal, A advances to next

symbol; if it matches T, it pops top of the stack.

Transitions:1. (s0,e,e) = {(f,s0)}2. (f,e,N) = {(f,w)} for each rule N w in grammar G3. (f,T,T) = {(f,e)} for each terminal T

106

Example 1 for CFG PDA

Transitions of PDA for CFG A is:1. (s0,e,e) = {(f,S)}2. (f,e,S) = {(f,aSa)} 3. (f,e,S) = {(f,bSb)}4. (f,e,S) = {(f,e)}5. (f,a,a) = {(f,e)}6. (f,b,b) = {(f,e)}

Grammar G with following rules:S eS aSaS bSbFor language L = {wwr | w {a,b}*}

107

PDA CFGTheorem :Given any PDA, A, there exists an algorithm that constructs a CFG, G, such that L(G) = L(A)Proof:Consider a simple PDA accepting by empty stack. A simple PDA is one where every transition replaces the top of the stack.For any transition (s,a,) --> (q,)Let be A1, A2, …, An. These represent the popped symbols.Then the transition (s,a,) --> (q,) can be replaced by:

(s,e,A1) --> (sA1,e) (sA1,e,A2) --> (sA1A2,e)

… (sA1…An-2,e,An-1) --> (sA1A2…An-1,e) (sA1…An-1,e,An) --> (q,)

Evelyne Tropper 108

The grammar can now be defined by the following rules:• For every sQ, S --> [s0, a0, s]• For every s,qQ, every a {e}, A, if (s,a,A) --> (q,e), then [s,A,q] --> a• For every s,qQ, every a {e}, A, if (s,a,A) --> (q,B1B2…Bk) then [s,A,qk] --> a[q,B1,q1] [q1,B2,q2]... [qk-1, Bk,qk]

Evelyne Tropper 109

RL PDA CFG

Theorem:

Every Regular Language is context free.

Proof:Given a FA, A, accepting a language, L.View A as a PDA that does not use its stack.Therefore L is accepted by a PDA.Because every PDA CFG, every Regular Language, or FA, is context free.

Evelyne Tropper 110

Languages that are not Context-Free

Take language:S --> uAz --> uvAyz --> uvxyz

Then it must be true that A --> x and A --> vAy

Then we can derive further, getting:S --> uAz --> uvAyz --> uv2Ay2z --> uv3Az3z --> .. --> uvnAynz

Now we can define a type of Pumping Lemma for context-free languages

Evelyne Tropper 111

Pumping Lemma for Context-Free Languages

Lemma:Let G = (, NT, R, S) be a context-free grammar. Then there exists a number n such that any string w L(G) with length |w| n can be written as w = uvxyz for some strings u, v, x, y, z , and such that

1. |v| > 0, or |y| > 02. |vxy| n3. For any k 0, uvkxykz L(G)

Evelyne Tropper 112

Example 1

Show that the language L = {akbkck | k = 0, 1, 2, …} is not context-free.Proof:If it is context-free, let w = anbncn with length |w| = 3nLet w = uvxyz with length |vxy| nThen vxy contains only a’s and b’s, or only b’s and c’s.If vxy contains only a’s and b’s, then v or y contains at least one symbol.Then the string uv2xy2z contains more than n a’s, or more that n b’sBut the number of c’s is still the same. Therefore the string is not in the language.