Theory of Computationjnujprdistance.com/assets/lms/LMS JNU/MCA/Sem V... · Theory of Computation 2...

Theory of Computation

Board of Studies

Prof. H. N. Verma Prof. M. K. GhadoliyaVice- Chancellor Director, Jaipur National University, Jaipur School of Distance Education and Learning Jaipur National University, JaipurDr. Rajendra Takale Prof. and Head AcademicsSBPIM, Pune

___________________________________________________________________________________________

Subject Expert Panel

Dr. Ramchandra G. Pawar Vaibhav BedarkarDirector, SIBACA, Lonavala Subject Matter ExpertPune

___________________________________________________________________________________________

Content Review Panel

Gaurav Modi Shubhada PawarSubject Matter Expert Subject Matter Expert

___________________________________________________________________________________________Copyright ©

This book contains the course content for Theory of Computation.

First Edition 2014

Printed byUniversal Training Solutions Private Limited

Address05th Floor, I-Space, Bavdhan, Pune 411021.

All rights reserved. This book or any portion thereof may not, in any form or by any means including electronic or mechanical or photocopying or recording, be reproduced or distributed or transmitted or stored in a retrieval system or be broadcasted or transmitted.

___________________________________________________________________________________________

I

Index

ContentI. ...................................................................... II

List of FiguresII. ..........................................................IV

List of TableIII. ............................................................. V

AbbreviationsIV. .........................................................VI

ApplicationV. ............................................................... 71

BibliographyVI. ........................................................... 76

Self Assessment AnswersVII. .................................... 78

Book at a Glance

II

Contents

Chapter I ....................................................................................................................................................... 1Introduction to Automata ............................................................................................................................ 1Aim ................................................................................................................................................................ 1Objectives ...................................................................................................................................................... 1Learning outcome .......................................................................................................................................... 11.1 Introduction .............................................................................................................................................. 21.2 Structural Representation ......................................................................................................................... 21.3 Automata and Complexity ....................................................................................................................... 21.4 Computability .......................................................................................................................................... 41.5 Introduction to Proofs .............................................................................................................................. 4 1.5.1 Deductive Proofs ...................................................................................................................... 4 1.5.2 Inductive Proofs ....................................................................................................................... 6Summary ....................................................................................................................................................... 9References ..................................................................................................................................................... 9Recommended Reading ............................................................................................................................... 9Self Assessment ........................................................................................................................................... 10

Chapter II ................................................................................................................................................... 12Finite Automata and Regular Languages ................................................................................................ 12Aim .............................................................................................................................................................. 12Objectives .................................................................................................................................................... 12Learning outcome ........................................................................................................................................ 122.1 Introduction ............................................................................................................................................ 132.2 Finite Automata ...................................................................................................................................... 132.3 Deterministic Finite Automaton (DFA) ................................................................................................. 142.4 Complexity of DFA String Recognition ................................................................................................ 142.5 Non-deterministic Finite Automata (NFA) ............................................................................................ 152.6 Difference between DFA and NFA ........................................................................................................ 152.7 Equivalence of NFAs and DFAs ............................................................................................................ 16 2.7.1 Proof Strategy ........................................................................................................................ 16 2.7.2 Subset Construction Algorithm .............................................................................................. 162.8 Finite Automata with ε-transition (Epsilon-transitions)......................................................................... 172.9 Regular Expressions and Languages ..................................................................................................... 172.10 Regular Languages ............................................................................................................................... 18 2.10.1 Properties of Regular Languages ........................................................................................ 18 2.10.2 Theorem (Properties of R) ................................................................................................... 182.11 Conversion of RE to FA ....................................................................................................................... 19Summary ..................................................................................................................................................... 21References ................................................................................................................................................... 21Recommended Reading ............................................................................................................................. 21Self Assessment ........................................................................................................................................... 22

Chapter III .................................................................................................................................................. 24Push Down Automata and Context Free Languages .............................................................................. 24Aim .............................................................................................................................................................. 24Objectives .................................................................................................................................................... 24Learning outcome ........................................................................................................................................ 243.1 Introduction ............................................................................................................................................ 253.2 Context Free Languages ........................................................................................................................ 253.3 Context Free Grammars ......................................................................................................................... 253.4 Ambiguity in CFG and Its Removal ...................................................................................................... 283.5 Chomsky Normal Form ......................................................................................................................... 313.6 Push Down Automata ............................................................................................................................ 33

III

3.6.1 Formal Definition of a PDA ................................................................................................... 33 3.6.2 PDA Transition Function ....................................................................................................... 33 3.6.3 Graphical Format of PDA ...................................................................................................... 35 3.6.4 Moves of a PDA ..................................................................................................................... 36 3.6.5 Language of PDA ................................................................................................................... 36 3.6.6 Equivalence of PDA and CFG ............................................................................................... 373.7 Non Context Free Languages ................................................................................................................. 38Summary .................................................................................................................................................... 40References ................................................................................................................................................... 40Recommended Reading ............................................................................................................................. 40Self Assessment ........................................................................................................................................... 41

Chapter IV .................................................................................................................................................. 43Turing Machines and Computability ....................................................................................................... 43Aim .............................................................................................................................................................. 43Objectives .................................................................................................................................................... 43Learning outcome ........................................................................................................................................ 434.1 Introduction ............................................................................................................................................ 444.2 Formal Definition of Turing Machine .................................................................................................... 454.3 Some Configuration on TM ................................................................................................................... 524.4 Variants of Turing Machines .................................................................................................................. 534.5 Equivalence with Other Models ............................................................................................................. 534.6 Example of Turing Machine .................................................................................................................. 544.7 Algorithm ............................................................................................................................................... 544.8 The Church-Turing Thesis ..................................................................................................................... 544.9 Universal Turing Machine ..................................................................................................................... 544.10 Universal TM as a 3-tape TM .............................................................................................................. 55Summary ..................................................................................................................................................... 56References ................................................................................................................................................... 56Reference Reading ..................................................................................................................................... 56Self Assessment ........................................................................................................................................... 57

Chapter V .................................................................................................................................................... 59Decidability, Undecidability and Reducibility ......................................................................................... 59Aim .............................................................................................................................................................. 59Objectives .................................................................................................................................................... 59Learning outcome ........................................................................................................................................ 595.1 Introduction ............................................................................................................................................ 605.2 Decidable Languages ............................................................................................................................. 605.3 Halting Problem ..................................................................................................................................... 61 5.3.1 The Halting Problem for Turing Machines ............................................................................ 615.4 The Post Correspondence Problem ........................................................................................................ 625.5 Undecidable Problems about Turing Machines ..................................................................................... 63 5.5.1 Rice Theorem ......................................................................................................................... 635.6 Reducibility ........................................................................................................................................... 645.7 Mapping Reducibility ........................................................................................................................... 65 5.7.1 The Halting Problem II .......................................................................................................... 665.8 Turing Reducibility ................................................................................................................................ 67Summary .................................................................................................................................................... 68References .................................................................................................................................................. 68Recommended Reading ............................................................................................................................. 68Self Assessment ........................................................................................................................................... 69

IV

List of Figures

Fig. 1.1 Complexity hierarchy ....................................................................................................................... 3Fig. 2.1 A finite automata ............................................................................................................................. 13Fig. 2.2 A state transition diagram ............................................................................................................... 14Fig. 4.1 Turing machine ............................................................................................................................... 44

V

List of Table

Table 2.1 Difference between DFA and NFA ............................................................................................... 15

VI

Abbreviations

BN - Backus-NaurCFG - Context Free LanguagesCNF - Chomsky Normal FormDFA - Deterministic Finite AutomatonFA - Finite AutomataGNF - Greibach Normal FormNFA - Non-Deterministic Finite AutomataPCP - Post Correspondence ProblemPDA - Push down AutomatonRAM - Random Access MemoryRE - Regular ExpressionTM - Turing MachineUNIX - Uniplexed Information and Computing System

1

Chapter I

Introduction to Automata

Aim

The aim of the chapter is to:

portray the theory of automata •

analyse automata and complexity•

study the different types of proofs•

Objectives

The objectives of the chapter are to:

examine computability•

determine deductive proofs•

ascertain inductive proofs•

Learning outcome

At the end of this chapter students will be able to:

understand the concept of automata•

explain complexity and computability •

recall proofs and types of proofs•


2

1.1 IntroductionAutomata theory is the study of abstract computing devices, or machines. It presents and studies roughly three types of automata:Finite automata - Finite automata are the simplest kind of automata; are widely used to model and analyze finite-state systems.Pushdown automata - Pushdown automata add to finite automata a restricted kind of unbounded memory in the form of a stack.Turing machines - Turing machines add to finite automata a more powerful notion of memory in the form of an unbounded tape.

In traditional automata theory, automata are treated as language acceptors. The idea is that a string accepted by the automaton represents a particular computation of the automaton, and the language accepted by it thus corresponds with the set of all computations of the automaton.

1.2 Structural RepresentationThere are two important notations that are not automaton-like, but they play an important role in the study of automation and their applications.

Grammars are useful models when designing software that processes data with a recursive structure. The • best-known example is a “parser”, the component of a complier that seals with the recursively nested features of the typical programming language, such as expressions- arithmetic, conditional, and so on. For example, a grammatical rule like states that an expression can be formed by taking any two expressions and connecting them by a plus sign; this rule is typical of how expressions of real programming languages are formed. Regular Expressions also denote the structure of data, especially text strings. The style of these expressions • differs significantly from that of grammars. For example, UNIX-style regular expression ‘[A-Z][a-z]*[] [A-Z][A-Z]’ represents capitalized words followed by space and two capital letters. This expression represents patterns in text that could be a city and state, for example, Ithaca NY. It misses multiword city names, such as Palo Alto, CA, which could be captured by more complex expression –

[A-Z][a-z]*([] [A-Z][A-Z*)*[][A-Z][A-Z]

While interpreting such expressions, we only need to know that [A-Z] represents a range of characters from capital ‘A’ to capital ‘Z’, and [ ] is used to represent the blank character alone. Also, the symbol * represents ‘any number’ of the preceding expression. Parenthesis is used to group components of the expression; they do not represent characters of the text described.

1.3 Automata and ComplexityComplexity is the study of the limits of computation. There are two important issues faced, which include:

Decidability: What can a computer do? The problems that can be solved by a computer in a realistic amount of time are called decidable. Some problems are undecidable, or only semi-decidable.For example: Suppose you have a procedure for generating (enumerating) members of a set S, but no general procedure for deciding if something is not in S (later we’ll see this has a relationship between recursively enumerable but not recursive sets). You want to know if x is an element of S; if it is in S, you can just generate elements of S until you produce x (which you will do after a finite number of steps). But suppose it is not in S. You can never be sure that you won’t generate it if S is not finite.

3

Intractability: What can a computer do efficiently? This studies the problems that can be solved by a computer using no more time than some slowly growing function of the size of the input. Typically, we will take all polynomial functions to be tractable, while functions that grow faster than polynomials intractable.

A diagram indicating some classes of complexity for different types of problems and machines are shown in the Fig 1.1. We’ll discuss only a few of the items on the diagram, in particular the relationship between problems that can be solved in polynomial time and those that may be solved in nondeterministic polynomial time.

Fig. 1.1 Complexity hierarchy

Arithmetiv Hierarch y

Recursiv e

co-r.e.co-r.e.

complete

co-NPcomplete

NPcompleteSO

co-NP NP

r.e.completer.e.FO (N)

FO

A

(N)

SO

A SOE

FO

E

(N)

Primitive Recursive

Polynomial-Time Hierarch y

NP

U

co-N P

EXPTIME

PSPACE FO(PFP) SO(TC)

SO(LFP)SO[2 n

o(1)

]

FO[n o(1)

]

FO[(log n) o(1)

] NC

NC 2

log(CFL)

Logspace

Regular NC 1

AC 0 FO

FO(TC)

FO(DTC)

SO-KromNondeterministic Logspace

Logrithmic-Time Hierarch y

FO(LFP)P

SO-Horn“truly feasible”

FO[2 n

o(1)

] SO[n o(1)

]


4

1.4 ComputabilityComputability theory asks questions such as; are there existing problems that are unsolvable by any effective procedure; unsolvable by any program in any conceivable programming language on any computer?

The programming intuitions may indicate a no answer, based on the experience that once a problem is made precise in the form of a specification, it is a more or less routine task to write a program to satisfy the specification. Indeed, a related intuition predominated the work of Hilbert on the foundation of mathematics; they conjectured that all of mathematics could be axiomatized. However, we shall see that both of these intuitions are disastrously wrong. There are certain problems that cannot be solved by effective procedures.

To prove this, we must make precise what is meant by an effective procedure and what is meant by a problem. It is not a prior obvious that any single formalization of effective procedure could be adequate; it might seem that any specific choice would be too narrow because it would exclude computing devices with special capabilities. Thus, different formalizations might lead to different theories of computability.

However, one of the great insights of computability theory was the gradual realization in the 1930’s that any reasonable choice of formalization of the notion of effective procedure leads, in a certain sense, to the same theory. This has been called the Church-Turing thesis, since Alonzo Church and Alan M. Turing first formulated and substantiated versions of this insight. Explaining why different formalizations lead to the same theory is itself one of the topics of computability theory; we thus devote considerable effort to the matter.

Granted precise definitions of the notions of problem and effective procedure, computability theory is concerned with the boundary between computability and uncomputability, and addresses questions such as:

Can every precisely stated problem be solved by some effective procedure?• What is the class of problems that can be solved by effective procedures and its basic properties?• What is the relationship between various problems that cannot be solved by effective procedures?•

If a problem can be solved by an effective procedure we shall say that is effectively solvable, or sometimes just solvable. The result that a certain computational problem is unsolvable is not a purely negative fact; for instance, it conveys the important knowledge that searching for an effective procedure to solve the problem is futile. This may indicate that one should try to find an approximate, solvable, solution to the problem at hand instead of trying to solve the exact, but unsolvable, problem.

1.5 Introduction to ProofsA proof is a convincing logical argument that a statement is true. In this class we will take a look on formal proofs (very rigorous and exact) and ‘personal feeling’ proofs (not very exact, but gets the general idea across). Both the forms have their pro’s and con’s. Very complex programs are too difficult to analyze formally, and instead we must rely on software engineering testing procedures. However, with a tricky recursive function or iteration is unlikely to be correct unless the programmer can prove it is correct or at least understand the process of proving it correct. Automata theory lends itself to both deductive and inductive proofs.

1.5.1 Deductive ProofsA deductive proof leads us from a hypothesis H to a conclusion C. Each step in the proof must follow, by some accepted logical principle, from either the given facts, or some of the previous statements in the deductive proof, or a combination of these.

The hypothesis may be true or false, typically depending on the values of its parameters. Often, the hypothesis consists of several independent statements connected by logical AND. In those cases, we talk of each of these statements as a hypothesis, or as a given statement. The theorem that is proved when we go from a hypothesis H to a conclusion C is the statement ‘if H then C’. We say that C is deduced from H. An example theorem of the form ‘if H then C’ will illustrate these points.

5

If,

Here, H is x>=4 and C is

Intuitively, it should not be difficult to convince yourself that this statement is true. Each time x increases by one, the left hand side doubles in size. However, the right side increases by the ratio ((x+1)/x)2. When x=4, this ratio is 1.56. As x increases and approaches infinity, the ratio ((x+1)/x)2 approaches 1. This means the ratio gets smaller as x increases. Consequently, if 1.56 is the largest then, the right hand side will increase. Since 1.56 < 2, the left side is increasing faster than the right side, therefore, the conclusion must be true as long as we start with a value like x=4 where the inequality is already satisfied.

Basic Formal LogicFirst, let’s look at some more deductive proofs using formal logic. An ‘If H, then C’ statement is typically expressed as:

or H implies C

The logic truth table for implication is :

Sometimes we will have the ‘If and only if statements’, e.g. ‘If and only if H, then C’ means that andSometimes this will be written as H⇔C or ‘H if C’. The truth table is:

Modus PonensGoing back to implication, let’s say that we are given . Further, suppose we are given that H is true. Then by logic of the truth table, we know that C must be true. Therefore, we can conclude C. This type of reasoning is called modus ponens (Latin for ‘method of affirming’) and can be used to form chains of logic to reach a desired conclusion. In other words, given

and H

Then we can infer CFor example, given: ‘If X and Y are siblings, then X and Y are related’ as a true assertion, and also given ‘X and Y


6

are siblings’ as a true assertion, then we can conclude that X and Y are related.

Modus TollensA slightly trickier but equally true form of reasoning is Modus Tollens (Latin for method of denying. This reason backwards across the implication. Cognitive psychologists have shown that 60% of college students have a solid intuitive understanding of Modus Tollens versus almost 100% for modus ponens.

Modus Tollens states the following. If we are given:

and ¬C

Then we can infer ¬H.For example, given: ‘If X and Y are siblings, then X and Y are related’ as a true assertion, and also given ‘X and Y are not related’ as a true assertion, then we can conclude ‘X and Y are not siblings.’

1.5.2 Inductive ProofsInductive proofs are essential when dealing with recursively defined objects. We can perform induction on integers, automata and concepts like trees or graphs.To make an inductive proof about a statement S(X) we need to prove two things:

Basis: Prove for one or several small values of X directly• Inductive step: Assume S(Y) for Y ‘smaller than’ X; then prove S(X) using that assumption•

Here is an example with integers that you may have seen before in a math class:Theorem: For all n≥0:

First, prove the basis. We pick n=0. When n=0, there is a general principle that when the upper limit (0) of a sum is less than the lower limit (1) then, the sum is over, no terms and therefore, the sum is 0.

That is

Next prove the induction. Assume n ≥ 0. We must prove that the theorem implies the same formula when n is larger. For integers, we will use n+1 as the next largest value. This means that the formula should hold with n +1 substituted for n.

7

How does this help us? Eq (2) should equal what we came up with for Eq (1) if we just add on an extra n+1 term

Expanding the right side of the equation -

This is the same as what we computed from the inductive step in (ii). Therefore, the theorem is true.

Let’s prove the example we looked at earlier with deduction:

Theorem: If

Basis: If x=4, then 2x is 16 and x2 is 16. Thus, the theorem holds -

Induction: Suppose for some x >=4 that 2x >= x2. With this statement as the hypothesis, we need to prove the same statement, with x+1 in place of x:

This is S(x+1)

We should rewrite it to make use of S(x). In this case,

2(x+1) =2*2x

And can conclude that,

2(x+1) =2*2x>= 2x2 (2)


8

By substituting (2) into (1) we need to show,

2x2 >=(x+1)2

2x2 >=x2+2x+1X2>=2x+1

Dividing both sides by x yields,

x>=2+1/x

Since x >=4, we get some value >=4 on the left side. The right side will equal at most 2.25 and in fact gets smaller and approaches 2 as x increases. Consequently, we have proven the theorem to be true by induction.

9

SummaryAutomata theory is the study of abstract computing devices or machines.• Finite automata are the simplest kind of automata; they are widely used to model and analyze finite-state state • systems.Pushdown automata add to finite automata a restricted kind of unbounded memory in the form of a stack.• Turing machines add to finite automata a more powerful notion of memory in the form of an unbounded tape• In traditional automata theory, automata are treated as language acceptors.• Grammars are useful models when designing software that processes data with a recursive structure.• Regular expressions also denote the structure of data, especially text strings.• Complexity is the study of the limits of computation.• A proof is a convincing logical argument that a statement is true.• A deductive proof leads us from a hypothesis H to a conclusion C given some statements.• Inductive proofs are essential when dealing with recursively defined objects.•

ReferencesAutomata: The Methods and the Madness. Available at< http://www.math.uaa.alaska.edu/~afkjm/cs351/handouts/• hop-chap1.pdf > [Accessed 16th, March 2011]Hopcroft. • Introduction to Automata Theory, Languages, and Computation, 3/E. Pearson Education India, 2008. pp 1-10.Michael Sipser. Introduction to the Theory of Computation. Available at • <http://n.ethz.ch/~ddukaric/zusammenfassungen/Buecher/Introduction%20to%20the%20Theory%20of%20• Computation.pdf> [Accessed 16th March, 2011].

Recommended ReadingDexter C. Kozen.• Theory of Computation. Springer; 1st ed. (December 15, 2010). 426 pages.Jeffrey E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman. • Introduction to Automata Theory, Languages, and Computation. Addison Wesley; 3rd ed. (July 15, 2006). 535 pages.Michael Sipser. • Introduction to the Theory of Computation. Course Technology; 2nd ed. (February 15, 2005). 456 pages.


10

Self Assessment

___________ theory is the study of abstract computing devices or machines.1. Automataa. Turningb. Hypothesisc. Modulard.

___________ automata are the simplest kind of automata; they are widely used to model and analyze finite-2. state systems.

Infinitea. Finiteb. Direct c. Indirectd.

___________ automata add to finite automata a restricted kind of unbounded memory in the form of a stack.3. Directa. Indirectb. Pushdownc. Pushupd.

___________ machines add to finite automata a more powerful notion of memory in the form of an unbounded 4. tape.

Directa. Indirectb. Pushdownc. Turingd.

State which of the following is true.5. Grammars are useful models when designing software that processes data with a recursive structure.a. Grammars are useful models when designing software that processes data with a non-recursive structure.b. Grammars are useful models when designing hardware that processes data with a recursive structure.c. Grammars are useful models when designing software that processes data with a similar structure.d.

State which of the following is true.6. The style of regular expressions are similar to that of grammars.a. The style of regular expressions differs significantly from that of grammars.b. The style of regular expressions differs significantly from that of irregular expressions.c. The style of regular expressions differs significantly from that of automata.d.

State which of the following is true7. The problems that can be solved by a computer in a realistic amount of time are called decibels.a. The problems that can be solved by a computer in a realistic amount of memory are called decidable.b. The problems that can be solved by a computer in a realistic amount of time are called decidable.c. The problems that can be solved by a computer in a realistic amount of time are called intractability.d.

11

Which of the following is a convincing logical argument that a statement is true?8. Verificationa. Testimonyb. Recordc. Proofd.

The hypothesis may be true or false, typically depending on which of the following?9. Values of its parametersa. Size if its parametersb. Number of its parametersc. Value of proofsd.

Which of the following proofs are essential when dealing with recursively defined objects?10. Deductivea. Inductiveb. Informativec. Deducibled.


12

Chapter II

Finite Automata and Regular Languages

Aim


understand the concept of finite automata (FA)•

analyse regular languages•

study conversion of regular expression to finite automata (FA)•

Objectives


examine non-deterministic finite automata (NFA)•

determine deterministic finite automata (DFA)•

determine finite automata with ε-transition•

Learning outcome


recall the equivalence of • NFA’s and DFA’s

explain finite automata and its types•

recall difference between DFA’s and NFA’s•

13

2.1 IntroductionThis chapter involves practical experimentation – specifically in regard to the time-efficiency implications of hard coding a finite automaton. However, by way of introduction, it is appropriate to locate the theoretical setting of the broad problem domain. Hence, this chapter introduces theoretical aspect of FAs and their relationship to various well-known computational problems that depend on various other kinds of automata as their primary model for the solution. We formally define FAs and explore two major characteristics of such devices, namely determinism and non-determinism. This helps to clarify the kinds of automata related to hard coding algorithms that shall be encountered in the remainder of the dissertation..

2.2 Finite AutomataFrom a practical point of view, a finite automaton is considered to be a computing device used to examine strings over an alphabet. The study of the complexity of such a device is of importance due to its wide area of applicability. The goal of its implementation is to accept or reject an input string. It is, therefore, of importance to consider how much memory the device uses for the processing of a given string, as well as its time complexity or processing speed.

An example of FA is shown in Fig. 2.1, the finite automaton has a transition function ‘δ’, where for each state ‘si’, and for zero or more characters ‘c’ in the device’s alphabet, the operation δ (si, c) maps to some state that depends on the value of c. Once it has read a character, the FA makes a transition δ (si, c) to some new state. The automaton halts when there are no more character to be read from the input stream, or when δ (si, c) in not defined.

When setting up or constructing an FA, a set of its states are designated as final states. The device is said to recognize a string if and only if there is a sequence of transitions of the device that leads to one such final state after reading the last character of the string. Otherwise, the device does not recognize the string. The characters read by the FA from the input stream are part of its alphabet. The set of all strings that an FA accepts is said to be the language recognized by the FA. An FA may be modeled as a transition graph as shown in Fig. 2.2, as long as its grammar is well defined.

The grammar of a finite automaton is a formal description of the structure of all finite and allowable sequences of symbols that can be obtained using the alphabet. The transition function is part of such a formal description.

Fig. 2.1 A finite automata


14

Fig. 2.2 A state transition diagram

2.3 Deterministic Finite Automaton (DFA)A deterministic finite automaton M is a quintuple (S, A,δ, s1, F)

Where:S = finite set of statesA = alphabets1 ∈ S is the initial stateF ⊆ S is the set of final statesThe transition function δ: S x A→S, is defined such that ∀s ∈ S,∀c ∈ A, δ(s, c) ∈ S is uniquely determined.

2.4 Complexity of DFA String RecognitionThe complexity of the computation of a string recognition operation by a deterministic finite automaton is measured in terms of complexity of each transition to be made (if any) for a given character in the string.

Assume that a string str is defined by str≡ c1c2…cn−1cn, and assume that M, is a DFA as defined above.

We denote by XM (str) the complexity of recognizing the string str.

Thus,

XM (str) = XM (c1c2…cn−1cn) = XM (c1) + XM(c2)+…+ XM(cn-1)+ XM(cn)

=

Where XM(ci) denotes the complexity of recognizing the single character ci. If we assume that the complexity of recognizing a single character can be characterized by some constant value, say K, irrespective of the state or the character, and then, both the worst case complexity and the average case complexity of recognizing a string may be regarded as linearly dependent on n, the length of the string – i.e. the complexity is K × n. However, in practical implementations (as will be seen later) the time taken to implement the operation δ (sj, ci) is a statistical variable.

The transition function δ of a deterministic finite automaton is said to be total if and only if,

∀si ∈ S, ∀c ∈ A, ∃sj ∈ S: δ(sj, c)= sj (sj is unique)

Then, if a string str is recognized by the automaton M, it follows that δ(si, cj) = sk, with i, k=1,…, m and j=1, 2…, n; where m is the total number of states, and n the numbers of characters in str.

The transition function of a deterministic finite automaton is said to be partial, if and only if,∃si ∈ S, ∃c ∈ A: δ(s, c) does not exist.

If a string str is recognized by M, the complexity of each ci exists and is totally determined.

S1 S2 S3 0

11

1

00

15

That is, .

If a string str is not recognized by M, then;

∃i, 1≤ i≤ :δ(si, cn)=sk ∈ S−F, δ(si, cj) does not exist.

In this second case, cj is the first character of str, that does not allow a transition at the state sk. This means that, for the present purposes, it may be assumed that XM (cl) = 0, j < l ≤ n. In particular, for i = 1, if δ(s1, c1) does not exists, then XM (str) = XM (c1).

2.5 Non-deterministic Finite Automata (NFA)A finite automaton is said to be non-deterministic, when the transition operation on a given state and a symbol returns, a set of states as result. A non-deterministic finite automaton (NFA) M is a quintuple (S, A,∆, s1, F)

Where,

S = finite set of statesA = alphabet,s1 ∈ S is the initial state,F ⊆ S is the set of final states, and∆ ⊆ S × A × S is the transition relation. Thus, (si, u, sj) ∈ ∆ means that when in state si and presented with the input character u the automaton M may transit to the state sj.

However, if it is also true that (si, u, sk) ∈ ∆, then when in state ‘s’i and presented with the input character ‘u’ the automaton M may transit to the state ‘sk’. The particular state that is selected cannot be predetermined. It is, in this sense, that the NFA is non-deterministic.

There are various practical reasons for using non-deterministic models. In some cases, a non-deterministic model reflects the underlying random nature of that solution, which is being modeled. In other cases, non-determinism is used in a specification context. The non-deterministic model concisely specifies several possible alternative options to be implemented, all alternatives being acceptable as implementations of some system. The implementer is then free to select any one of the nondeterministic possibilities that have been specified. But, DFAs and NFAs are equivalent (in their power).

2.6 Difference between DFA and NFADeterministic Finite Automata (DFA) Non deterministic Finite Automata (NFA)

All transitions are deterministic. Each transition leads to exactly one state.

All transitions are non-deterministic. Each transition leads to a subset of states.

For each state, transition on all possible symbols (alphabet) should be defined.

For each state, not all symbols necessarily have to be defined in the transition function.

Accepts input if the last state is in. Accepts input if one of the last states is in F.

Sometimes harder to construct because of the number of states. Generally easier than a DFA to construct.

Practical implementation is feasible. Practical implementation has to be deterministic (so needs conversion to DFA).

Table 2.1 Difference between DFA and NFA


16

2.7 Equivalence of NFAs and DFAsWe now show that DFAs and NFAs accept exactly the same set of languages. That is, nondeterminism does not make a finite automaton any more powerful. To show that NFAs and DFAs accept the same class of languages, we show two things:

Any language accepted by a DFA, can also be accepted by some NFA.• Any language accepted by a NFA, can also be accepted by some DFA.•

2.7.1 Proof Strategy

To show that any language accepted by a NFA is also accepted by some DFA.• We describe an algorithm that takes any NFA and converts it into a DFA that accepts the same language.• The algorithm is called the ’subset construction algorithm’.• We can use mathematical induction (on the length of a string accepted by the automaton) to prove that the DFA • that is constructed accepts the same language as the NFA.

2.7.2 Subset Construction Algorithm

Given a NFA, it constructs a DFA that accepts the same language.• The equivalent DFA simulates the NFA by keeping track of the possible states it could be in. Each state of the • DFA corresponds to a subset of the set of states of the NFA - hence, the name of the algorithm.If the NFA has ‘n’ states, the DFA can have as many as 2n states, although it usually has many less.•

Steps of Subset construction algorithmThe initial state of the DFA is the set of all states the NFA can be in without reading any input.• For any state {q• i, qj,…, qk} of the DFA and any input ‘a’, the next state of the DFA is the set of all states of the NFA that can result as next, states that if the NFA is in any of the states qi, qj,…, qk when it reads ‘a’. This includes states that can be reached by reading a followed by any number of λ-transitions. Use this rule to keep adding new states and transitions until it is no longer possible to do so.The accepting states of the DFA are those states that contain an accepting state of the NFA.•

ExampleHere is a NFA that we want to convert to an equivalent DFA -

Step 1The start state of the DFA is the set of states the NFA can be in before reading any input. This includes the start state of the NFA and any states that can be reached by an e-transition

Step 2For state {0, 1}, we create a transition for each possible input, a and b. In the process, we create state {2}.

a

λ

a

ab

b

b

b

0 1

2

{0,1}

17

Step 3For state {2}, we create a transition for each possible input, a and b. In the process, we create another state, {1, 2}.

Step 4For state {1, 2}, we create a transition for each possible input, a and b. At this point, a transition is defined for every state-input pair.

Step 5The last step is to mark the final states of the DFA

2.8 Finite Automata with ε-transition (Epsilon-transitions)We can allow e-transitions in finite automata i.e., a state can jump to another state without consuming any input symbol.Use:

Makes it easier sometimes for NFA construction• E -NFAs have to one more column in their transition table•

2.9 Regular Expressions and LanguagesRegular expressions are notations for regular languages. Regular languages are set of words, or strings, over an alphabet of characters valid for the language.

λ is the null string (there are no symbols in this string).•

{0,1}

{2}

b

a

{0,1}

{2}

{1,2}b

b

a

a

{0,1}

{2}

{1,2}b

b

a

a

b

a

{0,1}

{2}

{1,2}b

b

a

a

b

a


18

is the set of all strings of length greater than or equal to 0.•

Example:A = {a, b} // the alphabet is composed of a and ba* = {λ, a, b, aa, ab, ba, bb, aaa, aab ,…}The symbol * is called the Kleene star.∅ (empty set)λ (empty string)( ) delimiter∪ + union (selection)

ConcatenationThe order to describe languages over an alphabet A we introduces a new alphabet A, called a meta-alphabet, that has symbol that correspond to each symbol of A as well as other symbols used to form regular expressions. Namely, if A = {a0,…, an-1}, then;

A = {a0,…, an-1, (), A,Ø,∪, *}

DefinitionThe set of regular expressions over an alphabet A is the set of all words over the meta-alphabet A defined inductively as follows:

a• 0,…, an-1 are regular expressionsA is a regular expression• Ø is a regular expression•

Inductive stepsIf E• 0, E1 are regular expressions, then (E0 ∪ E1) is a regular expression.If E• 0, E1 are regular expressions, then (E0⋅E1) is a regular expression.If e is a regular expression, then (E*) is a regular expression.•

Regular expressions have an important role in computer science applications. In applications involving text, user may want to search for strings that satisfy certain patterns. Regular expressions provide a powerful method for describing such patterns.

2.10 Regular LanguagesA language is said to be regular if it can be marked by a finite-state automation, the class of regular languages is denoted by R. The class R is very important since it delimits the language that possesses automation representations that require finite memory when stored in a computer. In the other words, automata are a practical means of manipulating regular languages in analysis or controller synthesis problems. On the other hand, automata are not a practical means for representing non- regular languages, since they would require infinite memory.

2.10.1 Properties of Regular Languages The following theorem about the properties of the class of regular languages, along with the proof that we present, illustrates how well-behaved the class R is and how the duality between regular languages and finite-state automata, be they deterministic or non- deterministic, can be exploited.

2.10.2 Theorem (Properties of R)Let L1 and L2 be in R. then the following languages are also in R.

1. 2.

19

3. 4. 5. 6.

Proof: Let G1 and G2 be two finite-state automata that mark L1 and L2 respectively. We prove the first three properties by modifying G1 in order to obtain finite-state automata that mark , and respectively. We prove that the three properties by building from G1 and G2, a third finite-state automaton that marls the language obtained after the corresponding operation. Note that in all cases, it does not matter if a finite-state automaton is non- deterministic, as we know we can always build its deterministic and finite-state.

Take the trim of G• 1 and then mark all of its states.Kleene-closure• : add a new initial state, mark it and connect it to the old initial state of G1 by a ε-transition. Then add ε-transitions from every marked state of G1 to the old initial state. The new finite-state automaton marks .Union• : create a new initial state and connect it, with two ε-transition, to the initial states G1 and G2.The result is non- deterministic automation that marks the union of L1 and L2.Concatenation• : Connect the marked states of G1 to the initial state of G2 by ε-transitions. Unmark all the states of G1. The resulting non- deterministic automaton marks the language L1L2.

2.11 Conversion of RE to FAHere we see a method of converting regular expressions r into automata recognizing L(r). Here is the main result of this section:

Theorem 1. Every regular language is FA recognizable.Let r be a regular expression. We want to prove that L(r) is FA recognizable. In our proof, we just follow the inductive definition given for the language L(r) from the previous section and prove that L(r) is FA recognizable. Our proof uses NFA with silent moves.

Base caseAssume that r is defined in the base case. Then L(r) is either {aσ} for some σ ∈ Σ or ∅ or {λ}. Clearly in each case L(r) is FA recognizable.

Inductive stepIn this step there are three cases to consider.

Case 1: Assume that r is of the form (r1 +r2). We now use the inductive hypothesis applied to r1 and r2. By the hypothesis, the languages L (r1) and L (r2) are FA recognizable (and hence DFA recognizable). The language L (r1 +r2) is by definition L (r1) ∪ L (r2). The union of FA recognizable languages is again FA recognizable. Hence L(r) is FA recognizable.

Case 2: Assume that r is of the form (r1⋅r2). By the inductive hypothesis, the languages L (r1) and L (r2) are FA recognizable. The language L (r1⋅r2) is by definition L (r1) ⋅L (r2). We want to show that L (r1) ⋅L (r2) is FA recognizable.Let A1 = (S1, I1, T1, F1) and A2 = (S2, I2, T2, F2) be finite automata recognizing L (r1) and L (r2), respectively. Both automata are NFA that may have silent moves. We may assume that the state sets S1 and S2 have no states in common. Using these two automata we want to build a finite automaton recognizing L (r1) ⋅L (r2).We now use our method of pretending to be a machine to accept strings in L (r1) ⋅L (r2).Suppose w is an input string. We first pretend that we are the machine A1 and run A1 on w. If we never reach a final state of A1 then we reject w. Assume, that we have reached a final state after processing some prefix of ‘w’. In this case we need to non-deterministically choose one of the following actions:


20

Continue on running A• 1 on w.Make a silent transition to the initial state of A• 2 and start simulating A2.

This observation suggests that we can put the automata A1 and A2 together as follows. We keep the states and transition tables of both machines. We declare the initial states of A1 to be the initial states, and the final states of A2 to be the final states of our machine. Finally, we add the λ-transitions from the final states of A1 to the initial states of A2. Based on this description, we formally define the automaton A = (S, I, T, F) recognizing L (r1) ⋅L (r2) as follows:

S=S• 1∪S2

I=I• 1

On the input(s, • σ), where σ∈Σ ∪ {λ}, the transition function T is defined according to the following rules.If s � ∈ S1 and s is not a final state then T (s, σ) = T1(s, σ).If s � ∈ S1, s is a final state, and σ ≠ λ then T (s, σ) = T1(s, σ).If s � ∈ S1, s is a final state, and σ = λ then T (s, σ) = T1(s, σ) ∪ I2.If s � ∈ S2 then T (s, σ) = T2(s, σ).

F=F• 2It is not hard to check that A recognizes L (r1) ⋅L(r2)

Case 3: Assume that r is of the form (r1) ⋆. By the inductive hypothesis, the language L (r1) is FA recognizable. The language L (r⋆1) is by definition L (r1) ⋆. We want to show that L (r1)

⋆ is FA recognizable.

Let A1 = (S1, q1, T1, F1) be a finite automaton recognizing L (r1). We want to construct a finite automaton that recognizes the language L (r1)

⋆. As above, we use our method of pretending to be a machine that accepts strings in L (r1)

⋆. For an input string w, if it is the empty string then we accept it because the star of every language contains λ. Otherwise we simulate A1 and read w.

Every time we reach a final state of A1 we have to make a nondeterministic choice. We either continue on running A1 or make a silent move to one of the initial states of A1. Thus, we construct our automaton A recognizing L (r1)

⋆ as follows. We keep all the states of A1; we keep all the transitions of A1; we add λ-transitions from each final state of A1 to every initial state of A1; we add one new initial state qnew with no outgoing transitions and declare it to be a final state (this is needed to accept the empty string). Here is a more formal definition of the automaton A = (S, I, T, F) recognizing L (r1)

⋆:

S=S• 1 ∪ {qnew}, where qnew is a state L (r1)⋆:

I=I• 1 ∪ {qnew}On input (s, σ), where σ • ∈ Σ ∪ {λ}, the transition function T is defined according to the following rules.

If s � ∈ S1 and s is not a final state then T (s, σ) = T1(s, σ).If s � ∈ S1, s is a final state, and σ ≠ λ then T (s, σ) = T1(s, σ).If s � ∈ S1, s is a final state, and σ = λ then T (s, σ) = T1(s, σ) ∪ I1.If s = q � new then T (s, σ) = ∅.

F = F• 1 ∪ {qnew}.

Now it is not hard to check that A recognizes L (r1) ⋆. We have proved the theorem.

21

SummaryFrom a practical point of view, a finite automaton is considered to be a computing device used to examine • strings over an alphabet.When setting up or constructing an FA, a set of its states are designated as final states.• The characters read by the FA from the input stream are part of its alphabet.• The grammar of a finite automaton is a formal description of the structure of all finite and allowable sequences • of symbols that can be obtained using the alphabet.The complexity of the computation of a string recognition operation by a deterministic finite automaton is • measured in terms of complexity of each transition to be made (if any) for a given character in the string.A finite automaton is said to be non-deterministic when the transition operation on a given state and a symbol • returns a set of states as result.In some cases, a non-deterministic model reflects the underlying random nature of that solution which is being • modeled.Any language accepted by a DFA can also be accepted by some NFA.• Any language accepted by a NFA can also be accepted by some DFA.• A language is said to be regular if it can be marked by a finite-state automation the class of regular languages • is denoted by R.

ReferencesDan A. Simovici, Richard L. Tenney. • Theory of formal languages with applications. World Scientific, 1999. pp 147-148.Christos G. Cassandras, Stéphane Lafortune. • Introduction to discrete event systems. Springer, 2008.pp 94-95.Bakhadyr Khoussainov. Regular expressions and finite automata. Available at <http://www.cs.auckland.• ac.nz/~bmk/se211lectures/lecture14.pdf> [Accessed 20th March, 2011].

Recommended ReadingArnold L. Rosenberg. • The Pillars of Computation Theory: State, Encoding, Nondeterminism. Springer; 1st ed. (November 11, 2009). 326 pages.P.G. Spirakis. • Automata, Languages and Programming. Springer; 1st ed. (August 9, 2001). 1083 pages.Howard Straubing. • Finite Automata, Formal Logic, and Circuit Complexity. Birkhäuser Boston; 1st ed.. (May 3, 1994).226 pages.


22

Self AssessmentA language is said to be _________________ if it can be marked by a finite-state automation the class of regular 1. languages is denoted by R.

regulara. irregularb. customaryc. symmetricald.

A finite automaton is said to be _________________ when the transition operation on a given state and a symbol 2. returns a set of states as result.

deterministica. non-deterministicb. symmetricalc. non-symmetricald.

In some cases, a 3. _________________ model reflects the underlying random nature of that solution which is being modeled.

Symmetricala. Non-symmetricalb. non-deterministicc. deterministicd.

When setting up or constructing an FA, a set of its states are designated as _________________.4. initial statesa. machine stateb. state machinec. final statesd.

The grammar of a finite automaton is a _________________ description of the structure of all finite and allowable 5. sequences of symbols that can be obtained using the alphabet.

formala. informalb. officialc. structuredd.

State which of the following statement is true.6. Any language accepted by a FA can also be accepted by some NFA.a. Any language accepted by a DFA can also be accepted by some NFA.b. Any language accepted by a RE can also be accepted by some NFA.c. Any language accepted by a RE can also be accepted by some FA.d.

State which of the following statement is true.7. Any language accepted by a FA can also be accepted by some DFA.a. Any language accepted by a NFA can also be accepted by some FA.b. Any language accepted by a NFA can also be accepted by some DFA.c. Any language accepted by a FA can also be accepted by some RE.d.

23

State which of the following statement is true.8. The characters read by the NFA from the input stream are part of its alphabet.a. The characters read by the DFA from the input stream are part of its alphabet.b. The characters read by the FA from the input stream are part of its expressions.c. The characters read by the FA from the input stream are part of its alphabet.d.

In which of the following does each transition leads to exactly one state?9. DFAa. NFAb. FAAc. REd.

In which of the following does each transition lead to a subset of states?10. DFAa. NFAb. FAAc. REd.


24

Chapter III

Push Down Automata and Context Free Languages

Aim


portray context free languages •

identify context free grammars•

analyse push down automata•

Objectives


examine ambiguity in configuration (CFG) and its removal•

determine Chomsky’s normal form•

outline the definition and language of push down automaton•

Learning outcome

At the end of this chapter, students will be able to:

recall context free languages and grammar•

explain pushdown automata •

recognize non-context free languages •

25

3.1 IntroductionIn this chapter, we will introduce you to context free languages and context free grammar, a more powerful method, than finite automata and regular expressions of describing languages. Such grammar can describe certain features that have a recursive structure, which makes them useful in a variety of applications.

3.2 Context Free LanguagesSuperficially thinking, one might view modern computers as deterministic finite automata. Computers have finite memory, so there are only finitely many states the computer can be at, and the previous state of the computer determines the next state, so the machine has deterministic state transitions. This approach is, however, not reasonable. First of all, the number of states is indeed finite but that finite number is astronomical. And even worse, the memory of the computer is extendible, so the approach is not even theoretically correct. A more accurate mathematical model is needed.

Our next step into the direction of ’correcting’ the notion of computation is to overcome the limitations of finite memory by introducing new models that allow arbitrarily large memory. The memory can be organized and accessed in different ways. Our next model, context-free languages, assumes an infinite memory that is organized as a stack (=last-in-first-out memory). As we are going to see, this does not yet capture the full power of computation, but it provides a natural generalization of regular languages. Every regular language is a context-free language, but also, some non-regular languages are included. For example, {anbn | n≥0} is a context-free language. On the other hand, there are many simple languages which are not context-free, for example {anbncn | n≥0}.

Context-free languages have applications in compiler design (parsers). The syntax of programming languages is often given in the form of context-free grammar, or equivalent Backus-Naur form (BN-form).

Grammar consists of a collection of substitution rules, also called productions. Each rule appears as a line in the grammar and comprises a symbol and a string, separated by an arrow. The symbol is called a variable. The string consists of variables and other symbols called terminals.

3.3 Context Free GrammarsSimilar to regular languages, we have different ways of representing context-free languages. Let us first consider generative devices, so-called context-free grammars.

Let us begin with an example of a context-free grammar. It uses variables A and B, and terminal symbols a and b. We have following productions;A→AbAA→BB→aBaB→b

In a word containing variables, and possibly terminals, one can replace any variable with the word on the right hand side of a production for that variable. For example, in the word:

aBabAbA

One can, for example, replace the first occurrence of variable A by AbA, obtaining the word:aBab AbA bA

Then one can decide, for example, to replace the variable B by b, which gives the word:ab abAbAbA

Rewriting one variable is called a derivation step and we denote: aBabAbA⇒aBabAbAbA⇒ababAbAbA


26

Derivation is non-deterministic: usually one has many choices of how to proceed. We can continue the derivation as long as there exists variables in the word. Once the word contains only terminal symbols the derivation terminates. One variable is called the start symbol. In the case of this example, let A be the start symbol. All derivations start with the start symbol A, i.e., initially the word is A. The language defined by the grammar contains all strings of terminal symbols that can be obtained from the start symbol by applying the productions on the variables. For example, word aabaababa is in the language defined by our grammar since it is derived by; A⇒AbA⇒BbA⇒aBabA⇒aaBaabA⇒aabaabA⇒aabaabB⇒aabaabaBa⇒aabaababa

On the other hand, word aaa is not in the language because the only way to produce letter a is to use the production B → aBa, and it creates two a's, so the number of a's has to be even.Here is a precise definition of context-free grammars. A context-free grammar G = (V; T; P; S) consists of:

two disjoint finite alphabets, V and T, containing variables (=nonterminals) and terminals, respectively,• a finite set P of productions of the form A• →α

Where A ∈ V is a variable, and α ∈ (V ∪ T)* is a word of terminals and variables.A start symbol S• ∈ V.

For any α, β ∈ (T∪ V)*, we denote; a⇒β

If β is obtained from α by rewriting one variable in α using some production from P, that is, if α= uAv and β = uwv for some u, v, w ∈ (V ∪ T)* and A ∈ V, and A → w is in P. This is called a derivation step. The reflective, transitive closure of ⇒ is denoted by ⇒*, α⇒* β if there is a sequence of derivation steps,

α=α0⇒α1⇒α2⇒αn=β

That starts with α and leads to β. The number n of derivation steps can be zero, so α⇒* α always. We say that α derives β. If we want to emphasize that a derivation takes n steps, we may use the notation ⇒n.

A word α∈(V ∪ T)* is called a sentential form if it can be derived from the start symbol S, i.e., ifS⇒*α

The language L (G) generated by grammar G consists of all sentential forms that contain only terminals. In other words, L (G) = {w ∈ T*| S⇒* w}

A language is called a context-free language if it is L (G) for some context-free grammar G.

Example: Consider the grammar G = (V; T; P; S) where V = {S}, T = {a; b}, and P contains the productionsS→ aSb

S→ε

To make notations shorter we may use the following convention: productions of the same variable may be combined on one line, separated by symbol |. So we may write;

S→aSb|εWe easily see that L (G) = {anbn| n≥0}.

Let us determine L (G). Clearly every word that belongs to L (G) must contain equally many a’s and b’s. (Every production adds the same number of a’s and b’s to the sentential form.)

Let us now prove that L (G) contains all words w ∈ {a, b}* with equally many a's and b's. LetL = {w ∈ {a, b}*| w has equally many a's and b's}

We observe first that any non-empty word w ∈ L can be written as w = a u b v or w = b u a v

for some u, v ∈ L. Let us prove this fact. Assume w ∈ L and w starts with letter a (if it start with letter b the proof

27

is symmetric). Let au be the longest prefix of w with more a's than b's. The next letter coming after au has to be b because aua has more a's than b's and is longer than au. So w starts a u b…. In the word au the number of a's is exactly one greater than the number of b's: If the difference would be more than one then also prefix a u b would contain more a's than b's, and it is longer than au. So u has equally many a's and b's. The suffix v that follows a u b has also equally many a's and b's. So w = a u b v and u; v ∈ L.

Now we are ready to prove that every word w ∈ L is in L (G). We use induction on the length of w.If |w| = 0 then w=ε• ∈L (G) because S→ε is in P.Assume |w|>0, and we know that every shorter word with equally many a’s and b’s can be derived from S. We • demonstrated above that;

W=a u b v or w=b u a v

Where u and v are words that contain equally many a's and b's. According to the inductive hypothesis:S⇒*u and S⇒*v

If, w = a u b v then we can derive w as follows:S ⇒ a S b S ⇒* a u b v

The case, w = b u a v is similar. In either case, we have a derivation for w so w ∈ L (G).

For example, ababba has the following derivation:

S ⇒ aSbS ⇒ abSaSbS ⇒ abaSbS ⇒ ababS ⇒ ababbSaS ⇒ ababbaS ⇒ ababba

Let us prove next that every regular language can be generated by a context-free grammar.

Example: Let L be the regular language recognized by the NFA A

Let us construct a grammar G that generates the same language L. The variables of G are the states {Q; P} and the terminals are {a; b}. The start symbol is the initial state Q. Every transition of A is simulated by a production:

Q→bQQ→bPP→aQ

Whenever the automaton reads a letter and goes to state q = Q or P, the grammar produces the same letter and changes the variable to indicate the current state q.

Each sentential form consists of a terminal word w followed by one variable. Clearly, word wq is a sentential form if and only if the automaton can go to state q reading input w. For example, bbaQ is a sentential form because the machine can read bba ending up in state Q:

Q⇒ bQ⇒ bbP⇒ bbaQ

To terminate the simulation we add the transition


28

Q→ε

This is correct because Q is a final state: the generation is terminated if the corresponding computation by A is accepting. For example, when accepting the word bab, the automaton is in states Q, P, Q, Q, in this order. The corresponding derivation by the grammar is

Q⇒bP⇒baQ⇒babQ⇒babIt is easy to prove that L(G)= L.

A similar construction can be done on any NFA, proving the following theorem.

Theorem: Every regular language is context-free.Proof:Let L be a regular language and let A = (Q,Σ, δ, q0, F) be an NFA such that L = L(A). An equivalent grammar is G = (V; T; P; S)where,V=QT=ΣS=q0

and P contains a production q→ap for all q, p, ∈ Q and a∈Σ such that p ∈ δ(q; a), and a production q → ε for every q ∈ F. This grammar only derives sentential forms that contain at most one variable, and the variable must appear as the last symbol of the form. Moreover, using a straightforward induction on the length of w one sees that S ⇒*wq for w∈Σ* if and only if q ∈ δ (q0, w). Hence S⇒*w for w ∈ Σ* ,if and only if, w ∈ L(A).

A grammar is called right linear; if all productions are of the forms, A→wB and A→w

For strings of terminals w ∈T* and variables A, B ∈ V. In other words, the right hand side of a production may contain only one variable, and it has to be the last symbol.

The construction in the proof of above theorem always produces a right linear grammar, so every regular language is generated by a right linear grammar. The converse is also easily seen true: a language generated by a right linear grammar is always regular. This observation provides yet another equivalent way to define regular languages, in terms of right-linear grammars.

3.4 Ambiguity in CFG and Its RemovalGrammars G and G’ are called equivalent if L (G) = L (G). Certain types of productions are undesirable in a grammar, and we would like to find an equivalent grammar that does not contain such productions. In the following we want to remove productions of the forms:A→ε (‘ε-production’)A→B (’unit production’)

Where A and B are variables. We also want to remove symbols that are unnecessary in the sense m that they are not used in any terminating derivations. The simplification is done in the following order:

Remove ε -productions.• Remove unit productions.• Remove variables that do not derive any terminal strings.• Remove symbols that cannot be reached from the start symbol.•

29

All four steps are effective, i.e., doable by an algorithm. We use the following grammar as a running example:S→AC |aB| ADA→ε |ab| SB→Aa | ABC→Aaa | εD→EbDE→bb

Step 1: Remove ε-productionsIf ε • ∉ L (G), we can construct an equivalent grammar G’ that does not have any ε-productions. If ε • ∈ L(G), we can construct a grammar G' that does not have any ε-productions.L (G’) = L (G) \ {ε}.•

Let us call a variable X nullable if X ⇒* ε.Find all nullable variables using a marking procedure: First, mark nullable all variables X that have a production • X→ε. Then mark variables Y that have productions Y→α where α is a string of variables that have all been already marked nullable. Repeat this until no new variables can be marked. It is clear that all nullable variables are found. Note that ε∈L (G) if and only if the start symbol S is nullable. In the sample grammar above, variables A;C and S are nullable.Construct a new set P of productions as follows: • For each original production,

A→X1X2 …Xn

In P, where all Xi ∈ V ∪ T, we include in P’ all productions,A →α1α2 …αn

Where,α � i = Xi if Xi is not a nullable variableα � i = ε or αi = Xi if Xi is a nullable variableα � 1α2 …αn ≠ ε

If several Xi are nullable, all combinations of αi = ε and Xi are used. One original production can result in up to 2k new productions if the right-hand side contains k nullable variables. In our example, the new productions will be,

S→AC | C | A | aB | AD | DA→ ab | SB→ Aa | a | AB | BC→ AAa | Aa | aD→ EbDE→ bb

There are no ε-productions. The language generated by the new grammar is L (G) \ {ε}.The construction works because the empty string ε is directly "plugged in" the right hand side of the production, whenever there is a variable that can produce ". When we use that production during a derivation, we decide at that point whether the nullable variable on the right hand side will be used to produce an empty string, or a non-empty string. For example, an original derivation, S ⇒ AC ⇒ abC ⇒ ab

is replaced by the more direct derivation S ⇒ A ⇒ ab:Since C derived ε in the old derivation, we used S→A instead of S→AC.


30

Step 2: Remove unit productions. Assume we have removed all ε-productions from P. The right-hand side of every production in P is some non-empty word. Let us find an equivalent grammar that does not contain ε-productions or unit productions. The idea is similar to Step 1: We anticipate all possible unit derivations, and plug them in directly into the productions.

For every variable A, find all variables B, such that• A ⇒* B

All such variable-to-variable derivations A ⇒* B can be found using a reachability algorithm in the directed graph whose vertices are the variables and there is an edge from A to B if A→B is in P. Clearly, A ⇒* B if there is a path in the graph from A to B. In our example, after the simplification step 1, the unit productions are;S → C | A | DA → SB → B

So, we have the following letter-to-letter derivations:S⇒* S | C| A | DA⇒* A | S| C | DB⇒* BC⇒* CD⇒* DE⇒* E

For every pair of variables X and Y, X ≠ Y, such that X ⇒* Y and Y ⇒* X, we can remove one of the variables: They both derive exactly the same words. So we trim P by replacing Y by X everywhere, both left- and right-hand sides of all productions.

In our example we can remove A since it derives same words as S. This leaves us with a smaller number of variables:S → |SC | C | S | aB | SD | D | abB → Sa | a | SB | BC → SSa | Sa | aD→ EbDE→ bb

Now we construct P’ from the trimmed set P. For every non-unit production• Y→α

in P and every variable X such that X ⇒* Y , we include in P' the non-unit productionX→α.

We obtain an equivalent grammar whose derivations may be shorter: An original derivationβXϒ ⇒* βYϒ ⇒ βαϒ

that used unit productions X ⇒* Y is replaced by a single derivation stepβXϒ⇒ βαϒ

In our example we end up with following productions:S → |SC | aB | SD | ab | SSa | Sa | a | EbDB → Sa | a | SB C → SSa | Sa | aD → EbDE → bb

Step 3: Remove variables that do not derive any terminal strings.Such variables can be simply removed together with all productions containing the variable on either side of

31

the production. This does not affect the generated language since the variable is not used in any terminating derivation.

To find variables that do generate some terminal string, we apply similar marking procedure as with ε-productions: First we mark all variables X such that there is a production; X → w

in P, where w ∈ T* contains only terminals. Then we mark variables Y such that there is a production;Y → α

In P, where α contains only terminals, and variables that have already been marked. This is repeated until no more variables can be marked. The variables that have not been marked at the end can be removed.

In our sample grammar, variable D does not derive any terminal string and can be removed.We obtain the following grammar,S → SC | aB | SD | ab | SSa | Sa | a B → Sa | a | SB C → SSa | Sa | aE → bb

Step 4: Remove symbols (variables and terminals) that are not reachable from the start symbolS.We use a similar marking procedure: Initially mark the start symbol S. Then mark all symbols appearing on the right hand side of a production for a variable that has been already marked. Repeat until no new symbols can be marked. All symbols that have not been marked can not be reached from the initial symbol, and they can be removed. Note that it is important that step 3 is executed before step 4, not the other way around: Step 3 may introduce new unreachable symbols.

In our example, variable E is unreachable and can be removed. (But it became unreachable only after we removed D!) We end up with the grammar;S → SC | aB | ab | SSa | Sa | a B → Sa | a | SB C → SSa | Sa | a

After steps 1, 2, 3 and 4 we have found an equivalent (except the possible loss of the empty wordε) grammar that does not have ε-productions, unit productions, or useless symbols.If the empty word was in the original language and we want to include it in the new grammar we may introduce a non-recursive start symbol. That is, we add a new variable S', make it the start symbol, and add the productions S'→α; for every production S →α from the original start symbol S S'→ε

Where S is the old start symbol. The new grammar is equivalent to the original one, and it has only one ε-production that can be used only once.

3.5 Chomsky Normal FormIn many proofs it is useful to be able to assume that the productions in the grammar are of some restricted, simple form. Several such simplified types of grammars are known to be equivalent to general context-free grammars. They are called normal forms of grammars.

A grammar G is said to be in the Chomsky normal form (or CNF) if all productions are of the forms A → BC or A → aWhere A, B and C are variables, and a is a terminal. Right-hand-sides of productions consist of two variables, or one terminal.


32

Theorem: Every context-free language L without ε is generated by a grammar that is in the Chomsky normal form. An equivalent CNF grammar can be constructed effectivelyProof: Let G be a grammar that generates L. First we use the simplification steps 1, 2, 3 and 4 above to find an equivalent grammar G0 that does not contain ε-productions or unit productions. Because there are no unit productions, all productions with one symbol on the right hand side are terminal productions A→ a, i.e., they are already in the Chomsky normal form.We only need to work on productions;

A→ X1X2…Xn

Where n≥2, and Xi ∈ V ∪ T. For every terminal symbol ‘a’ that appears on the right hand side we introduce a new variable Va. If Xi = a this is replaced Xi by Va. After this is done for all terminals, the right hand side of the production contains only variables. For example, the production

S → cAbbSwill be replaced by the production

S →VcAVbVbSwhere Vb and Vc are new variables

Then, we add new terminal productionsVa → a

for all terminals a. The only word derivable from Va is a. Note that these productions are in the Chomsky normal form.

Clearly the new grammar is equivalent to the original one: Every application of the original production;S → cAbbS

is replaced by a sequence of derivation steps that first uses productionS → VcAVbVbS

and then applies productionsVc → c and Vb →b

to replace all occurrences of variables Vc and Vb by terminals c and b.

So far we have constructed an equivalent grammar whose productions are of formsA → B1B2 … Bn, and

A → aWhere n≥2, and A, B1, B2, …, Bn are variables, and a denotes a terminal symbol.

The only productions that are not in CNF are:A → B1B2 … Bn

Where n ≥3. For each such production we introduce n −2 new variables D1, D2, …Dn-2, and replace the production by the productions,A → B1D1D1 → B2D2D2 → B3D3…Dn−3 → Bn−2Dn−2Dn−2 → Bn−1Bn

One application of the original production gives the same result as applying the new productions sequentially, one after the other. The new grammar is clearly equivalent to the original one.

33

Another normal form is so-called Greibach normal form (or GNF). A grammar is in Greibach normal form if all productions are of the form A → aB1B2 BnWhere n ≥0 and A, B1, B2, …, Bn are variables, and a is a terminal symbol. In other words, all productions contain exactly one terminal symbol and it is the first symbol on the right hand side of the production.

3.6 Push Down AutomataA pushdown automaton (PDA) is a non-deterministic finite state automaton that has access to an infinite memory, organized as a stack. We’ll see that the family of languages recognized by PDA is exactly the family of context-free languages. A PDA consists of the following:

Stack: The stack is a string of symbols. The PDA has access only to the leftmost symbol of the stack. This is called the top of the stack. During one move of the PDA, the leftmost symbol may be removed (’popped’ from the stack) and new symbols may be added (“pushed”) on the top of the stack.

Input tape: Similar to finite automata; the input string is normally scanned one symbol at a time. But also ε-moves are possible.

Finite state control unit: The control unit is a non-deterministic finite state machine.

Transitions may depend on the next input symbol and the topmost stack symbol.

There are two types of moves: normal moves, and ε-moves.Normal moves: Depending on•

the current state of the control unit �the next input letter �the topmost symbol on the stack �

The PDA maychange the state �pop the topmost element from the stack �push new symbols to the stack �move to the next input symbol �

Spontaneous ε-moves don’t have (b) and (D), i.e., they are done without using the input tape.•

3.6.1 Formal Definition of a PDAFormally, a pushdown automaton M consists of

M = (Q, Σ, Γ, δ, q0, Z0, F)Where,

Q is a finite state set• Σ is the finite input alphabet• Γis the finite stack alphabet• q• 0 ∈ Q is the initial stateZ• 0 ∈ Γ is the start symbol of the stackF • ⊆ Q is the set of final accepting statesδ is the transition function from•

3.6.2 PDA Transition Functionδ = transition function, which takes the triple: δ (q, a, Z)Where,


34

q = state in Q• a = input symbol in • Σ

Z = stack symbol in • Γ

The output of δ is the finite set of pairs (p, 𝛾) where p is a new state and 𝛾 is a new string of stack symbols that replaces X at the top of the stack.

If • 𝛾 = ε then we pop the stackIf • 𝛾 = X the stack is unchangedIf • 𝛾 = YZ then X is replaced by Z and Y is pushed on the stackIf X = ε then we push on • 𝛾

Example 1As a first example, let us consider the following PDA that recognizes the language.L = {anbn | n≥1}. The PDA has two states, Sa and Sb, and the stack alphabet contains two symbols, A and Z0. In the beginning, the machine is in the initial state Sa, and the stack contains only one symbol Z0, called the start symbol of the stack. Possible moves are summarized in the following table:

An input word is accepted if the PDA can reach the empty stack after reading all input symbols.

Example 2Here is a formal description of the PDA that recognizes L = {0n1n | n ≥ 0}.

Q = {q• 1, q2, q3, q4}Σ• = {0, 1}Γ• = {0, Z0}F = {q• 1, q4}

35

And δ is described by the table below:

Input 0 1 ε

Top Stack 0 Z0 ε 0 Z0 ε 0 Z0 ε

q1 q1, Z0

q2 q2, 0 q3, ε

q3 q3, ε q4, ε

q4

An instantaneous description (ID) records the configuration of a PDA at any given time. It is a triplet (q, w, 𝛾)Where,

q is the state of the PDA• w is the remaining input, i.e. the suffix of the original input that has not been used yet• 𝛾• is the content of the stack

3.6.3 Graphical Format of PDAPushdown Automata uses the following format Input-Symbol, Top-of-Stack → String-to-replace-top-of-stack

Any of these may be empty,

Example 3Here is the graphical description of the PDA that accepts the language

L = {ww• R | w is in (0+1)*}Stays in state q• 0 when we are reading w and saving the input symbol. Every time we ‘guess’ that we have reached the end of w and are beginning wR by going to q1 on an epsilon-transition.In state q• 1 we pop off each 0 or 1 we encounter that matches the input. Any other input will ‘die’ for this branch of the PDA. If we ever reach the bottom of the stack, we go to an accepting state.


36

Requires, Z0 initially on the stack.

3.6.4 Moves of a PDA

To describe the process of taking a transition, we can adopt a notation similar to • δ like we used for DFA’s. In this case, we use the ’turnstile’ symbol ├ which is used as:

(q, aw, Xβ) ├ (p, w, αβ)

In other words, we took a transition such that we went from state q to p, we consumed input symbol a, and we • replaced the top of the stack X with some new string αWe can extend the move symbol to taking many moves, •

├* represents zero or more moves of the PDA.

3.6.5 Language of PDAThere are two ways to define the language of a PDA M = (Q, Σ, Γ, δ, q0, Z0; F)(L (M) ⊆ Σ*) because there are two notions of acceptance:

Acceptance by final stateL (M) = {w | (q0, w, Z0) ├*p (q, ∈, 𝛾) |^ q ∈ F}

That is the PDA accepts the word ‘w’ if there is any sequence of IDs starting from (q0, w, Z0), and leading to (q, ∈, 𝛾), where q ∈ F is one of the final states. Here, it doesn't play a role what the contents of the stack are at the end.

In this example, the PDA P0 would accept 0110 because (q0, 0110, #) ├* P0(q2, ε, ε) and q2 ∈ F. Hence we conclude 0110 2 ∈ L (P0).

On the other hand, since there is no successful sequence of IDs starting with (q0, 0011, #) we know that 0011 ∉ L (P0).

Acceptance by empty stackL (M) = {w | (q0, w, Z0) ├*p (q, ε, ε)}

That is the PDA accepts the word w if there is any sequence of IDs starting from (q0, w, Z0) and leading to (q, ε, ε), in this case the final state plays no role.

If we specify a PDA for acceptance by empty stack we will leave out the set of final states F and just use M = (Q, Σ, Γ, δ, q0, Z0).

Our example automaton P0 also works if we leave out F and use acceptance by empty stack.We can always turn a PDA which uses one acceptance method into one which uses the other. Hence, both acceptance criteria specify the same class of languages.

37

3.6.6 Equivalence of PDA and CFGA context-free grammar and pushdown automata are equivalent in power.

For every CFG, there is an equivalent PDAIt has been said many times that PDA’s are an abstract machine just powerful enough to recognize the languages generated by CFG’s. In order to prove this, one must show that for any CFG G there exists a PDA M

L (M) =L (G).

Furthermore, one must also show that for any PDA M there exists a CFG G

L (G) = L (M).

It is interesting to point out that PDA’s were formally introduced by Oettinger in 1961 and CFG’s by Chomsky in 1956. The equivalence between these two was first perceived by Chomsky in 1962 and Evey in 1963. In both directions you will see a constructive proof.

Theorem:If G = (V,Σ, S, P) is a CFG, there exists a PDA M = (Q, Σ, Γ, q0, Z0, δ, A) such that Lf (m) = L (G).

Proof:We define our PDA M as follows:

Q = {q0, q1, q2} - It is interesting to note that any CFG can be recognized (via final state) by a PDA with only three states.Γ = V ∪ Σ ∪ {Z0}, Z0 ∉ V ∪ Σ - Our stack symbols are the grammar variables, a stack start symbol and the input alphabet.

A = {q2} - Only need one accepting state for any CFG.

We must now define our PDA’s transition function δδ(q• 0, ε,Z0) = {(q1, SZ0)}For A • ∈ V , δ(q1, ε, A) = {(q1, α)|(A → α) ∈ P}For each a • ∈ Σ, δ(q1, a, a) = {(q1, ε)}δ(q• 1, ε,Z0) = {(q2,Z0)}

The proof can be completed by showing L (G) ⊆ Lf (M) and Lf (M) ⊆ L (G)

The basic idea of the construction is to use the start state and a required ǫ-transition first move to transit to state q1 while placing the grammar’s start symbol on the stack. Whenever a grammar variable is on the top of the stack we can pop it off replacing it with its right-hand side string. 1Whenever the input symbol matches the stack’s top (grammar terminals) we eat the input and pop the symbol off the stack. Finally when input is exhausted and Z0 is all that is left on the stack (nothing was ever placed under it), the PDA transits to the accepting state.

Hopefully, one sees that the constructed PDA will correctly guess the steps necessary for a leftmost string derivation for x ∈ L (G). Simply put if at some point in the derivation the current string looks like xα, where x ∈ Σ* (a string of terminals), then at some point in the PDA’s simulation of the leftmost derivation, the input string read so far is x and that stack contains α.


38

It should be clear that this PDA is NOT a DPDA. The non-determinism is introduced by there being multiple productions for any given variable A. Any nondeterminism inherent in the grammar is replicated in the PDA.

For every PDA, there is an equivalent CFGA word (or two) is in order before looking at how to construct a CFG out of any PDA M. Assume w.l.o.g. that the given PDAM accepts language L via empty stack L = Le(M). (Given a PDA M’ that accepts L via final state one can convert ‘M′ into a PDA M such that; L = Le (M) = Lf (M′)

We already know what the terminal symbols are, but what to use for the grammar’s variables? It should be obvious that one cannot use the PDA’s states; there are probably not enough of them for our purposes. Furthermore one cannot use either the set Γ of stack symbols (not enough information encoded) or all the possible stack configurations (infinitely many). Instead our variables will be things of the form [p, A, q] where p, q ∈ Q; A ∈ Γ. This will certainly provide more variables then needed.Whenever the PDA can move from state p to state q, read a ∈ Σ ∪ {ε} from the input stream and pop A, A ∈ Γ, off the stack (a true pop operation, not a replace) we introduce into the grammar the production [p, A, q] → a. Hence our grammar’s variables incorporate both state information and stack top symbol information.

Theorem If M = (Q, Σ, Γ, q0, Z0, δ, A) is a PDA there exists a CFG G = (V, Σ, S, P) such that Le (M) = L (G)Proof:The CFG G is defined as follows:V = {S} ∪ {[p, A, q] | A ∈ Γ; p, q ∈ Q} – more than is needed but it is harmless to include useless variables.

The productions of the grammar are:For every q • ∈ Q; (S → [q0,Z0, q]) ∈ PIf q• 1, q2 ∈ Q; a ∈ Σ ∪ {ε}; A ∈ Γ; and δ(q1, a, A) contains (q2, ε) then ([q1, A, q2] → a) ∈ PIf m ≥ 1; q• 0, q1 ∈ Q; a ∈Σ ∪ {ε}; A ∈ Γ; δ(q0, a, A) contains (q1,B1B2, ….Bm) for some string B1,B2,… , Bm ∈ Γ then for every possible choice (i.e. assignment) of qa, …, qm ∈ Q, the production

[q0, A, qm] → a [q1,B1, qa][qa,B2, qb]…[ql, Bm, qm]

Is included in PThe proof is completed by showing L (G) ⊆ Le (M) and Le (M) ⊆ L (G).

3.7 Non Context Free LanguagesNot all languages are context-free. The simplest example of a non-context-free language is

{an bn cn | n ≥ 0}.To process this would require two stacks.

Proving non-context-free by pumping lemma

Lemma: Consider a parse tree according to a CNF grammar with a yield of w ∈ Σ*. If the height of the tree is h then

|w| ≤ 2h−1.

Theorem: (The pumping lemma for context-free languages): Let A be a CFL. Then there exists a constant p (the pumping length) such that ∀s ∈ A with |z| ≥ p, we can write s = uvxyz such that,

|vy| > 0• |vxy|• ≤ p∀• i ≥ 0, string uvixyiz ∈ A

39

Proof: Given a CFL A, there is a CNF G = (V, Σ, R, S) such that L(G) = A−{ε}. Let m = |V| and choose p = 2m.

Suppose s∈ L (G) with |s| ≥ p. Any parse tree for s must have a height of at least m+1, otherwise |s| ≤ 2m−1 = p/2 by Lemma. Therefore, there must be a path A0(S), A1,…, Al ,a with at least m+1 variables, i.e., l ≥ m, in the parse tree for s. Since there are only m different variables in the grammar, by the pigeonhole principle, there must be at least two identical variables on the path. Choose the identical pair closest to the leaf, i.e., Ai = Aj with l −m ≤ i < j ≤ l.

Then, it is possible to divide the parse tree such that x is the yield of the sub tree rooted at Aj, vxy is the yield of the sub tree rooted at Ai, and s = uvxyz is the yield of the entire parse tree. Next we examine whether this partition satisfies all three conditions stated in the pumping lemma. First, since there is no unit production in CNF, v and y can not both be ε. So |vy| > 0.

Second, since the height of the sub tree rooted at Ai is at most m+1, its yield vxy has length at most 2m = p. So |vwx| ≤ p.

Third, since Ai = Aj = T, we have T vTy and T x from the parse tree. So, for any i ≥ 0, T vixyi, thus, S uTz uvixyiz. So uvixyiz ∈ A for i ≥ 0. This completes the proof of the pumping lemma.

Using the pumping lemma to prove that a language is not context-free:Assume that A was context-free by contradiction. Then, the pumping lemma applies to A. Let p be the constant • in the pumping lemma.Select s • ∈ A with |s| = f (p) ≥ p.By the pumping lemma, s = uvxyz with |vy| > 0, |vxy| • ≤ p, and uvixyiz ∈ A ∀i ≥ 0.For any u, v, x, y, z such that s = uvxyz, |vy| > 0, and |vxy| • ≤ p, find i ≥ 0 such that uv• ixyiz ∉ A.

Proving non-context-free by closure propertiesNote that the following closure properties of CFLs may also be used to prove that a given language is context free.

Closed under union: If A and B is context-free, so is A • ∪ B.Closed under concatenation: If A and B is context-free, so is AB.• Closed under star: If A is context-free, so is A*.• Closed under reverse: If A is context-free, so is A• R.Not closed under intersection: Consider A = {a• nbncm} and B = {ambncn}.Not closed under complementation: Note that A∩B =• .Not closed under difference: Note that • = Σ* −A.Intersect with a regular language: If A is context-free and B is regular, then A ∩ B is context-free.• Difference from a regular language: If A is context-free and B is regular, then A−B is context-free. Note that • A−B = A ∩ .

Example: A = {w ∈ {a, b, c}*|w has equal numbers of as, bs and cs} is not a CFL.


40

Summary Context free grammars can describe certain features that have a recursive structure which makes them useful • in a variety of applications.Context-free languages have applications in compiler design (parsers).• The syntax of programming languages is often given in the form of context-free grammar, or equivalent Backus-• Naur form.A grammar consists of a collection of substitution rules, also called productions.• The string consists of variables and other symbols called terminals.• Similar to regular languages, there are different ways of representing context-free languages.• A pushdown automaton (PDA) is a non-deterministic finite state automaton that has access to an infinite memory, • organized as a stack.The control unit is a non-deterministic finite state machine.• Transitions may depend on the next input symbol and the topmost stack symbol.• A context-free grammar and pushdown automata are equivalent in power.• Not all languages are context-free.• A grammar G is said to be in the Chomsky normal form (or CNF) if all productions are of the forms•

A → BC; orA ∈ a

ReferencesContext free languages. Available at <http://users.utu.fi/jkari/automata/part2.pdf >. [Accessed 22• nd March, 2011]Pushdown Automata. Available at < http://www.cs.nott.ac.uk/~txa/g51mal/notes-5x.pdf>. [Accessed 22• nd March, 2011]Pda CfgEquivalence. Available at <http://www.cs.xu.edu/csci250/06s/Theorems/pdaCfgEquivalence.pdf > • [Accessed 22nd March, 2011].

Recommended ReadingAlexander Meduna. • Automata and Languages: Theory and Applications. Springer; 1st ed. (August 2000). 920 pages.Laura Kallmeyer. • Parsing Beyond Context-Free Grammars. Springer; 1st ed. (August 24, 2010). 240 pages.Dexter C. Kozen. • Automata and Computability. Springer (August 1997). 413 pages.

41

Self Assessment

A grammar consists of a collection of substitution rules, also called ____________1. productionsa. reportb. languagec. alphabetsd.

The string consists of variables and other symbols called ____________2. nodesa. terminalsb. integersc. alphabetsd.

The control unit is a _____________ finite state machine.3. deterministica. Mooreb. non-deterministicc. Mealyd.

_____________ may depend on the next input symbol and the topmost stack symbol4. Conversiona. Alterationb. Variationsc. Transitionsd.

A context-free grammar and pushdown automata are equivalent in _____________5. powera. sizeb. lengthc. dimensiond.


42

State which of the following true.6. Not all languages are error-free.a. Not all languages are context-free.b. All languages are context-free.c. Not all languages are non context-free.d.

State which of the following is true.7. Context-free languages have applications in medical science.a. Context-free languages do not have applications in compiler design (parsers).b. Context-free languages have applications in compiler design (parsers).c. Non context-free languages have applications in compiler design (parsers).d.

State which of the following is false.8. The control unit is a non-deterministic finite state machine.a. A pushdown automaton (PDA) is a non-deterministic finite state automaton that has access to an infinite b. memory, organized as a stack.The syntax of programming languages is often given in the form of context-free grammar, or equivalent c. Backus-Naur form.A pushdown automaton (PDA) is a non-deterministic infinite state automaton that has access to a finite d. memory, organized as a stack.

A context-free grammar G = (V; T; P; S) consists of which one of the following?9. Finite set P of productions of the forma. Infinite set P of productions of the formb. Terminalsc. Set of terminalsd.

Which one of the following can describe certain features that have a recursive structure which makes them 10. useful in a variety of applications?

Non context free grammarsa. Context free grammarsb. Finite automatac. Pushdown automatad.

43

Chapter IV

Turing Machines and Computability

Aim


learn about Turing machine •

analyse variants of Turing machine•

analyse the Church-Turing Thesis•

Objectives


examine informal definition of Turing machine•

determine non-deterministic Turing machine•

identify the Church-Turing thesis•

Learning outcome


recall the concept of Turing machine•

explain variants of Turing machine •

recall Church-Turing Thesis•


44

4.1 IntroductionTuring machine (TM) is the next more powerful model of automata approach which recognizes more languages than pushdown automata models do. Also, phrase- structure model is the corresponding grammatical model that matches Turing machines in computational power.

TM is an abstract entity constituted of mathematical objects like sets and a (partial) function. However in order to help our understanding of the subject-matter of TMs, we can visualize a TM as a physical computing device that can be represented in the following diagram

Fig. 4.1 Turing machine

According to the above figure, TM consists of – A tape• , with an end on the left but infinite on the right side. The tape is divided into squares or cells, with each cell capable of holding one of the tape symbols including the blank symbol #. A finite control• , which can be in any one of the finite number of states.

The states in TM can be divided in three categories viz.The initial state � , the state of the control just at the time when TM starts its operations. The initial state of a TM is generally denoted by q

0 or s.The halt state, in which � TM stops all further operations. The halt state is generally denoted by h. The halt state is distinct from the initial state. Thus, a TM has at least two states.Other states �

A tape head • (or simply head), is always stationed at one of the tape cells and provides communication for interaction between the tape and the finite control. The head can read or scan the symbol in the cell under it. The symbol is communicated to the finite control. The control taking into consideration the symbol and its current state decides for further course of action including,

The change of the symbol in the cell being scanned and/or change of its state and/or �Moving the head to the left or to the right. The control may decide not to move the head. �

45

4.2 Formal Definition of Turing MachineSimilar to a finite automaton but with an unlimited and unrestricted memory, a Turing machine is a much more accurate model of a general purpose computer. A Turing machine can do everything that a real computer can do. Nonetheless, even a Turing machine cannot solve certain problems. In a very real sense, these problems are beyond the theoretical limits of computation.

The following list summarizes the differences between finite automata and Turning machines:A Turing machine can both write on the tape and read from it.• The read – write head can move both to the left and to the right.• The tape is infinite.• The special states for rejecting and accepting take immediate effect.•

DefinitionFormally a TM M = (Q, Σ, Γ, δ, s) where,

Q = a finite set of states• Σ = input alphabet not containing the blank symbol B• Γ = the tape alphabet of M• s in Q is the start state• δ : Q X Γ• →Q× Γ× {L, R} is the (partial) transition function

Deterministic Turing machines definitionA Deterministic Turing machine is a 7 – tuple (Q, Σ ,Γ, δ, q0, qaccept, qreject) Where Q,Σ ,Γ are all finite sets and:

Q is the set of states.• Σ is the input alphabet not containing the special blank symbol B• Γ is the tape alphabet, where B • ∈ Γ and Σ ⊆ Γd: Q• ×Γ→ Q×Γ×{L, R} is the transition functionq• 0 ∈ Q is the start stateq• accept ∈ Q is the accept stateq• reject ∈ Q is the reject state, where qreject ≠ qaccept $

Non-deterministic Turing machines definitionA non- deterministic Turing machine is a 7 – tuple (Q, Σ ,Γ, δ, q0, qaccept, qreject) Where Q, Σ , Γ are all finite sets and:

Q is the set of states.• Σ• is the input alphabet not containing the special blank symbol BΓ• is the tape alphabet, where B ∈ Γ and Σ ⊆ Γd: Q• ×Γ→ Q×Γ×{L, R} is the transition functionq• 0 ∈ Q is the start stateq• accept ∈ Q is the accept stateq• reject ∈ Q is the reject state, where qreject ≠ qaccept $For a Turing machine, d takes the form: d: Q• ×Γ→ Q×Γ×{L, R}. That is, when the machine is in a certain state q1 and the head is over a tape square containing a symbol a, and if (q• 1, a) = (q2, b, L) the machine writes the symbol b replacing a, and goes to state q2.Initially M receives its input w = w• 1 w2… wn ∈ Σ* on the leftmost n squares of the tape, and the rest of the tape is blank.


46

As a Turing machine computes, changes occur in the current state, the current tape contents, and the current • head location. A setting of these three items is called a configuration of the Turing machine.A Turing machine M accepts input w if a sequence of configurations C• 1, C2 ,..., Ck exists where

C � 1 is the start configuration of M on input wEach C � i yields Ci+1

C � k is an accepting configuration.

The collection of strings that M accepts is the language of M, denoted L(M).• Call a language Turing• – recognizable if some Turing machine recognizes it.When we start a TM on an input, three outcomes are possible. The machine may accept, reject, or loop. By • loop we mean that the machine simply does not halt. It is not necessarily repeating the same steps in the same way forever as the connotation of looping may suggest. Looping may entail any simple or complex behavior that never leads to a halting state.We prefer Turing machines that halt on all inputs; such machines never loop. These machines are called deciders • because they always make a decision to accept or reject. A decider that recognizes some language also is said to decide that language.Call a language Turing – decidable or simply decidable if some Turing machine decides it.• Every decidable language is Turing – recognizable but certain Turing – recognizable languages are not • decidable.

Example: TM for {0n1n|n>0}Description of how the machine works

Look for 0’s• If 0 found, change it to x and move right, else reject• Scan (to the right) 0’s and y’s until you reach 1• If 1 found, change it to y and move left, else reject.• Move left scanning past 0’s and y’s• If x found move right• If 0 found, loop back to step 2.• If 0 not found, scan (to the right) y’s and accept•

Head is on the left or start of the string. X and y are just variables to keep track of equalityHead is on the left or start of the string.

State Symbol Next state actionq0 0 (q1, x, R)q0 1 Halt/rejectq0 X Halt/rejectq0 y (q3, y, R)

Head is on the left or start of the string.State Symbol Next state action

q1 0 (q1, 0, R)q1 1 (q2, y, L)q1 X Halt/reject

47

q1 y (q1, y, R)Head is on the left or start of the string

State Symbol Next state action

q2 0 (q2, 0, L)q2 1 Halt/rejectq2 X (q0, x, R)q2 y (q2, y, L)

Head is on the left or start of the stringState Symbol Next state action

q3 0 Halt/reject

q3 1 Halt/reject

q3 X Halt/reject

q3 y (q3 y, R)

q3 B (q4, B, R)

Head is on the left or start of the stringState Symbol Next state action

q4 0 Illegal i/pq4 1 Illegal i/pq4 X Illegal i/pq4 y Illegal i/pq4 B Halt/accept


48

Input 1: 01


50

Input 2: 10

Input 3: 011


52

4.3 Some Configuration on TMStart configuration q• 0wAccepting configuration q• accept

Reject configuration q• reject

Halting configuration: accepting and rejecting configurations•

53

4.4 Variants of Turing MachinesThe original TM model and its reasonable variants all have the same power – they recognize the same class of • languages.To show that two models are equivalent we simply need to show that we can simulate one by the other.• Robustness: invariant power to certain changes.• Example: d: Q• ×Γ→ Q×Γ×{L, R, S}TM with multi-tapes• : A multitape TM is like an ordinary Turing machine with several tapes. Each tape has its own head for reading and writing. Initially the input appears on tape 1, and the others start out blank. Two machines are equivalent if they recognize the same language. Every multitape Turing machine has an equivalent single tape Turing machine.

Example:2-tape TM for copying

Assume there are 2-tapes• Move the head on the first tape left until a blank is found.• Move the head on both tapes to the right, copying each symbol on the first tape onto the second tape, until a • blank is found on the first tape. The first square of the second tape should be left blank.Move the head on the second tape to the left until a blank is found.•

Again move the heads on both tapes to the right, this time copying symbols from the second tape onto the �first tape. Halt when a blank is found on the second tape.

Nondeterministic TM’s: • A nondeterministic Turing machine is defined in the expected way. At any point in a computation the machine may proceed according to several possibilities. The transition function for a nondeterministic is

δ: Q×Γ→P (Q ×Γ×{L, R})

The computation of a nondeterministic Turing machine is a tree whose branches correspond to different possibilities for the machine. (If you want to simulate a nondeterministic TM with a ’normal’ TM you have to perform a breadth – first search through the tree, because with depth – first you can lose yourself in an infinite branch of the tree and miss the accept state). If some branch of the computation leads to the accept state, the machine accepts its input. Every nondeterministic Turing machine has an equivalent deterministic Turing machine. A language is Turing – recognizable if and only if some nondeterministic Turing machine recognizes it. We call a nondeterministic Turing machine a decider if all branches halt on all inputs. A language is decidable if and only if some nondeterministic TM decides it.

Enumerators• : Loosely defined, an enumerator is a Turing machine with an attached printer A language is Turing – recognizable if and only if some enumerator enumerates it.

From a Turing machine M, construct an enumerator E. �From an enumerator E, construct a Turing machine M. �

TM with multi-strings (and multi-cursors).• TM with multi-cursors.• TM with multi-tracks.• TM with two-way infinite tape.• TM with multi-dimensional tape.•

4.5 Equivalence with Other ModelsTuring machine has many variations.• Many other models of general purpose computation have been proposed•

They are the same in power as Turing machine. �


54

Power of programming languages are the same• Program of any language can be compiled to a program of other language. �

4.6 Example of Turing MachineExample 1TM M2 recognizes the language consisting of all strings of 0s whose length is a power of 2. It decides the language A = {02n |n ≥ 0}.

Solution: M2= ‘on input string w’

Sweep left to right across the tape, crossing off every other 0.• If in stage 1 the tape contained a single 0, accept.• If in stage 1 the tape contained more than a single 0 and the number of 0s was odd, reject• Return the head to the left-hand end of the tape.• Go to stage 1.•

4.7 AlgorithmDefinition:A collection of simple instructions for carrying out some task.

Polynomial a sum of terms, where each term is a product of certain variables and a constant called a • coefficient.Root assignment of values to its variables so that the value of the polynomial is 0.• Integral root value of root is integer.•

4.8 The Church-Turing ThesisThe thesis states that the Turing Machine is the formal equivalent of the informal concept of algorithm.• It is NOT a theorem, since it cannot be proved. (Algorithm is an informal concept).• Recall, algorithm -- a sequence of precise steps, each of which can be executed in a finite amount of time with • finite resources. Precise means no intelligence or judgment required can be executed literally.All reasonable formal models so far have turned out to be equivalent to TM.• Reasonable -- means no infinite resources or power.• Examples of other formal models:•

Post correspondence systems �Random access machine model �Unrestricted grammars. �Partial recursive functions, etc. �

4.9 Universal Turing MachineWe shall assume that there are fixed countable infinite sets K∞ = {q1, q2, q3, …}and Σ∞ = {a1, a2, a3, …}Such that for every TM, the state set is a finite subset of K∞ and the tape alphabet is a finite subset of Σ∞. If M halts on input ‘w’, then U halts on an input consisting of the encoding of M followed by the encoding of ‘w’. If ‘U’ halts on an input consisting of the encoding of M followed by the encoding of ‘w’, then M halts on input ‘w’.

55

4.10 Universal TM as a 3-tape TMLet ‘U’ = 3-tape TM,‘U’ uses it three tapes as follows:

The first tape contains the encoding of the tape of M.• The second tape contains the encoding of M itself.• The third tape contains the encoding of the state of M at the current point in the simulated computation•


56

SummaryTuring machine (TM) is the next more powerful model of automata approach which recognizes more languages • than pushdown automata models do.TM is an abstract entity constituted of mathematical objects like sets and a (partial) function.• A Turing machine can do everything that a real computer can do.• Even a Turing machine cannot solve certain problems.• A Turing machine can both write on the tape and read from it.• The computation of a nondeterministic Turing machine is a tree whose branches correspond to different • possibilities for the machine.Every nondeterministic Turing machine has an equivalent deterministic Turing machine.• A language is decidable if and only if some nondeterministic TM decides it.• Enumerator is a Turing machine with an attached printer.• A language is Turing – recognizable if and only if some enumerator enumerates it• Turing machine has many variations.• An algorithm collection of simple instructions for carrying out some task.• The Turing Machine is the formal equivalent of the informal concept of algorithm.•

ReferencesThe Church-Turing Thesis. Available at <http://www.networks.howard.edu/chunmei/CC/Chapter3.pdf> • [Accessed 25th March 2011].Turing Machines. Available at<http://www.cs.wm.edu/~wm/CS423/ln8.pdf>. [Accessed 24th March 2011].• Tae Young Choe. The Church-Turing Thesis. Available at<http://cespc1.kumoh.ac.kr/~choety/2010/al-ch03.• pdf>. [Accessed 21st March 2011].

Reference ReadingAlexander Meduna. • Automata and Languages: Theory and Applications. Springer; 1 edition (August 2000). 920 pages.Martin C. Carlisle. • An Introduction to Languages and Machines. CreateSpace (May 15, 2009). 126 pages.Jon Agar. • Turing and the Universal Machine: The Making of the Modern Computer (Revolutions of Science). Totem Books (April 23, 1997). 106 pages.

57

Self AssessmentA _________________ can do everything that a real computer can do.1.

Turing machinea. Turning machineb. Trail machinec. Tuner machined.

TM is an abstract entity constituted of 2. _________________objects like sets and a (partial) function.physicala. mathematicalb. logicalc. geometrical d.

The computation of a _________________ Turing machine is a tree whose branches correspond to different 3. possibilities for the machine

deterministica. PCPb. nondeterministicc. finited.

Every nondeterministic Turing machine has an equivalent _________________ Turing machine.4. finitea. infiniteb. churchc. deterministicd.

Enumerator is a Turing machine with an attached _________________.5. Printera. Scannerb. Fax machinec. Photo copier d.

State which of the following is true.6. The Turing Machine is the informal equivalent of the formal concept of algorithm.a. The Turing Machine is the formal equivalent of the informal concept of algorithm.b. The Turing Machine is the formal equivalent of the informal concept of flowchart.c. The Turing Machine is equivalent to algorithm.d.

State which of the following is true.7. A language is Turing – non recognizable if and only if some enumerator enumerates it.a. A language is not Turing – recognizable if some enumerator enumerates it.b. A language is Turing – recognizable if and only if some enumerator enumerates it.c. A language is Turing – recognizable if and only if some printer enumerates it.d.


58

State which of the following is false.8. Turing machine has many variations.a. Even a Turing machine cannot solve certain problems.b. A language is decidable if and only if some nondeterministic TM decides it.c. Turing machine do not have variations.d.

Which of the following is a collection of simple instructions for carrying out some task?9. Algorithma. Flowchartb. Datac. Reportd.

Which of the following is an abstract entity constituted of mathematical objects like sets and a (partial) 10. function?

TNMa. TMb. PCPc. TPd.

59

Chapter V

Decidability, Undecidability and Reducibility

Aim


learn what is of decidability •

recognize the concept of reducibility•

analyze the Turing reducibility•

Objectives


examine decidable languages•

determine the halting problem•

identify the mapping reducibility•

Learning outcome

At the end of this chapter, students will be able to:

recall the concept of decidability•

explain post correspondence problem•

outline mapping and Turing reducibility•


60

5.1 IntroductionA decision problem A is called decidable or effectively solvable if A is a recursive set. A problem is partially decidable, semi decidable, solvable or provable if A is a recursively enumerable set. Partially decidable problems and any other problems that are not decidable are called undecidable.

5.2 Decidable LanguagesDefinition 1: ATM M accepts a language L ⊆ Σ*, if L = {w: M (w) = accept}.

Definition 2: A TM M decides a language L ⊆ Σ*, if ∀w ∈ Σ*,

M (w) =

Definition 3: A language L is Turing-acceptable, or recursively enumerable if there is a TM M which accepts L.

Definition 4: A language L is Turing-decidable or recursive if there is a TM M which decides L

Examples of decidable languages

Theorem 1: ADFA= {<B, w>: B is DFA that accepts a string w} is a decidable languageProof: A description of a TM M which decides ADFA for input <B, w>

Run B on input w1. If B accepts w, M accepts <B, w>; Else M rejects <B, w>.2.

To construct such a TM the five components of DFA Q, Σ, δ, q0 and F can be used in the TM. When M receives its input, M first determines whether it properly represents a DFA B and a string w. if not rejects. Then M carries out simulation directly. It keeps track of B’s current position in the input w by writing this information down on its tape. Initially B’s current state is q0 and B’s current input position is the leftmost symbol of w. The states are updated according to specified transition functionδ. When M finishes processing the last symbol of w, M accepts the input if B is in an accepting state. M rejects the input if B is in a non-accepting state.

Theorem 2: ANFA= {<B, w>: B is an NFA that accepts w} is decidable languageProof: A description of a TM M which decides ANFA for input <B, w>

Construct a DFA C which is equivalent to B;1. Run C on w;2. If C accepts w, M accepts <B, w>; 3. Else M rejects <B, w>.

Theorem 3: AREX = {<R, w>: R is a regular expression that generates w} is a decidable language.Proof: A description of a TM M which decides AREX for input <R, w>

Construct a DFA C which accepts the language generated by R;1. Run C on w;2. If C accepts w, M accepts <B, w>;3.

Else M rejects <R, w>.

Theorem 4: EDFA= {<B>: B is a DFA and L (B) = Ø} is a decidable language.

61

This means that no string exists that DFA B accepts

Theorem 5: EQDFA = {<A, B>: A and B are DFA’s and L (A) = L (B)} is a decidable language.The theorem states whether two DFAs recognize the same language is decidable.

Theorem 6: ECFG = {<G>: G is a CFG and L (G) = Ø} is a decidable languageProof: If L (G) ≠ ∅, then there is a parse tree whose yield is a string of terminals and whose height ≤ |V|. This is because every parse tree of height ≥ |V | has a path with two nodes labeled with the same variable. Then, a smaller parse tree can be constructed by excising the part of the tree between the nodes with the same label. The process can be repeated until a parse tree of height ≤ |V | is constructed with the terminal yield. Thus, to learn if L (G) is empty, it is sufficient to search through all parse trees of height ≤ |V |. If none of them yields a terminal string, the language is empty

5.3 Halting ProblemWe have seen that the problems are said to be decidable if there exists an algorithm which can solve problem in finite amount of time. There are some specific problems that are algorithmically unsolvable. Though the computers are so powerful there are some limitations. For example computers can’t easily avoid crashing; or rather they can’t predict when they are about to crash and avoid it.

Turing machines: A Turing machine is an extremely powerful model of a computing device, and could be programmed to solve many problems. The Turing machine is a theoretical computer, which works by storing data on a tape of infinite length and moving back and forward along it according to the programme. Even the programme itself can be stored on the tape; these machines are called Universal Turing machines. Turing programme either halt when they have arrived at the results or else they continue forever. No Turing program can be said to have produced any results until it has halted.

5.3.1 The Halting Problem for Turing MachinesHalting problem of Turing machine is formulated by giving an arbitrary Turing machine and machine input to the machine.

In other words, Turing wondered whether it would be possible to write a Turing machine program that would take two inputs: < program P, input i> and answer ‘Yes’ if the TM halts when executing program P on input i and answer ‘No’ if it wouldn’t halt, that is it would loop for ever.

The solution can be found by giving a description number to every possible Turing machine so that no possible programs should be left out. One way to figure out if there’s a program that can solve the halting problem would be to look through all the whole numbers, interpreting each as the description number of a Turing machine program, and checking to see if it’s program that solves the halting problem.

Of course, this is completely impractical. But Turing realized that if we could prove that no whole number was the right one, then we could know that no program to solve the halting problem exists.

Theorem 1: A Turing machine is recognizable but not decidable. A recognizer of a TM is called the Universal Turing Machine U where U=<M, w>, where M is a TM and w is a string.

Run M on w1. If M ever enters it’s accept state, accept; if ever it enters its reject state, reject.2.

Note: U is universal because it stimulates any other TM from its description Important points to understand here are

All problems may be solvable but all are not decidable.1. Since a TM is undecidable, to solve this problem we need to expand out problem solving methodology by a 2. new method for proving undecidability.


62

5.4 The Post Correspondence ProblemThe Post correspondence problem (due to Emil Post) is another undecidable problem that turns out to be a very helpful tool for proving problems in logic or in formal language theory to be undecidable.

Let Σ be an alphabet with at least two letters. An instance of the Post Correspondence problem (for short, PCP) is given by two sequences U = (u1,…, um) and V = (v1,… vm), of strings ui, vi ∈ Σ*. The problem is to find whether there is a (finite) sequence (i1, …ip), with ij ∈ {1, …, m}forj = 1, …, p, so thatuil ui2…uip=vi1vi2…vip

Equivalently, an instance of the PCP is a sequence of pairs

For example, consider the following problem:

There is a solution for the string 1234556:abab aaabbb aab ba ab ab aa= ababaaa bb baab baa ba ba a

Theorem 1: (Emil Post, 1946) The Post correspondence problem is undecidable, provided that the alphabet Σ has at least two symbols.

There are several ways of proving the above theorem, but the strategy is more or less the same: Reduce the halting problem to the PCP, by encoding sequences of ID’s as partial solutions of the PCP.

For instance, this can be done for RAM programs. The first step is to show that every RAM program can be simulated by a single register RAM program.

Then, the halting problem for RAM programs with one register is reduced to the PCP (using the fact that only four kinds of instructions are needed). A proof along these lines was given by Dana Scott.

As an application, we prove the following result

Theorem 2: It is undecidable whether a context-free grammar is ambiguous.Proof: We reduce the PCP to the ambiguity problem for CFG’s. Given any instance U = (u1, …, um) and V = (v1, … vm) of the PCP, let c1, … ,cm be m new symbols, and consider the following languages:LU = {ui1…uip cip…ci1 | 1 ≤ ij ≤ m, 1 ≤ j ≤ p, p ≥ 1},LV = {vi1…vipcip…ci1 | 1 ≤ ij ≤ m, 1 ≤ j ≤ p, p ≥ 1},and LU,V = LU ∪ LV .We can easily construct a CFG, GU,V, generating LU,V . The productions are:S→SUS→SVSU→uiSU ciSU→uiciSV→viSVciSV→vici

It is easily seen that the PCP for (U, V) has a solution if LU ∩ LV ≠Ø if G is ambiguous.

63

5.5 Undecidable Problems about Turing MachinesThe following problems about Turing machines are not decidable:

Whether L (M) = • θ for any TM M. Whether L (M• 1) = L (M2) for any two TMs M1 and M2.Whether L (M) is finite for any TM M• Whether L (M) is regular for any TM M.• Whether L (M) is context-free for any TM M.•

Theorem 1: ETM = {< M > |M is a TM and L (M) = θ} is undecidable.Proof: Reduce ATM to ETM. Assume that ETM is decidable. Let R be the TM that decides ETM. We use R to construct TM S that decides ATM as follows: On input < M, w >,

Construct TM M• 1 which on input x, rejects if x ≠ w and simulates M on w if x = w.Run R on <M• 1>If R accepts, reject and if R rejects, accept.•

Theorem 2: NETM = {<M> | M is a TM and L (M) ≠ θ} is Turing-recognizable but not decidable.Proof: To prove that NETM is Turing-recognizable, we design a TM MNE to recognize NETM. On input <M>,

MNE systematically generates strings w: ε, 0, 1, 00, 01, • …. and use the universal TM U to test whether M accepts w. (What if M never halts on w? Run M on w1, … wi for i steps for i = 1,…)If M accepts some w, then M• NE accepts its own input M.

We next prove that NETM is not decidable. Assume that there is a TM MNE that decides NETM, i.e., TM MNE determines whether L (M) ≠θ for any TM M. We will use MNE to construct a TM Mu that would decide the undecidable ATM. On input < M, w >,

M• u constructs a new TM M’, which rejects if its input is not w and mimics M if its input is w.M’ is then fed to M• NE.M• NE accepts its input M’ if L(M’) ≠θ if M accepts w

5.5.1 Rice TheoremRice theorem is useful for understanding part of the reason for the pervasiveness of undecidability.

Definition: P: {0, 1}*→ {0, 1} is a nontrivial property of Turing-acceptable languages ifP (<M>) depends only on L (M) • −if L (M) = L (M') then P (<M>) = P (<M>').P is nontrivial— • ∃M1 such that P (<M1>) = 1, and ∃M0 such that P (<M0>) = 0.

If P is a property of Turing-acceptable languages then we say that a language L = L (M) has property P if P (<M>) = 1.

Theorem 1: If P is a nontrivial property of Turing-acceptable languages, thenLP = {<M>: P (<M>) = 1}

is undecidable. (In fact, LP is not Turing-acceptable if; has property P; and LP is not co-Turing-acceptable if θ does not have property P.)

Proof: Case 1: Assume Ø does not have property P, ie., P (<M0>) = 0, where M0 is a TM for which L(M0) = ;. We show ATM ≤ m LP. To show this, we must exhibit a Turing computable function f for which <M'> = f (<M, w>) is a machine


64

accepting a language with property P if M accepts w. We specify the behavior of M0 on input x to be:“Run M on w.• If M rejects, reject.• Run M• 1 on x, where M1 is a (fixed) machine for which P (<M1>) = 1. (Here we are using that P is nontrivial).If M• 1 accepts, accept; if M1 rejects, reject.”

Clearly M0 is Turing computable from M and w. Observe thatIf M accepts w then L (M’) = L (M• 1), which is a language with property P.If M does not accept w, then L (M’) = Ø which, by assumption, is a language which does not have property P.•

Case 2.Assume Ø does have property P. Proceed as above, reducing to LP.

5.6 Reducibility A reduction is a way of converting one problem into another problem in such a way that a solution to the second • problem can be used to solve the first problem.When A is reducible to B, solving A cannot be harder than solving B because a solution to B gives a solution to • A. In terms of computability theory, if A is reducible to B and B is decidable, A also is decidable. Equivalently, if A is undecidable and reducible to B, B is undecidable. This last version is key aspect for proving that various problems are undecidable.

Theorem 1: HALTTM is undecidable.Proof: Assume that TM R decides HALTTM.Then, the following TM S will decide ATM, which is not true.S= “On input <M, w>, an encoding of a TM M and a string w

Run TM R on input <M, w>.1. If R rejects, reject.2. If R accepts, simulate M on w until it halts.3. If M has accepted, accept; if M has rejected, reject.”4.

Theorem 2: ETM is undecidable.Def. ETM = {<M> | M is a Turing machine and L (M) = Ø}.Proof: Assume that TM R decides ETMMw = “On input x:If x ≠ w, rejectIf x =w, run M on input w and accept if M does”.

The following machine S will decide ATM, which is not true.S= “On input <M, w>, an encoding of a TM M and a string w:

Use the description of M and construct M• w as above.Run R on input <M• w>.If R accepts, reject; if R rejects, accept.”•

There are two possibilities:M accepts w• ⇔ L (Mw) = {w}M does not accept w • ⇔ L (Mw) = Ø

S rejects <M, w> ⇔ R accepts <Mw> ⇔ L (Mw) = ØS accepts <M. w> ⇔ R rejects <Mw> ⇔ L (Mw) ≠ Ø.

65

Theorem 3: RegularTM is undecidableDef: RegularTM= {<M>| M is a TM and L(M) is a regular language}Proof: Assume TM R decides RegularTMThe following s will decide ATM, which id not true.S=”On input <M, w>, where M is a TM and w is a string:

Construct TM M1. w, whereMw= “On input x:

If x has the form 0• n1n, accept.If x does not have this form, run M on input w and accept if M accepts w”.• Run R on input <M1. w>.If R accepts, accept; if R rejects, reject”.2.

Remark: L (Mw) is regular ⇔ M accepts wL (Mw) is not regular ⇔ M does not accept wWe assume that the input alphabet Σ contains 0 and 1 First we see that L (Mw) ⊇ {0n1n | n≥0}.If M accepts w, then we have L (Mw) = Σ*, which id trivially regular.If M does not accept w, then L (Mw) = {0n1n | n≥0}, which turns out to be non-regular by pumping lemma.

Theorem 4: EQTM is undecidable.Proof: Let TM R decide EQTM. The following TM S will decide ETM. S=” On input <M>, where M is a TM:

Run R on input <M, M1. 1>, where M1 is a TM that rejects all inputs.If R accepts, accept, if R rejects, reject.”2.

The theorem 2 tells us that ETM is undecidable. Thus we got contradiction

5.7 Mapping Reducibility Mapping Reducibility⇒ an computational approach to problem reduction.Definition: A function f: Σ∗ → Σ∗ is a computable function if some Turing machine M, on every input w, halts with just f (w) on its tape.This allows us to formally define mapping reducibility,Definition: Language A is mapping reducible to language B, written A ≤ m B, if there is a computable function f: Σ∗ → Σ∗, where for every w,w ∈ A ⇔ f(w) ∈ B.The function f is called as the reduction from A to B.Observation: The function f does not have to be a correspondence (neither one-to-one nor subjective). But, it is not allowed to map elements w ∉ A into B.

Theorem 1: If A ≤ m B and B is decidable, then A is decidable.Proof:Let M be a decider for B and let f be a reduction from A to B, then we can construct a decider N for A as follows:N = “On input w:

Compute f (w).• Run M on f (w) and output whatever M outputs.”•

w ∈ A if f (w) ∈ B since f is a reduction.

Theorem 2: If A ≤ m B and A is undecidable, then B is undecidableProof:Assume that B is decidable, let M be a decider for B and let f be a reduction from A to B, then we can construct a decider N for A as follows:N = “On input w:


66

Compute f (w).• Run M on f (w) and output whatever M outputs.”•

But, since A is undecidable by assumption this machine cannot exist and therefore our assumption that B is decidable must be wrong.

5.7.1 The Halting Problem IILet HALTTM = {<M, w>|M is a TM and halts on w}. We construct a reduction from ATM to HALTTM such that

<M, w> ∈ ATM ⇔ <M’, w> ∈ HALTTM.The following machine F computes the reduction:F = “On input <M, w>:

Construct the following machine M’1. M’ = “on input x:

Run M on x.• If M accepts, accept.• If M rejects, loop.”•

Output <M’, w> “2. Observe that <M’, w> ∈ HALTTM if and only if <M, w> ∈ ATM as required. If the input to F is not an element of A we assume that F maps it into some string not in B.

Reductions between languages do not always exist. That is, it is not always possible to specify a computable function that reduces one language to another.

Theorem 3: If A ≤m B and B is Turing-recognizable, then A is Turing recognizable.Proof:Let M be a recognizer for B and let f be a reduction from A to B, then we can construct a recognizer N for A as follows:N = “On input w:

Compute f (w).1. Run M on f (w) and output whatever M outputs.”2.

Therefore, if w ∈ A then f (w) ∈ B since f is a reduction. Thus M accepts f (w) whenever w ∈ A.

Theorem 4: The language EQTM = {<M1, M2>|M1 and M2 are TMs and L (M1) = L (M2)} is neither Turing-recognizable nor co-Turing-recognizable.Proof:To show EQTM is not Turing-recognizable we show that ATM is reducible to , a that is

<M, w> ∈ ATM ⇔ F (<M, w>) ∈ .The following machine accomplishes thatF = “On input <M, w>:Construct the two machines M1 and M2:M1 = “On any input: reject.”M2 = “On any input: run M on w, if it accepts, accept.”

Output <M1. 1, M2>.”To show EQTM is not Turing-recognizable we show that ATM is reducible to EQTM, that is <M, w> ∈ ATM ⇔ G (<M, w>) ∈ EQTM.The following machine accomplishes thatG = “On input <M, w>:

Construct the two machines M1 and M2:

67

M1 = “On any input: accept.”• M2 = “On any input: run M on w, if it accepts, accept.”•

Output <M1, M2>2.

5.8 Turing ReducibilityDefinition: An oracle for a language B is an external device that is capable of reporting whether a string w is a member of BInformally, an oracle for B is a hypothetical decider for B.Oracle TM is a modified TM that has the additional capability of querying an oracle.Notation: MB is an oracle TM for language B

Example: Consider an oracle for ATM

An oracle TM with an oracle for A• TM can decide more languages than an ordinary TM can.Such a TM can obviously decide A• TM itself by querying the oracle about its input.Such a TM can also decide E• TM with the following procedure .

Procedure . = “On input <M> where M is a TM:

Construct the following TM N:1. N = “On any input:

Run M in parallel on all strings in • Σ*If M accepts any of these strings, accept"• Query the oracle to determine whether <N, 0> 2. ∈ ATM.If the oracle answers NO, accept; if answers are Y ES, reject.”3.

We say that ETM is decidable relative to ATM.

Language A is Turing reducible to language B, A ≤T B, if A is decidable relative to B.

Example: We have shown that ETM ≤T ATM.

Note: Turing reducibility satisfies our intuitive concept of reducibility

Theorem 1: If A ≤T B and B is decidable then A is decidable.Proof: If B is decidable then we may replace the oracle for B by an actual procedure that decides B. Thus, we may replace the oracle TM that decides A by an ordinary TM that decides A.Conclusion:

Turing reducibility is a generalization of mapping reducibility.• If A ≤ • m B then A ≤T B because the mapping reduction may be used to give an oracle TM that decide A relative to B.An oracle TM with an oracle for A• TM can solve problems that are not solvable by ordinary TM.However, oracle Turing machines cannot decide all languages.•


68

Summary A decision problem ‘A’ is called decidable or effectively solvable, if A is a recursive set.• A problem is partially decidable, semi decidable, solvable or provable, if A is a recursively enumerable set.• Partially decidable problems and any other problems that are not decidable are called undecidable.• There are some specific problems that are algorithmically unsolvable.• A Turing machine is an extremely powerful model of a computing device, and could be programmed to solve • many problems.Turing programme either halt when they have arrived at the results or else they continue forever.• No Turing program can be said to have produced any results until it has halted.• The Post correspondence problem (due to Emil Post) is another undecidable problem that turns out to be a very • helpful tool for proving problems in logic or in formal language theory to be undecidable.Rice theorem is useful for understanding part of the reason for the pervasiveness of undecidability.• A reduction is a way of converting one problem into another problem in such a way that a solution to the second • problem can be used to solve the first problem.

References Phillip Rogaway. Theory of Computation. Available at <http://www.cs.ucdavis.edu/~rogaway/classes/120/• spring04/rice.pdf>. [Accessed 29 March 2011].Sunitha. • Formal Languages & Automata Theory. Tata McGraw-Hill Education. pp 285-290Decidable Languages. Available at <http://www.cs.rpi.edu/~goldberg/10Models/16-decid.pdf>. [Accessed 28 • March 2011].

Recommended ReadingSong Y. Yan. • An Introduction to Formal Languages and Machine Computation. World Scientific Publishing Company (December 1996). 420 pages.Raymond Greenlaw, H. James Hoover• . Fundamentals of the Theory of Computation. Morgan Kaufmann; 1st ed. (May 15, 1998). 385 pages.Maribel Fernandez. • Models of Computation: An Introduction to Computability Theory. Springer; 1st ed. (May 29, 2009). 184 pages.

69

Self AssessmentA ____________ is partially decidable, semi decidable, solvable or provable if A is a recursively enumerable 1. set.

problema. queryb. equationc. theoremd.

A decision problem A is called ____________ or effectively solvable if A is a recursive set.2. non- decidablea. decidableb. solvablec. unsolvabled.

____________ programme either halt when they have arrived at the results or else they continue forever3. Turninga. Toggleb. Turingc. Decodabled.

____________ theorem is useful for understanding part of the reason for the pervasiveness of undecidability.4. Reida. Rayb. Ryanc. Riced.

A ____________ is a way of converting one problem into another problem in such a way that a solution to the 5. second problem can be used to solve the first problem.

reductiona. diminutionb. post correspondencec. mappingd.

State which of the following is true.6. There are some specific problems that are algorithmically solvable.a. There are some specific problems that are algorithmically unsolvable.b. All problems that are algorithmically are unsolvable.c. There are some specific problems that are unsolvable.d.

State which of the following is true.7. Partially decidable problems and any other problems that are not decidable are called unsolvable.a. Partially decidable problems and any other problems that are not decidable are called solvable.b. Partially decidable problems and any other problems that are not decidable are called undecidable.c. Partially undecidable problems and any other problems that are not decidable are called decidable.d.


70

State which of the following is false.8. A language L is Turing-non decidable or recursive if there is a TM M which decides L.a. A Turing machine is recognizable but not decidable.b. A Turing machine is an extremely powerful model of a computing device, and could be programmed to c. solve many problems.Though the computers are so powerful there are some limitations.d.

Under which condition Language A is Turing reducible to language B, A ≤9. T B?If A is decidable relative to Ba. If A is not decidable relative to Bb. If B is decidable relative to Ac. If A is non decidable to Bd.

B is undecidable under which of the following condition?10. If A ≤ a. m B and A is decidableIf A ≤ b. m B and A is undecidableIf A ≤ c. m B and B is decidableIf A ≥ d. m B and A is undecidable

71

Application I

Recognizing Behavioral Patterns at Runtime Using Finite Automata

Abstract During reverse engineering, developers often required to understand the undocumented design of software. In particular, recognizing design patterns in the software can provide reverse engineers with considerable insight on the software structure and its internal characteristics. Researchers have therefore proposed techniques based on static analysis to automatically recover design patterns in a program. Unfortunately, most design patterns comprise not only structural, but also significant behavioral aspects. Although static analysis is well suited for the recognition of structural aspects, it is typically limited and imprecise in analyzing behavior. To address this limitation, a new technique is presented that complements our existing static analysis with a dynamic analysis, so as to perform more accurate design pattern recognition. The dynamic analysis is based on:

Transforming behavioral aspects of design patterns into finite automata,• Identifying and instrumenting relevant method calls• Monitoring relevant calls at runtime and matching them against the automata. •

The results of the dynamic analysis are then used to compute the likelihood of a pattern to be in the code. This paper describes our technique and presents a preliminary empirical study performed to assess the technique.

From behavioral patterns to finite automataBehavioral patterns can be interpreted as regular expressions. The symbols used in these regular expressions consist of a combination of the method call, its caller, and callee. We use finite automata to match the regular expressions. In this section, we describe how behavioral patterns are transformed transformed into Deterministic Finite Automata (DFA) that get method call events as input and check method traces for their conformance to the behavioral patterns.

To generate DFAs, we define, for each syntactic element of a behavioral pattern (e.g., method call, optional, and alternative fragment), a transformation into parts of a Non-Deterministic Finite Automaton (NFA). The complete NFA for a behavioral pattern is constructed by combining the NFA parts for each element of the pattern. The NFA is then transformed into a DFA. For the sake of space, we do not present the complete set of transformations for all syntactical elements. Instead, we illustrate a few of these transformations to give the reader the intuition of how our transformation works as shown in below Fig. Specifically, we show how we (1) transform a method call that occurs in a behavioral pattern into its corresponding part of an NFA, (2) combine two partial. NFAs that correspond to two consecutive calls and (3) transform a loop into its corresponding NFA fragment

Method calls are the simplest syntactic element in a behavioral pattern. We transform a method call into two states and a transition between them, as shown in Figure 5(1). The symbol accepted by the transition consists of the


72

concatenation of variables derived from the behavioral pattern: the caller’s object, (a: A), a call symbol, “->”, the callee’s object, (b: B), and the called method, m1 (the latter two separated by a dot). The variables will be bound to concrete values incrementally during the dynamic analysis.

Questions:What are the limitations of static analysis and what technique is presented to overcome this limitation?1. Answer:Static analysis is well suited for the recognition of structural aspects; it is typically limited and imprecise in analyzing behavior. To address this limitation, a new technique is presented that complements the existing static analysis with a dynamic analysis, so as to perform more accurate design pattern recognition.

Dynamic analysis is based on which points?2. Answer:The dynamic analysis is based on:Transforming behavioral aspects of design patterns into finite automata,• Identifying and instrumenting relevant method calls• Monitoring relevant calls at runtime and matching them against the automata. •

What is Deterministic Finite Automata (DFA)?3. AnswersA deterministic finite automaton M is a quintuple (S, A,δ, s1, F)Where:S = finite set of states,A = alphabet,s1 ∈ S is the initial state,F ⊆ S is the set of final states, andThe transition function δ: S x A→S, is defined such that ∀s ∈ S,∀c ∈ A, δ(s, c) ∈ S is uniquely determined

73

Application II

Applications of Automata Theory in Biology

Automata theory is the basis for the theory of formal languages. A proper treatment of formal language theory begins with some basic definitions:

A symbol is simply a character, an abstraction that is meaningless by itself.• An alphabet is a finite set of symbols.• A word is a finite string of symbols from a given alphabet.• Finally, a language is a set of words formed from a given alphabet.•

To the casual observer, biology is an impossibly complex science. Traditionally, the intricacy and variation found in life science has been attributed to the notion of natural selection. Species become “intentionally” complex because it increases their chance for survival. For example, a camouflage-patterned toad will have a far lower risk of being eaten by a python than a frog colored entirely in orange. This idea makes sense, but automata theory offers a simpler and more logical explanation, one that relies not on random, optimizing mutations but on a simple set of rules.

Basic automata theory shows that simplicity can naturally generate complexity. Apparent randomness in a system results only from inherent complexities in the behavior of automata, and seemingly endless variations in outcome are only the products of different initial states. A simple mathematical example of this notion is found in irrational numbers. The square root of nine is just 3, but the square root of ten has no definable characteristics. One could compute the decimal digits for the lifetime of the universe and never find any kind of recurring patter or orderly progression; instead, the sequence of numbers seems utterly random. Similar results are found in simple two-dimensional cellular automaton. These structures form gaskets and fractals that sometimes appear orderly and geometric, but can resemble random noise without adding any states or instructions to the set of production rules.

The most classic merging of automata theory and biology is John Conway’s Game of Life. “Life” is probably the most frequently written program in elementary computer science. The basic structure of Life is a two-dimensional cellular automaton that is given a start state of any number of filled cells. Each time step, or generation, switches cells on or off depending on the state of the cells that surround it. The rules are defined as follows:

All eight of the cells surrounding the current one are checked to see if they are on or not.• Any cells that are on are counted, and this count is then used to determine what will happen to the current • cell:

Death � : if the count is less than 2 or greater than 3, the current cell is switched off.Survival � : if (a) the count is exactly 2, or (b) the count is exactly 3 and the current cell is on, the current cell is left unchanged.Birth � : if the current cell is off and the count is exactly 3, the current cell is switched on.

Like any manifestation of automata theory, the Game of Life can be defined using extremely simple and concise rules, but can produce incredibly complex and intricate patterns.In addition to the species-level complexity illustrated by the Game of Life, complexity within an individual organism can also be explained using automata theory. An organism might be complex in its full form, but examining constituent parts reveals consistency, symmetry, and patterns. Simple organisms, like maple leaves and star fish, even suggest mathematical structure in their full form. Using ideas of automata theory as a basis for generating the wide variety of life forms we see today, it becomes easier to think that sets of mathematical rules might be responsible for the complexity we notice every day.

Inter-species observations also support the notion of automata theory instead of the specific and random optimization in natural selection. For example, there are striking similarities in patterns between very different organisms:

Mollusks and pine cones grow by the Fibonacci sequence, reproducible by math.•


74

Leopards and snakes can have nearly identical pigmentation patterns, reproducible by two-dimensional • automata.

With these ideas in mind, it is difficult not to imagine that any biological attribute can be simulated with abstract machines and reduced to a more manageable level of simplicity.

Questions:Define automata.1. List the applications of automata theory.2. How is automata theory applicable in biology?3.

75

Application III

Applications of Regular Expression

Regular expressions in Unix1. In the UNIX operating system various commands use an extended regular expressions language that provides shorthand for many common expressions. In this we can write character classes (A character class is a pattern that defines a set of characters and matches exactly one character from that set.) to represent large set of characters. There are some rules for forming this character classes:

The dot symbol (.) is to represent ‘any character’

The regular expression a+b+c+…+z is represented by [abc…z].

Within a character class representation, - can be used to define a set of characters in terms of a range. For example, a-z defines the set of lower-case letters and A-Z defines the set of upper-case letters. The endpoints of a range may be specified in either order (i.e. both 0-9 and 9-0 define the set of digits).

If our expression involves operators such as minus then we can place it first or last to avoid confusion with the range specifier.i.e. [-.0-9]. The special characters in UNIX regular language can be represented as characters using \ symbol i.e. \ provides the usual escapes within character class brackets. Thus [[\]] matches either [or], because \ causes the first] in the character class representation to be taken as a normal character rather than the closing bracket of the representation.

Special notations[: digit:] - Same as [0-9][: alpha:] - same as [A-Za-z][: alnum:] - Same as [A-Za-z0-9]Operators| - Used in place of +? - 0 or 1 of R? Means 0 or 1 occurrence of R

+ - 1 or more of R+ means 1 or more occurrence of R

{n} - n copies of R {3} means 1 or more occurrence of R^ - Compliment of

If the first character after the opening bracket of a character class is ^, the set defined by the remainder of the class is complemented with respect to the computer’s character set. Using this notation, the character class represented by ‘.’ can be described as [^\n]. If ^ appears as any character of a class except the first, it is not considered to be an operator. Thus [^abc] matches any character except a, b, or c but [a^bc] or [abc^] matches a, b, c or ^.When more than one expression can match the current character sequence, a choice is made as follows:

The longest match is preferred.1. Among rules, which match the same number of characters, the rule given first is preferred.2.

Questions:

Define Regular expressions.1. Explain how Regular expressions can be used in UNIX.2. What are the rules for forming the character classes?3.


76

Bibliography

ReferencesAutomata: The Methods and the Madness. Available at< http://www.math.uaa.alaska.edu/~afkjm/cs351/handouts/• hop-chap1.pdf > [Accessed 16th, March 2011].Bakhadyr Khoussainov. Regular expressions and finite automata. Available at <http://www.cs.auckland.• ac.nz/~bmk/se211lectures/lecture14.pdf> [Accessed 20th March, 2011].Christos G. Cassandras, Stéphane Lafortune. • Introduction to discrete event systems. Springer, 2008.pp 94-95.Context free languages. Available at <http://users.utu.fi/jkari/automata/part2.pdf >. [Accessed 22• nd March. 2011].Dan A. Simovici, Richard L. Tenney. • Theory of formal languages with applications. World Scientific, 1999. pp 147-148.Decidable Languages. Available at <http://www.cs.rpi.edu/~goldberg/10Models/16-decid.pdf>. [Accessed 28 • March 2011].Hopcroft. • Introduction to Automata Theory, Languages, and Computation, 3/E. Pearson Education India, 2008. pp 1-10.Michael Sipser. • Introduction to the Theory of Computation. Available at< http://n.ethz.ch/~ddukaric/zusammenfassungen/Buecher/Introduction%20to%20the%20Theory%20of%20Computation.pdf> [Accessed 16th, March 2011].Pda CfgEquivalence. Available at <http://www.cs.xu.edu/csci250/06s/Theorems/pdaCfgEquivalence.pdf > • [Accessed 22nd March. 2011].Phillip Rogaway. Theory of Computation. Available at <http://www.cs.ucdavis.edu/~rogaway/classes/120/• spring04/rice.pdf>. [Accessed 29 March 2011].Pushdown Automata. Available at < http://www.cs.nott.ac.uk/~txa/g51mal/notes-5x.pdf>. [Accessed 22• nd March. 2011]Sunitha. • Formal Languages & Automata Theory. Tata McGraw-Hill Education. pp 285-290Tae Young Choe. The Church-Turing Thesis. Available at<http://cespc1.kumoh.ac.kr/~choety/2010/al-ch03.• pdf>. [Accessed 21st March 2011].The Church-Turing Thesis. Available at <http://www.networks.howard.edu/chunmei/CC/Chapter3.pdf> • [Accessed 25th March 2011].Turing Machines. Available at<http://www.cs.wm.edu/~wm/CS423/ln8.pdf>. [Accessed 24th March 2011].•

Recommended ReadingAlexander Meduna. • Automata and Languages: Theory and Applications. Springer; 1 edition (August 2000). 920 pages.Arnold L. Rosenberg. • The Pillars of Computation Theory: State, Encoding, Nondeterminism. Springer; 1 edition (November 11, 2009). 326 pages.Dexter C. Kozen. • Automata and Computability. Springer (August 1997). 413 pages.Dexter C. Kozen. • Theory of Computation. Springer; 1st Edition. (December 15, 2010). 426 pages.Howard Straubing. • Finite Automata, Formal Logic, and Circuit Complexity. Birkhäuser Boston; 1 edition (May 3, 1994).226 pages.Jeffrey E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman. • Introduction to Automata Theory, Languages, and Computation. Addison Wesley; 3 edition (July 15, 2006). 535 pages.Jon Agar. Turing and the Universal Machine: The Making of the Modern Computer (Revolutions of Science). • Totem Books (April 23, 1997). 106 pages.Laura Kallmeyer. • Parsing Beyond Context-Free Grammars. Springer; 1st Edition. Edition (August 24, 2010). 240 pages.

77

Maribel Fernandez. • Models of Computation: An Introduction to Computability Theory. Springer; 1st Edition. (May 29, 2009). 184 pages.Martin C. Carlisle. An Introduction to Languages and Machines. CreateSpace (May 15, 2009). 126 pages.• Michael Sipser. • Introduction to the Theory of Computation. Course Technology; 002 edition (February 15, 2005). 456 pages.P.G. Spirakis. • Automata, Languages and Programming. Springer; 1 edition (August 9, 2001). 1083 pages.Raymond Greenlaw, H. James Hoover• . Fundamentals of the Theory of Computation. Morgan Kaufmann; 1 edition (May 15, 1998). 385 pages.Song Y. Yan. • An Introduction to Formal Languages and Machine Computation. World Scientific Publishing Company (December 1996). 420 pages.


78

Self Assessment Answers

Chapter Ia1. b2. c3. d4. a5. b6. c7. d8. a9. b10.

Chapter IIa1. b2. c3. d4. a5. b6. c7. d8. a9. b10.

Chapter IIIa1. b2. c3. d4. a5. b6. c7. d8. a9. b10.

Chapter IVa1. b2. c3. d4. a5. b6. c7. d8. a9. b10.

79

Chapter Va1. b2. c3. d4. a5. b6. c7. d8. a9. b10.

Theory of Computationjnujprdistance.com/assets/lms/LMS JNU/MCA/Sem V... · Theory of Computation 2...

Documents

Transcript of Theory of Computationjnujprdistance.com/assets/lms/LMS JNU/MCA/Sem V... · Theory of Computation 2...