::ICS 804::Theory of

Computation

- Ibrahim Otieno -

iotieno@uonbi.ac.ke

+254-0722-429297

SCI/ICT Building Rm. G15

Course OutlineCourse Outline Mathematical Preliminaries Turing Machines Recursion Theory Markov Algorithms Register Machines Regular Languages and finite-state

automata Aspects of Computability

Last week: Register Last week: Register MachinesMachines Register machines Register machines and formal

languages Model-independent

characterization of computational feasibility

Course OutlineCourse Outline Mathematical Preliminaries Turing Machines

◦ Additional Varieties of Turing Machines Recursion Theory Markov Algorithms Register Machines Regular Languages and finite-

state automata Aspects of Computability

Regular Languages and finite-state Regular Languages and finite-state automataautomata

Regular Expressions and regular languages

Deterministic FSA Non-deterministic FSA Finite-state automata with epsilon moves Generative grammars Context-free, Context-sensitive languages Chomsky Hierarchy

Regular Expressions and Regular Regular Expressions and Regular LanguagesLanguages

Characterizing formal Characterizing formal languageslanguagesPlain words: the language of all and

only those words over ={a,b} of length 2 (aa,bb,ab,ba,bb)

Set abstraction: {w|w* and |w| = 2}

New way: regular expressions denote languages

Regular Expressions Denote Regular Expressions Denote Languages Languages a* denotes language {an|n 0}(a).(b) or just ab denotes unit

language {ab}a*b* denotes {anbm|n,m 0}a2b3 denotes {aabbb}an not regular expressiona+bb denotes {anbb|n 1}a?bb denotes {anbb| 0 n 1}a|b denotes {a,b}

ExerciseExercisea* denotes language {an|n 0}(a).(b) or just ab denotes unit

language {ab}a*b* denotes {anbm|n, m 0}a2b3 denotes {aabbb}an not regular expressiona+bb denotes {anbb|n 1}a?bb denotes {anbb| 0 n 1}a|b denotes {a,b}

What does ((a.b)*)|((b.a)*) mean? Give a few examples

ExerciseExercisea* denotes language {an|n 0}(a).(b) or just ab denotes unit

language {ab}a*b* denotes {anbm|n, m 0}a2b3 denotes {aabbb}an not regular expressiona+bb denotes {anbb|n 1}a?bb denotes {anbb| 0 n 1}a|b denotes {a,b}

What does ((a.b)*)|((b.a)*) mean? Give a few examples

The language containing all and only even-length words consisting of alternating a’s and b’s {,ab,ba,abab,baba,…}

Kleene-Closure Operator Kleene-Closure Operator (recap)(recap)

Symbol *: certain unary operation on languages

Given language LL* = def {w| for some n 0, w is the

concatenation of n words of L}L*: is the result of concatenating 0 or more

words of L

Language forming operations Language forming operations (recap)(recap)

Binary concatenation operation: .L1.L2 = def. {w1w2|w1 L1 & w2 L2}

The language that results from taking a word from L1 and appending to it a word from L2

Definition - Regular Definition - Regular ExpressionExpression

(i) is regular expression (over ) and denotes language

(ii) is regular expression (over ) and denotes language {}

(iii) If s is in * then s itself is a regular expression and denotes language {s}

(iv) Suppose s and r are regular expressions that denote languages Lr and Ls, then

(a) (r|s) is a regular expression that denotes LrLs

(b) (r.s) is a regular expression that denotes Lr.Ls

(c) (r*) is a regular expression that denotes (Lr)*

(v) No expression is a regular expression unless it is obtainable from (i) – (iv)

DefinitionDefinitionWhat about (r+)

What about (r?)

DefinitionDefinition

What about (r+) = ((r*).r)

What about (r?)= (|r)

NotationNotation Usually forget about parentheses:

(ab) = ab (a|b|c): 3-word language {a,b,c} parentheses > superscript > concatenation

> alternationab* = a(b*)

(ab)*a|ba = (a|(b.a))

((a|b).a)

Regular LanguagesRegular LanguagesLet L be a language over

alphabet , i.e., L *. Then L is said to be a regular language if L is denoted by some regular expression over

Let be a finite alphabet and L1 and L2 regular languages over . Then L1 L2, L1.L2, and L1

* are also regular languages

RemarksRemarksif is a finite alphabet and w is

any word over *. Then unit language {w} is regular.

if is a finite alphabet. Then any finite language over is regular.

Deterministic Finite State Deterministic Finite State AutomataAutomata

Finite State AutomataFinite State Automata

New model of computation: analysis of the kind of computation that requires a fixed (finite) amount of memory for arbitrary input

Also called finite-state machines

Deterministic Finite-State Deterministic Finite-State AutomataAutomata

q0

q2

q1

ab

a b

b a

Figure 9.2.1(a)

= {a,b}• Vertices and arcs• Labels of arcs are

members of • No tapes, but input• Input: (possibly

empty) word over e.g. abb

Deterministic Finite-State Deterministic Finite-State AutomataAutomata

q0

q2

q1

ab

a b

b a

Figure 9.2.1(a)

• Accepting configuration: FSA halts in state q1

• The FSA accepts word abb• e.g. aba

• q2: trap state

• L = {abn|n0}

DeterminismDeterminismFor each state/symbol pair, FSA M

has exactly one instructionFSA M has at least one instruction.

This makes M fully definedDeterminism means that, within

any state diagram for FSA, the path labeled by given word w is unique: for word w *, there is exactly one path starting at q0 and labeled by w

ExerciseExercise0 1 2 3

4

a a

b

a

a

abbb

Figure 9.2.1(b)

b

start

• Which regular language is accepted by this FSA?• What are the accepting states?• Is an accepted word?• What is the trap state?• Is the trap state a sink?• Is the language finite?

ExercisExercisee

0 1 2 3

4

a a

b

a

a

abbb

Figure 9.2.1(b)

b

start

• Which regular language is accepted by this FSA? a(a|b)a?

• What are the accepting states? q2 and q3

• Is an accepted word? no• What is the trap state? q4

• Is the language finite? yes

ExerciseExercise

3

1 20a b

a

ab

b ba

Figure 9.2.1(c)

• Which regular language is accepted by this FSA?

• What are the accepting states?• Is an accepted word?• What is the trap state?• Is the language finite?

ExerciseExercise

3

1 20a b

a

ab

b ba

Figure 9.2.1(c)

• Which regular language is accepted by this FSA? (aba)*

• What are the accepting states? q0

• Is an accepted word? yes• What is the trap state? q3

• Is the language finite? no

Alternate descriptionAlternate description

M(q0,a) = q1

M(q0,b) = q2

M(q1,b) = q1

M(q1,a) = q2

M(q2,a) = q2

M(q2,b) = q2

q0

q2

q1

ab

a b

b a

Figure 9.2.1(a)

Formal DefinitionFormal DefinitionA deterministic FSA is a quintuple ,Q,qinit,F, ◦ is the input alphabet◦Q is a finite, nonempty set of states ◦qinit Q is the initial state or start

state◦F Q is a (possibly empty) set of

accepting or terminal states◦: Q Q transition function

(total and single valued)

Word AcceptanceWord Acceptance

A deterministic finite-state automaton M accepts word w * if there is a unique path starting at qinit and labeled by w that leads to some member of F

Language AcceptanceLanguage AcceptanceThe language accepted by M is

the set of all and only those words over that are accepted by M

L(M) for the language accepted by M.

FSAs are language acceptors only

Nondeterministic Finite State Nondeterministic Finite State AutomataAutomata

Non-determinismNon-determinismCf. Turing MachinesExistence of alternative

instructions for a given state/symbol pair

A Nondeterministic A Nondeterministic MachineMachine

q1q2

q3

q4

b

a

a

b

Figure 9.3.3

b

q0

b

a b

q0 q1

q2

a

b

b a

a b

L = (ab)*

L = (ab)* {a}

= (ab)*|a

Non-determinismNon-determinismNondeterministic FSA are usually

easier to design but run the risk of accepting unintended words

: QQ is a transition mapping Assumed to be total but permitted

to be multi-valuedCf. difference between function and

mapping!

Formal DefinitionFormal DefinitionA nondeterministic FSA is a quintuple ,Q, qinit,F,◦ is the input alphabet◦Q is a finite, nonempty set of states ◦qinit Q is the initial state or start state◦F Q is a (possibly empty) set of accepting

or terminal states◦: Q Q transition mapping (total and

possibly multi-valued)

Word AcceptanceWord AcceptanceWord w * is accepted by FSA M

provided there exists some path, labeled by w, in the state diagram of M leading from qinit to a terminal state

Cf. deterministic definition of word acceptance: unique path

Language AcceptanceLanguage AcceptanceThe language accepted by a

nondeterministic FSA is the set of words accepted by M.

Nondeterminism Nondeterminism determinismdeterminism

Nondeterministic FSA are easier to design

For every nondeterministic FSA, there exists an equivalent deterministic FSA

We can automatically convert the nondeterministic FSA to an equivalent deterministic FSA through subset construction

Finite-state automata with epsilon Finite-state automata with epsilon movesmoves

Epsilon movesEpsilon movesExecuting arcs labeled do not

advance input-arcs may or may not introduce

nondeterminism

ExampleExample

(a*b*c*)

3

210

a

a a

a

b

b

b

b

c

c

c

c

Equivalence ResultEquivalence ResultLet M be FSA with -moves. Then

there exists a FSA M´ with no -moves such that L(M) = L(M´)

Non-determinismNon-determinism

-moves do not necessarily imply nondeterminism

210 a

Regular languagesRegular languages

The family of regular languages is identical to the family of FSA-acceptable languages

!!!!!!!!!!!!!!!!!

Generative GrammarsGenerative Grammars

Generative GrammarsGenerative GrammarsAlternative characterization of the family of

(regular) languagesExample with just 2 productions

(1) S aSb(2) S

Generates all words of form anbn for n 0e.g. aaabbb

S aSb (1)aaSbb (1)aaaSbbb (1)aaabbb (2)

DefinitionDefinitionempty productionsgrammar terminals (usually

lowercase)terminal alphabet grammar non-terminals (usually

uppercase)Non-terminal alphabet start symbol S in production set

Second ExampleSecond Example

(1) S aaXcc (2) X aXc (3) X b

Generates all words of form anbcn for n 2e.g. aaaabcccc

S aaXcc (1)aaaXccc (2)aaaaXcccc (2)aaaabcccc (3)

Third ExampleThird Example(1) S aS´bc(2) S (3) S´ aS´bC(4) S´ (5) Cb bC(6) Cc cc

Generates language {anbncn|n 0} e.g. aaabbbcccS aS’bc (1)

aaS’bCbc (2)aaaS’bCbCbc (2)aaabCbCbc (4)aaabbCCbc (5)aaabbCbCc (5)aaabbbCCc (5)aaabbbCcc (6)aaabbbccc (6)

EquivalenceEquivalenceTwo generative grammars G and G´

are said to be equivalent if L(G) = L(G´).

Right-linear grammarsRight-linear grammarsA generative grammar where the

production rules are either of the form

X wYor X wwhere X,Y are nonterminals and w is a (possibly empty) word

Equivalence resultEquivalence resultIf L is generated by a right-linear

grammar, then L is regular

A language generated by a right-linear grammar (i.e. a regular language) can always be accepted by a FSA

Context-free Context-free LanguagesLanguages

There are languages that can be generated by a context-free grammar that are not regulare.g (1) S aSb

(2) S

L = {anbn|n 0}Context-free grammars have a single non-terminal

on the left-hand side of the productionContext-free languages: the class of languages

that can be generated by some context-free grammar

Context-sensitive Context-sensitive languageslanguages

e.g. (1) S aS´bc(2) S (3) S´ aS´bC(4) S´ (5) Cb bC(6) Cc cc

There are languages that cannot be generated by a context-free grammar

Language L is a context-sensitive language if there exists a context-sensitive grammar G such that either L = L(G) or L = L(G) {}

Context-sensitive languages: the class of languages that can be generated by some context-sensitive grammar

An equivalence resultAn equivalence result

Any context-sensitive language is Turing-recognizable but not vice-versa

There exists Turing recognizable languages that are not context-sensitive: recursively enumerable languages

The Chomsky HierarchyThe Chomsky Hierarchy

Regular

languages

Context-

sensitive

languages

Context-free

languages

Recursively

enumerable

languages

Accepted by deterministic FSAG: Right-linear grammars

Accepted by linear-bounded automatonG: context-sensitive grammars

Accepted by nondeterministic push-down stack automatonG: Context-Free Grammars

Accepted by deterministic 1tape Turing Machine