MATHEMATICAL FOUNDATIONS OF COMPUTER SCIENCE COMPSCI 350

Fred Kroon – Philosophy

MATHEMATICAL FOUNDATIONS OF

COMPUTER SCIENCECOMPSCI 350


1. THE THEORY OF TURING MACHINES

Standard One-tape Turing MachinesDefinition 1. A Turing machine (TM) is a 7-tuple (Q, ,

, , qs, qaccept, qreject), where Q, , are finite sets, and 1) Q is the set of states of the machine,2) 2) is the input alphabet ( U, the blank symbol, ),3) is the tape alphabet (U , ),4) : Q x Q x x {L, R} is the machine’s transition

function,5) qs Q is the start state, usually taken to be q0 or q1,6) qaccept Q is the accept state, and7) qreject Q is the reject state (qs ≠ qaccept ≠ qreject).

This is the definition in Sipser’s text. Strictly speaking, Sipser takes to be a function from Q \ { qaccept, qreject } x to Q x x {L, R}, since he assumes that is not defined when the state is either qaccept or qreject .


The conventions for Turing machines are as follows. A Turing machine begins in the start state qs looking at the left-most symbol a of the input on a tape which is infinite to the right (apart from the input the tape is blank, ie contains only U’s). If the machine’s transition function specifies that ( qs, a) = (qj, b, R), the machine head then writes symbol b in the square it is scanning, and moves Right into state qj. The machine then continues to follow the other instructions in the same way.

Suppose, for example, that the machine head now detects a symbol g. And suppose that the machine’s transition function specifies that ( qj, g) = (qr, t, L).


In this case, the head writes t in the square (overwriting the previously written copy of g) and moves Left into state qr (here, of course, it meets the previously written b). (Note that we follow the convention that if the head is instructed to move left when the head is already on the leftmost square, it stays in the same place for that move and awaits the next instruction.) The machine halts when it enters either state qaccept or qreject . If it never enters those states, the computation continues for ever.


Definition 2. (a) Given a Turing machine M, a configuration (‘cf’, for short) for M is a (finite) string vqw, where q is a state of M, and where v and w are strings over the machine’s tape alphabet. [vw represents the current tape contents of the machine, in the sense that it gives the symbols on a left-most segment of the tape, with everything on the tape to the right of vw being blank; the machine is assumed to be in state q, scanning the first symbol of w.] We say that q is the state of the configuration, and the symbol to the right of q is the scanned symbol of the configuration [if w is the empty string, the scanned symbol is the blank symbol U].

Intuitively, one configuration yields another, given a machine M, if we get to the second by applying the transition function of M to the state and scanned symbol of the first. We now define this idea formally.


(b) If C and D are configurations of M, we say that

C yieldsM D (C ├M D) if one of the following holds (where v, w are strings * and a, b, c ):

1) C = vaqi bw and (qi, b)= ( qk, c, L), then D = vqkacw,

2) C= vqi bw and (qi, b) = ( qk, c, R), then D = vcqk w,

3) C = qi bw and (qi, b) = ( qk, c, L), then D = qk cw [can’t go left, so leave head where it is],

4) C = vqi and (qi, U) = ( qk, c, R), then D = vcqk,

5i) C = vaqi and (qi, U) = ( qk, b, L) and b U, then D = vqk ab,

ii) C = vaqi and (qi, U) = ( qk, U, L), then D = vqk a .


(c) Let C be a configuration.

1) C is the start configuration of M on input w if C = qsw.

2) C is an accepting configuration if the state of the configuration is qaccept.

3) C is a rejecting configuration if the state of the configuration is qreject.

4) C is a halting configuration if C is an accepting or rejecting configuration.


Definition 3 (a) A Turing machine M with input alphabet accepts w, where w *, if there is a sequence of configurations C1, ..., Ck such that C1 is the start configuration of M on w, each Ci yieldsM Ci+1 (0< i < k), and Ck is an accepting configuration.

(b) A Turing machine M with input alphabet rejects w, where w *, if there is a sequence of configurations C1, ..., Ck such that C1 is the start configuration of M on w, each Ci yieldsM Ci+1 (0< i < k), and Ck is a rejecting configuration.


Note that our TMs are deterministic devices, so no machine ever accepts and rejects the same string w.

(c) If M is a Turing machine with input alphabet , L(M) is the set of strings over accepted by M. If L is a language over , we say that M recognises L if L = L(M).

(d) A language L is Turing-recognisable if there is a Turing machine M such that L=L(M), ie such that M recognises L.

Examples: Let A1={0, 1}*; A2 = Ø; A3 = {} be languages over {0, 1}. Show them to be Turing-recognisable.


Definition 4 (a) A Turing machine M with input alphabet is a decider if, for all strings w over , M either accepts or rejects w.

(b) M decides language L if M is a decider and L=L(M).

(c) A language L is Turing-decidable if there is a Turing machine M such that M decides L.

Note that it follows immediately that any Turing-decidable language is also Turing-recognisable.


Examples: A1= {0,1}*, A2=, A3= {},

A4= {w {0,1}* | w contains at least one 0},

A5= {w {0,1}* | w contains exactly one 0},

A6= {wwR | w {0,1}*},

A7= {w {0,1}*| w A6}


Example 4: Consider the language A4 = { w | w {0, 1}* and w contains at least one 0}. Show that A4 is Turing-decidable, and that the construction is correct, ie, that the machine M you construct is such that A4 = L(M).

Proof: Consider machine M = (Q, {0, 1}, {0, 1, U}, , q1, qaccept, qreject), where Q ={q1, q2, qaccept, qreject}, and transition function is as follows.

[The accompanying notes help to explain the way the function behaves. Note that the transition function could also have been specified in a state diagram.]


(q1, U) = (qreject, U, R) (no 0 spotted before the end of string is reached; reject)

(q1, 0) = (qaccept, 0, R) (M has seen its first 0; that is enough, so accept)

(q1, 1) = (q1, 1, R) (keep going until a 0 is found)


We now show that the construction is correct by proving that A4 = L(M), ie that both A4 L(M) and L(M) A4.

To show that A4 L(M), let w=w1w2...wn and suppose that w A4. So w contains at least one 0. Let wk be the first 0 (1 k n). Consider the following cfs for M.

1. Let C1 = q1w1w2...wn, C2 = w1q1w2...wn, … , Ck = w1w2...q1wk… wn.

2. Let Ck+1 = w1w2...wkqaccept wk+1…wn. [Of course, if wk is wn then Ck+1 = w1w2...wkqaccept .]


It is obvious that Ci yieldsM Ci+1 for 1 i < k , since until the machine encounters the first 0, ie wk, it simply stays in q1 and moves right, and that is what is described in C1, C2, …, Ck. In addition, Ck yieldsM Ck+1 since (q1, wk) = (qaccept, wk, R) (wk is a 0).

Note also that C1 is the start configuration of M on w, and Ck+1 is an accepting configuration. It follows immediately from definition 3 that M accepts w, and hence that A4 L(M).


ii) To show that L(M) A4, consider w L(M). Let w = w1w2...wn; we allow this to cover the case of n = 0, ie w is the empty sequence ). Because w L(M), there is a sequence of cfs C1, C2, … , Ck+1 such that C1 is the start cf of M on w, each Ci yields Ci+1 (1 i ≤ k n), and Ck+1 is an accepting cf. We thus have C1 = q1w1w2...wn, …, Ci = w1 … q1wi ...wn, … , Ck+1 = w1 ...wk qaccept … . Clearly, the application of that yields Ck+1 from the preceding cf must have been (q1, 0) = (qaccept, 0, R) (since that is the only place qaccept occurs in ). Since Ck+1 = w1 ...wk qaccept … it follows that wk (the kth symbol of w) must be a 0. Hence w A4, so that L(M) A4.

Combining i) and ii), it follows that A4 = L(M).


2. VARIANTS OF TURING MACHINES

Multi-tape Turing Machines and Non-deterministic Turing Machines

Definition 6. (a) A multitape Turing machine is a 7-tuple (Q, , , , qs, qaccept, qreject), where Q, , , qs, qaccept, qreject are as before, and where : Q x k Q x k x {L, R}k is the machine’s transition function, where k is the number of tapes (1 k).


[The expression (qi, a1, …, ak) = (qj, b1, …, bk, L, R, …, L) means that if the machine is in state qi and the read-write heads on tapes 1 to k are reading symbols a1through ak , the machine goes to state qj, writes symbols b1 through bk, and moves each head to the Left or Right as specified.


Note that the definition given is that from the first edition of Sipser. The second edition gives a definition of multitape Turing machines that uses three moves: L, R and S, for stay-put. It is easy, of course, to show that having S makes no difference whatsoever, for a machine that only has L and R can always simulate the stay-put move S by first moving R, then straight away moving L. ]


(b) A nondeterministic Turing machine is a 7-tuple (Q, , , , qs, qaccept, qreject), where Q, , , qs, qaccept, qreject are as before, , and where : Q x (Q x x {L, R}) is the machine’s transition function.

[ (Q x x {L, R}) is the set of all subsets of (Q x x {L, R}). This means that the transition function for nondeterministic TMs can allow more than one choice: each of the members of (Q x x {L, R}) determines a possible next move.]


The notion of a configuration (cf) is the same as for single tape machines. The yield operation can be defined much as usual, except that a cf for a NTM can yield 0 or more cfs. We say that a NTM, N say, accepts w if there is a sequence C1, …, Ck of cfs for N such that i) C1 is the start cf on w, ii) Ci yields Ci+1 for all I, 0 < I < k, and iii) Ck is an accepting cf. We say that N rejects w if every sequence C1, C2, …, of cfs for N such that i) C1 is the start cf on w, and ii) Ci yields Ci+1 (for I > 0), is finite, and none terminates with an accepting cf.


Theorem 1.Every multitape TM M has an equivalent standard one-tape TM S, one that accepts a string w iff M accepts it and rejects w iff M rejects it.

Corollary: (a) A language is Turing-recognisable iff some multitapeTM recognises it.

(b) A language is Turing-decidable iff some multitape TM decides it.

Proof: The left to right direction for both (a) and (b) is immediate since any standard TM is a multitape TM. The other direction follows from the theorem.


Theorem 2.

Every NTM (nondeterministic Turing machine) N has an equivalent standard one-tape TM D, one that accepts a string w iff N accepts it and rejects w iff N rejects it.

Note (as in the case of multitape machines) that this is stronger than just saying that N and D are equivalent, ie, that N and D recognise the same language, for it implies that if N decides a language then D decides it as well.


Proof of Theorem 2.

Most of the proof is in the textbook (proof of Theorem 3.16). We view N’s computation on an input w as a tree, each node of which is a configuration and each branch of which represents a sequence of possible configurations ordered by the ‘yields’ operation.


Let b = maxqQ, s || q,s) || . We assign addresses to nodes of such a tree by means of strings over the aphabet {1, 2, …, b}. 132, for example, represents the node we arrive at by starting at the root of the tree, going to its first ‘child, then going to that node’s third child, then to that node’s second child. Each symbol in the string tells us which choice to make next when simulating a step in one branch in N’s nondeterministic computation.


Note that some of these addresses are invalid, since a symbol may not correspond to any choice (there may be far fewer than b choices available to go from one node to one of its children, and there may even be none, if the node is a halting configuration).


We describe a multitape Turing machine D that simulates N’s computation, accepting when N accepts, rejecting when N rejects, and not halting otherwise. Since every multitape Turing machine can be simulated by a standard single tape TM (by Theorem 1), the existence of this machine D would prove the theorem.


D has three tapes: tape 1 contains the input w and its contents are never changed, tape 2 is the working tape on which D simulates N’s computation on w, and tape 3, which contains an address over {1, 2, …, b}, keeps track of D’s location in N’s nondeterministic computation. Note that initially tape 1 contains input w, with tapes 2 and 3 empty.

D operates as follows:


» 1. Copy tape 1 to tape 2.» 2. On tape 3, write the first

string over alphabet {1, …, b} (using the lexicographic ordering of such strings).

» 3. On tape 2, simulate N on input w, following the branch of its nondeterministic computation given by the address on tape 3. Before each step of N, consult the next symbol on tape 3 to determine which choice to make from those allowed by N’s transition function.


» 3. (Cont.) Go to stage 4 if either no more symbols remain on tape 3 OR if this address is invalid OR if a rejecting configuration is encountered. If an accepting configuration is encountered, accept.

» 4. Replace the string on tape 3 with the next string in the lexicographic ordering of strings over {1, …, b} [Sipser p. 14].

» 5. If each of the branches of nondeterministic computation terminates with a rejecting configuration, then reject.


(Re: stage 5. Note that if each of the branches terminates with a cf that is not an accepting cf, each branch has finitely many nodes; it then follows that the entire tree has finitely many nodes, so that D can search N’s entire computation tree to determine that each branch terminates with a rejecting configuration.)


By construction, D accepts a string w iff N accepts it, so that D recognises the same language that N recognises. In addition, D rejects a string iff N rejects it.

Corollary: (a) A language is Turing-recognisable iff some NTM recognises it.

(b) A language is Turing-decidable iff some NTM decides it.

Proof: The left to right direction is immediate since any standard TM is an NTM. The other direction follows from the theorem.


3. CLOSURE THEOREMS FOR THE CLASS OF TURING-DECIDABLE LANGUAGES AND FOR THE CLASS OF TURING-RECOGNISABLE LANGUAGES.


Theorem 3 [First Closure Theorem]

The class of Turing-decidable languages over a fixed alphabet is closed under the operations of i) union, ii) intersection, iii) complement, iv) concatenation, and v) star.


Proof of Theorem 3:

i) Suppose A and B are Turing-decidable. Let M1 decide A and M2 decide B. Then M decides AB, where M is as follows:

M = “On input w,

1. Make a second copy of w. Run M1 on the first copy of w. If M1 accepts, accept. If M1 rejects, run M2 on the second copy of w. If M2 accepts, accept. Otherwise, reject.”

Clearly, M accepts w iff either M1 or M2 accepts w, ie, iff w is in AB.


ii) Suppose A and B are Turing-decidable. Let M1 decide A and M2 decide B. Then M decides AB, where M is as follows:

M = “On input w,

1. Make a second copy of w. Run M1 on the first copy of w. If M1 accepts, go to 2. If M1 rejects, reject.

2. Run M2 on the second copy of w. If M2 accepts, accepts. Otherwise, reject.”

Clearly, M accepts w iff both M1 and M2 accept w, ie, iff w is in AB.


iii) Suppose A is Turing-decidable. Let M decide A. Then MC decides the complement AC of A:

MC = “On input w,

1. Run M on w. If M accepts, reject. If M rejects, accept.”

Clearly, MC is a decider (since M is) and MC accepts w iff M rejects w iff w is in AC. Hence MC decides the complement AC of A.


iv) Suppose A and B are Turing-decidable. Let M1 decide A and M2 decide B. Then N, a NTM, decides AoB [the concatenation of A and B], where N is as follows:

M = “On input w,1. Nondeterministically split w into two parts, x and y. Go to 2.

2. Run M1 on the left part, x. If M1 accepts, go to 3. If M1 rejects, go to qreject.

3. Run M2 on the right part y. If M2 accepts, accept, ie, go to qaccept. If M2 rejects, go to qreject.”


Clearly N accepts w just so long as w is the concatenation of two strings x and y, the first of which is in A and the second in B, and N rejects w otherwise. Hence N decides AoB. According to the corollary to Theorem 2, a language is Turing-decidable iff some NTM decides it. Hence AoB is Turing decidable.


v) Suppose A and B are Turing-decidable. Let M decide A. Then N, a NTM, decides A*, where N is as follows:

N = “On input w,

1. If w = , go to qaccept. Otherwise, nondeterministically split A into k parts (0<k≤|w|). Go to 2.

2. Run M on each of these parts. If M accepts each of them, go into qaccept. If M rejects any of them, go into qreject.”


It is not hard to see that N accepts an input w just so long as w is the concatenation of k strings, 0≤k≤|w|, all of which are in A. Furthermore, N is a decider, so that, by the corollary to Theorem 2, A* is Turing-decidable.


Theorem 4 [Second Closure Theorem]

The class of Turing-recognisable languages over a fixed alphabet is closed under the operations of i) union, ii) intersection, iii) concatenation, and iv) star.


Proof of Theorem 4:

i) Suppose A and B are Turing-recogisable. Let M1 recognise A and M2 recognise B. Then M recognises AB, where M is as follows:

M = “On input w,

1. Make a second copy of w, and run M1 and M2 alternatively on the two copies of w, one step at a time. If either accepts, accept. If both machines reject, reject.”


Note: M doesn’t spend all its time first simulating M1 on w. It might never finish!

M = “On input w,

1. Make a second copy of w, and run M1 and M2 alternatively on the two copies of w, one step at a time. If either accepts, accept. If both machines reject, reject.”

Clearly, M accepts w iff either M1 or M2 accepts w, ie, iff w is in AB. Note that if neither M1 nor M2 accepts or rejects w, then M will loop.


ii) Suppose A and B are Turing- recognisable. Let M1 recognise A and M2 recognise B. Then M recognises AB, where M is as follows:

M = “On input w,

1. Make a second copy of w, and run M1 and M2 alternatively on the two copies of w, one step at a time. If both accept, accept. If either machine rejects, reject.”

Note that neither M1 nor M2 are guaranteed to be deciders, so this machine M may never halt. In any case, it is clear that M accepts w just in case both M1 and M2 accept w, ie w is in AB.


iii) Suppose A and B are Turing- recognisable. Let M1 recognise A and M2 recognise B. Then N, a NTM, recognises AoB [the concatenation of A and B], where N is as follows:

N = “On input w,

1. Nondeterministically split w into two parts, x and y. Go to 2.

2. Run M1 and M2 alternatively on the two parts x and y of w, one step at a time. If at any point both parts are accepted, accept, ie go into qaccept. If at any point one part is rejected, go into qreject.”


Once again, neither M1 nor M2 are guaranteed to be deciders, so this machine N may never halt. In any case, it is clear that N accepts w just in case w is the concatenation of two strings x and y such that x is in A and y is in B, ie such that w is in AoB.


iv) Suppose A is Turing-recognisable. Let M recognise A. Then N, a NTM, recognises A*, where N is as follows:

N = “On input w,


2. Run M alternatively on each of these parts, one step at a time. If at some point M has accepted each of them, accept, ie, go into qaccept. If M rejects any of them, go into qreject.”


N = “On input w,


2. Run M alternatively on each of these parts, one step at a time. If at some point M has accepted each of them, accept, ie, go into qaccept. If M rejects any of them, go into qreject.”

It is not hard to see that N accepts an input w just so long as w is the concatenation of k strings, 0 ≤ k ≤ |w|, each of which is accepted by M and hence are in A.


What happened to closure under complementation? Well, the proof of the closure of Turing-decidability can’t be adapted to the case of Turing-recognisability. (Our machines needn’t be deciders, so they might fail to accept certain inputs without rejecting them.) Indeed, it can be proved that the class of Turing-recognisable languages is not closed under complementation. This is a corollary of the immensely important fact that the Halting Problem for Turing machines is undecidable, to be proved later.


4.ENCODING TURING MACHINES, AND A UNIVERSAL TURING MACHINE

We are going to adopt certain conventions, without loss of generality. Assume that there are fixed infinite sets Q = {qaccept , qreject , q1, q2, q3, ...} and = {a1, a2, a3, ...}, such that for every Turing machine, the state set is a finite subset of Q and the tape alphabet (and hence also the input alphabet) is a finite subset of . We can take the blank symbol U to be a designated member of , say a1. In addition, let the start state of every machine be the state q1.


ENCODING TURING MACHINES, AND A UNIVERSAL TURING MACHINE

We first assign a code to each separate primitive symbol. Here is a way of doing it that assigns a different code to each state symbol and a different code to each alphabet symbol.

qaccept I L I

qreject II R II

qi Ii+2 ai Ii


We encode Turing machines by using the 2-symbol alphabet {c, 1}. Suppose M is a Turing machine (Q,, , , q1, qaccept , qreject), where Q Q and . Note that Q, , , , can all be written as sequences whose members are arranged in lexicographic order:

Eg, if Q = {q2, q7, q1, qaccept} we can write Q as the sequence (qaccept, q1, q2, q7). Similarly we can write as a sequence. Suppose (q1 , a2) = (q1, a5, R), and (q1 , a5) = (qreject, a5, R), and suppose there are no other parts to . We can write as ((q1 , a 2, q1, a5, R), (q1 , a5, qreject, a5, R)).


We can now define the code of a symbol, or sequence built up out of symbols, as follows:

If A is a state or alphabet symbol or a ‘move left/right’ symbol (L or R), we let the code for A, or < A> be g(A), where g is defined as above.

If A is a sequence (A1, .., Ak) we define the code of A as follows: <(A1, A2 .., Ak) > = c< A1>c< A2>c…c< Ak>c


(This encoding method described can also be used to encode other kinds of Turing machines, e.g., multitape Turing machines and nondeterministic Turing machines, as well as restricted kinds of automata such as DFAs and NFAs.)


We thus obtain a unique code <M> for each Turing machine M, and a unique code for each word w over some alphabet . Note that Sipser ch. 4 and beyond uses the notation <M, w>, where M is a TM and w a string over some finite alphabet. We can take this to be identical to <(M, w)> = c<M>c<w>c.


A UNIVERSAL TURING MACHINE

Theorem 5 There is a universal Turing machine (over some language containing at least 2 symbols) such that, presented with <M,w> as input, where M is a Turing machine and w a string over , halts in the accept state if M accepts w, halts in the reject state if M rejects w, and doesn’t halt if M doesn’t halt on w.


Proof: We construct a universal Turing machine that is actually a three-tape machine, but since we know (by Theorem 1) that every multitape machine can be simulated by a standard one-tape TM, we know that there is a standard TM that accepts what accepts, rejects what rejects, and doesn’t halt when doesn’t halt.

The machine uses its three tapes as follows. The first tape contains the encoding of what is on M’s tape, the second tape contains the encoding of M itself (ie, <M>) and the third tape contains the encoding of the state of M at the current point in the simulated computation.


The machine is started with some string <M,w> on its first tape and the other tapes blank. then first moves <M> onto the second tape, and moves <w> to the left of the first tape. ( can easily determine where <M> ends and <w> begins.) then copies the encoding < q1> of the start state q1 of M to the third tape.


first determines whether w is a string over the input alphabet of M. If it isn’t, it enters qreject. If it is, begins simulating the steps of the computation of M on w on the first tape. Between such simulated steps, keeps the heads on the second and third tapes at the left ends of those tapes, and the head on the first tape scanning the beginning of the encoded version of the symbol that M would be scanning at the corresponding time.


Suppose that the string on the third tape is < qi > and that the current head position on the first tape is at the beginning of a string < aj > that encodes the symbol aj. Suppose that now finds on its second tape a string that encodes (qi , aj) = (ql, ak, L/R). then changes the first tape accordingly, replacing < aj > with < ak >, and (depending on whether L or R is encoded) moving left or right to the preceding or next string encoding a symbol.


also puts the string < ql > on the third tape, checking to see whether this is the encoding of state qaccept or of qreject. If it is, halts in the appropriate state. If the third tape does not contain the encoding of a halting state, simulates another computational step of M.

By construction, accepts <M, w> if M accepts w; rejects <M, w> if M rejects w; and doesn’t halt on <M, w> if M doesn’t halt on w.


5. SOME CARDINALITY RESULTS (see Sipser section 4.2)

Definition 8 Given sets A and B, we say that(i) A is equivalent to B (or: have the same cardinality or the same size) iff there is a one-to-one, onto function f: A B. (We sometimes also call such a function a [one-one] correspondence [between A and B].)

(ii) A is finite iff either A is empty, or {1, …, n} is equivalent to A for some natural number n.

(iii) A is infinite iff A is not finite.


(iv) A is countable iff either A is finite or N is equivalent to A (where N = the set of natural numbers {1, 2, …}.

(v) A is countably infinite iff A is countable and infinite.

(vi) A is uncountably infinite iff A is infinite but not countable


Theorem 6 a) The set of all Turing machines with input alphabets is countably infinite;b) the set of all Turing-recognisable languages over alphabets is countably infinite.

Proof: a) Consider all encodings <M> of Turing machines. Replace ‘c’ by ‘2’ and ‘I’ by ‘1’. Then every such encoding <M> becomes a string of 1s and 2s, and so represents a unique natural number in decimal notation. Call this the number code of such a <M>.


Only some natural numbers will be number-codes of the relevant machines, of course. Let m1 be the smallest such number, m2 the next smallest such number, and so on. Then we can let M1 be the machine with number-code m1, M2 be the machine with number-code m2, and so on. The set of all such machines is obviously countably infinite (ie, the same size as N).


b) According to b). the set of all Turing-recognisable languages over alphabets is countably infinite. To prove this, consider the sequence of languages L(M1), L(M2), ..., where L(Mi), is the language recognised by Mi (the i-th machine in our listing of Turing machines). Removing languages from this sequence that have already occurred earlier in this sequence yields a non-repeating sequence L1, L2, ... of all and only Turing-recognisable languages. This shows that the set of all such languages is countably infinite.


Theorem 7. The class of all languages (over some alphabet ) is uncountably infinite.

Proof: Consider the set of all strings of length n. It is easy to prove, by induction, that there are ||||n of these (where |||| is the size or cardinality of ). We can now list all strings of length 0, then of length 1, then of length 2, etc, in standard lexicographic order (p. 14 Sipser). Let the listing of all strings over be w1, w2, w3, w4, .... Hence the class of all strings over is countably infinite.


Now suppose, contrary to what we are trying to establish, that the class of all languages (over ) is countably infinite. Then we have a (one-one) correspondence between {1, 2, 3, … } and the class of all such languages. For arbitrary n, let Ln be the language corresponding to n in this correspondence. We now construct a new language L as follows:

(*) For each i ≥ 1, put wi in L iff wi Li

(i.e., wi L iff wi Li


By construction, L is a language over and so L = Lk for some k 1. Then we can derive a contradiction as follows: Either wk Lk or wk Lk. If wk Lk, then since Lk = L it follows from (*) that wk Lk. But if wk Lk then, again by (*), wk L and so wk Lk after all (since L = Lk).

Hence the assumption that the class of all languages over is countably infinite results in a contradiction. It follows that the class of all languages over is not countably infinite. Since it is certainly infinite, it must therefore be uncountably infinite.


Corollary to Theorem 7: We already know from Theorem 6 that the class of all Turing-recognisable languages (over ) is only countably infinite. It immediately follows that there are languages over that are NOT Turing-recognisable.


6.DECIDABLE LANGUAGES (section 4.1 Sipser)

We focus on decidable problems concerning regular languages.

We shall represent computational decision problems as problems about languages. Consider first the problem of deciding whether a particular finite automaton accepts a given string. This problem can be represented as the problem of whether the following language is (Turing-) decidable:

ADFA = {<B,w> | B is a DFA that accepts input string w}


Theorem 8 (= Theorem 4.1 Sipser) ADFA is a decidable language.

Proof: Consider the following machine, for which we give an implementation-level description:

M = “On input v of the form <B,w>, where B is a DFA and w a string over the alphabet of B, go to 1. (If v is not of this form, reject.).

1. Simulate B on w.2. If the simulation ends in an accept state, accept the input <B,w>. If it ends in a non-accepting state, reject.”


By construction, given input <B,w>, where B is a DFA and w a string over the alphabet of B, M accepts the input if B, presented with w, terminates its processing of w in an accepting or final state, and rejects the input otherwise. Hence M is a decider for ADFA, and hence ADFA is a decidable language.


Theorem 9 (= Theorem 4.2 Sipser) ANFA = {<B,w> | B is an NFA that accepts input string w} is a decidable language.

Proof: Consider the following machine, which uses M above as a subroutine:

N = “On input <B,w>, where B is a DFA and w a string over the alphabet of B,

1. Convert NFA B to an equivalent DFA C, using a fixed effective conversion procedure (eg, the one given in Theorem 1.19 [Sipser edition 1] =Theorem 1.39 [edition 2]). 2. Run M on input <C,w>. 3. If M accepts, accept; otherwise, reject.”


By construction, given input <B,w>, where B is a NFA and w a string over the alphabet of B, N accepts the input if the equivalent DFA C, presented with w, terminates its processing of w in accepting state, and rejects the input otherwise. Hence N is a decider for ANFA, so that ANFA is a decidable language.


Theorem 10 (=Theorem 4.4 Sipser) EDFA = {<B> | B is a DFA & L(B) = } is decidable.

Proof: A DFA accepts some string iff it is possible to reach an accept state from the start state by travelling along the transition arrows of the DFA.

T= “On input <B>,

1. Mark the start state of B.

2. Mark any state that has a transition coming into it from a marked state. If no new states get marked this way, go to 3. Else, go to 2 [ie, repeat].

3. If no accept state is marked, accept. Otherwise, reject.”


Note that T accepts <B> if no string is accepted by B, i.e., L(B) = , and rejects B otherwise. Hence T decides EDFA.


Theorem 11 (=Theorem 4.5 Sipser) EQDFA = {<A,B,> | A, B are DFAs & L(A) = L(B)} is decidable.

Proof: Suppose we are given DFAs A and B. Note that L(A) = L(B) iff (L(A)c L(B)) (L(A) L(B)c) = (where L(A)c is the complement of L(A)). From the proofs of the closure of the class of regular languages under intersection, complement, and union, we can construct a DFA C which recognises (L(A)c L(B)) (L(A) L(B)c)). This construction can be carried out by a TM. Once we have such a C, we can test to see if L(A) = L(B) by testing to see if L(C) = , using the TM T of the previous theorem.


In short, E = “On input <A,B>, where A and B are DFAs:

1. Construct a DFA C such that L(C) = (L(A)c L(B)) (L(A) L(B)c).2. Run TM T from the previous theorem on input <C>.3. If T accepts (ie, if L(C) = ), then accept. If T rejects, reject.”

It is clear that E accepts <A,B>, where A and B are DFAs, iff L(A) = L(B), and rejects <A,B> otherwise. [We assume that E also rejects all inputs not of the form <A,B>, where A and B are DFAs. Hence E decides EQDFA, and EQDFA is therefore decidable.


7 UNDECIDABLE LANGUAGES (section 4.2, Sipser)

Remarkably, everything changes when we turn to decision problems involving Turing machines, for example whether an arbitrary Turing machine accepts a given string, whether it performs a given task, and so on. These problems turn out not to be decidable.


Thus consider the first problem. This problem can be represented as a problem about whether a certain language ATM is Turing-decidable or not, where ATM = {<M,w> | M is a TM and M accepts w}. ATM is the Turing machine version of ADFA, which we know to be decidable (Theorem 8). (We can assume that ATM is based on Turing machines over any possible alphabet, as in the encoding method we used. Nothing of any importance changes if we restrict the alphabet to a fixed alphabet.)


Theorem 12 ATM is Turing-recognisable.

Proof: Theorem 13 is an easy corollary of Theorem 5, the Universal Turing Machine Theorem. According to Theorem 5, there is a universal Turing machine which, presented with <M,w> as input, where M is a Turing machine and w a string over the alphabetof halts in the accept state if M accepts w, halts in the reject state if M rejects w, and doesn’t halt if M doesn’t halt on w. therefore recognises the language ATM.

We now show that ATM is Turing-undecidable.


Theorem 13 (= Theorem 4.11 Sipser) ATM is Turing-undecidable.

Proof: Assume ATM is Turing-decidable, contrary to what we are trying to prove. Suppose that H decides ATM. As a decider, H accepts <M,w> if M accepts w, and H rejects <M,w> if M does not accept w (it rejects all strings not of the form <M,w>). (We can represent H as follows:

accept if M accepts w

H(<M,w>) =

reject if M does not accept w.)


We now construct a new Turing machine with H as subroutine. This new machine, which we dub ‘D’, for ‘Diagonaliser’ (we will see why in a moment), calls subroutine H to determine what M does when M is given the string <M> as argument, ie its own encoding, and then does the very opposite to what M does.

D = “On input <M>, where M is a TM,» 1. Run H on input <M,<M>>.» 2. If H accepts this input (ie, M

accepts <M>), then reject. If H rejects this input (ie, M does not accept <M>), then accept.”


We can represent D as follows:

accept if M does not accept <M>

D(<M>) =

reject if M accepts <M>.

Now consider what happens if we use <D> as input to D:

accept if D does not accept <D>

D(<D>) =

reject if D accepts <D>.


Thus D accepts <D> iff D does not accept <D>. But this is a contradiction. Hence the assumption we began with, namely that ATM is Turing-decidable, is false. It follows that ATM is Turing-undecidable.

Note why this is called a proof by diagonalisation.


Consider first Theorem 7, which showed us that there are languages that are not Turing-recognisable. That construction proceeded as follows. Consider the 2-dimensional array

L1 L2 L3 …

w1 no yes no …

w2 yes yes yes …

w3 yes no no …… … … … …where the entry at row i and column j tells us whether wi Lj.


Language L was defined by ensuring that a string wi belongs to L just when the diagonal at (i, i) says No, and doesn’t belong when it says Yes. If L is one of the Li, say Lk, this means that wk L just when wk L, a contradiction.


Now suppose that whether a TM M accepts, rejects or fails to halt on its own code <M> is specified by the following 2-dimensional array.

<M1> <M2> <M3> <M4> …

M1 accept reject accept …

M2 reject accept accept …

M3 accept …

M4 accept accept reject …


Assuming H exists, the following figure then tells us the results of running H on input <Mi, <Mj>>:

<M1> <M2> <M3> <M4> …

M1 accept reject reject accept …

M2 reject reject accept accept …

M3 reject reject accept reject …

M4 accept accept reject reject …


Now consider D, which was deliberately made to behave differently on <M> from the way M behaved on < M > , as specified along the diagonal. D rejects <M> exactly when H accepts <M, <M>>, ie ie exactly when M accepts <M>. Hence if D is one of the machines in this array, it follows that D rejects <D> exactly when D accepts <D>, a contradiction.

Recall that this is just what happened earlier on with L.


A TURING-UNRECOGNISABLE LANGUAGE

We already know, by Theorem 7, that there are Turing-unrecognisable languages, but so far we don’t know that any of them are theoretically very interesting. We now show that there are are theoretically interesting Turing-unrecognisable languages. Call a language co-Turing-recognisable if it is the complement of a Turing-recognisable language.

Theorem 14 (=Theorem 4.22, Sipser) A language A is Turing-decidable iff it is Turing-recognisable and co- Turing-recognisable.


Proof: (a) : Suppose A is Turing-decidable. Then A is obviously Turing-recognisable (being decidable means that there is a decider that recognises the language). In addition, the complement Ac is also Turing-decidable (since the class of Turing-decidable languages is closed under complementation), so that Ac is also Turing-recognisable.


(b) : Suppose both A and the complement of A are Turing-recognisable. Let M1 recognise A and M2 the complement of A. Now consider Turing machine M, where

M = “On input w,

1. Run M1 and M2 on w in parallel. 2. If M1 accepts, accept. If M2 accepts, reject.”


M decides A. For either w is in A or in its complement, but not both. If the former, then since M1 recognises A, it accepts w. By construction, M then also accepts w. If the latter, then since M2 recognises the complement of A, M2 accepts w. By construction, M then rejects w. Hence M is a decider, one which accepts all strings in A and rejects all strings not in A. It follows that A is Turing-decidable.


Corollary: The complement of ATM is not Turing-recognisable (ie, ATM is not co-Turing-recognisable).

Proof: We know, by Theorem 12, that ATM is Turing-recognisable. If the complement of ATM is Turing-recognisable, then by Theorem 14 ATM is Turing-decidable. But by Theorem 13 ATM is Turing-undecidable. It follows that the complement of ATM is not Turing-recognisable.


We are now ready to tackle the Halting problem for Turing machines. The method we will use is the method of reduction. Remember how we showed that the problem of deciding whether L(A) = L(B), where A and B are DFAs, was solvable by reducing it to the problem of deciding whether the language of a DFA is empty (we already knew we had a solution to that problem). This kind of reduction showed that the first problem was no harder than the second: a solution to the second yields a solution to the first.


We now show that a problem like the Halting problem is not solvable by showing that the problem of deciding whether a TM M accepts a string w is no harder: if we had a solution to the Halting problem, we would have a solution to this problem as well.


Since we know that this latter problem is unsolvable (that is, we know that ATM is undecidable), it follows that the Halting problem is unsolvable as well.

Let HALTTM = {<M,w> | M is a TM that halts on input w}

Theorem 15 (=Theorem 5.1 Sipser) HALTTM is undecidable.

Proof: We show that ATM is reducible to HALTTM ; that is, that if we have a way of deciding HALTTM we also have a way of deciding ATM.


Suppose there is a TM R that decides HALTTM . We now use this machine to devise a TM S that decides ATM.

S = “On input <M,w>, an encoding of a TM M and a string w:

1. Run R on <M,w>.2. If R accepts, simulate M on w until it

halts; if M halts in an accepting configuration, accept, while if M halts in a rejecting configuration, reject.

3. If R rejects, reject.”


Clearly, if R decides HALTTM, then S decides ATM. But there can’t be such a TM S: ATM is undecidable. Hence there can’t be such a TM as R either. It follows that HALTTM must also be undecidable.

MATHEMATICAL FOUNDATIONS OF COMPUTER SCIENCE COMPSCI 350

Documents

Transcript of MATHEMATICAL FOUNDATIONS OF COMPUTER SCIENCE COMPSCI 350