Decidability (What, stuff is unsolvable?)cobweb.cs.uga.edu/~potter/theory/5_decidability.pdf ·...

Decidability (What, stuff is unsolvable?)

CSCI 2670

University of Georgia

Fall 2014

CSCI 2670 Decidability (What, stuff is unsolvable?)

Outline

I Decidability

I Decidable Problems for Regular Languages

I Decidable Problems for Context Free Languages

I The Halting Problem

I Countable and Uncountable Sets

I Diagonalization

I Unrecognizable Problems


Turing Machines

Definition

A Turing machine is a 7-tuple (Q,Σ, Γ, δ, q0, qaccept , qreject) such that

1. Q is a finite set of states;

2. Σ is the input alphabet, and t /∈ Σ;

3. Γ is the tape alphabet, where t ∈ Γ and Σ ⊆ Γ;

4. δ : (Q × Γ)→ (Q × Γ× {L,R}) is the transition function;

5. q0 ∈ Q is the start state;

6. qaccept ∈ Q is the accept state;

7. qreject ∈ Q is the reject state, and qreject 6= qaccept .


Configurations

Definition

The state, tape contents, and tape head position constitute a configuration.An accepting configuration (rejecting configuration) is one in which themachine is in state qaccept (qreject). The start configuration has the tape headin the leftmost position, and the input string is written on it.

I A string 01010q7010111 may be used to indicate a configuration.

I The transition function maps one configuration to another.

Definition

A Turing Machine M accepts a string w if there exists a sequence ofconfigurations C1, C2, . . .Cn such that

1. C1 is the starting configuration of M on input w ,

2. Cn is an accepting configuration,

3. for each 1 ≤ i < n, Ci yields Ci+1.

L(M), language of M is the set of strings accepted by M.


Turing Recognizable and Decidable Languages

Definition

A language L is Turing–recognizable (recursively enumerable) if thereis a Turing machine M such that L(M) = L.

I It is possible for a TM to never reach a halting configuration. On anygiven input w , it might instead loop.

I These are the three options for any input: accept, reject, loop.

I A TM M that halts on every input is called a decider.

I If L(M) = L, then M is said to decide L.

Definition

A language L is Turing–decidable (also called recursive) if there is aTuring machine M that decides L.


Decidability and Undecidability

I The definitions indicate that not all languages are decidable.

I The Church-Turing Thesis equates “algorithmically solvable” withsolvable by Turing machines.

I A yes/no problem is algorithmically decidable if there is some Turingmachine that will decide it.

I Given any instance of the problem (input), the machine will always halt infinite time with the correct answer (yes or no).

I A problem is not algorithmically solvable if every TM loops on at least oneinput.

I Chapter 4 discusses problems that are solvable/unsolvable by TMs.

I Knowing which problems are unsolvable is important (and pretty cool).

I Why try to write a program to solve a problem when someone’s alreadytold you it can’t be done.



“I can’t find an ... algorithm, I guess I’m just too lame.”

I source: MR Garey and DS Johnson. Computers and Intractability. W. H. Freeman, 1970.

I image: http://max.cs.kzoo.edu/~kschultz/CS510/ClassPresentations/NPCartoons.html


http://max.cs.kzoo.edu/~kschultz/CS510/ClassPresentations/NPCartoons.html


“I can’t find an ... algorithm, because no such algorithm is possible.”

I source: MR Garey and DS Johnson. Computers and Intractability. W. H. Freeman, 1970.

I image: http://max.cs.kzoo.edu/~kschultz/CS510/ClassPresentations/NPCartoons.html


http://max.cs.kzoo.edu/~kschultz/CS510/ClassPresentations/NPCartoons.html

Decidable Problems for Regular Languages

The following languages/problems are all decidable.

I ADFA = {〈B,w〉|B is a DFA and w ∈ L(B)}I For any DFA B, L(B) is decidable.

I ANFA = {〈B,w〉|B is an NFA and w ∈ L(B)}I AREX = {〈R,w〉|R is a regular expression that generates w}I EDFA = {〈B〉|B is a DFA and L(B) = ∅}I EQDFA = {〈A,B〉|A,B are DFAs and L(A) = L(B)}

I In each case, the input is a string representing, for instance, a DFA andan input string w .

I We design a TM to decide the problem (typically by simulating themachines described in the input).

I A reasonable representation of the DFA is needed. E.g.,

q0, . . . , qn#a1, . . . , am#(q0, a0, qi ), . . .#q0#qi , qj , . . .#w0 . . .wk



Theorem

ADFA is decidable, where ADFA = {〈B,w〉|B is a DFA and w ∈ L(B)}



Theorem

ADFA is decidable, where ADFA = {〈B,w〉|B is a DFA and w ∈ L(B)}

(Proof idea)

I We construct a Turing machine M to decide the problem.

I Scan the input string 〈B,w〉, determining whether the inputconstitutes a valid DFA and string w . If not, then reject.

I Scan the input string repeatedly.

I Keep track of the current DFA state and position of w .I Find the appropriate transition to update the state

appropriately.

I After processing wk , check whether B is in an accept state.

I If so, then accept 〈B,w〉. Otherwise reject 〈B,w〉.



Theorem

If B is a DFA, then L(B) is decidable.

I This is clearly true, as every DFA halts on every input.

I We can easily create a TM MB to recognize L(B) by modifying thetransition function of B to fit the form for TMs.

I If δB(q, a) = q′, then δMB (q, a) = (q′, a,R).

Alternatively, we can use M from the previous proof in MB .

I On input w , run M on 〈B,w〉.I If M accepts 〈B,w〉, accept w .

I Otherwise reject w .



Theorem

ANFA is decidable, where ANFA = {〈B,w〉|B is an NFA and w ∈ L(B)}



Theorem

ANFA is decidable, where ANFA = {〈B,w〉|B is an NFA and w ∈ L(B)}

(Proof idea)

I We construct another Turing machine N to decide ANFA:

I Scan the input string 〈B,w〉, determining whether the inputconstitutes a valid NFA and string w . If not, then reject.

I Convert B to a DFA C (using the construction described in Ch 1).

I Run M on 〈C ,w〉.I Accept 〈B,w〉 if M accepts 〈C ,w〉. Otherwise reject 〈B,w〉.



Theorem

AREX is decidable, where

AREX = {〈R,w〉|R is a regular expression that generates w}



Theorem

AREX is decidable, where

AREX = {〈R,w〉|R is a regular expression that generates w}

(Proof idea)

I We construct another Turing machine P to decide AREX :

I Scan the input string 〈R,w〉, determining whether the inputconstitutes a valid reg-exp and string w . If not, then reject.

I Convert R to an NFA A (using the construction described in Ch 1).

I Run N on 〈A,w〉.I Accept 〈R,w〉 if N accepts 〈A,w〉. Otherwise reject 〈R,w〉.



Theorem

EDFA is decidable, where

EDFA = {〈A〉|A is a DFA and L(A) = ∅}

Idea: Check whether an accept state is reachable from A’s start state.



Theorem

EDFA is decidable, where

EDFA = {〈A〉|A is a DFA and L(A) = ∅}

Idea: Check whether an accept state is reachable from A’s start state.

(Proof idea)

I We construct Turing machine T to decide EDFA:

I Scan the input string 〈A〉, determining whether the input constitutesa valid DFA. If not, then reject.

I We assume states of A are listed left to right, with q0 being leftmost.

I Mark q0. Using the transition function of A, mark any unmarkedstate q′ such that δ(q, a) = q′ and q is already marked.

I Continue until no new nodes can be marked.

I If an accept state is marked, then reject 〈A〉. Otherwise accept it.



Theorem

EQDFA is decidable, where EQDFA = {〈A,B〉|A,B are DFAs and L(A) = L(B)}

I Regular languages are closed under complementation, union, intersection.

I Given A and B, we can construct a DFA for the symmetric difference:

Diff = (L(A) ∩ L(B)) ∪ (L(B) ∩ L(A))

I If Diff is nonempty, then L(A) 6= L(B).



Theorem

EQDFA is decidable, where EQDFA = {〈A,B〉|A,B are DFAs and L(A) = L(B)}

I Regular languages are closed under complementation, union, intersection.

I Given A and B, we can construct a DFA for the symmetric difference:

Diff = (L(A) ∩ L(B)) ∪ (L(B) ∩ L(A))

I If Diff is nonempty, then L(A) 6= L(B).

(Proof idea)

I We construct a Turing machine F to decide EQDFA:

I Scan the input string 〈A,B〉 determining whether the input constitutestwo valid DFAs A and B. If not, then reject.

I Construct a string representing the DFA C , where L(C) is the symmetricdifference of L(A) and L(B).

I Run Turing Machine T on 〈C〉. (Recall that T decides EDFA.)

I If T accepts 〈C〉, accept 〈A,B〉. Otherwise reject it.



Answer all questions for the following DFA M and give reasons for youranswers.

1. Is 〈M, 0110〉 ∈ ADFA?

2. Is 〈M, 001〉 ∈ ADFA?

3. Is 〈M〉 ∈ ADFA?

4. Is 〈M, 0110〉 ∈ AREX?

5. Is 〈M〉 ∈ EDFA?

6. Is 〈M,M〉 ∈ EQDFA?


Decidable Problems for Context Free Languages

The following languages/problems are decidable.

I ACFG = {〈G ,w〉|G is a CFG generating w}I ECFG = {〈G 〉|G is a CFG and L(A) = ∅}I If G is a CFG, then L(G ) is decidable.

The following language/problem is NOT decidable.

I EQCFG = {〈G ,H〉|G ,H are CFGs and L(G ) = L(H)}

Why won’t using (L(G ) ∩ L(H)) ∪ (L(H) ∩ L(G )) work in this case?


An Undecidable Problem for Context Free Languages

The following language/problem is NOT decidable.

I EQCFG = {〈G ,H〉|G ,H are CFGs and L(G ) = L(H)}

Context free grammars are not closed under complementation orintersection, and so we cannot use

(L(G ) ∩ L(H)) ∪ (L(H) ∩ L(G ))

as was done for EQDFA.



Theorem

ACFG is decidable, where ACFG = {〈G ,w〉|G is a CFG generating w}.



Theorem

ACFG is decidable, where ACFG = {〈G ,w〉|G is a CFG generating w}.

Idea: If G is in CNF, then it takes at most 2n − 1 steps to generate w .

(Proof idea)

I We construct a Turing machine S to decide the problem.

I Scan the input string 〈G ,w〉, determining whether the inputconstitutes a valid CFG and string w . If not, then reject.

I Convert G to CNF, using the procedure described in Ch 2.

I If w = ε, look for grammar rule S → ε. If present, then accept.Otherwise reject.

I If w 6= ε, systematically generate all derivations of at most 2n − 1steps, where |w | = n.

I If any of these derivations produce w , then accept. If not, reject.



Theorem

ECFG is decidable, where ECFG = {〈G 〉|G is a CFG and L(G ) = ∅}.



Theorem


Idea: Work backwards from terminal symbols to start symbol S .

(Proof idea)

I We construct a Turing machine R to decide the problem.

I Scan the input string 〈G 〉, determining whether the input constitutesa valid CFG. If not, then reject.

I Convert G to Chomsky Normal Form, as described in Ch 2.

I For each rule A→ a, mark every occurrence of A in the rules.

I For each rule of the form A→ BC , if both B and C are marked,then mark every occurrence of A in the rules.

I Continue until no new variables are marked.

I If S is NOT marked, then accept 〈G 〉. Otherwise Reject.



Theorem


An easier alternative.

(Proof idea)

I We construct a Turing machine R to decide the problem.

I Scan the input string 〈G 〉, determining whether the input constitutesa valid CFG. If not, then reject.

I Mark every terminal in every rule.

I For each rule of the form A→ w , if every symbol of w is marked,mark every occurrence of A in the rules.

I Continue until no new variables are marked.

I If S is NOT marked, then accept 〈G 〉. Otherwise Reject.



Theorem

If G is a CFG, then L(G ) is decidable.



Theorem

If G is a CFG, then L(G ) is decidable.

We can use the machine S , which decides ACFG = {〈H,w〉|H is a CFGgenerating w}, to construct a TM MG to decide L(G ).

I On input w , run S on 〈G ,w〉.I If S accepts 〈G ,w〉, accept w .

I Otherwise reject w .


Closure Property

?Is it closed under Set of LanguagesOperations RL CFL T-Recog T-DeciUnion Y Y Y YConcatenation Y Y Y YStar Y Y Y YIntersection Y N Y YComplementation Y N N Y

I RL: Regular Languages; CFL: Context-Free Languages; T-Recog:Turing-Recognizable Languages; T-Deci: Turing Decidable Languages.

I The intersection of a CFL and a RL is a CFL.

I It is recommended by Ian Lindsey Berrigan.


Undecidable Problems

I There are problems/languages that are undecidable.

I In general, program verification (deciding whether a program runs“correctly”) is undecidable.

I The Halting Problem: The most famous undecidable problem in CS.

Theorem

ATM is undecidable, where

ATM = {〈M,w〉|M is a TM and M accepts w}.

I The proof of this is a proof by contradiction.


Undecidable Problems

I Before we get to the proof, let us first observe that ATM isTuring-recognizable.

I Thus this theorem shows that recognizers are more powerful thandeciders.

I The following TM U recognizes ATM .

U = “On input 〈M,w〉, where M is a TM and w is a string:

1. Simulate M on input w .

2. If M ever enters its accept state, accept; if M ever enters its reject state,reject.”

Note that this machine loops on 〈M,w〉 if M loops on w . Hence ATM

sometimes called the halting problem.


The Halting Problem

Theorem

ATM is undecidable, where

ATM = {〈M,w〉|M is a TM and M accepts w}.

Proof.

Suppose ATM is decidable. Then there’s a machine H deciding it.

I The input of H is a string 〈M,w〉.I M is a TM, and w is a string.

I H runs M on w .

I If M accepts w , then H stops and accepts.I If M does not accept w , then H stops and rejects.

Observe that there is no string on which H does not halt.


The Halting Problem

continued.

Given H, we construct a new Turing machine D:

I The input of D is a string 〈M〉.I D runs H on 〈M, 〈M〉〉.

I If H accepts 〈M, 〈M〉〉, then D rejects 〈M〉.I If H rejects 〈M, 〈M〉〉, then D accepts 〈M〉.

I Given the definition of H and D:

I On input 〈M〉,I D accepts 〈M〉 if M does not accept 〈M〉.I D rejects 〈M〉 if M accepts 〈M〉.


The Halting Problem

I Given the definition of H and D:

I On input 〈M〉,I D accepts 〈M〉 if M does not accept 〈M〉.I D rejects 〈M〉 if M accepts 〈M〉.

I On input 〈D〉, we then find

I D accepts 〈D〉 if D does not accept 〈D〉.I D rejects 〈D〉 if D accepts 〈D〉.

I Under the assumption that ATM is decidable, there is a machine Dwhich simultaneously accepts and rejects its own specification 〈D〉.

I This is a contradiction!

I And so the assumption that ATM is decidable is false.

I We conclude that ATM is undecidable.

�


Countability and Uncountability/Diagonalization

I Implicitly, the proof that ATM is undecidable relies on Cantor’sdiagonalization method.

I This is used to prove that a set is uncountable.

I Recall that a set S is countable if either it is finite or there is aone-to-one correspondence between elements of S and those of N.

I That is, there is a bijective function f from N to S .

I injective: For all a, b ∈ N, if a 6= b, then f (a) 6= f (b).I surjective: For all b ∈ S , there is an a ∈ N such that f (a) = b.

I Intuitively, S is countable if we can write the elements of S as a list,and each element of S appears exactly once.

I A set A is countable if either it is finite or it has the same size as N.

I The set of positive even integers is countable, as is the set of positiveodd numbers.

I The set of positive rational numbers is countable as well.


The set of finite strings are countable

The set of finite strings S over any finite alphabet Σ is countably infinite.

I Assume an ordering of symbols in A.

I The strings of S can be ordered as follows:

I For any n ≥ 0, all string of length n come before all those oflength n + 1.

I For any given n, arrange the strings of length n in according tothe ordering of the alphabet symbols (as in a dictionary).

I Crucially, for any n, only a finite number of strings have length n.

I As a corollary to this, the set of Turing machines is countablyinfinite.

I Each TM can be written as a finite string over a given alphabet.


The set of infinite strings are uncountable

I In contrast, the set of infinite strings S over a finite alphabet Σ isnot countable.

I We’ll prove this using Cantor’s diagonalization method, a type ofproof by contradiction. Assume Σ = {0, 1} as the alphabet.

I An infinite binary sequence is an unending sequence of 0s and 1s.

I Each element of S is an infinitely long string of 0s and 1s.

I E.g., 01010101011111010101010111000101 . . ..I We will refer to particular bits of a string: b1b2b3 . . ..

Theorem

The Set B of infinitely long strings over Σ = {0, 1} is uncountable.



Theorem

The Set B of infinite strings over Σ = {0, 1} is uncountable.

Proof.

I Assume B is countable.

I As such, the elements of B can be listed as a sequence s1, s2, . . .(there’s a first element, a second element, a third ...).

I Every element of B appears exactly once in the sequence.

I s1 = b1,1b1,2b1,3b1,4b1,5 . . .I s2 = b2,1b2,2b2,3b2,4b2,5 . . .I s3 = b3,1b3,2b3,3b3,4b3,5 . . .

I...

I Above, bi,j refers to the jth bit of the ith element of B.



Theorem


Proof, continued.

I We construct an infinite bitstring X = c1c2c3 . . . by examining thediagonal:

I s1 = b1,1b1,2b1,3b1,4b1,5 . . .I s2 = b2,1b2,2b2,3b2,4b2,5 . . .I s3 = b3,1b3,2b3,3b3,4b3,5 . . .

I...

I For each i ≥ 1, if bi,i = 1, then ci = 0. Otherwise ci = 1.

I For instance:

If b1,1b2,2b3,3b4,4b5,5 . . . = 01101 . . .Then c1c2c3c4c5 . . . = 10010 . . .

I Observe that for any k , bk,k 6= ck .



Theorem


Proof, continued.

I We construct an infinite bitstring X = c1c2c3 . . .:

If b1,1b2,2b3,3b4,4b5,5 . . . = 01101 . . .Then c1c2c3c4c5 . . . = 10010 . . .

I Observe that X is an infinite bitstring and so must be some sn onthe list (it must appear somewhere on the list).

I And so bn,n (the nth bit of sn) is the same as cn.

I However, by construction of X , bn,n 6= cn.

I A contradiction!

I And so the assumption that B is countable must be false.

�


Characteristic Sequences

I For any finite Σ, Σ∗ is a set of finite strings and so is countable.

I That is, we can list out the strings of Σ∗: s1, s2, s3, . . ..

I Any language L over Σ can thus be represented as an infinitebitstring b1b2b3 . . ..

I si ∈ L if and only if bi = 1.I si /∈ L if and only if bi = 0.

I The string b1b2b3 . . . is called the characteristic sequence for L.

I It is clear that, for any Σ, there is a one-to-one correspondence betweenthe languages over Σ and the set of infinite bit-strings.

I Given the previous proof, the following may be inferred.

Theorem

For any Σ, the set of languages over Σ is uncountable.

Corollary

There exist languages that are not Turing Recognizable.


Back to the Halting Problem

I The set of Turing Machines is countable, and so we can list them.

I The set of finite strings over Σ is also countable.

I We then can construct the following table.

〈M1〉〈M2〉〈M3〉〈M4〉 . . .M1 1 0 1 1 . . .M2 0 1 1 0 . . .M3 0 1 0 1 . . .M4 1 0 1 1 . . .

......

......

... . . .

I A 1 in a cell i , j means that Mi accepts 〈Mj〉.I A 0 in a cell i , j means that Mi does not accept 〈Mj〉.I And so the table gives a partial description of H.



I Recall D: On input 〈Mi 〉,I D accepts 〈Mi 〉 if Mi does not accept 〈Mi 〉.I D rejects 〈Mi 〉 if Mi accepts 〈Mi 〉.

〈M1〉〈M2〉〈M3〉〈M4〉 . . .M1 1 0 1 1 . . .M2 0 1 1 0 . . .M3 0 1 0 1 . . .M4 1 0 1 1 . . .

......

......

... . . .

I So, the output of D can be found by flipping bits in the diagonal.



I Recall D: On input 〈Mi 〉,I D accepts 〈Mi 〉 if Mi does not accept 〈Mi 〉.I D rejects 〈Mi 〉 if Mi accepts 〈Mi 〉.

〈M1〉〈M2〉〈M3〉〈M4〉 . . . 〈D〉 . . .M1 1 0 1 1 . . . 1 . . .M2 0 1 1 0 . . . 0 . . .M3 0 1 0 1 . . . 1 . . .M4 1 0 1 1 . . . 1 . . .

......

......

... . . .... . . .

D 0 0 1 0 . . . ??? . . ....

......

...... . . . . . . . . .

I So, the output of D can be found by flipping bits in the diagonal.

I But D must be equal to some Mn and so appear on the list.

I At cell (n, n), the contents is both 1 and 0. A contradiction!


Unrecognizable Languages

Definition

Language L is co-Turing recognizable (co-recursively enumerable) ifit is the complement of a Turing recognizable language.

Theorem

L is Turing decidable if and only if L is recognizable and co-recognizable.

Proof.

I If L is decidable, then it is clearly recognizable. Also, if M is the machinedeciding L, we construct a new machine M ′ to decide L by simply doingthe opposite of M. So M ′ recognizes L. As such, L is co-recognizable.

I Now assume that L is recognizable and co-recognizable. Then L isrecognizable, and TMs M1 and M2 recognize L and L. We construct TMM deciding L. On input w , M runs M1 and M2 in parallel. If M1 acceptsw , then M accepts. If M2 accepts w , then M rejects. It is clear that itaccepts all strings of L and reject all those not in L.


Unrecognizable Languages

Theorem

L is Turing decidable if and only if L is recognizable and co-recognizable.

I ATM = {〈M,w〉|M is a TM and M accepts w} is undecidable.

I ATM is clearly recognizable, however. H runs M on w and accepts if andonly if M accepts w .

I From this, the below holds.

Theorem

ATM is not recognizable.

Proof.

If ATM is recognizable, then a machine M recognizes it. We canconstruct a machine N to decide ATM by running M and Hsimultaneously. On any input w , one of them must accept w . If Haccepts w , then N accepts. If M accepts, then N rejects. But ATM isnot decidable, and so no such machine M exists.


More Decidable Problems

Show the following languages/problems are decidable.

I EQDFA,REX = {〈A,R〉|A is a DFA, R is a regular expression andL(A) = L(R)}

I A = {〈M〉|M is a DFA which does not accept any string containingan odd number of 1s }

I Aε−CFG = {〈G 〉|G is a CFG that generates ε}




I S = {〈M〉|M is a DFA that accepts wR whenever it accepts w}I ALLDFA = {〈A〉|A is a DFA and L(A) = Σ∗}I A = {〈R,S〉|R and S are regular expressions and L(R) ⊆ L(S)}I A = {〈R〉|R is a regular expression describing a language containing

at least one string w that has 111 as a substring(i.e., w = x111y forsome x and y }




I INFINITEDFA = {〈A〉|A is a DFA and L(A) is an infinite language}I INFINITEPDA = {〈B〉|B is a PDA and L(B) is an infinite language}



Theorem

INFINITEDFA is decidable, where

INFINITEDFA = {〈A〉|A is a DFA and L(A) is an infinite language}.

Proof.

The following TM X decides INFINITEDFA.X = “On input 〈A〉 where A is a DFA:

1. Let k be the number of states of A.

2. Construct a DFA D that accepts all strings of length k or more.

3. construct a DFA M such that L(M) = L(A) ∩ L(D).

4. Test L(M) = ∅, using the EDFA decider T .

5. If T accepts, reject; if T rejects, accept.”



Theorem

INFINITEPDA is decidable, where

INFINITEPDA = {〈B〉|B is a PDA and L(B) is an infinite language}.

Proof.

The following TM Y decides INFINITEPDA.Y = “On input 〈B〉 where B is a PDA:

1. Convert B to a CFG G and compute G ’s pumping length p.

2. Construct a regular expression E that contains all strings of length por more.

3. construct a CFG H such that L(H) = L(G ) ∩ L(E ).

4. Test L(H) = ∅, using the ECFG decider R.

5. If R accepts, reject; if R rejects, accept.”


Decidability (What, stuff is unsolvable?)cobweb.cs.uga.edu/~potter/theory/5_decidability.pdf ·...

Documents

Transcript of Decidability (What, stuff is unsolvable?)cobweb.cs.uga.edu/~potter/theory/5_decidability.pdf ·...