Lecture 5 Dynamic Programming. Dynamic Programming Self-reducibility.
Chapter 6 Reducibility
Transcript of Chapter 6 Reducibility
Chapter 6
Reducibility
We demonstrated in the last chapter a direct
process of showing that ATM is not decidable,
thus unsolvable.
We will now use this result to dig out more
unsolvable problems, and particularly, we will
introduce reducibility, the primary technique of
showing a problem is not solvable. /
Reducibility always involves two problems, say
A and B. If A reduces to B, A ≤m B, then a
solution for B can be used to solve A. ,
Since B → A ≡ ¬A → ¬B, if A is not solvable,
nor will be B. ,
It says nothing about the actual solving of
problems A and B, but only deals with the
solvability of problem A, given a solution of
problem B.
1
Examples
If I made the statement that “If I have five
bucks, I will buy you lunch”, and, if you know
that I have five bucks, then you know, logically
speaking, your lunch is on me. ,
But, what do you know if I don’t buy you
lunch? ,
Leaves are on your yard.... To clean it up, you
only need to have a rake or a blower. Hence,
the problem of cleaning the yard reduces to the
problem of finding a tool.
If you can solve the latter problem, i.e., if you
have found a rake or a blower, then the original
problem, i.e., cleaning the yard can also be
solved.
What if you know for sure that you cannot
clean your yard. Can you find a rake or a
blower? ,
2
Back to the business
Reduction is a way of converting one problem
to another so that a solution to the latter can
be used to solve the former.
The insight is that if we know that the former
problem is unsolvable, so must be the latter.
The unsolvable problem we will use most of
the times is ATM, which is the set of 〈M, w〉
such that M accepts w.
We have proved in the last chapter that it is
not decidable, or solvable.
Let HALTTM be the problem that, for any
given M and w, if M halts on w.
We will prove that HALTTM is unsolvable, by
reducing ATM to HALTTM.
3
Informally speaking...
By the reduction idea, if HALTTM has a solu-
tion, then we can also solve the ATM problem,
which is a contradiction, since we know that
ATM is unsolvable.
The key is thus how to reduce ATM to HALTTM.
Assume that R is the TM that decides HALTTM,
i.e., for any given M and w, R will tell us if M
halts on w. We will construct another TM S,
that decides if M accepts w : For any given M
and w, we run R first, if R says M does not
halt on w, S rejects 〈M, w〉, since then there is
no way M will accept s. On the other hand, if
R says that M does halt on w, we then simply
run M on w, and accepts 〈M, w〉, if M accepts
w; rejects otherwise.
Therefore, any solution to the HALTTM prob-
lem indeed provides a solution for the ATM.
Hence, ATM reduces to HALTTM.
4
HALTTM is undecidable
Assume that R decides HALTTM. We construct
the following S that decides ATM.
S = “On input 〈M, w〉,1. Run R on input 〈M, w〉.2. If R rejects 〈M, w〉, reject 〈M, w〉.3. If R accepts 〈M, w〉, then we know that M
does halt on w. We thus run M on w until it
halts.
4. If M accepts w, accept 〈M, w〉; otherwise,
reject.”
But, as ATM is unsolvable, we reach a contra-
diction. Therefore, there could not exist such
a TM R, and HALTTM is not solvable. /
5
Another problem
Let ETM be {〈M〉 : M is a TM and L(M) = ∅}.
We saw earlier that both EFA and ECFL are de-
cidable (Cf. Pages 4 and 11 in the Decidability
notes).
Theorem ETM is undecidable.
Again, we are trying to reduce ATM to ETM.
Let R be the TM that decides ETM, we are try-
ing to construct another TM, S, that decides
ATM, with the help of R.
Given a TM M, R either accepts or rejects M .
One thing is easy, for any given M and w, if R
accepts M, then L(M) = ∅, namely, M accepts
nothing. In particular, M definitely will not
accept w. /
6
The other side...
If R rejects M, the only thing we know is thatL(M) is not empty, so, it accepts something.
Question: Does it accept w?
We have no clue. /
What we will do is to construct another TM,M1, such that the only string it might acceptis w,, i.e., L(M1) ⊆ {w}, depending on whetherM accepts w. We then run R on M1, instead.
If R accepts M1, L(M1) = ∅, thus, M doesn’taccept w. On the other hand, if R rejects M1,
then M accepts something, which must be w,
since this is the only string that could be ac-cepted. ,
We can thus solve ATM, which is a contra-diction. As a result, ETM cannot be solved,either.
Question: What does M1 look like?
7
The accessory
We construct M1 as follows: M1 = “On input
x
1. If x 6= w, reject;
2. otherwise, run M on w and accept w if M
does.
Thus, L(M1) ⊆ {w}, and the equality holds iff
M accepts w.
We are now ready for our final kick.
8
ETM cannot be solved...
When R rejects M1, L(M1) 6= ∅, then L(M1) =
{w}, i.e., M accepts w. On the other hand, if
R accepts M1 then L(M1) = ∅, which happens
when M does not accept, maybe rejects, w.
We thus can construct a TM, S, that decides
ATM, to accomplish a mission impossible. ,
S = “On input 〈M, w〉.1. Construct M1.
2. Run R on input 〈M1〉.3. If R accepts 〈M1〉, i.e., L(M1) = ∅, implying
L(M) = ∅; reject 〈M, w〉;4. otherwise, L(M1) = {w}, i.e., M accepts w;
thus accept 〈M, w〉.”
9
Yet another problem
Let RegularTM be {〈M〉 : M is a TM and L(M)
is regular}, where M is an FA, i.e., all the TMs
that accept a regular language.
Theorem RegularTM is undecidable.
Although an FA is a TM, there is no way we
can tell if a given TM is a FA or not. /
Assume that R decides RegularTM, we con-
struct a TM S that decides ATM.
The trick is to design an M2 so that L(M2) =
{0n1n|n ≥ 0} if M does not accept w and
L(M2) = Σ∗ if M accepts w.
Notice that {0n1n|n ≥ 0} is not regular, while
Σ∗ is. Then, for any given M and w, L(M2) is
a regular language if and only if M accepts w.
Hence, ATM reduces to RegularTM. ,
10
The details...
Let R decide RegularTM, and construct S that
decides ATM..
S = “On input 〈M, w〉 :
1. Construct the following M2
M2 = “On input x:
1.1. If x has the format 0n1n, accept.
1.2. Otherwise, run M on w and accept x if
M accepts w.
2. Run R on input 〈M2〉.3. If R accepts 〈M2〉, accept 〈M, w〉; otherwise,
reject 〈M, w〉.”
If M rejects w, then M2 will accept only those
strings in the form of 0n1n; i.e., L(M2) =
{0n1n|n ≥ 0}; which is not regular (Cf. Page 101
of the RL notes). But, if M does accept w,
M2 accepts everything, i.e., Σ∗, which is regu-
lar (Cf. Page 77 of the RL notes).
Therefore, S accepts 〈M, w〉 iff R accepts 〈M2〉iff L(M2) is regular iff M accepts w. ,
11
Want to have a look?
Let M2 be defined as discussed, the following
chart shows the construction of the machine S
that decides ATM.
Again, since ATM is not decidable, there is no
way to construct S, nor is R.
Hence, RegularTM is not decidable, either.
12
Not always ATM
So far, we always reduce ATM to the problem
that we are interested in. This is certainly not
necessary. ,
Question: Can we tell if two machines speak
the same language?
We saw that EQDFA is decidable (Cf. Page 6
of the decidability chapter), and it is stated on
Page 12 of the decidability chapter that EQCFL
is not decidable. For a detailed discussion of
the latter problem, check out the link on the
course page.
Let EQTM be {〈M1, M2〉 : L(M1) = L(M2)},
we reduce ETM to EQTM to show that EQTM
is not decidable, either. /
13
The reduction process
Assume R decide EQTM, we construct the fol-
lowing S to decide ETM.
S = “ On input 〈M〉 :
1. Run R on inputs 〈M, M1〉, where M1 rejects
everything, i.e., L(M1) = ∅.
2. If R accepts 〈M, M1〉, accepts 〈M〉; other-
wise rejects 〈M〉. ”
Thus, S decides ETM. ,
14
So what?
Hence S accepts M if and only if R accepts
〈M, M1〉 if and only if L(M) = L(M1) = ∅, i.e.,
S decides ETM, which contradicts the undecid-
ability of the emptiness problem of a Turing
machine. (Cf. Page 6)
Theorem EQTM is undecidable.
So, the whole structure looks like a tree, rooted
at the ATM problem.
This structure certainly can grow to include
more problems that are not decidable, starting
with the very beginning of ATM.
15
Computation history
The computation history method is another
important technique of showing that ATM re-
duces to other languages. It is often used to
test if something exists, when showing a prob-
lem is unsolvable.
Definition: Let M be a Turing machine and w
an input string. An accepting computation his-
tory for M on w is a sequence of configurations,
C1, . . . , Cl, where C1 is the start configuration
of M on w, Cl is an accepting configuration
of M, and each Ci legally follows from Ci−1
according to the rules of M.
Similarly, we can define rejecting computation
history.
Question: Did we talk about this stuff before?
Answer: Yes, we did on Page 13 of the Com-
putability notes, when addressing the yield re-
lation. ,
16
Linear bounded automata
A linear bounded automaton (LBA) is a re-
stricted type of Turing machine, in which the
tape head is not allowed to move off the por-
tion of the tape containing the input. If it tries
to, it will stay where it is.
A LBA comes with a quantitatively limited,
but qualitatively unrestricted, memory. Using
a tape alphabet larger than the input alpha-
bet allows the available space increased up to
a constant factor. Hence, for a given input of
length n, the amount of memory available is
linear in n.
As we discussed on Page 69 of the CFL notes,
CSL, the collection of context sensitive lan-
guages, corresponds to LBA.
17
LBA is powerful
Despite its constrained memory, LBA is quite
powerful. For example, the deciders for ADFA,
ACFG, EDFA, and ECFG are all LBAs.
Every CFL can also be decided by an LBA.
Given an CFG, Γ, we convert it to a Chomsky
normal form grammar, Γ′, first, as we discussed
on Page 18 of the CFL notes.
Then, for a word w, it only need to check all
the derivations in Γ′, with their length being
exactly 2|w| − 1, setting a limit as how many
cells we need to check.
We will accept w iff at least one of such deriva-
tions accepts w.
This again is the basis of a compiler of any
programming language (Cf. Page 10 of the De-
cidiability notes).
18
Let’s work on LBA a little
We have the following simple result.
Lemma: Let M (= (Q, Σ,Γ, δ, qaccept, qreject))
be an LBA with q states, i.e., |Q| = q, and
g symbols in its tape alphabet, i.e., |Γ| = g.
There are exactly qngn configurations of M for
a tape of length n.
Proof: Consider a configuration upiv. The state
pi can be any of q states, the R/W head points
to any of the n symbol, each of which can be
any of the g symbols.
Each such combination can be a configuration,
thus there are a total of qngn configurations.
We will now check the decidability of several
LBA related problems.
19
ALBA is decidable
Let ALBA be {〈M, w〉|M is an LBA that accepts
w.} We have the following
Theorem: ALBA is decidable.
If M goes into an infinite loop, it will get locked
up in that loop. On the other hand, the maxi-
mum length of an accepting computation his-
tory for M on w, by the lemma, is q|w|g|w|.
Hence, the following decider is to decide L.
L = “On input 〈M, w〉,
1. Simulate M on w for q|w|g|w| many steps or
until it halts.
2. If M has halted, accept if M accepts, and
reject otherwise. If M has not halted, reject.
The key here is the finiteness of the configu-
ration, which does not hold for a general TM.
20
ELBA is undecidable
Let ELBA be {〈M〉 : M is an LBA where L(M) =
∅.} We have the following
Theorem: ELBA is not decidable.
The proof is by reduction from ATM. For a
Turing machine M and w, we will construct an
LBA B that accepts all the accepting compu-
tation history of M on w, then testing if L(B)
is empty, using the assumed decider for ELBA.
If L(B) is empty, M does not accept w, other-
wise, it does. This leads to the contradiction
we need.
Below shows a possible tape content of B, the
length of which is proportional to M and |w|,
thus it is an LBA.
21
How does B work?
When it receives an input x, B is supposed to
accept x if x is an accepting computation for
M on w.
For a given 〈M〉 and w, B will designate C1 ≡
q0w as the initial configuration, where q0 is the
starting state of M, and it will accept a con-
figuration uqv if q is the accepting state of M.
For any given configuration Ci, it will gener-
ate a next configuration Ci+1 with the rules as
given in M.
For the specific construction and operation de-
tails, check out the proof of Theorem 5.10 in
the book.
The language of B, L(B), is the collection of
all the accepting computations of M on w.
22
Now what?
Clearly, L(B) 6= ∅ iff there is at least one ac-
cepting computation of M on w iff M accepts
w.
Now let R decide ELBA. Then the following S
decides ATM.
S = “On input 〈M, w〉,
1. Construct the LBA B, as described
2. Run R on 〈B〉.
3. If R rejects 〈B〉, accept (There exists at
least one accepting computation, thus M ac-
cepts w.); reject otherwise (There does not
exist any accepting computation, thus M does
not accept w /).
23
A simple problem?
We will show that some rather natural prob-
lem is also undecidable. Particularly, we will
show that the so-called Post Correspondence
Problem (PCP) is not decidable.
Given a collection of dominos, each of which
contains two strings, one on each side, an in-
stance of the PCP problem is a collection, P,
of dominos:{[
t1
b1
]
,
[
t2
b2
]
, . . . ,
[
tk
bk
]}
,
and a match is a sequence, i1, i2, . . . , il, of such
dominos in P, such that ti1ti2 · · · til = bi1bi2 · · · bil.
For example, given the following collection:{[
b
ca
]
,
[a
ab
]
,
[ca
a
]
,
[abc
c
]}
.
We could have a match:{[
a
ab
]
,
[b
ca
]
,
[ca
a
]
,
[a
ab
]
,
[abc
c
]}
.
24
Guess what?
Let PCP = {〈P 〉 : P is an instance of the Post
correspondence problem with a match.}.
Theorem: PCP is undecidable.
The proof is actually quite conceptually simple
but detailed with six parts. It basically shows
that, for any given Turing machine, M, and
string, w, we can construct an instance P of
PCP such that P has a match iff M accepts
w.
That is to say, we simply reduce ATM to PCP.
For the details of this proof, check out the
proof of Theorem 5.15 in the book.
25
What is really going on?
To solve a problem A, we reduce it to an al-
ready solved problem B.
On the other hand, to prove that a problem
B is unsolvable, we reduce an already proved
unsolvable problem A to B.
We have been showing how to apply the re-
ducibility technique to prove the undecidability
of several problems, by reducing, e.g., ATM, to
such a problem.
Now, we formalize this handy notion of reduc-
tion so that we can understand it better (or
worse /).
Definition: A function f : Σ∗ 7→ Σ∗ is a com-
putable function if some Turing machine M,
on every input w, halts with f(w) on its tape.
26
An example
If we represent any number n with n + 1 ‘1’s,
i.e., 0 is ‘1’, 1 is ‘11’, etc. Then, we have the
following Turing machine:
On input
m+1︷ ︸︸ ︷
1 · · ·1B
n+1︷ ︸︸ ︷
1 · · ·1,
1. Scan all the symbols. If the second segment
contains only one ‘1’, change it to ‘B’
2. Otherwise, go over the first segment of ‘1’s.
3. Fill the blank ‘B’ with 1.
4. Go over to the end of the second segment
of ‘1’s, and change the last two ‘1’s with ‘B’.
Thus, the addition function is unsurprisingly
computable. ,
27
Formalize reduction
We used to say that, when a problem A reduces
to another one B, we mean a solution to B can
be used to solve A.
Technically, we have the following definition.
Let A and B be languages. By saying that A
is mapping reducible to B, (A ≤m B), we mean
that, for a computable function, f : Σ∗ 7→ Σ∗,
for every w ∈ Σ∗,
w ∈ A iff f(w) ∈ B.
The function f is called the reduction of A to
B.
In other words, to find out if w ∈ A, we just use
the computable function f to figure out f(w),
then test if f(w) ∈ B.
28
What does it mean?
A mapping reduction of A to B provides a
method to convert questions about member-
ship testing in A to membership testing in B.
To test w ∈ A, we can use the reduction to
effectively compute f(w) and test if f(w) ∈
B. By definition, we can provide a definitive
answer for the former question on A based on
the result obtained from the latter question on
B : w ∈ A iff f(w) ∈ B.
In particular, if one problem is mapping re-
ducible to another, previously solved, problem,
we can obtain a solution to the original one.
On the other hand, if we can map reducible a
known unsolvable problem to a problem, then,
the latter problem must be unsolvable, as well.
29
Theorem: If A ≤m B and B is decidable, so is
A.
Proof: Let M decide B, and let f be the re-
duction of A to B. We construct the following
decider, N, for A.
N = “On input w
1. Compute f(w). //That’s why f has to be
computable.
2. Run M on f(w).
3. Accept, if M accepts; reject otherwise.”
While we use the above one most of the time,
we use the following one in a negative sense,
again based on the idea that
A → B ≡ ¬B → ¬A.
Corollary: If A ≤m B and A is undecidable, so
is B.
30
An example
We once informally showed that the HALTTM
is undecidable by reducing ATM to it. Now, we
look into this result from this new perspective
of mapping reduction.
First, we have to present a computable func-
tion, f, which, for any input 〈M, w〉, will provide
an output 〈M ′, w〉 such that
〈M, w〉 ∈ ATM iff 〈M ′, w〉 ∈ HALTTM.
Below shows F, a Turing machine that com-
putes f :
F =“ On input 〈M, w〉
1. Construct the following machine M ′
M ′ =“On input x
1.1. Run M on x.
1.2. If M accepts x, accept 〈M, w〉.
1.3. If M rejects x, enter a loop.
2. Output 〈M ′, w〉.”
31
So what?
Now let’s show that, with this function, ATM
mapping reduces to HALTTM, i.e.,
〈M, w〉 ∈ ATM iff 〈M ′, w〉 ∈ HALTTM.
Assume 〈M, w〉 ∈ ATM, i.e., M accepts w, then
by the construction, M ′ also accepts w, thus
halts with w, i.e., 〈M ′, w〉 ∈ HALTTM.
On the other hand, if 〈M, w〉 6∈ ATM, i.e., either
M rejects w or gets into an infinite loop, M ′
would get into an infinite loop in both cases,
thus, 〈M ′, w〉 6∈ HALTTM.
This shows that ATM ≤m HALTTM.
Finally, by the corollary, HALTTM is not solv-
able.
Guess we need to stop here. ,
32