NP-Completenessgsyoung/CS530/Notes/CS530NP-Completenes… · NP-Completeness 1 NP-Completeness...

1

Young CS 530 Ad. Algo. D&A

NP-Completeness 1

NP-Completeness

Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness

by Garey and Johnson, W.H. Freeman and Company, 1979.


NP-Completeness 2

General Problems, Input Size and Time Complexity

• Time complexity of algorithms :

polynomial time algorithm ("efficient algorithm") v.s.

exponential time algorithm ("inefficient algorithm")

f(n) \ n 10 30 50

n 0.00001 sec 0.00003 sec 0.00005 sec

n5 0.1 sec 24.3 sec 5.2 mins

2n 0.001 sec 17.9 mins 35.7 yrs

2


NP-Completeness 3

“Hard” and “easy’ Problems• Sometimes the dividing line between “easy” and “hard”

problems is a fine one. For example– Find the shortest path in a graph from X to Y. (easy)– Find the longest path in a graph from X to Y. (with no

cycles) (hard)

• View another way – as “yes/no” problems– Is there a simple path from X to Y with weight <= M? (easy)– Is there a simple path from X to Y with weight >= M? (hard)

• First problem can be solved in polynomial time. • All known algorithms for the second problem (could)

take exponential time .


NP-Completeness 4

• Decision problem: The solution to the problem is "yes" or "no". Most optimization problems can be phrased as decision problems (still have the same time complexity).Example : Assume we have a decision algorithm X for 0/1 Knapsack problem with capacity M, i.e. Algorithm X returns “Yes” or “No” to the question “is there a solution with profit P subject to knapsack capacity M?”

3


NP-Completeness 5

We can repeatedly run algorithm X for various profits(P values) to find an optimal solution. Example : Use binary search to get the optimal profit,

maximum of lg pi runs. (where M is the capacity of the knapsack optimization problem)

Min Bound Optimal Profit Max Bound

0 pi

|___________________|_________________|

Search for the optimal solution


NP-Completeness 6

The Classes of P and NP

• The class P and Deterministic Turing Machine• Given a decision problem X, if there is a

polynomial time Deterministic Turing Machine program that solves X, then X is belong to P

• Informally, there is a polynomial time algorithm to solve the problem

4


NP-Completeness 7

• The class NP and Non-deterministic Turing Machine • Given a decision problem X, if there is a

polynomial time Non-deterministic Turing machine program that solves X, then X belongs to NP

• Given a decision problem X. For every instance I of X, (a) guess solution S for I, and (b) check “is S a solution to I?”. If (a) and (b) can be done in polynomial time, then X belongs to NP.


NP-Completeness 8

• Obvious : P NP, i.e. A problem in P does not need “guess solution”. The correct solution can be computed in polynomial time.

• Some problems which are in NP, but may not in P :• 0/1 Knapsack Problem

• PARTITION Problem : Given a finite set of positive integers Z.

Question : Is there a subset Z' of Z such that

Sum of all numbers in Z' = Sum of all numbers in Z-Z' ?

i.e. Z' = (Z-Z')

NP

P

5


NP-Completeness 9

• One of the most important open problem in

theoretical compute science : Is P=NP ?Most likely “No”. Currently, there are many known problems in NP, and there is no solution to show anyone of them in P.


NP-Completeness 10

NP-Complete Problems• Stephen Cook introduced the notion of NP-

Complete Problems. This makes the problem “P = NP ?” much more interesting to study.

• The following are several important things presented by Cook :

6


NP-Completeness 11

1. Polynomial Transformation (" ")

• L1 L2 : There is a polynomial time transformation that transforms arbitrary instance of L1 to some instance of L2.

• If L1 L2 then L2 is in P implies L1 is in P (or L1 is not in P implies L2 is not in P)

• If L1 L2 and L2 L3 then L1 L3


NP-Completeness 12

2. Focus on the class of NP – decision problems only. Many intractable problems, when phrased as decision problems, belong to this class.

3. L is NP-Complete if L NP and for all other L' NP, L' L

• If a problem in NP-complete can be solved in polynomial time then all problems in NP can be solved in polynomial time.

• If a problem in NP cannot be solved in polynomial time then all problems in NP-complete cannot be solved in polynomial time.

Note that an NP-complete problem is

one of those hardest problems in NP.NP

L

7


NP-Completeness 13

• So, if an NP-complete problem is in P then P=NP

• if P != NP then all NP-complete problems are in NP-P

Question : how can we obtain the first NP-complete problem L?

4. Cook Theorem : SATISFIABILITY is NP-Complete. (The first NP-Complete problem)

Instance : Given a set of variables, U, and a collection of clauses, C, over U.

Question : Is there a truth assignment for U that satisfies all clauses in C?


NP-Completeness 14

Example :

U = {x1, x2}

C1 = {(x1, ¬ x2), (¬ x1, x2)}

= (x1 OR ¬ x2) AND (¬ x1 OR x2)

if x1 = x2 = True C1 = True

C2 = (x1, x2) (x1, ¬ x2) (¬ x1) not satisfiable

“¬ xi ” = “not xi ” “OR” = “logical or ” “AND” = “logical and ”This problem is also called “CNF-Satisfiability” since the expression is in CNF – Conjunctive Normal Form (the product of sums).

8


NP-Completeness 15

• With the Cook Theorem, we have the following property :

Lemma : If L1 and L2 belong to NP, L1 is NP-complete, and L1 L2 then L2 is NP-complete.

i.e. L1, L2 NP and for all other L' NP, L' L1 and L1 L2 L' L2


NP-Completeness 16

• So now, to prove a problem L to be NP-complete problem , we need to

• show L is in NP

• select a known NP-complete problem L'

• construct a polynomial time transformation f from L' to L

• prove the correctness of f (i.e. L’ has a solution if and only if L has a solution) and that f is a polynomial transformation

9


NP-Completeness 17

• P: Problems solvable by deterministic algorithms in polynomial time

• NP: Problems solved by non-deterministic algorithms in polynomial time

• A group of problems, including all of the ones we have discussed (Satisfiability, 0/1 Knapsack, Longest Path, Partition) have an additional important property:

If any of them can be solved in polynomial time, then they all can!

NP

P

NP-Complete

• These problems are called NP-complete problems.


NP-Completeness 18

• Some NP-complete problems :• SATISFIABILITY

• 0/1 Knapsack

• PARTITION

• Two-Processor Non-Preemptive Schedule Length

• CLIQUE : An undirected graph G=(V, E) and a positive integer J |V|

Question : Does G contain a clique (complete subgraph) of size J or more?

10


NP-Completeness 19

Proving NP-Completeness Results

• Example 1 : Show that the PARTITION problem is NP-complete.

Given a known NPC problem - Sum of Subset Problem (SS), show that PARTITION problem is NPC.

SS ProblemInstance : Let A = {a1, a2, …, an} be a set of n positive numbers.Question : Given M, is there a subset A' A such that A' = M


NP-Completeness 20

PARTITION ProblemInstance : Given a finite set of m positive

integers Z. Question : Is there a subset Z' Z such that

Z' = (Z-Z')• PARTITION is in NP

guess a subset Z' O(m)verify Z' = (Z-Z')? O(m)

Total O(m)

11


NP-Completeness 21

• SS PARTITION

Given an arbitrary instance of SS, i.e. A = {a1, a2, …, an} and M,

Construct an instance of PARTITION as follows :

B={b1, b2, …, bn, bn+1, bn+2} of m = n+2 positive numbers

where

bi = ai for 1 i n

bn+1 = M + 1

bn+2 =A + (1 – M)

Note : bi = 2 A + 2. Also, the transformation can be done in polynomial time (based on input size of A & M)


NP-Completeness 22

To show the transformation is correct : The SS problem has a solution if and only if the PARTITION problem has a solution.

If SS problem has a solution, then the PARTITION problem has a solution

assume A' is the solution for SS problem then

Z' = A' {bn+2} and Z-Z' =A-A' {bn+1}

Z' = M + A + (1 – M) = A + 1 = (Z-Z')

12


NP-Completeness 23

If the PARTITION problem has a solution then the SS problem has a solution

if Z' is the solution then Z' = A + 1

exactly one of bn+2 or bn+1 Z'

if bn+2 Z' then Z' – { bn+2 } = A' and A' = M

if bn+1 Z', then use Z - Z' to obtain A'


NP-Completeness 24

• Example 2 : Show that the Traveling Salesman (TS) Problem is NP-complete.

Given a known NPC problem - Hamiltonian Circuit (HC), show that TS problem is NPC.

Hamiltonian Circuit (HC) problemInstance : Give an undirected graph G=(V, E)Question : Does G contain a Hamiltonian circuit, i.e. a sequence < v1, v2, …, vn > of all vertices in V which is a simple cycle.

13


NP-Completeness 25

Traveling Salesman (TS) ProblemInstance : Give an undirected complete graphG=(V, E) with distance d(i,j) 0 for each edge (i,j) for i j and a positive integer B.Question : Is there a tour of all cities (a simple cycle with all vertices) having total distance no more than B.

• TS is in NPguess a tour, i.e. sequence of all vertices O(|V|)

verify that it is a cycle covering all vertices and total distance B O(|V|)


NP-Completeness 26

• HC TS

Given arbitrary instance of HC, i.e. G=(V, E).

Construct an instance of TS as follows :

G’ = (V , E’ ), where (u,v) E’

for all u, v V and u v

d(u,v) = 0 if (u,v) E

d(u,v) = 1 if (u,v) E

and B = 0

Note : The transformation can be done in polynomial time (based on input size of V and E)

14


NP-Completeness 27

To show the transformation is correct : The HC problem has a solution if and only if the TS problem has a solution.

If HC problem has a solution, then TS problem has a solution

Assume a < v1, v2, …, vn > is the solution for HC It is a simple cycle which contains all vertices Each edge (u,v) in this cycle has d(u,v) = 0 Total distance is 0 Solution for TS

If TS problem has a solution then HC problem has a solution

Obvious to see.


NP-Completeness 28

• Example 3 : Show that the Vertex Cover (VC) Problem is NP-complete.

Given 3SAT problem is NPC, show that VC problem is NPC.

3SAT Problem Instance : Given a set of variables U = {u1, u2, …, un} and a collection of clauses C = {c1, c2, …, cm} over U such that | ci | = 3 for 1 i m.Question : Is there a truth assignment for U that satisfies all clauses in C? Note : 3SAT problem is a restricted problem of SATISFIABILITY problem.

15


NP-Completeness 29

Vertex Cover (VC) ProblemInstance : Given an undirected graph G=(V, E) and a positive integer K |V| Question : Is there a vertex cover of size K or less for G, i.e. a subset V’ V such that |V’| K and, for each (u,v) E, at least one of u or v V’.

• VC is in NPguess a set of vertices V’ V O(|V|)verify that |V’| K and, for each (u,v) E, u V’ or v V’ O(|V|+|E|)


NP-Completeness 30

• 3SAT VC

Given arbitrary instance of 3SAT, i.e. U = {u1, u2, …, un} and C = {c1, c2, …, cm}. Construct an instance of VC as follows :

G = (V , E ) and K = n+2m

V = Vu Vc

Vu = {u1t, u1f, u2t, u2f, …, unt, unf} and

Vc = {a11, a12, a13} {a21, a22, a23} … {am1, am2, am3}

16


NP-Completeness 31

E = Eu Ec Euc

Eu = {(u1t, u1f) , (u2t, u2f) , …, (unt, unf)}

Ec = {(a11, a12) , (a12, a13), (a13, a11)} … {(am1, am2), (am2, am3), (am3, am1)}

Assume ci = (xi, yi, zi) for 1 i m,

find the corresponding vertices, xi, yi, zi, in Vu

Euc = {(x1, a11), (y1, a12), (z1, a13)} … {(xm, am1), (ym, am2), (zm, am3)}


NP-Completeness 32

|V| = 2n+3m and |E| = n+3m+3m

The transformation can be done in polynomial time (based on input size n and m)

Example : U = {u1, u2, u3 , u4} and C = {{u1, ¬ u3, ¬ u4}, {u1, u2, ¬ u4}}

E

u1tu1f u2t u2f u3t u3f u4t u4f

a11

a12

a13 a21

a22

a23

Eu

Ec

Euc

17


NP-Completeness 33

Major Property : if there is a vertex cover set V’ K = n+2m, then

a) |V’| = n+ 2m, and b) V’ must include exactly 1 vertex in {uit, uif}

for 1 i n from Vu and at exactly 2 vertices in {ai1, ai2, ai3} for 1 i m from Vc,

i.e. n vertices from Vu and 2m vertices from Vc Look at edges in Eu and Ec, a vertex cover set V’ must

include at least 1 vertex {uit, uif} for 1 i n, and

at least 2 vertices from {ai1, ai2, ai3} for 1 i m.

Since |V’| K |V’| = K


NP-Completeness 34

To show the transformation is correct : The 3SAT problem has a solution if and only if the VC problem has a solution.

If VC problem has a solution then 3SAT problem has a solution

From the above property, V’ contains n vertices from Vu and 2m vertices from Vc

From Vu , the truth assignment for {u1, u2, …, un} in 3SAT is

ui = T if uit V’ui = F if uif V’ for 1 i n

18


NP-Completeness 35

To see that this is a solution for 3SAT :we must show that for each ci = (xi, yi, zi), there is at least one variable i {xi, yi, zi} which set ci to TRUE, 1 i n

From the above property, exactly 2 vertices from {ai1, ai2, ai3} in V’, for 1 i m

only cover 2 edges from {(xi, ai1), (yi, ai2), (zi, ai3)} from Euc

assume the edge (xi, ai1) is not covered by 2 vertices from {ai1, ai2, ci3} then xi V’ since V’ is a vertex cover set

xi set the clause ci to True for 1 i n


NP-Completeness 36

If the 3SAT problem has a solution, then the VC problem has a solution

The vertex cover set V’ with exactly n+2m vertices can be obtained as follows :

From the truth assignment for {u1, u2, …, un} in 3SAT, we get n vertices from Vu,

i.e. uit V’ if ui = T; otherwise uif V’ for 1 i n

This covers all edges in Eu and at least one edge in {(xi, ai1), (yi, ai2), (zi, ai3)} for 1 i m

From Vc, include 2 vertices from each {ai1, ai2, ai3} into V’, for 1 i m. These 2 vertices cover all edges {(ai1, ai2) , (ai2, ai3), (ai3, ai1)} and also cover edges in Euc that are not covered previously.

19


NP-Completeness 37

• Example 4 : Show that the Square Packing (SP) Problem is NP-complete.

Motivation : truck loading, processor allocation problem in a partitionable mesh connected system, the design of VLSI chips and etc.

Square Packing ProblemGiven a packing square S and a set of packed squares L = {s1, s2, ..., sn}. Question : is there and orthogonal packing of L into S?

Note : orthogonal packing the sides of squares are parallel to the vertical and horizontal axes


NP-Completeness 38

3-Partition Problem

Given a list A = {a1, a2, ..., a3z} of 3z positive integers such that sum of all numbers is zB and B/4 < ai < B/2 for each 1 i 3z.

Question : Can A be partitioned into z groups such that the sum of all numbers in each group is B.

Note : each group must have exactly 3 numbers

Proof : Refer to the following research paper

Leung, Tam, Wong, Young and Chin, “Packing Squares into a Square”, Journal of Parallel and Distributed Computing, 1990.

20


NP-Completeness 39

Coping with NP-Completeness Problem• NP-hard Problem

Note : Refer to Chapter 5 of Garey and Johnson

If L' L and L' is an NP-complete problem then L is called NP-hard problem.

All NPC problems are NP-hard.


NP-Completeness 40

There are some NP-hard decision problems that are not in NP.

Example : Kth Largest Subset Problem is not in NP

Instance : Given a set of positive integers A = {a1, a2, …, an}, and two non-negative numbers, B A and K 2|A|.

Question : Are there at least K distinct subsets A’ A such that each subset has total sum >= B.

Note : PARTITION problem Kth Largest Subset Problem

21


NP-Completeness 41

• Pseudo-polynomial time algorithm

Note : Refer to Chapter 4 of Garey and Johnson

Some NP-complete problem may be solved in "polynomial" time (based on input size and magnitude).

Example : PARTITION problem


NP-Completeness 42

Dynamic Programming Algorithm :

• assume B = (sum of n integers)/2 • construct a table of size (approx.) n x B • fill in the table row by row • for each row, add a new element

mark sum of all possible subsets

if there is a subset with sum = B, stop.

Time Complexity : O(nB)

22


NP-Completeness 43

• NP-completeness in the strong senseNote : Refer to Chapter 4 of Garey and Johnson

If L is NP-complete problem in the strong sense, then L cannot be solved by a pseudo-polynomial time algorithm unless P=NP

L is NP-complete in the strong sense if L' L , L' is NP-complete in the strong sense and L is in NP

Example : 3-partition problem is NPC in the strong sense.


NP-Completeness 44

• Solving more restricted problems

If we restricted the problem L to problem L'

i.e. L' is a special/restricted case of L,

then we may solve L' in polynomial time.

For example : PARTITION problem

if we assume that each input integer ai n, where n is the number of input integers, then the pseudo polynomial time algorithm becomes polynomial time algorithm, i.e. O(n3)

NP-Completenessgsyoung/CS530/Notes/CS530NP-Completenes… · NP-Completeness 1 NP-Completeness...

Documents

Transcript of NP-Completenessgsyoung/CS530/Notes/CS530NP-Completenes… · NP-Completeness 1 NP-Completeness...