1
Design and Analysis of Algorithms
Yoram MosesLecture 11
June 3, 2010
http://www.ee.technion.ac.il/courses/046002
2
Nondeterministic Polynomial Time(NP)
3
Shortest Path: Search, Existence, Verification
Search problem: Input: (G,w,s,t): a directed graph G with weight function w,
a source s, and a sink t. Goal: find a shortest path from s to t. (or reject if none exists) Complexity: our solution runs in O(VE) = O(n2)
(Notice n = size of input = O(V+E)) Existence problem:
Input: (G,w,s,t,k): G,w,s,t are as before + a number k. Goal: decide whether there is a path from s to t of length ≤ k. Complexity: our solution runs in O(VE) = O(n2)
Verification problem: Input: (G,w,s,t,k,p): G,w,s,t,k as before. p is a path in G. Goal: decide whether p is a simple path from s to t of length ≤ k. Complexity: O(V) = O(n).
4
Max Flow: Search, Existence, Verification
Search problem: Input: (G,c,s,t): a directed graph G with capacity function c, a
source s, and a sink t. Goal: find a maximum flow in G. (or reject if none exists) Complexity: O(VE2) = O(n3)
Existence problem: Input: (G,c,s,t,k): G,c,s,t as before + a number k. Goal: decide whether there is a flow in G with value ≥ k. Complexity: O(VE2) = O(n3)
Verification problem: Input: (G,c,s,t,k,f): G,c,s,t,k as before. f is a function from edges of
G to real numbers. Goal: decide whether f is a legal flow with value ≥ k. Complexity: O(E) = O(n).
5
Hamiltonian Cycle: Search, Existence, Verification
Search problem: Input: an undirected graph G. Goal: find a Hamiltonian cycle in G (or reject if none exists). Complexity: O(VxV!) = O(n2n log n)
Existence problem: Input: an undirected graph G. Goal: decide whether G has a Hamiltonian cycle. Complexity: O(VxV!) = O(n2n log n)
Verification problem: Input: (G,p): an undirected graph G and a sequence of nodes p. Goal: decide whether p is a Hamiltonian cycle in G. Complexity: O(V) = O(n).
6
3-Coloring: Search, Existence, Verification
Search problem: Input: G: an undirected graph Goal: find a 3-Coloring of G. (or reject if none exists) Complexity: O(E 3V) = O(n2n log 3)
Existence problem: Input: G: as before. Goal: decide whether G has a 3-Coloring. Complexity: O(E 3V) = O(n2n log 3)
Verification problem: Input: (G,): G as before and : V {1,2,3}. Goal: decide whether is a 3-Coloring of G. Complexity: O(E) = O(n).
7
Search and Existence vs. Verification
Conclusion: in many natural examples:Search and existence are computationally
equivalentVerification is easier
Sometimes it’s just a little easier (Shortest Path, Max flow)
Sometimes it’s a lot easier (Hamiltonian cycle, 3-Coloring)
8
Verification Relations Language: L {0,1}*
Definition: A verification relation for L is a relation R {0,1}* {0,1}* s.t. for all x {0,1}*: x L there is at least one y {0,1}* s.t. (x,y) R. x L there is no y {0,1}* s.t. (x,y) R.
y is called the “certificate” for x A.k.a. its “witness” or “proof”
Remarks: Every input x L has at least one certificate y. If (x,y) R, then y is a certificate for x. An input x L may have several certificates. A language L has many verification relations.
9
Verification Relations: Examples Shortest path:
x = (G,w,s,t,k), y = a path p Language: {(G,w,s,t,k): G has an s-t path of length ≤ k} Certificate: s-t path of length ≤ k Verification relation:
{((G,w,s,t,k),p): p is an s-t path of length ≤ k in G}
Hamiltonian cycle: x = undirected graph G, y = a path p Language: G that has a Hamiltonian cycle Certificate: a Hamiltonian cycle in G Verification relation: {(G,p): p is a Hamiltonian cycle in G}
10
Nondeterministic Polynomial Time
Definition: A binary relation R is polynomially bounded, if there exists some c > 0 s.t. for every (x,y) R, |y| ≤ |x|c.
Definition: L is polynomial-time verifiable, if it has a verification relation R, which satisfies both: R is polynomially bounded, and R is polynomial-time decidable.
Definition: The class NP (Nondeterministic Polynomial Time) is the set of all polynomial-time verifiable languages.
11
NP: Examples
Examples of languages in NP: Decision Shortest Path, Decision Max Flow, Decision LP Hamiltonian Cycle, TSP, 3-Coloring, SAT, k-SAT, Clique
Examples of languages not known to be in NP: HC-complement: given a graph G, decide whether G has no
Hamiltonian cycles.
12
Definition: A nondeterministic algorithm is an algorithm N that, on input x, First, N “nondeterministically” guesses a “witness” y. Then, N runs a deterministic “verification” algorithm on (x,y). Note: N may make different nondeterministic guesses in different
runs on the same input x.
Nondeterministic Algorithms
Nondeterministic guess
Nondeterministic Algorithm N
Verificationxy
yes/no(x,y)
13
Decision by Nondeterministic Algorithms
Definition: A nondeterministic algorithm N is said to decide language L if: For every input x L, there is at least one guess y s.t. N accepts (x,y). For every input x L, the verification algorithm N rejects (x,y), for all
guesses y.
A polynomial-time nondeterministic algorithm is one in which The guesses (y’s) are of polynomial size (in |x|), and The verification algorithm runs in polynomial time.
Lemma: L NP iff L is decidable by a polynomial-time nondeterministic algorithm.
14
An NP Algorithm for Clique
Nondeterministic guess (input: x = (G,k))1. for i = 1,…,k 2. vi nondeterministic guess of a node in V=V(G)3. output y = (v1,…,vk)
Verification algorithm (input: (x,y))1. If x is not a valid encoding of a graph G and an integer k, reject.2. If y is not a valid encoding of k nodes v1,…,vk in G, reject.3. If v1,…,vk are not distinct, reject.4. for i 1,…,k-1 do5. for j i+1,…,k do6. if {vi,vj} E reject.7. accept
15
Lemma: P NP Biggest open problem of computer science:
is P = NP?
Two possibilities:
Current belief: P NP Search & Existence strictly harder than Verification.
P = NP?
P vs. NP
P = NPP
NP
16
f: N N: a complexity measure. Time(f(n)) = all languages decidable in time O(f(n)).
Lemma: Let f(n),g(n) be two complexity measures. If there exists a constant c, s.t. for all n > c, f(n) ≤ g(n), then
Time(f(n)) Time(g(n)).
Theorem (Time Hierarchy)Let f(n),g(n) be two complexity measures. If there exists a constant c, s.t. for all n > c, f(n) ≤ g(n)1/2, then
Time(f(n)) Time(g(n)).
Time Hierarchy
17
P
Definition:
Lemma: P EXP but P EXPLemma: NP EXP (exercise)Open problem: is NP = EXP? 3 Possibilities:
P, NP, and EXP
P
EXP
NP
P
EXP = NP
NP =P
EXP
18
NP-Completeness (NPC)
Problems in NP not known to be in P:
Hamiltonian Cycle, Clique, SAT, k-SAT (k ≥ 3), k-Coloring (k ≥ 3), TSP, …. (many others)
All of these are “NP-Complete” NP-Complete Problems:
Belong to NP If any of them belongs to P, then NP = P.
Two possibilities:
NPCP
NPNP =
NPC = P
19
NP-Hardness (NPH)
Definition: A language L is NP-hard if L’ ≤p L holds for all L’ NP.
NPH = class of all NP-hard problems.
Lemma: If any NP-hard problem belongs to P, then NP = P. If one NPH problem is easy, then all of NP is easy.
Lemma: If L NPH and L ≤p L’, then L’ NPH.
20
NP-Completeness
Definition: A language L is NP-complete if both L NP and L is NP-hard
NPC = class of NP-complete problems NPC = NP NPH
Theorem: If some NPC language is in P, then P = NP.
(P NPC NP = P = NPC). If some NPC language is not in P, then no NPC language is in P.
(NPC P P NPC = NP P).
21
NP-Completeness
NPC: “hardest” problems in NP Behave as a “single block”: either all in P
or all outside P
Lemma: If L1,L2 NPC, then both
L1 ≤p L2 and L2 ≤p L1.
22
Proving NP-Completeness
How to prove that a given language L is NPC? Show that L NP, and Show that L’ ≤p L holds for every L’ NP.
Easier alternative: Show that L NP, and Find some NPC problem L’ and show L’ ≤p L.
How do we obtain the first NPC problem? Using the first alternative Cook-Levin theorem: Circuit-SAT is NP-complete.
23
NP-Completeness: the Full Recipe
To show that L is NPC:Prove L is in NP
Show a polynomial time nondeterministic algorithm for L
Select an NPC problem L’Show a polynomial-time reduction f from L’ to L
Prove that x L’ iff f(x) L Show a polynomial-time algorithm to compute f
24
Example: Clique is NPC
Clique is in NP (seen today) 3-SAT is NPC (will show this later on) 3-SAT ≤p Clique (seen in previous lecture) Therefore: Clique is also NP-Complete!
25
End of Lecture 11
Top Related