Time Complexity. p2. Measuring Complexity Worst-case analysis Average-case analysis Define M: a det....
-
date post
21-Dec-2015 -
Category
Documents
-
view
230 -
download
0
Transcript of Time Complexity. p2. Measuring Complexity Worst-case analysis Average-case analysis Define M: a det....
Time Complexity
p2.
Measuring Complexity Worst-case analysis Average-case analysis
Define M: a det. Turing machine that halts on all input The running time or time complexity of M is the
function f: NN, where f(n) is the maximum number of steps that M uses on any input of length n
If f(n) is the running time of M, we say M runs in time f(n) and that M is an f(n) time Turing machine.
p3.
Big-O and small-o Notation Def:
Def:
Def:
0 0
f(n)=O(g(n)), if there exists positive integersc and n so that for every n n , f(n) cg(n)
. ( )( ) ( ), ( ) ( )
eg f n n n n
f n O n f n O n
3 2
4 35 2 22 6
( ), : . ( ) ( ( )) lim
( ) Say that if
n
f nf g N R f n o g n
g n
0
. ( ), ( ), log log ( log ) eg n o n n o n n n o n n 2 3
: a functionTIME(t(n)): time complexity class
L is a language decided by a O(t(n)) time= L Turing machine
t N N
p4.
Complexity relationships among models
( ))2
Thm: t(n): a function n Every t(n) time multitape Turing machine has an equivalent O(t time single-tape Turing machine.
n
O(t(n))
Thm: t(n): a function n Every t(n) time non deterministic single-tape TM has an equivalent 2 time deterministic single-tape TM.
p5.
The class PP is the class of languages that are decidable in polynomial time on a deterministic single-tape TM
In other words, ( )k
k
P TIME n
p6.
, , is a directed graph tha
has a directed path from sto t
GPATH G s t
s t
:Thm PATH P
, | , are relatively primeRELPRIME x y x y :Thm RELPRIME P
p7.
The Class NP Hamiltonian path in a directed graph G is a directed
path that goes through each node once
Note: It is not hard to find an exponential time algorithm for the
HAMPATH problem The HAMPATH problem can be verified in polynomial time
G is a directed graph with aHAMPATH= , ,
Hamiltonian path from s to tG s t
s t
p8.
Not all problem can be verified in poly. time.
Def: A verifier for a language A is an algorithm V, whereA = { | V accepts < ,c> for some string c} A polynomial time verifier runs in polynomial time in the
length of A language A is polynomially verifiable if it has a polynomial
time verifier. A verifier uses additional information c to verify that a string
is member of A C: certificate or proof of membership in A
, for integers, p,q >1COMPOSITES x x pq
12653 = 123 x 111
verifiable in poly. time
p9.
Def: NP is the class of languages that have polynomial time verifiersNP: Non-deterministic Polynomial time
Def:
Thm: A language is in NP iff it is decided by some non-deterministic polynomial time Turing machine
Pf:
is a language decided by ( ( ))
a O(t(n)) time non-det TM
LNTIME t n L
k
Let V be the polynomial time verifier for A that exists by t
" ": Let A NP and show that A is decided by a polynomial
time
he
definition of NP.
Assume V runs in time n with input of
NTM N
si
.
ze n
p10.
k
Construct N:
N = "On input of length n
1. Nondeterministically guess the string c of length n
2. Run V on input < , c>
3. If V accepts, ACCEPT; otherwise, REJECT."
" ": Assume that A is decided by a poly. time NTM
N and construct a poly time verifier V as follows.
"On input < ,c>, where and c are strings:
treating each symbol of c 1. Simulate N o an , s a
V
description of the non-det choice to make at each
step.
2. If this branch of N's accepts, ACCEPT; otherwise
REJECT"
p11.
Cor:
Examples of problem in NP Def: clique is a graph, where every two nodes are
connected by an edge A k-clique is a clique that contains k nodes
( )k
k
NP NTIME n
A graph of 5-clique
p12.
Def:
Thm: CLIQUE is in NPpf :
is an undirected graph,
with a k-clique
GQUE GC kLI
" On input <G,k>, where G is a graph:
1. Nondeterministically guess a subset c of k nodes of G
2. Test whether G contains all edges connecting node in
C
3. If yes, ACCEP
N
T; otherwise, REJECT."
p13.
Def:
Thm: SUBSET-SUM is in NPPf:
1
1 i
{ ,...., } and for some,
{y ,...., } , we have yk
k
S x xSUBSET SUM S t
y S t
1 l
: <{ 4, 11, 16, 21, 27 } , 25 > SUBSET-SUM
4+
Note: S and {y ,...., y } can be
21
mu
= 25
ltisets
Eg
"On input <S,t>"
1. Non-deterministrically guess a subset c of the
numbers in S
2. Test whether c is a collection of numbers
that sum to t
3. If both pass, ACCE
N
PT; otherwise, REJECT."
p14.
P = the class of languages where membership can be decided quickly
NP= the class of languages where membership can be verified quickly
BIG Question!! P = NP?
p15.
NP-completeness
The satisfiability problem is to test whether a Boolean formula is satisfiable
_ _
( ) ( ) Boolean formula
a boolean formula is satisfiable if some assigment
of 0's and 1's to the variables makes the formula
evaluate to 1
x y x z
{ | is a satisfiable Boolean formula}SAT
p16.
Thm: Cook-Levin Theorem
Def:
SAT P iff P=NP
* *: is a polynomial time computable
function if some polynomial time Turing Machine
M exists that halts with just ( ) on its tape, when
started on any input w.
f
f
p17.
Def:
p
* *
Language A is polynomial time mapping reducible
to language B, A B, if a polynomial time
computable function : exists, where
for every ,
A ( )
f
f B
The function is called the polynomial time reduction
of A to B
f
A B
p18.
Thm:
Pf: pIf A B and B P, then A P
Let M be the polynomial time algorithm deciding
B and be the polynomial time reduction from
A to B
N="On input :
1. comput ( )
2. Run M on input and ( ) output whatever M
outputs.
f
f
f
"
p19.
Def: A language B is NP-complete if it satisfies two condition:1. B is in NP
2. Every A in NP is polynomial time reducible to B(with only 2, it is called NP-hard)
Thm:
Pf:
If B is NP-complete and B P, then P=NP
Follows from the definition of polynomial time reducibility
p20.
Thm:
Pf:
pIf B is NP-complete and B C for C in NP, then
C is NP-complete
p p p
B is NP-complete, every language in NP is polynomial
time reducible to B
Need to show that every A in NP is polynomial time
reducible to C
Hence every lan
A B, B C A
guage in NP s po
C
i
lynomial time reducible
to C
p21.
Def
__
__ __
1 2 3 4
__ __ __
1 2 3 4 5 3 6
1
( )
literal: a boolean variable or a negated boolean
variable or
clause:
conjunctive n
( )
ormal form: (cnf )
3cnf-formula:
( ) ( )
(
x x
x x x x
x x x x x x x
x x
__ __ __
2 3 3 5 6 3 6 4) ( ) ( )x x x x x x x
3SAT={< | is a satisfiable 3cnf-formula}
p22.
Thm: 3SAT is polynomial time reducible to CLIQUEPf:
1x
2x
2x
1x
2x
__
1x__
2x
__
1x
__
2x
1 1 2 1 2 2
1
1
1
2 2
1 2 2 2
Let be a formula with k clauses:
( ) ( ) ...
( ) ( )
)
( )
(
,
k k k
x
a b c a b
x
f
c a b c
x x x x x x x
G k
Need to explain
1. f is poly. time reduction
2. is satisfiable
iff <G,k> CLIQUE
p23.
Cook-Levin Theorem : SAT is NP-complete.Proof :
1> SAT NP :A NTM can guess an assignment to a given formula and accept if the assignment satisfies . 2> For any A NP and show that A is polynomial time reducible to SAT.
Let N be a non-deterministic TM that decides A in nk time for some constant k.
A tableau for N on w is an nkXnk table whose rows are the configurations of a branch of the
Computation of N on input w.
###
#
q0 w1 nww2 #
##
#
start configuration
2nd configuration
window
nkth configuration
kn
kn
cell
A tableau is accepting if any row of the tableau is accepting configuration.
Every accepting tableau for N on w corresponds to a computation branch of N on w.
* Determine whether N accepts w is equivalent to determine whether an accepting tableau for N on w exists.
f : polynomial time reduction from A to SAT.On input w, the reduction produces a formula .Let Q and be the state set and the tape alphabet of N.Let For and each we have a variable such variables. {#}.C Q knji ,1 ,Cs .,, sjix kn2~
If , it means cell[i,j] contains an s. Design so that a satisfying assignment to the variable does correspond to an accepting
tableau
for N on w.
1,, sjix
cell start move accept
(1)
, , , , , ,,1 , k
x x xcell i j s i j s i j ts C s t Ci j n
s t
si
j
(a)
#,,1,1,,3,1,2,1,4,1,3,1,2,1#,1,11210 kkn nnxnwnwwqstart xxxxxxxx
Ensure that the first row of the table is the starting configuration of N on w.
(3) Guarantees that an accepting configuration occurs in the tableau.accept
acceptk qjinji
accept x ,,,1
(4) This part is more complicated !:moveguarantee that each row of the table corresponds to a configuration that legally follows the preceding
Row’s configuration according to N’s rules.
)},,,{(),( 11 Rbqaq
)},,{(),( 22 Rbqaq
)}.,,(),,,{(),( 221 RaqLcqbq
###
b a
b
b
q2
b
q1
a
q2
b a
c a
a
a
a
a
ca q1 bq2 a
(b)a q1 b
q2a a
(c)a a q1
ba a
(d)
a# b a# b
(b)a b a
q2a b
(c)b b b
bc b
legal windows
Examples of illegal windows :
(a)
aa b aa a
(b)a q1 bq1 aa
(c)a a
bq1
q2 q2
Claim : If the top row of the table is the start configuration and every window in the table is legal, each row
of the table is a configuration that legally follows the preceding one.
##
b a
b q2
q1
a
b a
c a
a
a
##
)legal is winodwj)(i, the(,1 knji
move
)legal is winodwj)(i, the()(
65432161
,1,1,1,,1,1,,1,,,,1,...,
ajiajiajiajiajiajiaa
xxxxxx
is a legal window
a1 a3
a5
a2
a4 a6
i+1ii-1
jj+1
Size of : fixed. and Non depends ~Let ),( 2 lCnO k Total number of variables .)( 2knO
).(: , )(: 2 kstart
kcell nOnO
).(:, 2kmoveaccept nO Thus the reduction is poly.
p27.
Cor :3SAT is NP-complete.
Cor :CLIQUE is NP-complete.
Proof :
3SAT is in NP.
:3SATSAT p
)()()( 43214321 aazzaaaaaa
3 )( 321 laaaa l
1 2 1 1 3 2 2 4 3 3 1( ) ( ) ( ) ( )l l la a z z a a z a z z a a
How about )?( 21 aa
p28.
Vertex cover of G :
If G is an undirected graph, a vertex cover of G is a subset of the
nodes where every edge of G touches one of those nodes.
1 4
3
{2,3} is a vertex cover.
{1,3} is not a vertex cover.
VERTEX-COVER={<G,k>|G is an undirected graph that has a k-node vertex cover}.
Thm :
VERTEX-COVER is NP-complete.
Proof :
VERTEX-COVER is in NP.
3 .pSAT VERTEX COVER
)()()( 221221211 xxxxxxxxx
1x 1x
1x
1x 2x
1x
2x 2x
2x 2x
1x
2x 2x
Variable gadgets
Clause gadgets
Need to show that is satisfiable iff G has a vertex cover with k nodes.
Let have m variables and l clauses. Let k=m+2l.(1) One true variable from each variable gadget.
two node from each clause gadget.
(2) Each of the 3 edges connecting the variable gadgets with each clause gadget is covered.
p30.
SUBSET-SUM :
1 1 l i{ , | { ,..., } and for some { ,...,y } S, y }. kSUBSET SUM S t S x x y t
Thm :
SUBSET-SUM is NP-complete.
Proof :SUBSET-SUM is in NP.
SUMSUBSETSAT p 3Let be a boolean formula with variables x1,…,xl and clauses c1,…,ck. kccc ...21
., tS
1y1z2y2z
1x
2x
3y3z
3x
lylz
lx
1g1h2g2h
1c
2c
kgkh
kc
1 2 3 4 ….. l1 0 0 0 ….. 01 0 0 0 ….. 0
1 0 0 ….. 01 0 0 ….. 0
1 0 ….. 01 0 ….. 0
11
C1 C2 ….. Ck
1 0 ….. 0 0 0 ….. 0 0 1 ….. 0 1 0 ….. 0 1 1 ….. 0 0 0 ….. 1
0 0 ….. 0 0 0 ….. 01 0 ….. 01 0 ….. 0
1 ….. 01 ….. 0
11
t 1 1 1 1 ….. 1 3 3 ….. 3
iy 1 0 ….. 0
il
iz 1 0 ….. 0
jc
….. 1 ….….. 0 ….
clause cj contains xi
Suppose is satisfiable.(1)
Construct a subset S as follows.
If xi is assigned TRUE, select yi
else select zi. I.e. for each I, we select either yi or zi.
Last k digits add up between 1 and 3.
Select enough of g and h numbers to make each of the last k digits up to 3.
(2)Suppose a subset of S sums to t.We construct a satisfying assignment to .
If the subset contains yi , we assigned xi TRUE
else assign xi FALSE.
This assignment satisfied ! Why??
The table has size )(~).)(( 22 nOlkO
p33.
MAX-SAT:Given a boolean formula , and an integer k, is there a
truthassignment that satisfies at least k clauses?
Thm: MAX-SAT is NP-completePf:
SAT p MAX-SAT
= c1 c2 … cm
k = m Thm: DOUBLE-SAT:
Given a boolean formula , are there at least 2 truthassignment for ?
DOUBLE-SAT is NP-complete
p34.
Def: Hamiltonian path in a directed graph G is a directed path that goes through each node once.
HAMPATH:Given a directed graph and its 2 vertices s and t, is
there a Hamiltonian path from s to t?
Thm: HAMPATH is NP-completePf: (long…later)
3SAT p HAMPATH
UHAMPATH:Given a undirected graph and its 2 vertices s and t, is
therea Hamiltonian path from s to t?
p35.
Proof: (HAMPATH is NPC) The following is a 3cnf-formula with k clauses:=(a1∨b1∨c1) ∧ (a2∨b2∨c2) ∧ …∧ (ak∨bk∨ck)
Each a,b,c is a literal xi or , and x1,…,xl are l variables of .
xi : ci :
ix
p36.
Proof conti.s
c1
1x
c2
2x
t
cl
lx
p37.
Proof conti.
1c 2c
13 k
jc
jc
jc
jc
ij xc contains
ij xc contains
p38.
Proof conti.
If there is a satisfying assignment, “select exactly one of the literals” in a clause and assign it TRUE.
If there is a truth assignment, then there exists an Hamiltonian path from s to t.
) trueis when (
zagzig
ix
)false is when (
zigzag
ix
p39.
Proof conti. If the Hamiltonian path is normal, that is, it
goes through the diamonds in order from the top one to the bottom one, we can easily get the satisfying assignment. Because each clause node appears on the path, by
observing the diamond at which the detour to it is taken, we may determine which of the literals in the corresponding clause is TRUE.
If the path zigzag, we assign corresponding variable TRUEIf the path zagzig, we assign FALSE.
p40.
Proof conti. We show that a Hamiltonian path must be
normal. c
1a 3a2a
1b 2b
at. points aat th
node availableonly theis a because
a fromenter cannot path the:2 casein
nodes thesefrom
elsewhere goespath thebecause
cor a fromenter cannot path the: 1 casein
acontain cannot path thecase,both In
seperator is a :2 case
seperator a is a :1 case
2
3
3
1
2
3
2
p41.
Thm: UHAMPATH is NP-completePf:
HAMPATH p UHAMPATH
directed G undirected G’u V(G)vertices of G’: u uin, umid, uout
s sout ; t tin
edges of G’: u v E(G) uin – umid – uout – vin – vmid – vout
if s u1 u2 … uk t is a Hamiltonian path in G,
then sout – u1in – u1
mid – u1out – u2
in – u2mid – u2
out – …
– ukin – uk
mid – ukout – tin is an undirected
Hamiltonian path in G’
su
v
t
vmid
sout
uin
umiduout
vin
vout
tin
p42.
Undirected Hamiltonian Cycle Thm:
Undirected Hamiltonian Cycle is NP-comlpletePf: UHAMPATH p UHAMCYCLE
Traveling Salesman Problem: (TSP)Given an integer n2, an nn distance matrix of some
cities,and an integer B0, is there a tour that visits every cityexactly once and returns to the starting city by traveling within distance B?
Thm: TSP is NP-completePf: UHAMCYCLE p TSP
p43.
Longest Cycle:Given a graph and integer k, is there a cycle, with no repeated nodes, of length at least k?
Thm: LONGEST CYCLE is NP-complete
SUBGRAPH ISOMORPHISM:Given 2 undirected graphs G and H, is G a subgraph of
H? Thm:
SUBGRAPH ISOMORPHISM is NP-complete
p44.
Coping with NP-completeness:Approximation algorithms: Let x be an instance of an optimization problem Opt(x): the optimum solution of x A: a poly. time algorithm for x : positive real numberIf A satisfies
For all x, then we say A is an -approximation algorithm
)(
|)()(|
xopt
xAxopt
p45.
Eg. The following is a 1-approximation algorithm for the
vertex cover problem.G: C={2,3}
Algorithm: 1. C :=;2. while there is an edge [u,v] in G add u, v to C and delete them from
GLet be the optimum vertex cover
c={ , , ,…, }
1
23
4
c
1|ˆ|
|ˆ|
|ˆ|
||||ˆ||
|ˆ| |||ˆ| 2
|||ˆ|
c
c
c
cc
cccc
c
p46.
Polynomial Time Approximation Scheme(PTAS)We say that an approximation scheme is PTAS, if for
any fixed >0, the scheme runs in time polynomial in the size n
of itsinput instance.
Ratio bound: Inapproximable problem:
If there is no -approximation algorithm for them withhowever large , unless P=NP
)(),max(*
*ne
c
c
c
c
p47.
Thm: TSP is inapproximable unless P=NPPf:
Let G be a graph with n nodes.UHAMCYCLE p TSP
If G has a Hamiltonian cycle, then the optimum cost of a tour is n; otherwise if G has no Hamiltonian cycle, then the optimum cost of a tour > n(1+)
o/w 2
),( if 1
if 0
nGvv
ji
d jiij
1
1
2+3
p48.
If TSP had an -approximation algorithm A, then we would be able to tell whether G has a Hamiltonian cycle.Run A for TSP:i> if the returned cost n(1+)+1: No cycleii> if the returned cost n(1+): has a cycleUnless P=NP, TSP is inapproximable.
p49.
Backtracking and Branch&Bound:A={S0}
while A is not empty dochoose a subproblem S and delete it from A choose a way of branching out of S , say to subproblems
S1, S2,…, Sr
for each subproblem Si in this list do
if test(Si) returns “solution found”
else if test(Si) returns “?” then add Si to A
Return “no solution”
p50.
Eg.
)()()()()( zyxxzzyyxzyx
zzyzyzyzyy )()( )()(
yyzz
x = T x = F
y = T y = F z = T z = F
p51.
Branch&Bound algorithm:A:={S0} , best_so_far:=
while A is not empty dochoose a subproblem S and delete it from Achoose a way of branching out of S , say to subproblems
S1, S2,…, Sr
for each subproblem Si in this list do
if |Si|=1 then update best_so_far
else if lowerbound(Si) < best_so_far then add Si to A
Return best_so_far
p52.
Local improvement algorithm:S= initial solutionwhile there is a solution S’ such that
S’ is a neighbor of S and cost(S’) < cost(S)do S:=S’
Return S
p53.
Simulated annealing:S= initial solution; T:=T0
repeatgenerate a random solution S’ such that
S’ is a neighbor of S and let = cost(S’)cost(S)if 0 then S:=S’
else S:=S’ with probability e-/T
update (T)until T=0Return the best solution seen!
T: temperature