Robbie Hott www.cs.virginia.edu/~jh2jf

Department of

Computer Science

University of Virginia

www.cs.virginia.edu/~njb2b/theory

Theory of Computation CS3102 – Spring 2015

A tale of computers, math, problem solving, life, love and tragic death

PSP

AC

E-co

mp

lete

QB

F

The Extended Chomsky Hierarchy

Finite {a,b} Regular a* Det. CF anbn

Context-free wwR P anbncn

NP

PSPACE

EXPSPACE

Rec

ogn

izab

le

No

t R

eco

gniz

able

H H

Decidable Presburger arithmetic

NP

-co

mp

lete

SAT

No

t fi

nit

ely

de

scri

bab

le

?

2S*

EXPTIME

EXP

TIM

E-co

mp

lete

Go

EXP

SPA

CE-

com

ple

te =

RE

Turing Context sensitive LBA

H*

degrees

Computability Complexity

~1800s – 1960s 1900: Hilbert’s Problems 1936: Turing’s Computable Numbers 1957: Chomsky’s Syntactic Structures

(Mostly) “Dead” field

1960s – 2150? 1960s: Hartmanis and Stearns: Complexity class 1971: Cook/Levin, Karp: P=NP? 1976: Knuth’s O, Ω, Θ

Very Open and Alive

Decidable

Undecidable Intractable

Tractable

Resource-Bounded Computation

Previously: can something be done?

Now: how efficiently can it be done?

Goal: conserve computational resources:

Time, space, other resources?

Resource-Bounded Computation

Def: L is decidable within time O(t(n)) if some TM M

that decides L always halts on all w* within

O(t(|w|)) steps / time.

Def: L is decidable within space O(s(n)) if some TM

M that decides L always halts on all w* while

never using more than O(s(|w|)) space / tape cells.

Complexity Classes

Def: DTIME(t(n))={L | L is decidable within

time O(t(n)) by some deterministic TM}

Def: NTIME(t(n))={L | L is decidable within

time O(t(n)) by some non-deterministic TM}

Def: DSPACE(s(n))={L | L decidable within

space O(s(n)) by some deterministic TM}

Def: NSPACE(s(n))={L | L decidable within

space O(s(n)) by some non-deterministic TM}

Examples of Space & Time Usage

Let L1={0n1n | n>0}:

For 1-tape TM’s:

L1 DTIME(n2)

L1 DSPACE(n)

L1 DTIME(n log n)

For 2-tape TM’s:

L1 DTIME(n)

L1 DSPACE(log n)

Examples of Space & Time Usage

Let L2=S*

L2 DTIME(n)

Theorem: every regular language is in DTIME(n)

L2 DSPACE(1)

Theorem: every regular language is in DSPACE(1)

L2 DTIME(1)

Let L3={w$w | w in S*}

L3 DTIME(n2)

L3 DSPACE(n)

L3 DSPACE(log n)

Multi-Tape vs. One-Tape TM

Is one more powerful than the other?

Space is Tape Independent Theorem: The space does not depend on the # tapes.

Proof: 1 0 1 1 0 0 1 0 1

1 1 1 1 0 0 1

0 1 0 1 1 0

0 1 0 1

1 1 1 1 0 0 1

0

Note: This does not asymptotically increase the overall space (but can increase the total time).

Theorem: A 1-tape TM can simulate a t(n)-time- bounded k-tape TM in time O(kt2(n)).

Idea: Tapes can be “interlaced” space-efficiently:

Multi-Tape vs. One-Tape TM

Are there problems that are in TIME(t(n)) for a multi-tape TM, but not in TIME(t(n)) for a one-tape TM?

Copy Input Problem Input: w, a string of N bits Output: ww

Obvious multi-tape algorithm that involves 2N steps: N steps: walk over the input, copying it to the second tape N steps: continue to move right, copying the second tape contents onto the input tape after the input

Best (?) single-tape algorithm that involves ~2N2 steps: N iterations: move over the input, marking each symbol N steps: move to the first non-blank square, write that symbol N steps: move back to the rightmost marked input symbol

Intuitively is seems impossible to do much better, but hard to prove!

Theory is about Big Questions

If little tweaks to our model change the answers, we might as well focus on answering the practical questions for a real system and specific problem instance instead.

Making things Robustier?

• Find a more robust computing model than a TM – Church-Turing thesis says all mechanical models are

equivalent (computing power) to a TM

– But, this doesn’t mean there might not be better models for complexity

• Make the complexity classes bigger – Define a complexity class big enough so the little tweaks

to TMs do not change the answers

Complexity Class P

P = U k

DTIME(nk)

P is the class of languages that can be decided in Polynomial Time on a deterministic, single-tape Turing machine.

Classes in P

a) TIME(n2)

b) TIME(O(n7))

c) TIME(O(2n))

d) Class of languages that can be decided in Polynomial Time by a 2-tape TM

e) Class of languages that can be decided in Polynomial Time by a nondeterministic TM

Unknown! This is the P = NP question. Focus of next class…

Yes! We can simulate each step of a 2-tape TM by making 2 passes over the whole tape ~ 2(n+t(n))

(See Theorem 7.8)

P Examples

• What are some examples of problems in P?

Traveling Salesperson Problem

• You have to visit n cities

• You want to make the shortest trip

• How could you do this?

• What if you had a machine that could guess?

Non-deterministic polynomial time

• Deterministic Polynomial Time: The TM takes at most O(nc) steps to accept a string of length n

• Non-deterministic Polynomial Time: The TM takes at most O(nc) steps on each computation path to accept a string of length n

Complexity Class NP

NP = U k

NTIME(nk)

P is the class of languages that can be decided in Polynomial Time on a non-deterministic, single-tape Turing machine.

Complexity Class NP

• Properties of NP

– Easy to verify an answer (verifier in P)

– Hard to compute an answer

NP Example

• Subset Sum Problem

– Given a set of integers, determine if a subset sums to 0

Example: {-7, -3, -2, 5, 8, 11, 15}

– Verify that {-3, -2, 5} is a solution

NP Example?

• Traveling Salesperson Problem (decision version)

– Given a list of n cities to visit and a distance k, is there a route to visit all cities with distance less than k?

NP Example

• Traveling Salesperson Problem (decision version) – Even the simpler version is in NP

– Verifier: given an ordered list of cities, sum up distance and check < k

– Solver: • Non-determinism: at each city, “guess” the next correctly.

After visiting all, verify cost less than k ( linear algorithm )

• Deterministic: check all possible routes ( exponential time )

NP Example?

• Traveling Salesperson Problem (decision version)

– Given a list of n cities to visit and a distance k, is there a route to visit all cities with distance less than k?

The Class P and the Class NP

• P = { L | L is accepted by a deterministic Turing Machine in polynomial time }

• NP = { L | L is accepted by a non-deterministic Turing Machine in polynomial time }

• They are sets of languages

P vs NP?

• Are non-deterministic Turing machines really more powerful (efficient) than deterministic ones?

• Essence of P vs NP problem

Does Non-Determinism matter?

DFA ≈ NFA DFA not ≈ NFA

(PDA)

Finite Automata?

No!

Push Down Automata?

Yes!

Perelman 2006

??

Progress

• P = NP if every NP problem has a deterministic polynomial algorithm

• We could find an algorithm for every NP problem

• Seems… hard…

• We could use polynomial time reductions to find the “hardest” problems and just work on those

Reductions

• Real world examples:

– Finding your way around the city reduces to reading a map

– Traveling from Richmond to Cville reduces to driving a car

– Other suggestions?

Polynomial time reductions • PARTITION = { n1,n2,… nk | we can split the

integers into two sets which sum to half } • SUBSET-SUM = { <n1,n2,… nk,m> | there exists a

subset which sums to m }

• 1) If I can solve SUBSET-SUM, how can I use that to solve an instance of PARTITION?

• 2) If I can solve PARTITION, how can I use that to solve an instance of SUBSET-SUM?

Polynomial Reductions

• 1) Partition REDUCES to Subset-Sum

– Partition <p Subset-Sum

• 2) Subset-Sum REDUCES to Partition

– Subset-Sum <p Partition

• Therefore they are equivalently hard

• How long does the reduction take?

• How could you take advantage of an exponential time reduction?

NP-Completeness

• How would you define NP-Complete?

• They are the “hardest” problems in NP

P NP

NP-Complete

Definition of NP-Complete

• Q is an NP-Complete problem if:

• 1) Q is in NP

• 2) every other NP problem polynomial time reducible to Q

Getting Started

• How do you show that EVERY NP problem reduces to Q?

• One way would be to already have an NP-Complete problem and just reduce from that

P1

P2

P3

P4

Mystery NP-Complete Problem

Q

SAT

• SAT = { f | f is a Boolean Formula with a satisfying assignment }

• Is SAT in NP?

Cook-Levin Theorem (1971)

• SAT is NP-Complete

If you want to see the proof it is Theorem 7.37 in Sipser (assigned reading!) or you can take CS 660 – Graduate Theory. You are not responsible for knowing the proof.

3-SAT

• 3-SAT = { f | f is in Conjunctive Normal Form, each clause has exactly 3 literals and f is satisfiable }

• 3-SAT is NP-Complete

• (2-SAT is in P)

NP-Complete • To prove a problem is NP-Complete show a polynomial

time reduction from 3-SAT • Other NP-Complete Problems:

– PARTITION – SUBSET-SUM – CLIQUE – HAMILTONIAN PATH (TSP) – GRAPH COLORING – MINESWEEPER (and many more)

NP-Completeness Proof Method

• To show that Q is NP-Complete:

• 1) Show that Q is in NP

• 2) Pick an instance, R, of your favorite NP-Complete problem (ex: Φ in 3-SAT)

• 3) Show a polynomial algorithm to transform R into an instance of Q

Savitch’s Theorem Theorem: NSPACE(f(n)) DSPACE (f2(n))

Proof: Simulation: idea is to aggressively conserve and reuse space while sacrificing (lots of) time. Consider a sequence of TM states in one branch of an NSPACE(f(n))-bounded computation:

Computation time / length is bounded by cf(n) (why?) We need to simulate this branch and all others too! Q: How can we space-efficiently simulate these? A: Use divide-and-conquer with heavy space-reuse!

Walter Savitch

Savitch’s Theorem Pick a midpoint state along target path:

Verify it is a valid intermediate state by recursively solving both subproblems. Iterate for all possible midpoint states! The recursion stack depth is at most log(cf(n))=O(f(n)) Each recursion stack frame size is O(f(n)). total space needed is O(f(n)*f(n))=O(f2(n)) Note: total time is exponential (but that’s OK).

non-determinism can be eliminated by squaring

the space: NSPACE(f(n)) DSPACE (f2(n))

Walter Savitch

Savitch’s Theorem Corollary: NPSPACE = PSPACE

Proof: NPSPACE = NSPACE(nk) k>1

DSPACE(n2k) k>1

= DSPACE(nk) k>1

= PSPACE

i.e., polynomial space is invariant with respect to

non-determinism!

Q: What about polynomial time?

A: Still open! (P=NP)

Walter Savitch