summary - Caltech Computingcourses.cms.caltech.edu/cs20/a/slides/summary/summary.pdf · If this...
Transcript of summary - Caltech Computingcourses.cms.caltech.edu/cs20/a/slides/summary/summary.pdf · If this...
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 1
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary
• Finite automata– Deterministice FA– Nondeterministic FA, with e-transitions– Regular expressions– Equivalence of Regex, eNFA, NFA, DFA– Pumping Lemma
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 2
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Determistic Finite Automata
• A finite automaton is a 5-tuple (Q,Σ, δ, q0, F)
– Q is a finite set of states
– Σ is an alphabet
– δ : Q× Σ→ Q is a transition function
– q0 ∈ Q is the initial state
– F ⊆ Q is a set of final or accepting states
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 3
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Transition diagrams
Q = {q0, q1, q2, q3}Σ = {0,1}δ =
State Symbol Goto
q0 0 q2
q0 1 q1
q1 0 q3
q1 1 q0
q2 0 q0
q2 1 q3
q3 0 q1
q3 1 q2
q0 = q0
F = {q0}
1
1
0 0
1
1
0 0
q0 q1
q2 q3
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 4
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
A finite automaton
0 1 1 0 0 1 0 1
FiniteControl
Read-only tape
Tape head
1. In state q a. read a symbol c b. move the tape head right b. goto state delta(q, c)2. Accept iff the FA is in a final state after reading the last symbol
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 5
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Regex->eNFA
• Prove by structural induction: induction on the size of the regular expression
• Base cases:– Show the empty RE has an e-NFA– Show the RE has an e-NFA
– Show the a (symbol) RE has an e-NFA
• Induction:– Show (xy) has an e-NFA– Show (x + y) has an e-NFA– Show x* has an e-NFA
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 6
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Choice
ε
ε ε
ε
Machine 1
Machine 2
Start state Any (originally)final state
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 7
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Equivalence classes of RM
L(M)
Equivalence classes of RM
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 8
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Myhill-Nerode
Theorem (The Myhill-Nerode theorem). The followingthree statements are equivalent:
• The set L ⊆ Σ∗ is accepted by a FA.
• The set L is the union of some of the equivalenceclasses of a right-invariant equivalence relation withfinite index.
• Let equivalence relation RL be defined as xRLy iffxz ∈ L� yz ∈ L. Then RL has finite index.
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 9
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Regular languages
• Intuition: if a FA accepts a string that is “long enough,” it must repeat a state– But it can’t remember that the state was repeated– So it can be forced to repeat the state over and over
q0 qmqj=qk
a1,...,aj
aj+1,...,ak
ak+1,...,am
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 10
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Pumping lemma
Lemma (the Pumping Lemma)
• Let L be a regular set.
• There is a constant n s.t. for any z where |z| ≥ n,then z can be written z = uvw, where
– |uv| ≤ n– |v| ≥ 1
– For all i ≥ 0, uviw ∈ L– n is bounded by |Q|
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 11
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary
• Context-free languages– Context-free grammars– Grammar simplification– Pushdown automata– Equivalence of CFGs and PDAs
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 12
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Sentence diagramming; derivation trees
FRUIT FLIES LIKE A BANANA
adjective noun verb noun
noun-phrase
verb-phrase
noun-phrase
sentence
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 13
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
CFG: formal definition
• A CFG is a four-tuple (V , T , P, S)
– V is a finite set of nonterminals
– T is a finite set of terminals (V and T are dis-joint)
– P is a finite set of productions
– S is a nonterminal called the start symbol
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 14
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Ambiguity
e ::= e + e | e * e | NUMBER
e
e e
e e
1 2
3
+
*
e
ee
e e
2 3
1
*
+
(1 * 2) + 3 1 * (2 + 3)
Leftmost derivation Rightmost derivation
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 15
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Ambiguity
• A leftmost-derivation is a derivation in which a production is always applied to the leftmost symbol– A rightmost derivation applies to the rightmost symbol
• In general, a string may have multiple left and rightmost derivations
• A grammar in which some word has two parse trees is said to be ambiguous
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 16
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Simplification
• A nonterminal A isuseless iff– S * xAy * xzy– Otherwise, it is useless
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 17
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Eliminating epsilon-productions
Theorem All ε-productions can be eliminated, exceppossibly a production of the form S → ε
Definition A nonterminal A is nullable if A→∗G ε
Algorithm (finding nullable nonterminals)
• Base: if A→ ε then A is nullable
• Step: if A → α1 · · ·αn and α1 · · ·αn are all nulable, then A is nullable
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 18
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
PDA machine
1 + 2 * 3 + 4 ;
FiniteControl
Read-only tape
Tape head
1. In state q a. read a symbol c, stack symbol Z b. move the tape head right b. goto state delta(q, c, Z).1 d. replace Z with delta(q, c, Z).2--or: a. goto state delta(q, ε, Z).1 b. replace Z with delta(q, c, Z).22. Accept iff the FA is in a final state after reading the last symbol
Stack
2
+1
Z
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 19
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
PDA: formal definition
A (nondetermistic) PDA is a 7-tuple (Q,Σ, Γ , δ, q0, Z0, F)
• Q is a set of states
• Σ is an input alphabet
• Γ is a stack alphabet
• q0 is the start state
• Z0 is a stack symbol called the start symbol
• F ⊆ Q is a set of final states
• δ : Q× (Σ∪ {ε})× Γ → 2Q×Γ∗)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 20
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Balanced parentheses
M = ({q1}, {), (}, {S, (}, δ, S, {})
δ(q1, (, S) = {(q1, ()}δ(q1, (, () = {(q1, ()}δ(q1, ), () = {(q1, ε)}δ(q1, ε, S) = {(q1, ε)}
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 21
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Empty stack vs final state
• Final state PDA -> empty stack PDA– Simulate: whenever the PDA reaches a final state, empty
the stack
• Empty stack PDA -> final state PDA– Add a special marker $ at the bottom of the stack– Add transitions delta(q, epsilon, S) qf for some new
state qf in F
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 22
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Compiling a CFG to a PDA
Theorem If G = (V , T , P, S), then there is a PDA Mwhere L(G) = L(M) (empty stack)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 23
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Building the PDA from the GNF
To build the PDA for a grammar G = (V , T , P , S) in GNF
• Each production has the formA→ a1 . . . anX1 . . . Xm
• Define M = ({q}, T , V , δ, q, S, {})• Let δ(q,a,A) contains (q, γ) iff A→ aγ ∈ P
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 24
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Building a CFG from a PDA
Theorem IfM is a PDA, then there is a CFGG = (V , T , P , S)s.t. L(M) = L(G)
• Let M = (Q,Σ, Γ , δ, q0, Z0, F)
• Let V = {[q,A,p] | q,p ∈ Q∧A ∈ Γ} ∪ S• Productions
– S → [q0, Z0, q] for each q ∈ Q– [q,A, qm+1]→ a[q1, B1, q2] · · · [qm,Bm,qm+1]
for each
(q1, B1 . . . Bm) ∈ δ(q,a,A)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 25
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Pumping lemma for CFL
Lemma Let L be a CFL. Then there is a constant n suchthat for any string z ∈ L where |z| ≥ n, then z =uvxyz and
• |vx| ≥ 1
• |vwx| ≤ n• uviwxiy ∈ L for i ≥ 0
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 26
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Pumping
A
A
A
A
A
S
S
vu y
vu y
w x v w x
x
Pump
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 27
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Ogden’s lemma
Ogden's Lemma Let L be a CFL. Then there is a constantn, then for any z ∈ L we can mark n or more positionsof z = uvwxy , such that:
• v and x have at least one marked symbol
• vwx has at most n marked symbols
• uviwxiy ∈ L for i ≥ 0
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 28
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Building the path
Constructed pathb
b
b
P
* * * ****
Branch points
Marked symbols
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 29
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary: Turing Machines
• Turing Machines– Executions– Diagonalization– The halting problem– Rice’s theorem
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 30
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Turing machines
0 1 1 0 0 b b b
FiniteControl
Infinite read-write tape
Tape head
1. In state q a. read a symbol c b. print a new symbol c' c. move the tape head one position left or right d. goto state delta(q, c)2. Accept iff the TM ever enters a final state
Blanks
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 31
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Turing machine formal definition
A Turing Machine is a 7-tuple M = (Q,Σ, Γ , δ, s, b, F)
• Q is a finite set of states,
• Γ is a finite set of tape symbols,
• b ∈ Γ is the blank symbol,
• Σ ⊆ Γ − {b} is the set of input symbols,
• δ : Q × Γ → Q × Γ × {L,R} is a partial transitionfunction,
• s ∈ Q is the start state,
• F ⊆ Q is the set of final states
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 32
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Instantaneous descriptions
• An instantaneous description ID (σ ) isα1qα2, where
– q is the current state of the TM,
– α1α2 ∈ Γ∗ if the contents of the tape (to thelast non-blank symbol)
– The current symbol is the first symbol of α2
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 33
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Universal Turing Machines
Next, build a universal Turing machine U that accepts
L(U) ≡ {M#x | x ∈ L(M)}
• U checks that M#x is a valid description
• Copy M to one tape, and x to another tape
• Use a scratch tape to represents current state, headposition, etc.
• Simulate M : for each move:
– Figure out δ function from M
– Move the head,
– modify the tape x
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 34
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Another diagonalization argument
Place a * if M halts on w
Input string w0 1 2 3 4 5 ...
M0
M1
M2
M3
M4...
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
Machine MTuring Machine K = { M#w | M halts on w}
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 35
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Halting problem
• Problem: determine entries in the matrix• Find a recursive machine K, given M#x
– K halts and accepts if M accepts x– K halts and rejects if M does not terminate
• Is there such a machine?• Suppose so, then build a new machine N, that,
given x, runs K on Mx#x and:– Accepts if K rejects– Loops forever of K accepts
• If K exists, then N does too; let N = My
– N#y halts and accepts if N#y does not halt– N#y loops forever if N#y halts
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 36
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Halting problem
N(x) = if K(Mw#w) accepts then loop else halt
0 1 2 3 4 5 ...
M0
M1
M2
M3
M4...
*
*
*
*
*
*
*
*
*
*
*
*
*
Turing Machine K = { M#w | M halts on w}
N * * ?
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 37
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Rice’s theorem
• A property P is a set of r.e. languages (so it is a set containing sets of strings)
• For any r.e. language L, we say P(L) is true iff L is a member of P
• Theorem: Any nontrivial property of the r.e. languages is undecidable– Nontrivial means: the property is neither always true
nor always false
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 38
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Proof of Rice’s theorem
• Consider a property P– Assume P({}) = false, otherwise invert P
• Since P is nontrivial, there is a language L in P– Let ML be a TM accepting L
• To determine if M halts on input w, build M’– Ignore the input, and simulate M on w– If M halts, then start ML on the input string– So P(M’) iff M halts on w
w
x
Halts
MML yes/no
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 39
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary: alternate models
• Primitive and partial recursive programs– For-programs– While-programs– Godel numbering– The recursion theorem– Rice’s theorem
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 40
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Primitive recursion
Functions N tuples → N tuplesThe following functions are primitive recursive:
• s(x) = x + 1 (the successor function)
• z(x) = 0 (the zero function)
• πmi (x1, . . . , xm) = xi (projection)
• For any f : Nm → Nm, and g1, . . . , gm : Nk → N,the composition f(g1(x), . . . , gm(x)) is primitiverecursive
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 41
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
For-programs
• Variables x,y, z over N
• Simple assignments:
– x ← y
– x ← s(y)
– x ← 0
• Induction: if p,q are for-programs, so are:
– p;q (sequential composition)
– if x = y then p else q (conditional)
– fory do p done (for-loop)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 42
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
For programs and primitive recursion
Theorem The for-programs define exactly the primitive-recursive functions.
Encoding p.r. functions with for-loops. By inductionon the size of the p.r. function.
f(0, x) = h(x)f(s(y), x) = g(y,x, f (y,x))
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 43
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Partial recursive functions (Godel)
• To capture r.e. computations, we need more• A partial recursive computation includes the
prim-rec computations, plus unbounded minimization
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 44
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Unbounded minimization
• Find the least y s.t. g(x,y) = 0
f(x) = µy.g(x,y) = 0
where g is primitive recursive
• Alternatively:
f (x) = µy.(g(x,y) = 0∧g(x, z) is defined for z ≤ y)
where g does not have to be primitive recursive
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 45
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
While-programs
The while programs are defined inductively:
• All for-programs are while-programs
• Add whilex ≠ y do p done, where p is a while-program
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 46
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Godel numbering
Gödel numbering:
• There is a universal function U(〈i, x〉) = ϕi(x)
• For anym,n there is a total recursive function Smn(a partial recursive function defined on all inputs),s.t. for any i and x1, . . . , xn, y1, . . . , ym
ϕSmn (i,x1,...,xn)(y1, . . . , ym) = ϕi(x1, . . . , xn,y1, . . . , ym)
An indexing is acceptable iff it satisfies these two prop-erties.
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 47
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Recursion theorem
Theorem (Fixed-point theorem)For all total recursive functions f : N → N, there is anindex i s.t. ϕi = ϕf(i).
ProofTake any v ∈ N, and consider a recursive function thatdoes the following on x:
1. Compute ϕv(v)
2. If it halts, apply f to get f (ϕv(v))
3. Use it as an index: compute ϕf(ϕv(v))(x)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 48
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Rice’s theorem
Theorem (Rice's theorem)Every non-trivial property of the partial recursive func-tions is undecidable. That is, there is no total recursivefunction p : N→ {0,1} s.t.
• p(k) ≠ p(l)
• For any i, j, if ϕi =ϕj , then p(i) = p(j)
Proof Suppose p exists. Define
f(i) ={k if p(i) = p(l)l if p(i) = p(k)
f is total, and it has no fixpoint.
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 49
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary: lambda calculus
• Syntax• Substitution• Reduction
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 50
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Lambda calculus
• A starting point for reasoning about functions• The foundation of most functional programming
languages, including the Lisp and ML languages
v ranges over a countable number of variables
e ::= v (variables)| e1 e2 (function application)| λv.e (function abstraction)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 51
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Single-step evaluation
• A single-step reduction is just a substitution• Called “beta-reduction”
(λv.e1) e2 →β e1[e2/v]
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 52
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary: logic
• Propositional logic• Predicate calculus• Godel’s incompleteness theorem (Rice’s theorem
in disguise)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 53
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Defining the syntax
Start with a countable set of propositional letters P,Q,R, . . .Define the propositions inductively:
• (true) is a proposition
• ⊥ (false) is a proposition
• Any propositional letter is a proposition
• If A is a proposition, so it ¬A (negation)
• If A and B are propositions, so are
– A∧ B (conjunction)
– A∨ B (disjunction)
– A⇒ B (implication)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 54
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Standard syntax definition
Propositions:
e ::= | ⊥| P,Q,R, . . .| ¬e| e∧ e| e∨ e| e ⇒ e
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 55
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Rule table
Γ � ∆1, ,∆2 Γ1,⊥, Γ2 � ∆
Γ � ∆1, α,∆2 Γ � ∆1, β,∆2
Γ � ∆1, α∧ β,∆2
Γ1, α, β, Γ2 � ∆Γ1, α∧ β, Γ2 � ∆
Γ � ∆1, α, β,∆2
Γ � ∆1, α∨ β,∆2
Γ1, α, Γ2 � ∆ Γ1, β, Γ2 � ∆Γ1, α∨ β, Γ2 � ∆
Γ , α � ∆1, β,∆2
Γ � ∆1, α⇒ β,∆2
Γ1, β, Γ2 � ∆ Γ1, Γ2 � α,∆Γ1, α⇒ β, Γ2 � ∆
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 56
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Pierce’s law
A � B,A 4
� A⇒ B,A 3 A � A 5
(A⇒ B) ⇒ A � A 2
� ((A ⇒ B)⇒ A)⇒ A 1
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 57
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Formalizing arithmetic
The language of arithmetic:
e ::= i | v | e+ e | · · · | e/eP ::= e = e | e < e | · · ·
| ¬P| P ∧ P | P ∨ P | P ⇒ P| ∃v.P[v] | ∀v.P[v]
Example:
∀x1.∃x2.∀x3.∃x4.(x3 + x2 ≤ (1+ x1)∗ x4)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 58
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Indexing the arithmetic formulas
Define �ϕ� is the Gödel index of formula ϕDefine the normal way:
• �xi� = [0 :: i]
• �e1 + e2 = 1 :: �e1�@�e2�
• �e1 < e2� = 10 :: �e1�@�e2�
• �P1 ∧ P2� = 20 :: �P1�@�P2�
• · · ·
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 59
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Fixpoint theorem for arithmetic
Theorem (Fixpoint theorem)For any formula ψ(y) with free variable y , there is asentence ϕ with no free variables s.t. ϕ� ψ(�ϕ�).
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 60
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Godel’s Incompleteness Theorem
Theorem (Gödel's Incompletness theorem)There is a formula that is not provable.Proof
• Define a formula unprovable(�ϕ�) iff ϕ is notprovable.
• This is a formula in arithmetic
ψ(y) ≡ unprovable(y) ≡ unprovable(�ϕy�)
• Consider, for some ϕ, the formula
ϕ� unprovable(�ϕ�)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 61
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary: complexity
• Time & space bounded machines• NP computations
– SAT
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 62
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Space-bounded TMs
• Input tape is read-only• The number of storage
tapes is an arbitrary constant k
• M is DSPACE(T(n)) if:– M is deterministic– For any input of length n,
M scans at most T(n) cells on any storage tape
• NSPACE(T(n)) is the nondeterministic bound
Read-only input
FiniteControl
Storage tapes
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 63
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Time-bounded TMs
• All tapes are 2-way infinite
• M is DTIME(T(n)) if– M is deterministic– For any input of length n,
M takes at most T(n) steps
• M is NTIME(T(n))– Nondeterministic case
FiniteControl
Storage tapes
R/W inputput inpR/W
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 64
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Asymptotic complexity: big-Oh notation
Let f ,g : N→ N
• f is O(g) if
∃c ∈ N.∞∀n ∈ N.f (n) ≤ c · g(n)
∞∀ means ``for all but finitely many.'' Asympoti-cally, f grows no faster than g within a constantmultiple.
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 65
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
CNF satisfiability
• A propositional formula is in conjunctive normalform (CNF) if it is a conjunction of disjunctions ofliterals.
• A literal is a propositional letter, or the negationof a propositional letter.
• Every propositional formula can be represented inCNF.
(A∨¬B ∨ C)∧ (D ∨¬B ∨¬C)∧ (¬A∨ B ∨ E ∨¬F)...∧ (E ∨ F ∨¬D ∨A∨¬B)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 66
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Satisfiability phase transition (Selman)
• Using modern algorithms, plot how long it takes to solve a SAT formula as ratio of clauses/literals
1 4.3 10
t
clauses/literal
% clauses satisfiable
CPU time
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 67
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary
• Graph theory– Undirected, directed graphs– Storngly-connected components– Solving 2SAT in linear time
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 68
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Formal definition of graphs
• A graph G is defined as a pair G = (V , E) where
– V is a set of vertices
– E is a set of edges (vi, vj), . . .
• n = |V | is the size of the graph
• |E| is the number of edges
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 69
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Path example
A simple path
A simple cycle
A non-simple path
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 70
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Graph components
A connected graph
A disconnected graph
Articulation point
V1 V2
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 71
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
DFS: Finalize
2
2
2
2
0
1 3
45
6
7
0
1 3
45
6
7
0
1 3
45
6
7
0
1 3
45
6
7
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 72
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
DFS backedges
2
0
1 3
45
6
7Back edges
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 73
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Compute low values
3 2
1
4
5 7
6
8
low[3] = 1
low[2] = 1
low[1] = 1
low[4] = 3
low[5] = 4
low[6] = 6
low[7] = 7 low[8] = 6
x = min{low(w) | w is an immediate descendent of v}y = min{z | z is reachable by a back-edge from v}
low(v) = min(x,y)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 74
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Term summary
• Reductions– 3SAT Clique– 3SAT Coloring
• Cook’s theorem– NP SAT
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 75
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Conflict graph for 3SAT
x
¬x¬y x
¬y
yC1
C1
C2
C2
C3
C3
(x ∨ y) ∧ (¬x ∨ ¬y) ∧ (x ∨ ¬y)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 76
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
SAT/clique example
x
¬x¬y x
¬y
yC1
C1
C2
C2
C3
C3
(x ∨ y) ∧ (¬x ∨ ¬y) ∧ (x ∨ ¬y)
Satisfying assignment:(Both cases): (x ∧ ¬y)
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 77
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
More NP problems: graph coloring
Definition Consider an undirected graph G = (V , E),and a finite set C . A coloring is a map ϕ : V → C suchthat ϕ(u) ≠ ϕ(v) for (u,v) ∈ E. Given G and a con-stant k, the k-coloring problem is to determine whetherthere exists a coloring using no more than k colors.
Reduction We show that coloring is hard by showingSAT ≤pm 3-colorability.
• Use the colors R, G, B.
• Build a triangle for each literal.
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 78
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Unsatisfiable clause
¬x y ¬z w
Conflict
Suppose top row is entirely red
G G G G
G G
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 79
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Satisfiable clause
¬x y ¬z w
Suppose at least one is green
G G G G
G G
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 80
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Cook’s theorem
Theorem If R ∈ NP then R ≤pm SAT
Preparation
• Let M be the TM for R
• Assume the depth of the configuration tree is atmost N = |x|k for some constant k
• Assume thatM uses a single tape, delimited at theleft by a special symbol �
• Also, assume when M accepts, it erases its input,and moves all the way left
• Note, M can scan at most N tape cells
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 81
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Boolean variables
• Qqi : at time i, the machine is in state q
• Hij : at time i, the tape head is at position j
• Saij : at time i, the symbol in cell j is a
Computation, Computers, and Programs Cook's Theoremhttp://www.cs.caltech.edu/courses/cs20/a/ December 2, 2002 82
CALI
FOR
NIA
INSTITUTE OF TECH
NO
ILO
GY
If this page displays slowly, try turning off the “smooth line art” option in Acrobat, under Edit->Preferences
Boolean variables
S00Qq00
Q : at time i, the machine is in state qqi
00
S : at time i, tape cell j contains symbol a aij
Sa101 Sa2
02 SaN0N
H
S10Qq11 Sa1
11 Sa212 SaN
1N
11H
SN0QqNN
N0
S bN1 S b
N2 S bNN
H
S20Qq22 S x
21 Sa222 SaN
2N
22H
ijH : at time i, the tape head is at cell j