Theory Of Computation
description
Transcript of Theory Of Computation
THEORY OF COMPUTATIONDr. Adam P. AnthonyLectures 25 and 26
Overview Computer Science: do we need
computers? Computation Theory Functions Turing Machines Universal Programming Languages The Halting problem
Computer Science and ComputersComputer science is no more
about computers than astronomy is about telescopes.
Edgser W. Dijkstra
Computability Insight: the computation is separate, in concept, from
the computer A computer, then, is just some object that can carry
out the computation Humans
Brain, often supplemented by pencil, paper Charles Babbage
Difference engine, analytical engine Controlled using clockwork-type components
ENIAC Controlled using vacuum tubes
Intel 8080 Controlled using micro-transistors
Strength of Computers Can a simple calculator help you find
your way around cleveland? How about a (dumb) phone?
Aside from making calls How about a smart phone? How about a laptop? How about a desktop? Which of these count as computers?
Specific-Purpose vs. General Computers
Some ‘computers’ are designed only to achieve a limited number of specific tasks, and to do that either at high speed or at a low cost: Digital Phones (Cell and otherwise)
Encryption chips too! Various scientific measuring devices
Others are considered General Purpose Computers Anything that can be computed, can be
done on one of these machines
Theory of Computation The theory of computation aims to
answer the following questions: 1. What is a general purpose computer?2. What problems can I solve with a general
purpose computer?3. Is this specific computer general purpose?4. Given a general purpose computer, how
difficult will it be to solve a specific problem?
Alan Turing: “The Father of Computer Science” Successful mathematician Cryptographer Helped build some early
(classified) computation devices Many ideas predated the first
computers Turing Machine Computability
Helped define what is possible on a computer, and what is not
Computable Functions A function is a mapping of inputs to outputs
Sum(2+2) = 4 Feet-Centimeter(500) = 15,240 Sort([1,3,2,3,6,9,7,8,0]) = [0,1,2,3,3,6,7,8,9] Father(Bill Smith) = Edward Smith
Some functions are computable Given the input, an algorithmic process can
always be applied to get an exact answer for the output
A general purpose computer can compute any computable function, and no others
Turing Machines
Control Unit: The actual machine Tape: infinitely long memory Read/Write Head: Used to read information from the tape,
erase information, write new information Reads one character at a time Moves left/right one position at a time
State: Description of current situation, based on tape values
State = START
How a Turing Machine Works1. Each new Turing Machine has an alphabet
of characters that it understands, and a set of states that help it make decisions
2. Given the state the current character read by the the read/write head, and a program of execution, the Control Unit decides to:
a) Stop running (HALT state)b) Write over the current characterc) Move one space left/rightd) Change States
A More Complex Machine Alphabet: {0,1,*}
A single binary number is represented as *101010*
States: ADD,RETURN,CARRY,OVERFLOW,HALT
Program to increase a positive binary integer by 1:
Useful Turing Machine Facts Multiple Turing machines are no more powerful
(though possibly faster) than a single Turing Machine
Any Turing Machine can ‘simulate’ another Turing Machine
Result: We can use unambiguous complex commands in the control unit’s program! Command: “Move 5 Spaces to the left”
Turing Machine reads: “Execute the Turing Machine routine that moves 5 spaces to the left”
Theoretically speaking, one should typically demonstrate the sub-program is computable first
Church-Turing Thesis
Any function that can be computed using a Turing Machine is also computable using any other general purpose computer (i.e., the function is computable)
SO WHAT???Who Cares?
Impact of Church-Turing Thesis1. If Power = ‘number of functions I can
compute,’ then a Turing machine is the most powerful computer imaginable Or, at least, it ties with any other computer It computes ALL of the computable functions!
2. If a Turing machine can’t solve a problem, then neither can a real computer, no matter how ‘powerful’ it is
Modern Computers As Turing Machines
Control Unit = Processor Alphabet = {0,1} States = Op Codes Read/Write Head = BUS Programs = Software Tape = RAM
Infinite????? No, but for most purposes it is long enough to solve the
problem Tape = External Storage
Only limited by the number of natural resources we can obtain from the entire universe (so, probably infinite!)
Bare-Bones Computer Language Programming languages usually market their
‘features’ Meant to make programming easier
Bare-Bones Language: Only includes features that are 100% necessary to be
equivalent to a Turing machine: Variable names: all variables are in binary clear statement: set a variable = 0 (clear X;) incr statement: increase a variable by 1 ( incr X; ) decr statement: decrease a variable by 1 (decr X;) While/end: continue execution until a variable = 0
while X not 0 do;….end;
Group Work! Can you use Bare-Bones to:
Set the variable Z = 4? Add X + Y = Z? (use one variable each for
X,Y,Z) Copy the value of X into Y?
About the Bare-Bones Language Computer scientists have proven that any
computer that can execute the Bare-Bones language is equivalent in power to a Turing Machine Heaven forbid!
Useful conclusion: Any programming language does at least
the same as Bare-Bones (hopefully more!) will also be Turing Equivalent
The extra features are just for convenience
Where We’ve Been Computers are just tools for completing
computations Theory of computation: what is possible/impossible
for all computers? What is computable? Turing Machine: imaginary ‘all powerful’ computer
Church-Turing thesis states no computer can do better Modern computers are equivalent to Turing
Machines Any algorithm we implement on a computer is
computable
Where We’re headed It’d be nice to know, before we start if a
problem is noncomputable Halting problem as an example
Even if a problem is computable, it would be nice to know in advance if it is easy or hard to solve
Even if we can solve a problem, it would be nice to know how long it will take to solve it Save effort in solving complex problems Take advantage of complexity
The Halting Problem Some problems can’t be solved. Consider: Given the source code for any
computer program, can you analyze the code and decide if it will it ever stop running?
The Halting ProblemDoes this program halt?
int X = 3while( X > 0)x = x -1
The Halting ProblemDoes this program halt?
int X = 3repeatx = x +1;until x = 0
The Halting Problem
How about this program?virtual void estimate_sigmas(){sigmas = std::vector< std::vector<Matrix> >(num_clusters);for(int i = 0; i<num_clusters; i++){sigmas[i] = std::vector<Matrix>(num_clusters);}for(int i = 0; i<num_clusters; i++){for(int j = 0; j<num_clusters; j++){sigmas[i][j] = zero_matrix<double>(proper_size,proper_size);}}Vector temp_v(proper_size);edge_iterator ebg,end;int c_i,c_j;for(tie(ebg,end) = edges(data); ebg!=end; ++ebg){if(data[*ebg].type == edge_type && data[*ebg].exists){c_i = data[source(*ebg,data)].clustering();c_j = data[target(*ebg,data)].clustering();temp_v = get_edge_vector(*ebg) - ic_means[c_i][c_j]sigmas[c_i][c_j] = sigmas[c_i][c_j] + outer_prod(temp_v,trans(temp_v))/observed_edge_prob(c_i,c_j);
}}}
Computability Explained Sometimes, we can work out answers for simple,
example inputs of hard problems, but: What algorithm did you use to decide for the first
two programs? Can you generalize it to the third?
To prove something is not computable, we’ll use the following strategy: 1. Assume that there is an algorithm that can solve the
problem all the time2. Show that, regardless of how the algorithm works, that
there is at least one case where the algorithm will fail1. Contradicts part 1, which claimed it ‘always works’
The Halting Problem Is Not Computable (PROOF!)
1. Assume, for the sake of contradiction, that there exists a computer program that, given any other computer program as input, can tell us if it stops:
STOPS(Program)2. Make a new program:
Opposite(X): 1. If STOPS(X), then run forever. 2. Otherwise, stop!
3. Opposite is a program, which itself accepts program code as input. What happens when we try to run
Opposite(Oppisite)?1. If STOPS(Opposite), then Opposite will run forever2. Otherwise, stop!
Halting Problem Implications Existence proof: Since there’s one program that
exists which we can’t compute if it halts, then there may be (probably are) others
If there’s one problem that seems computable, but is not, then there are others Look up Wang Tiles for an interesting example!
Program Analysis: “Does my program compute X?” Any place in the code where X is computed, add a
HALT command Changes to “Does my program ever halt?”
Algorithmic Complexity Algorithmic Complexity refers to how
many resources (time and memory) a computer will need to solve a problem How long will it take to process all the data? How much space (Memory) will we need? If we use more space, will it take less time? Are some problems harder to solve than
others? Can we figure that out before we try to solve them?
How can we take advantage of complexity?
Problems Vs. Solutions It’s one thing to take a specific algorithm and say it’s complex (or
not):PROCEDURE Add(X,Y):
sum = X + YRETURN sum
Because solving the problem, and doing so efficiently, are two different things:
PROCEDURE BadAdd(X,Y):Z = 1000000000sum = 0REPEAT Z = Z - 1UNTIL Z = 0sum = X + YRETURN sum
To say that a PROBLEM is difficult, you need to prove that there are no easy ways to solve it
Run-Time Complexity Looked at briefly in chapter 5 Principal method for analyzing algorithm complexity:
How many steps does it take to complete the entire algorithm?
Steps are often based on the size of the input: PROCEDURE add-all(L):
sum = 0count = 0WHILE count < length(L):sum = sum + L[count]RETURN sum
How many steps to add a list with 10 numbers? 1000 numbers?
Space Complexity Many algorithms only need enough
space to hold the input data Procedure add-all (L) Only needs enough
space to store L Others, because the problem is more
difficult, use supplementary data EX: Binary Search Trees
Still others use extra memory to be faster EX: Dynamic Programming Fibonacci
sequence
Complexity Classes The most reoccurring algorithmic runtimes are (in order)
constant, log(N), N, N*log(N), N2, N3, and an
A polynomial problem is any problem for which the best known algorithm for solving it has time complexity that is no worse than a polynomial function f(N) = Nd where d can be any number and N is the size of the input.
All problems that we can solve with an exact solution is a reasonable amount of time are in the polynomial class
Problems outside this class are referred to as intractible For short, we refer to the entire set of all
polynomial problems in the world a the set P
Complexity Classes, continued. A Nondeterministic machine: is a theoretical
machine that just knows how to solve a problem, no matter how hard it may be
A Nondeterministic Polynomial Problem is a problem for which the best known algorithm for solving has a polynomial runtime, but its execution would require a nondeterministic machine
Another intuition: these are problems for which finding the solution is hard, but checking the solution for correctness is easy
We refer to the set of problems in this domain as NP
NP DOES NOT MEAN “NOT POLYNOMIAL”
NP-Completeness In General, the class of problems in NP consists of
problems that are difficult, but useful Traveling Salesman example—best solution is exponential
Within a single class, some problems are harder than others In P, it is harder to sort a list than it is to add two
numbers In the class NP, we identify a set of problems that are
the most difficult to solve in the entire set Called NP-Complete problems The speed at which we can solve these problems
determines how fast we can solve the lesser problems
P vs. NP All problems that are in the set P must also be
in the set NP Why?
Big, unknown question in Computer Science: DOES P = NP???
What does it mean if P = NP? One approach: Find a polynomial solution for
an NP-Complete problem Thousands have tried, all have failed
Most people believe, but can’t prove P NP