CSE 200 - Computability and ComplexityWhat is computation?

Russell Impagliazzo and Jiawei Gao

Sept, 2016

What is this class about?

Computability Which problems can be solved throughcomputation? Which problems cannot be solved?

Complexity If a solution to a problem can be computed, howexpensive is this computation in terms of theresources (e.g., time, memory, energy) required toperform it?

What is computation? What is a computer?

Is this a computer?

Are these computers?

Are these computers?

How about these?

One definition

From Wikipedia“A computer is a device that can be instructed to carry out anarbitrary set of arithmetic or logical operations automatically. ”

In other words a computer should be a physical object, but capableof carrying out an arbitrary (rather than specific) algorithm,automatically (without active supervision by an outsider).

Algorithm?

This begs the question : What is an algorithm? (Discussion)

1. What does an algorithm get from the user?

2. What does an algorithm produce for the user?

3. When is an algorithm solving a problem?

4. How does the user specify which algorithm to use?

5. What can an algorithm actually do (atomically)?

6. How does the algorithm “know” what to do next?

1. What does an algorithm get from the user?

Finitely specified information, in the form of a finite lengthsequence of symbols, x = x1...xn.

2. What does an algorithm produce for the user?

Finitely specified information, in the form of a finite lengthsequence of symbols y = y1...ym.

3. When is an algorithm solving a problem?

Some relationship R definining the problem holds, R(x , y)

4. How does the user specify which algorithm to use?

An algorithm must have a finite specification that completelydetermines it.

5. What can an algorithm actually do (atomically)?

Each step must be automatic, a fixed finitely describedoperation with no room for judgement or higher cognition.

6. How does the algorithm “know” what to do next?

The series of steps to be performed must be unambiguouslydetermined by the algorithm description.

1. What does an algorithm get from the user?Finitely specified information, in the form of a finite lengthsequence of symbols, x = x1...xn.

2. What does an algorithm produce for the user?

Finitely specified information, in the form of a finite lengthsequence of symbols y = y1...ym.

3. When is an algorithm solving a problem?

Some relationship R definining the problem holds, R(x , y)

4. How does the user specify which algorithm to use?

An algorithm must have a finite specification that completelydetermines it.

5. What can an algorithm actually do (atomically)?

Each step must be automatic, a fixed finitely describedoperation with no room for judgement or higher cognition.

6. How does the algorithm “know” what to do next?

The series of steps to be performed must be unambiguouslydetermined by the algorithm description.

1. What does an algorithm get from the user?Finitely specified information, in the form of a finite lengthsequence of symbols, x = x1...xn.

2. What does an algorithm produce for the user?Finitely specified information, in the form of a finite lengthsequence of symbols y = y1...ym.

3. When is an algorithm solving a problem?

Some relationship R definining the problem holds, R(x , y)

4. How does the user specify which algorithm to use?

An algorithm must have a finite specification that completelydetermines it.

5. What can an algorithm actually do (atomically)?

Each step must be automatic, a fixed finitely describedoperation with no room for judgement or higher cognition.

6. How does the algorithm “know” what to do next?

The series of steps to be performed must be unambiguouslydetermined by the algorithm description.

1. What does an algorithm get from the user?Finitely specified information, in the form of a finite lengthsequence of symbols, x = x1...xn.

2. What does an algorithm produce for the user?Finitely specified information, in the form of a finite lengthsequence of symbols y = y1...ym.

3. When is an algorithm solving a problem?Some relationship R definining the problem holds, R(x , y)

4. How does the user specify which algorithm to use?

An algorithm must have a finite specification that completelydetermines it.

5. What can an algorithm actually do (atomically)?

Each step must be automatic, a fixed finitely describedoperation with no room for judgement or higher cognition.

6. How does the algorithm “know” what to do next?

The series of steps to be performed must be unambiguouslydetermined by the algorithm description.

1. What does an algorithm get from the user?Finitely specified information, in the form of a finite lengthsequence of symbols, x = x1...xn.

2. What does an algorithm produce for the user?Finitely specified information, in the form of a finite lengthsequence of symbols y = y1...ym.

3. When is an algorithm solving a problem?Some relationship R definining the problem holds, R(x , y)

4. How does the user specify which algorithm to use?An algorithm must have a finite specification that completelydetermines it.

5. What can an algorithm actually do (atomically)?

Each step must be automatic, a fixed finitely describedoperation with no room for judgement or higher cognition.

6. How does the algorithm “know” what to do next?

The series of steps to be performed must be unambiguouslydetermined by the algorithm description.

1. What does an algorithm get from the user?Finitely specified information, in the form of a finite lengthsequence of symbols, x = x1...xn.

2. What does an algorithm produce for the user?Finitely specified information, in the form of a finite lengthsequence of symbols y = y1...ym.

3. When is an algorithm solving a problem?Some relationship R definining the problem holds, R(x , y)

4. How does the user specify which algorithm to use?An algorithm must have a finite specification that completelydetermines it.

5. What can an algorithm actually do (atomically)?Each step must be automatic, a fixed finitely describedoperation with no room for judgement or higher cognition.

6. How does the algorithm “know” what to do next?

The series of steps to be performed must be unambiguouslydetermined by the algorithm description.

1. What does an algorithm get from the user?Finitely specified information, in the form of a finite lengthsequence of symbols, x = x1...xn.

2. What does an algorithm produce for the user?Finitely specified information, in the form of a finite lengthsequence of symbols y = y1...ym.

3. When is an algorithm solving a problem?Some relationship R definining the problem holds, R(x , y)

4. How does the user specify which algorithm to use?An algorithm must have a finite specification that completelydetermines it.

5. What can an algorithm actually do (atomically)?Each step must be automatic, a fixed finitely describedoperation with no room for judgement or higher cognition.

6. How does the algorithm “know” what to do next?The series of steps to be performed must be unambiguouslydetermined by the algorithm description.

Church and Turing

Alonzo Church and Alan Turing were among the first to try tomake mathematical definitions of “computation”. Church’slambda calculus was an abstract programming language, wherealgorithms could be specified. Turing machines were an abstractcomputer architecture making it precise what qualified as a “step”of computation.

Church-Turing thesis

Kleene observed that the functions describable in lambda calculuswere exactly those computable by Turing Machines. He formulatedthe famous “Church-Turing thesis”, that this formal notion ofcomputability captures the human intuition about what counts ascomputation. In other words, all reasonable models of computationshould define the same class of computable functions. This is not amathematical conjecture, in that it cannot be provedmathematically, but it could be refuted if we had a formal model ofalgorithm that was intuitively performing computation, but couldnot be simulated by a Turing machine. IMO, there is no seriouschallenger to the Church-Turing thesis.

Efficient Church-Turing thesis?

In complexity, we could extend “computable” to “efficientlycomputable”. But as we shall see shortly, the best we could hopefor is “reasonable models of computation approximately agree inhow efficient the best algorithms are”. More precisely, we shall seethat most models of efficient computation agree on the complexityof problems up to polynomials. That is, if one model can solve aproblem in T (n) time, another can solve it in at most (n + T (n))c

time for some fixed c .We can summarize this as “All reasonable models of computationagree on which problems are solvable in polynomial time”.Even this has challenges, notably randomized and quantumcomputation.

TM definition

Tapes A number of tapes k ≥ 1. Each has a tape head thatis initially at cell 0 of the tape.

Input alphabet Σ. The input x1...xn, will in cells 1..n of the firsttape.

Tape alphabet Γ. Σ ⊂ Γ, and in addition, it contains at least twonew symbols ⊥ marking the beginning of the tape,and β representing an empty tape cell. Each tape cellholds a symbol from Γ at all times, the cells that arenot part of the input or marking the start of the tapeare initialized to β.

TM definition continued

Actions At each time step, the machine can perform one ofthe following actions:

I Lefti : move the tape head on tape i to the left,Righti : same, to the right;

I Writei (σ): write a σ to the current position ofthe tape head on tape i ;

I Returni : return the contents of tape i as outputand halt;

I Accept: halt and return “Accept”;I Reject: halt and return “Reject”.

States Q the states of the control, one of which isdesignated the starting state, q0

Transition Function δ gets as input q ∈ Q, σ1, ..σk from Σ andoutputs q′ (new state) and a ∈ Actions (what themachine should do this step).

Runs of TM’s

A configuration of a TM M consists of the following information

1. The position of each tape head

2. The contents of each cell of each tape

3. The state of the control

On input x = x1...xn, xi ∈ Σ, the initial configuration of the TMM is determined: input on tape 1 preceded by ⊥, and followed byindefinite number of β’s, all other tapes empty (i.e., all β’s), withtape heads at position 1, and state q0. Each step, we can applythe transition function to the current contents of the cells underthe tape heads and current state to get a new state and an action.Performing the action updates the configuration, to give us thenext configuration.The output is determined when one of the three halting actionsAccept,Reject,Return is performed.

Decidability and recognizability

Most of the problems we will consider are either decision problemsor can be rephrased as a decision problem. A decision problem isone where the correct answer is either “Yes” or “No”. In this case,the TM will either Accept for “Yes” or Reject for “No”.We can identify a decision problem with a language L ⊂ Σ∗, theset of “Yes” inputs. The machine M decides L if on any x ∈ L, iteventually Accepts and any x 6∈ L, it eventually Rejects. (If italways Accepts x ∈ L, but might not halt if x 6∈ L, we say it“recognizes” L).

DefinitionREC or COMP is the class of all decision problems decidable bysome machine, RE or CE the class recognizable.

Time Complexity

The time of the machine M on x , TM [x ] is the number of steps Mtakes until M halts; if it never happens, the time is infinite.TM(n) = maxx ,|x |=nTM [x ] is the complexity function of the TM.

DefinitionFor a function T (n), TIME (T (n)) is the class of decision problemsdecidable by a TM M with TM(n) ∈ O(T (n)).

DefinitionP = ∪iTIME (ni ).

Church-Turing thesis more precisely

We can (and will) define analogs of these classes for other modelsof computation. We’ll use the subscript “Model” to say the classrelative to some other model.The Church-Turing thesis can be then stated: For any reasonablemodel of computation, COMPModel = COMP.The Time-bounded version is: PModel = P.It would really be nice if we had a fine-grained time boundedversion, e.g., TIMEModel(T (n)) = TIME (T (n)). We will see soonwhy this is too much to hope for.

Does a TM meet our criteria?

1. What does an algorithm get from the user?Finitely specified information, in the form of a finite lengthsequence of symbols, x = x1...xn.

2. What does an algorithm produce for the user?Finitely specified information, in the form of a finite lengthsequence of symbols y = y1...ym.

3. When is an algorithm solving a problem?Some relationship R definining the problem holds, R(x , y)

4. How does the user specify which algorithm to use?An algorithm must have a finite specification that completelydetermines it.

5. What can an algorithm actually do (atomically)?Each step must be automatic, a fixed finitely describedoperation with no room for judgement or higher cognition.

6. How does the algorithm “know” what to do next?The series of steps to be performed must be unambiguouslydetermined by the algorithm description.

But are TM’s a good model for computers?

Reasons why TMs are too strong:

1.

2.

3.

Reasons why TMs are too weak:

1.

2.

3.

Sandwiching

We might never have a single model that addresses all concernssimultaneously. But if we can show that both a model that seemstoo weak and one that seems too strong are equivalent to TM’s,that is evidence that TM’s approximate whatever real-worldcomputation is.Our too weak model will be one-tape TM, and our too strongmodel will be a variant of RAM.