LimitationsLimitations

Limitations of ComputationLimitations of Computation

Hardware precisionHardware precision Software - we’re humanSoftware - we’re human ProblemsProblems

– complex problem formulation complex problem formulation (weather?)(weather?)

– complexitycomplexity– inherent uncomputabilityinherent uncomputability

HardwareHardware

Read text section 17.1Read text section 17.1 arithmetic precisionarithmetic precision

– but use extra wordsbut use extra words– be attentive to numerical roundoff effectsbe attentive to numerical roundoff effects

physical componentsphysical components– build in redundancy, test, maintainbuild in redundancy, test, maintain

communicationscommunications– check-digits, parity bits, error-correcting codescheck-digits, parity bits, error-correcting codes

are we reaching the limit of Moore’s law?are we reaching the limit of Moore’s law?

SoftwareSoftware

Read text section 17.2Read text section 17.2 Software complexitySoftware complexity

– software engineeringsoftware engineering– formal i/o specificationformal i/o specification– code reviews, inspection, testingcode reviews, inspection, testing– formal verificationformal verification

often too complex, but has been usefuloften too complex, but has been useful

ProblemsProblems

Complex problemsComplex problems– many factors and variablesmany factors and variables– difficulty modelingdifficulty modeling– no “closed form” no “closed form”

temp tomorrow = temp. today * 1.1 - pressure…temp tomorrow = temp. today * 1.1 - pressure…

– solved via large-scale simulation with lots solved via large-scale simulation with lots of math dictating behavior of “cells” based of math dictating behavior of “cells” based on adjacent cells. turn crank, see what on adjacent cells. turn crank, see what happens.happens.

– as moments increase, accuracy decreasesas moments increase, accuracy decreases

ProblemsProblems

Computational ComplexityComputational Complexity

Running time of an algorithm is a function of input Running time of an algorithm is a function of input sizesize

Some functions grow faster than othersSome functions grow faster than others WAY fasterWAY faster For some problems, we have efficient algorithms For some problems, we have efficient algorithms

(functions that grow moderately). (functions that grow moderately). For others, the best known algorithms have running For others, the best known algorithms have running

times that grow so fast that they are impractical, and times that grow so fast that they are impractical, and always will be.always will be.

Some problems provably cannot be solved with a Some problems provably cannot be solved with a faster algorithmfaster algorithm

Spreadsheet of running Spreadsheet of running timestimes

and execution times at 1 billion and execution times at 1 billion steps/secsteps/sec

exercise: exercise:

- create columns for n, n- create columns for n, n22, n, n33, 2, 2nn, 3, 3nn, , n!n!

- compute running times for - compute running times for n=10,20,…n=10,20,…

UNCOMPUTABLE problemsUNCOMPUTABLE problems

previous category were problems previous category were problems that could be solved, but known that could be solved, but known methods took too longmethods took too long

this category are problems for which this category are problems for which we know: we know:

THEY CANNOT BE SOLVED BY THEY CANNOT BE SOLVED BY

ANY ALGORITHM ANY ALGORITHM WHATSOEVERWHATSOEVER

HistoryHistory

Hilbert’s “program”Hilbert’s “program”

Gödel’s undecidability theoremGödel’s undecidability theorem

Turing’s halting problemTuring’s halting problem

Natural problems and consequencesNatural problems and consequences

David HilbertDavid Hilbert

famous famous mathematicianmathematician

23 problems23 problems

founder offounder of “ “metamathematics”metamathematics”

goal to “mechanize” goal to “mechanize” mathematicsmathematics

Kurt GödelKurt Gödel

LogicianLogician Proved in 1931Proved in 1931…(age 25)…(age 25)

““there are true propositions that there are true propositions that cannot be proved” cannot be proved” (i.e., “no” to (i.e., “no” to Hilbert’s program)Hilbert’s program)

Shook the foundations ofShook the foundations of– mathematicsmathematics– philosophyphilosophy– sciencescience– everythingeverything

Alan TuringAlan Turing

British MathematicianBritish Mathematician

Seminal work in Seminal work in Computation, AI, Computation, AI, CryptanalysisCryptanalysis

Subject of several books, Subject of several books, moviesmovies

A Halting ProblemA Halting Problem

x = input (“please enter a positive x = input (“please enter a positive number)number)

WHILE x > 1:WHILE x > 1:

IF x is even:IF x is even:

x = x/2x = x/2

ELSE:ELSE:

x = 3x + 1x = 3x + 1

The Halting ProblemThe Halting Problem

Given: Program, Input to programGiven: Program, Input to program

Does the program on the input ever Does the program on the input ever halt?halt?

Critical question that pervades all of Critical question that pervades all of computationcomputation

Has much more broad consequences Has much more broad consequences than it at first appears than it at first appears

Hilbert’s 10th problemHilbert’s 10th problem

Finding integer solutions to a polynomial with Finding integer solutions to a polynomial with integer coefficients:integer coefficients:

15x15x44yy22 - 3xy - 3xy33zz22 + 2x + 2x22z = 1z = 1

No algorithm exists for finding such solutions, No algorithm exists for finding such solutions, or determining whether there are any.or determining whether there are any.

in some cases you can find solutions, or tell that there is in some cases you can find solutions, or tell that there is no solution. By “No algorithm exists” we mean there isn’t no solution. By “No algorithm exists” we mean there isn’t a general method that works for all possible polynomials.a general method that works for all possible polynomials.

Mathematical theorems…Mathematical theorems…

Given a possible theorem T, we’d like to Given a possible theorem T, we’d like to know if T is true.know if T is true.

We need to find a proof of T.We need to find a proof of T. Can write a computer program P to Can write a computer program P to

enumerate all possible proofs for T, and enumerate all possible proofs for T, and check to see if any is a correct one.check to see if any is a correct one.

If P ever halts, it has found a correct If P ever halts, it has found a correct proof.proof.

Thus, halting problem for P is question of Thus, halting problem for P is question of whether or not T is provably true.whether or not T is provably true.

Towards proving that the Towards proving that the Halting Problem is Halting Problem is

unsolvable…unsolvable…

Warm-ups: Self-reference Warm-ups: Self-reference leads to paradoxleads to paradox

In a certain town, there is a barber who In a certain town, there is a barber who shaves all and only those who don’t shaves all and only those who don’t shave themselves.shave themselves.

Homogenous words: self-describingHomogenous words: self-describing– ““english”, “short”, “polysyllabic”, …english”, “short”, “polysyllabic”, …

Heterogenous words: non-self-Heterogenous words: non-self-describingdescribing– ““spanish”, “long”, “monosyllabic”, …spanish”, “long”, “monosyllabic”, …

Warm-ups: Self-reference Warm-ups: Self-reference leads to paradoxleads to paradox

I can force you to lie, if you answer I can force you to lie, if you answer my question with only “yes” or “no”:my question with only “yes” or “no”:

Q: Will the next word you say be Q: Will the next word you say be “no”?“no”?

““Proof” of Halting problemProof” of Halting problem

Suppose there was an algorithm that Suppose there was an algorithm that could tell whether a program halted could tell whether a program halted on given input……(and worked for all on given input……(and worked for all inputs)inputs)

We will show how this leads to a We will show how this leads to a logical contradiction. When an logical contradiction. When an assumption leads to a contradiction, assumption leads to a contradiction, the assumption must be false.the assumption must be false.

““Proof” of Halting problemProof” of Halting problem

Suppose there was an algorithm that Suppose there was an algorithm that could tell whether a program halted could tell whether a program halted on given input……on given input……

program P

data D

“yes, P(D) halts”

“no, P(D) doesn’t halt”

halt-checker

Program QProgram Q

program P

data P

“yes, P(P) halts”

“no, P(P) doesn’t halt”

program P

purposely get into infinite loop!!

halt and print “Q halts”

halt-checker

Program Q when given itselfProgram Q when given itself

program Q

data Q

yes, Q(Q) halts

no, Q(Q) doesn’t halt

program Q

purposely get into infinite loop!!

halt and print “Q halts”

So, Program Q when run with program Q as input halts if and only if Program Q when run with program Q as input doesn’t halt!

halt-checker

This is a contradiction. Thus, no “halt-checker” can exist!!

SummarySummary

Impending hardware limitations ??Impending hardware limitations ?? Software limitations due to how large Software limitations due to how large

and complex programs can become.and complex programs can become. Running-time limitations because Running-time limitations because

sometimes best algorithm runs too slow.sometimes best algorithm runs too slow. Some problems too complex to be Some problems too complex to be

modeled.modeled. Some problems just have no algorithm.Some problems just have no algorithm.