Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity
description
Transcript of Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity
![Page 1: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/1.jpg)
Church, Kolmogorov and von Neumann:Their Legacy Lives in Complexity
Lance FortnowNEC Laboratories America
![Page 2: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/2.jpg)
1903 – A Year to Remember
![Page 3: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/3.jpg)
1903 – A Year to Remember
![Page 4: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/4.jpg)
1903 – A Year to Remember
Kolmogorov Church von Neumann
![Page 5: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/5.jpg)
Andrey Nikolaevich Kolmogorov
Born:April 25, 1903Tambov, Russia
Died:Oct. 20, 1987
![Page 6: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/6.jpg)
Alonzo Church
Born:June 14, 1903Washington, DC
Died:August 11, 1995
![Page 7: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/7.jpg)
John von Neumann
Born:Dec. 28, 1903Budapest, Hungary
Died:Feb. 8, 1957
![Page 8: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/8.jpg)
Frank Plumpton Ramsey
Born:Feb. 22, 1903Cambridge, England
Died:January 19, 1930
Founder of Ramsey Theory
![Page 9: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/9.jpg)
Ramsey Theory
![Page 10: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/10.jpg)
Ramsey Theory
![Page 11: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/11.jpg)
Applications of Ramsey Theory
Logic Concrete Complexity Complexity Classes Parallelism Algorithms Computational Geometry
![Page 12: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/12.jpg)
John von Neumann
Quantum Logic Game Theory Ergodic Theory Hydrodynamics Cellular Automata Computers
![Page 13: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/13.jpg)
The Minimax Theorem (1928)
Every finite zero-sum two-person game has optimal mixed strategies.
Let A be the payoff matrix for a player.
max min min maxT T
Y YX XX AY X AY
![Page 14: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/14.jpg)
The Yao Principle (Yao, 1977)
Worst case expected runtime of randomized algorithm for any input equals best case running time of a deterministic algorithm for worst distribution of inputs.
Invaluable for proving limitations of probabilistic algorithms.
![Page 15: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/15.jpg)
Making a Biased Coin Unbiased
Given a coin with an unknown bias p, how do we get an unbiased coin flip?
![Page 16: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/16.jpg)
Making a Biased Coin Unbiased
Given a coin with an unknown bias p, how do we get an unbiased coin flip?
TAILS
HEADS
or Flip Again
![Page 17: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/17.jpg)
Making a Biased Coin Unbiased
Given a coin with an unknown bias p, how do we get an unbiased coin flip?
TAILS
HEADS
or Flip Again
(1-p)p
p(1-p)
![Page 18: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/18.jpg)
Weak Random Sources
Von Neumann’s coin flipping trick (1951) was the first to get true randomness from a weak random source.
Much research in TCS in 1980’s and 90’s to handle weaker dependent sources.
Led to development of extractors and connections to pseudorandom generators.
![Page 19: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/19.jpg)
Alonzo Church
Lambda Calculus Church’s Theorem
No decision procedure for arithmetic.
Church-Turing Thesis Everything that is
computable is computable by the lambda calculus.
![Page 20: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/20.jpg)
The Lambda Calculus
Alonzo Church 1930’s A simple way to define and manipulate
functions. Has full computational power. Basis of functional programming
languages like Lisp, Haskell, ML.
![Page 21: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/21.jpg)
Lambda Terms
x xy x.xx
Function Mapping x to xx xy.yx
Really x(y(yx)) xyz.yzx(uv.vu)
![Page 22: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/22.jpg)
Basic Rules
-conversion x.xx equivalent to y.yy
-reduction x.xx(z) equivalent to zz
Some rules for appropriate restrictions on name clashes (x.(y.yx))y should not be same as y.yy
![Page 23: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/23.jpg)
Normal Forms
A -expression is in normal form if one cannot apply any -reductions.
Church-Rosser Theorem (1936) If a -expression M reduces to both A and B
then there must be a C such that A reduces to C and B reduces to C.
If M reduces to A and B with A and B in normal form, then A = B.
![Page 24: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/24.jpg)
Power of -Calculus
Church (1936) showed that it is impossible in the -calculus to decide whether a term M has a normal form.
Church’s Thesis Expressed as a Definition An effectively calculable function of the
positive integers is a -definable function of the positive integers.
![Page 25: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/25.jpg)
Computational Power
Kleene-Church (1936) Computing Normal Forms has equivalent
power to the recursive functions of Turing machines.
Church-Turing Thesis Everything computable is computable by a
Turing machine.
![Page 26: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/26.jpg)
Andrei Nikolaevich Kolmogorov
Measure Theory Probability Analysis Intuitionistic Logic Cohomology Dynamical
Systems Hydrodynamics
![Page 27: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/27.jpg)
Kolmogorov Complexity
A way to measure the amount of information in a string by the size of the smallest program generating that string.
min :p
K x p U p x
![Page 28: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/28.jpg)
Incompressibility Method
For all n there is an x, |x| = n, K(x) n. Such x are called random. Use to prove lower bounds on various
combinatorical and computational objects. Assume no lower bound. Choose random x. Get contradiction by giving
a short program for x.
![Page 29: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/29.jpg)
Incompressibility Method
Ramsey Theory/Combinatorics Oracles Turing Machine Complexity Number Theory Circuit Complexity Communication Complexity Average-Case Lower Bounds
![Page 30: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/30.jpg)
Complexity Uses of K-Complexity
Li-Vitanyi ’92: For Universal Distributions Average Case = Worst Case
Instance Complexity Universal Search Time-Bounded Universal Distributions Kolmogorov characterizations of
computational complexity classes.
![Page 31: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/31.jpg)
Rest of This Talk
Measuring sizes of sets using Kolmogorov Complexity
Computational Depth to measure the amount of useful information in a string.
![Page 32: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/32.jpg)
Measuring Sizes of Sets
How can we use Kolmogorov complexity to measure the sizeof a set?
![Page 33: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/33.jpg)
Measuring Sizes of Sets
How can we use Kolmogorov complexity to measure the sizeof a set?
Strings of length n
![Page 34: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/34.jpg)
Measuring Sizes of Sets
How can we use Kolmogorov complexity to measure the sizeof a set?
Strings of length n
An
![Page 35: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/35.jpg)
Measuring Sizes of Sets
How can we use Kolmogorov complexity to measure the sizeof a set?
The string in An of highest Kolmogorov complexity tells us about |An|.
Strings of length n
An
![Page 36: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/36.jpg)
Measuring Sizes of Sets
There must be a string x in An such that K(x) ≥ log |An|.
Simple counting argument, otherwise not enough programs for all elements of An.
Strings of length n
An
![Page 37: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/37.jpg)
Measuring Sizes of Sets
If A is computable, or even computably enumerable then every string in An hasK(x) ≤ log |An|.
Describe x by A and index of x in enumeration of strings of An.
Strings of length n
An
![Page 38: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/38.jpg)
Measuring Sizes of Sets
If A is computable enumerable then
Strings of length n
An max log n
x AK x A
![Page 39: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/39.jpg)
Measuring Sizes of Sets in P
What if A is efficiently computable?
Do we have a clean way to characterize the size of A using time-bounded Kolmogorov complexity?
Strings of length n
An
![Page 40: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/40.jpg)
Time-Bounded Complexity
Idea: A short description is only useful if we can reconstruct the string in a reasonable amount of time.
min : in stepst
pK x p U p x t x
![Page 41: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/41.jpg)
Measuring Sizes of Sets in P
It is still the case that some element x in An has Kpoly(x) ≥ log |A|.
Very possible that there are small A with x in A with Kpoly(x) quite large.
Strings of length n
An
![Page 42: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/42.jpg)
Measuring Sizes of Sets in P
Might be easier to recognize elements in A than generate them.
Strings of length n
An
![Page 43: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/43.jpg)
Distinguishing Complexity
Instead of generating the string, we just need to distinguish it from other strings.
( , ) 1
min : ( , ) 0 for ( , ) uses (| | | |) time
t
p
U p xKD x p U p y y x
U p y t x y
![Page 44: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/44.jpg)
Measuring Sizes of Sets in P
Ideally would like
True if P = NP. Problem: Need to
distinguish all pairs of elements in An
Strings of length n
An max logpoly n
x AKD x A
![Page 45: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/45.jpg)
Measuring Sizes of Sets in P
Intuitively we need
Buhrman-Laplante-Miltersen (2000) prove this lower bound in black-box model.
2max log 2logpoly n n
x AKD x A A
![Page 46: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/46.jpg)
Measuring Sizes of Sets in P
Buhrman-Fortnow-Laplante (2002) show
We have a rough approximation of size
max 2logpoly n
x AKD x A
log max 2logn poly n
x AA KD x A
![Page 47: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/47.jpg)
Measuring Sizes of Sets in P
Sipser 1983: Allowing randomness gives a cleaner connection.
Sipser used this and similar results to show how to simulate randomness by alternation.
max | logpoly n
x AKD x r A
![Page 48: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/48.jpg)
Useful Information
Simple strings convey small amount of information. 00000000000000000000000000000000
Random string have lots of information 00100011100010001010101011100010
Random strings are not that useful because we can generate random strings easily.
![Page 49: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/49.jpg)
Logical Depth
Chaitin ’87/Bennett ’97 Roughly the amount of time needed to
produce a string x from a program p whose length is close to the length of the shortest program for x.
![Page 50: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/50.jpg)
Computational Depth
Antunes, Fortnow, Variyam and van Melkebeek 2001
Use the difference of two Kolmogorov measures.
Deptht(x) = Kt(x) – K(x) Closely related to “randomness deficiency”
notion of Levin (1984).
![Page 51: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/51.jpg)
Applications
Shallow Sets Generalizes random and sparse sets with
similar computational power. L is “easy on average” iff time required is
exponential in depth. Can easily find satisfying assignment if
many such assignments have low depth.
![Page 52: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/52.jpg)
1903 – A Year of Geniuses
Several great men that helped create the fundamentals of computer science and set the stage for computational complexity.
![Page 53: Church, Kolmogorov and von Neumann: Their Legacy Lives in Complexity](https://reader036.fdocuments.us/reader036/viewer/2022062521/568167f1550346895ddd6391/html5/thumbnails/53.jpg)
2012 - The Next Celebration
Alan Turing Born:
June 23, 1912London, England
Died:June 7, 1954