1

Time-space tradeoff lower bounds for

non-uniform computation

Paul BeameUniversity of Washington

4 July 2000

2

Why study time-space tradeoffs?

To understand relationships between the two most critical measures of computation unified comparison of algorithms with

varying time and space requirements.non-trivial tradeoffs arise frequently in

practice avoid storing intermediate results by re-

computing them

3

e.g. Sorting n integers from [1,n2]

Merge sort S = O(n log n), T = O(n log n)

Radix sort S = O(n log n), T = O(n)

Selection sort only need - smallest value output so far

- index of current element

S = O(log n) , T = O(n2)

4

Complexity theory

Hard problems prove L P prove non-trivial time lower bounds for

natural decision problems in PFirst step

Prove a space lower bound, e.g. S=(log n), given an upper bound on time T, e.g. T=O(n) for a natural problem in P

5

An annoyance

Time hierarchy theorems imply unnatural problems in P not solvable in

time O(n)

Makes ‘first step’ vacuous for unnatural problems

6

Non-uniform computation

Non-trivial time lower bounds still open for problems in P First step still very interesting even without

the restriction to natural problems

Can yield bounds with precise constants

But proving lower bounds may be harder

7

Talk outlineThe right non-uniform model (for now)

branching programs

Early success multi-output functions, e.g. sorting

Progress on problems in P Crawling

restricted branching programs

That breakthrough first step (and more)true time-space tradeoffs

The path ahead

8

Branching programs

x1

x4

x2

x3

x5x5

x3

x7

x1

x2 x8x7

1

0

10

9

Branching programs

x1

x4

x2

x3

x5x5

x3

x7

x1

x2 x8x7

1

0

10

To computef:{0,1} n {0,1}on input (x1,…,xn)follow path fromsource to sink

x=(0,0,1,0,...)

10

Branching program properties

Length = length of longest pathSize = # of nodes

Simulate TM’s node = configuration with input bits erased time T= Length space S=log2Size =TM space +log2n (head)

= space on an index TM polysize = non-uniform L

11

TM space complexity

x1 x2 x3 x4 … xn read-only input

working storage

output

Space = # of bitsof working storage

12

Branching program properties

Simulate random-access machines (RAMs) not just sequential access

Generalizations Multi-way version for xi in arbitrary domain D

good for modeling RAM input registers Outputs on the edges

good for modeling output tape for multi-output functions such as sorting

BPs can be leveled w.l.o.g. like adding a clock to a TM

13

Talk outlineThe right non-uniform model (for now)

branching programs

Early success multi-output functions, e.g. sorting

Progress on problems in P Crawling

restricted branching programs

That breakthrough first step (and more)true time-space tradeoffs

The path ahead

14

Success for multi-output problems

Sorting T S = (n2/log n) [Borodin-Cook 82] T S =(n2) [Beame 89]

Matrix-vector product T S = (n3) [Abrahamson 89]

Many others including Matrix multiplication Pattern matching

15

Proof ideas: layers and treesm outputs on input xat least m/r outputs in

some tree Tv

Only 2S trees Tv

Typical Claim if T/r = n, each tree Tv

outputs p correct answers on only a c-p fraction of inputs

Correct for all x implies 2Sc-

m/r is at least 1S=(m/r)=(mn/T)

v0

vr-1

v

v1

vr10

T

T/r

T/r

16

Limitation of the technique

Never more than T S = (nm) where m is number of outputs

“It is unfortunately crucial to our proof that sorting requires many output bits, and it remains an interesting open question whether a similar lower bound can be made to apply to a set recognition problem, such as recognizing whether all n input numbers are distinct.” [Cook: Turing Award Lecture, 1983]

17

Talk outlineThe right non-uniform model (for now)

branching programs

Early success multi-output functions, e.g. sorting

Problems in P Crawling

restricted branching programs

That breakthrough first step (and more)true time-space tradeoffs

The path ahead

18

Restricted branching programs

Constant-width - only a constant number of nodes per level

[Chandra-Furst-Lipton 83]

Read-once - every variable read at most once per path

[Wegener 84], [Simon-Szegedy 89], etc.

Oblivious - same variable queried per level[Babai-Pudlak-Rodl-Szemeredi 87],

[Alon-Maass 87], [Babai-Nisan-Szegedy 89]

BDD = Oblivious read-once

19

BDDs and best-partition communication complexity

Given f:{0,1}8->{0,1}Two-player game

Player A has {x1,x3,x6,x7}

Player B has {x2,x4,x5,x8} Goal: communicate fewest bits

possible to compute fPossible protocol: Player A sends

the name of node.BDD space # of bits sent for

best partition into A and B

10

x7

x1

x6

x3

x2

x5

x4

x8

A

B

20

Communication complexity ideas

Each conversation for f:{0,1}Ax{0,1}B {0,1} corresponds to a rectangle YAxYB of inputs YA {0,1}A YB {0,1}B

BDD lower bounds size min(A,B) # of rectangles in tiling of inputs

by f-constant rectangles with partition (A,B)

Read-once bounds same tiling as BDD bounds but each rectangle in

tiling may have a different partition

21

Restricted branching programs

Read-k - no variable queried > k times onany path - syntactic read-k[Borodin-Razborov-Smolensky 89],

[Okol’nishnikova 89], etc.

any consistent path - semantic read-kmany years of no results

nothing for general branching programs either

22

Uniform tradeoffs

SAT is not solvable using O(n1-) space if time is n1+o(1). [Fortnow 97] uses diagonalization works for co-nondeterministic TM’s

Extensions for SAT S=logO(1) n implies T= (n1.4142..-) deterministic

[Lipton-Viglas 99]

with up to no(1) advice [Tourlakis 00]

S= O(n1-) implies T=(n 1.618..-). [Fortnow-van Melkebeek 00]

23

Non-uniform computation [Beame-Saks-Thathachar FOCS 98]

Syntactic read-k branching programs exponentially weaker than semantic read-twice.

f(x) = “xTMx=0 (mod q)” x GF(q)n

nloglog n time (n log1-n) space for q~n

f(x) = “xTMx=0 (mod 3)” x {0,1}n

1.017n time implies (n) spacefirst Boolean result above time n for general branching

programs

24

Non-uniform computation

[Ajtai STOC 99] 0.5log n Hamming distance for x

[1,n2]n kn time implies (n log n) spacefollows from [Beame-Saks-Thathachar 98]improved to (nlog n) time by [Pagter-00]

element distinctness for x [1,n2]n

kn time implies (n) spacerequires significant extension of techniques

25

That breakthrough first step!

[Ajtai FOCS 99] f(x,y) = “xTMyx (mod 2)”

kn time implies (n) space

First result for non-uniform Boolean computation showing time O(n) space (log n)

x {0,1}n

y {0,1}2n-1

26

Ajtai’s Boolean function

0

y1

y2n-1y8y7y6yn

y4

y3

y2

My

f(x,y)= xTMyx (mod 2)

My is a modified Hankel matrix

27

Superlinear lower bounds

[Beame-Saks-Sun-Vee FOCS 00] Extension to -error randomized

non-uniform algorithms Better time-space tradeoffs

Apply to both element distinctness and f(x,y) = “xTMyx (mod 2)”

)(n/S)log/loglog(nT

28

(m,)-rectangles

An (m,)-rectangle R DX is a subset defined by

disjoint sets A,B X, DAUB SA DA, SB DB such that

R = { z | zAUB = , zA SA, zB SB }|A|,|B| m|SA|/|DA|, |SB|/|DB|

29

An (m,)-rectangle

SA

A B

SB

x1 xnmm

In general A and B may be interleaved in [1,n]

SA and SB each have density at least

DA

DB

SA

SB

30

Key lemma [BST 98]

Let program P use time T = kn space S accept fraction of its inputs in Dn

then P accepts all inputs in some (m,)-rectangle where m = n is at least 2-4(k+1) m - (S+1) r

-1 ~ 2k and r ~ k2 2k

31

Improved key lemma [Ajtai 99 s]

Let program P use time T = kn space S accept fraction of its inputs in Dn

then P accepts all inputs in some (m,)-rectangle where m = n is at least -1 and r are constants depending on k

Srm1/50k

2

32

Proving lower bounds using the key lemmas

Show that the desired function f evaluates to 1 a large fraction of the time

i.e., is large

evaluates to 0 on some input in any large (m,)-rectanglewhere large is given by the lemma bounds

or ... do the same for f

33

Our new key lemmaLet program P use time T = kn space S and

accept fraction of its inputs in Dn

Almost all inputs P accepts are in (m,)-rectangles accepted by P where m = n is at least

-1 and r are

no input is in more than O(k) rectangles

Srm2

1/8k2

)O(k2

k

34

Proving randomized lower bounds from our key lemma

Show that the desired function f evaluates to 1 a large fraction of the time

i.e, is large

evaluates to 0 on a fraction of inputs in any large-enough (m,)-rectangle

or ... do the same for f

Gives space lower bound for O(/k)-error randomized algorithms running in time kn

35

Proof ideas: layers and trees

v0

vr-1

v2

v1

vr10

kn

kn/r

kn/r

(v1,…,vr-1)

f = (v1,…,vr-1)f

# of (v1,…,vr-1) is 2S(r-1)

(v1,…,vr-1)f =

i=1

r

vi-1vif

vi-1vif can be computed inkn/r height

36

(r,)-decision forest

The conjunction of r decision trees (BP’s that are trees) of height nEach is a computed by a

(r,k/r)-decision forestOnly 2S(r-1) of themThe various accept disjoint sets of inputs

(v1,…,vr-1)f

(v1,…,vr-1)f

37

Decision forest

Assume wlog all variables read on every inputFix an input x accepted by the forestEach tree reads only a small fraction of the

variables on input xFix two disjoint subsets of trees, F and G

kn/r

T1 T2 T3 TrT4

38

Core variables

Can split the set of variables into core(x,F)=variables read only in F (=not read outside F)

core(x,G)=variables read only in G (=not read outside G)

remaining variablesstem(x,F,G)=assignment to remaining variables

General idea: use core(x,F), core(x,G), and stem(x,F,G) to define (m,)-rectangles

kn/r

T1 T2 T3 TtT4

39

A partition of accepted inputs

Fix F, G,x accepted by P Rx,F,G={ y | core(y,F)=core(x,F),

core(y,G)=core(x,G), stem(y,F,G)=stem(x,F,G),

and P accepts y}

For each F, G the Rx,F,G partition the accepted inputs into equivalence classes

Claim: the Rx,F,G are (m,)-rectangles

40

Classes are rectangles

Let A=core(x,F), B=core(x,G), =stem(x,F,G) SA={yA| y in Rx,F,G }, SB={zB| z in Rx,F,G }

Let w=(,yA,zB) w agrees with y in all trees outside G

core(w,G)=core(y,G)=core(x,G) w agrees with z in all trees outside F

core(w,F)=core(z,F)=core(x,F) stem(w,F,G)==stem(x,F,G) P accepts w since it accepts y and z

So... w is in Rx,F,G

41

Few partitions suffice

Only 4k pairs F,G suffice to cover almost all inputs accepted by P by large (m,)-rectangles Rx,F,G

Choose F,G uniformly at random of suitable size, depending on access pattern of inputprobability that F,G isn’t good is tinyone such pair will work for almost all inputs with

the given access pattern

Only 4k sizes needed.

42

Special case: oblivious BPs

core(x,F), core(x,G) don’t depend on xChoose Ti in F with prob q

G with prob q neither with prob 1-2q

43

xTMyx on an (m,)-rectangle

My

A Bx

A

B

x

For every on AUB, f(xAUB,,y)

= xAT MAB xB

+ g(xA,y) + h(xB,y)

44

Rectangles, rank, & rigidity

largest rectangle on which xATMxB is

constant has 2-rank(M)

[Borodin-Razborov-Smolensky 89]

Lemma [Ajtai 99] Can fix y s.t. every nxn minor MAB of My has rank(MAB) cn/log2(1/) improvement of bounds of

[Beame-Saks-Thathachar 98] & [Borodin-Razborov-Smolensky 89] for Sylvester matrices

45

High rank implies balance

For any rectangle SAxSB {0,1}Ax{0,1}B with (SAxSB) |A||B|23-rank(M)

Pr[ xATMxB= 1 | xA SA, xB SB] 1/32

Pr[ xATMxB= 0 | xA SA, xB SB] 1/32

derived from result for inner product in r dimensions

So rigidity also implies balance for all large rectangles and so

Also follows for element distinctness [Babai-Frankl-Simon 86]

)(n/S)log/loglog(nT

46

Talk outlineThe right non-uniform model (for now)

branching programs

Early success multi-output functions, e.g. sorting

Progress on problems in P Crawling

restricted branching programs

That breakthrough first step (and more)true time-space tradeoffs

The path ahead

47

Improving the bounds

What is the limit? T=(nlog(n/S)) ? T=(n2/S) ?

Current bounds for general BPs are almost equal to best current bounds for oblivious BPs ! T=(nlog(n/S)) using 2-party CC [AM]

T=(nlog2(n/S)) using multi-party CC [BNS]

48

Improving the bounds

(m,a)-rectangles a 2-party CC idea insight: generalizing to non-oblivious BPs yields same bound as [AM] for oblivious BPs

Generalize to multi-party CC ideas to get better bounds for general BPs? similar framework yields same bound as [BNS]

for oblivious BPs

Improve oblivious BP lower bounds? ideas other than communication complexity?

49

Extension to other problems

Problem should be hard for (best-partition) 2-party communication complexity (after most variables fixed). try oblivious BPs first

Prime candidate: (directed) st-connectivity Many non-uniform lower bounds in structured

JAG models [Cook-Rackoff], [BBRRT], [Edmonds], [Barnes-Edmonds], [Achlioptas-Edmonds-Poon]

Best-partition communication complexity bounds known

50

Limitations of current method

Need n>T/r = decision tree height else all functions trivial so r > T/n

A decision forest works on a 2-Sr fraction of the accepted inputs

•only place space bound is usedSo need Sr<n else d.f. need only work on

one inputimplies ST/n < n, i.e. T < n2/S