Physical Limits of ComputingPhysical Limits of ComputingDr. Mike FrankDr. Mike Frank

CIS 6930, Sec. #3753X CIS 6930, Sec. #3753XSpring 2002Spring 2002

Lecture #28Lecture #28Reversible Computing Theory II:Reversible Computing Theory II:Emulating Irreversible MachinesEmulating Irreversible Machines

Fri., Mar. 22Fri., Mar. 22

Administrivia & OverviewAdministrivia & Overview• Don’t forget to keep up with homework!Don’t forget to keep up with homework!

– We are We are 9 out of 14 weeks into the course.9 out of 14 weeks into the course.• You should have earned You should have earned ~64 points by now.~64 points by now.

• Course outline:Course outline:– Part I&II, Part I&II, BackgroundBackground, , Fundamental LimitsFundamental Limits - done - done– Part III, Part III, Future of Semiconductor TechnologyFuture of Semiconductor Technology - done - done– Part IV, Part IV, Potential Future Computing TechnologiesPotential Future Computing Technologies - done - done– Part V, Part V, Classical Reversible ComputingClassical Reversible Computing

• Limits of adiabatics: Friction,Leakage,Power supplies. - last Mon.Limits of adiabatics: Friction,Leakage,Power supplies. - last Mon.• RevComp theory I: Reversible Logic Models - last Wed.RevComp theory I: Reversible Logic Models - last Wed.• RevComp theory II: Emulating Irreversible MachinesRevComp theory II: Emulating Irreversible Machines - TODAY - TODAY

RevComp theory II: Bounds on Space-Time OverheadsRevComp theory II: Bounds on Space-Time Overheads • Physics-based models of computing - Mon. 3/25Physics-based models of computing - Mon. 3/25• Reversible scaling advantages, low-leakage limit. - Wed. 3/27Reversible scaling advantages, low-leakage limit. - Wed. 3/27• (plus ~5 more lectures…)(plus ~5 more lectures…)

– Part VI, Part VI, Quantum ComputingQuantum Computing– Part VII, Part VII, Cosmological Limits, Wrap-UpCosmological Limits, Wrap-Up

Reversible Computing Theory II:Reversible Computing Theory II:Emulating Irreversible MachinesEmulating Irreversible Machines

Motivation for this studyMotivation for this study• We want to know how to carry out any We want to know how to carry out any arbitraryarbitrary

computation in a way that is reversible to an computation in a way that is reversible to an arbitrarily high degreearbitrarily high degree..– Up to limits set by leakage, power supply, Up to limits set by leakage, power supply, etc.etc.

• We want to do this as We want to do this as efficientlyefficiently as possible: as possible:– Using as few “device ticks” as possible (spacetime)Using as few “device ticks” as possible (spacetime)

• Minimizes HW cost, & leakage lossesMinimizes HW cost, & leakage losses– Using as few adiabatic transitions as possible (ops)Using as few adiabatic transitions as possible (ops)

• Minimizes frictional lossesMinimizes frictional losses

• But, a desired computation may be originally But, a desired computation may be originally spec’d in terms of spec’d in terms of irreversibleirreversible primitives. primitives.

General-Case vs. Special-CaseGeneral-Case vs. Special-Case• We’d like to know two kinds of things:We’d like to know two kinds of things:

– For arbitrary general-purpose computations,For arbitrary general-purpose computations,• How to How to automaticallyautomatically emulate them in a fairly efficient emulate them in a fairly efficient

reversible way, reversible way, – w/o needing new intelligent/creative design work in each case?w/o needing new intelligent/creative design work in each case?

• Topic of today’s lectureTopic of today’s lecture– For various specific computations of interest,For various specific computations of interest,

• What are the most efficient reversible algorithms?What are the most efficient reversible algorithms?– Or at least, the most efficient that we can find?Or at least, the most efficient that we can find?

• Note: These may not look anything like the most Note: These may not look anything like the most efficient efficient irreversibleirreversible algorithms! algorithms!

• More on this laterMore on this later

The Landauer embeddingThe Landauer embedding• The obvious embedding of irreversible ops into The obvious embedding of irreversible ops into

“expanding” reversible ones leads to a linear “expanding” reversible ones leads to a linear increase in space through time. (Landauer ‘61)increase in space through time. (Landauer ‘61)– Or, increase in width of an input-consuming circuitOr, increase in width of an input-consuming circuit

“Garbage”bits

input

Circuit depth, or time

“Expanding”operations

(e.g., AND)

Desiredoutput

Lecerf ReversalLecerf Reversal• Lecerf (‘63) was interested in the group-theory Lecerf (‘63) was interested in the group-theory

question of whether an iterated permutation of question of whether an iterated permutation of items would eventually return to initial item.items would eventually return to initial item.– Proved Proved undecidableundecidable by reducing Turing’s by reducing Turing’s halting halting

problemproblem to this question, w. a reversible TM. to this question, w. a reversible TM.• Reversible TM Reversible TM reverses directionreverses direction instead of halting. instead of halting.• Returns to initial state iff irreversible TM would halt.Returns to initial state iff irreversible TM would halt.

• Only problem:Only problem:No useful output data!No useful output data!

f f 1

Desiredoutput

Garbage

Input Copy ofInput

The Bennett TrickThe Bennett Trick• Bennett (‘73) pointed out that you could simply Bennett (‘73) pointed out that you could simply

fan-out (reversibly copy) the desired output fan-out (reversibly copy) the desired output before reversing.before reversing.

• Note Note OO((TT)) storage is still storage is still temporarilytemporarily needed! needed!

f f 1

Garbage

Input Copy ofInput

Desired output

Improving Spacetime EfficiencyImproving Spacetime Efficiency• Bennett ‘73 transforms a computation taking Bennett ‘73 transforms a computation taking

spacetime spacetime SS··TT to one taking to one taking ((SS··TT22) in the worst ) in the worst case. case. – Can we do better?Can we do better?

• Bennett ‘89: Described a technique that takes Bennett ‘89: Described a technique that takes spacetimespacetime– Actually, can generalize slightly and arrange for exponent Actually, can generalize slightly and arrange for exponent

on on TT to be 1+ to be 1+, where , where 0 (very slowly)0 (very slowly)

• Lange, McKenzie, Tapp ‘97: Space Lange, McKenzie, Tapp ‘97: Space ((SS) is possible, ) is possible, if you use time if you use time (exp((exp(((SS))))))– Not any more spacetime-efficient than Bennett.Not any more spacetime-efficient than Bennett.

)log()log( 59.13log2 TTSTTS

Reversible “Climbing Game”Reversible “Climbing Game”• Suppose a guy armed w. hammer, Suppose a guy armed w. hammer,

NN spikes, & rope is trying to climb a spikes, & rope is trying to climb acliff, w. the following rules.cliff, w. the following rules.– How high can he climb?How high can he climb?

• Rules:Rules:– Standing on ground or spike, canStanding on ground or spike, can

insert & remove spike 1 meter higher.insert & remove spike 1 meter higher.– Can raise & lower himself betweenCan raise & lower himself between

spikes & ground using his rope.spikes & ground using his rope.• Can’tCan’t insert or remove a spike while insert or remove a spike while

dangling from a higher spike!dangling from a higher spike!– Not enough leverage/stability?Not enough leverage/stability?

Analogy w. Emulation ProblemAnalogy w. Emulation Problem• Height on cliff = How much progress haveHeight on cliff = How much progress have

we made through the irreversiblewe made through the irreversiblecomputation?computation?

• Number of spikes = Memory of rev. mach.Number of spikes = Memory of rev. mach.• Spike at height Spike at height HH = Using memory to = Using memory to

record state of irreversible machinerecord state of irreversible machineafter after HH steps. steps.

• Adding/removing spike @Adding/removing spike @HH+1+1if spike is @if spike is @HH = Computing/ = Computing/uncomputing state uncomputing state HH+1 from +1 from HH..

1. Insert spike @ 1.

Let’s Climb!Let’s Climb!0. Standing on ground.

2. Insert spike @ 2.3. Remove spike @ 1.4. Insert spike @ 3.5. Insert spike @ 4.6. Remove spike @ 3.7. Insert spike @ 1.8. Remove spike @ 2.9. Remove spike @ 1.10. Can use remaining 3 spikes to climb up another 4 if desired!

How high can we climb?How high can we climb?• Using only Using only NN spikes, and the strategy spikes, and the strategy

illustrated, we can climb to height 2illustrated, we can climb to height 2NN11 (wow!) (wow!)• Li & Vitanyi: (Theorem) Li & Vitanyi: (Theorem) This is the optimal This is the optimal

strategy for this game.strategy for this game.• Open question: Open question:

– Are there more efficient emulation techniques that Are there more efficient emulation techniques that are are notnot based on this game model? based on this game model?

““Pebble Game” RepresentationPebble Game” Representation

Triangle representationTriangle representation

k = 2n = 3

k = 3n = 2

Analysis of Bennett AlgorithmAnalysis of Bennett Algorithm• nn = # of recursive levels of algorithm = # of recursive levels of algorithm• kk = # of lower-level iterations to go forward 1 higher- = # of lower-level iterations to go forward 1 higher-

level steplevel step• TTrr = # of reversible lowest-level steps executed = # of reversible lowest-level steps executed

= 2(2= 2(2kk1)1)nn

• TTii = # of irreversible steps emulated = # of irreversible steps emulated

= = kknn

• So, So, nn = log = logkk TTii, and so , and so

TTrr = 2(2 = 2(2kk1)1)log log TTi/log ki/log k = 2 = 2eelog(2log(2kk1)log(1)log(TTi)/log i)/log kk

= = 22TTiilog(2log(2k k 1)/log 1)/log kk

(n+1 spikes)

Linear-Space EmulationLinear-Space Emulation(Lange, McKenzie, Tapp ‘97)

Unfortunately, the tree may have 2S nodes!

Can we do better?Can we do better?• Bennett ‘73 takes order-Bennett ‘73 takes order-TT time, LMT ‘97 takes order- time, LMT ‘97 takes order-

SS space. space.– Can some technique achieve both, simultaneously?Can some technique achieve both, simultaneously?

• Theorem:Theorem: (Frank & Ammer ‘97) The problem of (Frank & Ammer ‘97) The problem of iterating a black-box function iterating a black-box function cannotcannot be done in time be done in time TT & space & space SS on a reversible machine. on a reversible machine.– Proof really covers Proof really covers allall possible algorithms! possible algorithms!– Also provides loose lower bounds on extra space required Also provides loose lower bounds on extra space required

by a time-by a time-TT simulation. simulation.

• Results Results maymay also apply to the problem of iterating a also apply to the problem of iterating a cryptographic cryptographic one-wayone-way function. function.

One-Way FunctionsOne-Way Functions• ……are invertible functions are invertible functions ff such that such that ff is easy to is easy to

compute (compute (e.g.e.g., takes polynomial time) but , takes polynomial time) but f f 11 is hard to is hard to compute (compute (e.g.e.g., exponential)., exponential).

• Example:Example:– Consider: Consider: ff((pp,,qq) = ) = pq pq with with pp,,qq prime. prime.

• Multiplication is easy.Multiplication is easy.• Factoring is hard (except with quantum computers).Factoring is hard (except with quantum computers).

– The “one-way-ness” of this function is essential to the The “one-way-ness” of this function is essential to the security of the RSA public-key cryptosystem.security of the RSA public-key cryptosystem.

• No function has been No function has been provenproven to be one-way. to be one-way.– Certain kinds of one-way functions are known to exist if Certain kinds of one-way functions are known to exist if PP

NPNP..

Elements of Frank-Ammer ProofElements of Frank-Ammer Proof• Consider a chain of bit-strings (size Consider a chain of bit-strings (size SS each) that is each) that is

incompressibleincompressible by a certain compressor. by a certain compressor.– Easily proven to exist. (See next slide.)Easily proven to exist. (See next slide.)– Machine’s job is to follow this chain from one node to Machine’s job is to follow this chain from one node to

the next by using a black-box function.the next by using a black-box function.

• The compressor can run a reversible machine The compressor can run a reversible machine backwards, to reconstruct earlier nodes in the chain backwards, to reconstruct earlier nodes in the chain from later machine configurations.from later machine configurations.– If the reversible machine only uses order-If the reversible machine only uses order-SS space in its space in its

configurations, then the chain is compressible!configurations, then the chain is compressible!– Contradicts choice of incompressible chain; QED.Contradicts choice of incompressible chain; QED.

Existence of IncompressiblesExistence of Incompressibles• A A decompressordecompressor or or description systemdescription system ss:{:{00,,11}* }*

maps any bit-string maps any bit-string descriptiondescription dd to the to the described described string string xx..– Notation Notation ff::DD means a unary operator on means a unary operator on DD, , ff::DDDD

• xx is is compressiblecompressible is is ss iff iff dd: : ss((dd)=)=xx, |, |dd|<||<|xx||– Notation |Notation |bb| means the length of bit-string | means the length of bit-string bb in bits. in bits.

• Theorem:Theorem: EveryEvery decompressor has an decompressor has an incompressible string for any given length incompressible string for any given length ..– Proof:Proof: There are 2 There are 2 -bit strings, but only-bit strings, but only

shorter descriptions. shorter descriptions.1221

0

i

i