Tricks of the Trade - Recursion to Iteration
Transcript of Tricks of the Trade - Recursion to Iteration
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
1/49
Tom Moertels Blog Home About Arch
Tricks of the trade:
Recursion to Itera-tion, Part 1: The Sim-ple Method, secret
features, and accu-mulatorsBy Tom Moertel
osted on May 11, 2013
Tags: programm!ng, recurs!on, !terat!on, python, google code jam, puzzles, recurs!on-to-!terat
er!es
Alternat!ve t!tle: I w!sh Python had ta!l-call el!m!nat!on.
Recursive programming is powerful because it maps so easily to proof by
nduction, making it easy to design algorithms and prove them correct. I
mportant because getting stuff right is what programming is all about.
But recursion is poorly supported by many popular programming lan-
guages. Drop a large input into a recursive algorithm in Python, and you
probably hit the runtimes recursion limit. Raise the limit, and you may
out of stack space and segfault.
These are not happyoutcomes.
https://plus.google.com/103957126301745359889?rel=authorhttp://blog.moertel.com/http://en.wikipedia.org/wiki/Mathematical_inductionhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/puzzles.htmlhttp://blog.moertel.com/tags/google%20code%20jam.htmlhttp://blog.moertel.com/tags/python.htmlhttp://blog.moertel.com/tags/iteration.htmlhttp://blog.moertel.com/tags/recursion.htmlhttp://blog.moertel.com/tags/programming.htmlhttps://plus.google.com/103957126301745359889?rel=authorhttp://blog.moertel.com/archive.htmlhttp://blog.moertel.com/about.htmlhttp://blog.moertel.com/http://blog.moertel.com/ -
7/23/2019 Tricks of the Trade - Recursion to Iteration
2/49
Therefore, an important trick of the trade is knowing how to translate re
ursive algorithms into iterative algorithms. That way, you can design,
prove, and initially code your algorithms in the almighty realm of recurs
Then, after youve got things just the way you want them, you can transl
your algorithms into equivalent iterative forms through a series of mech
cal steps. You can prove your cake and run it in Python, too.
This topic turning recursion into iteration is fascinating enough that
m going to do a series of posts on it. Tail calls, trampolines, continuatio
passing style and more. Lots of good stuff.
For today, though, lets just look at one simple method and one supporti
rick.
The Simple MethodThis translation method works on many simple recursive functions. Wh
t works, it works well, and the results are lean and fast. I generally try it
irst and consider more complicated methods only when it fails.
n a nutshell:
1. Study the function.
2. Convert all recursive calls into tail calls. (If you cant, stop. Try anot
method.)
3. Introduce a one-shot loop around the function body.
4. Convert tail calls into continue statements.5. Tidy up.
An important property of this method is that its incrementally correct
after every step you have a function thats equivalent to the original. So
you have unit tests, you can run them after each and every step to make
ure you didnt make a mistake.
http://en.wikipedia.org/wiki/Tail_callhttp://blog.moertel.com/tags/recursion-to-iteration%20series.html -
7/23/2019 Tricks of the Trade - Recursion to Iteration
3/49
Lets see the method in action.
Example: factorialWith a function this simple, we could probably go straight to the iterativ
version without using any techniques, just a little noggin power. But the
point here is to develop a mechan!cal processthat we can trust when our
unctions arent so simple or our noggins arent so powered. So were go
o work on a really s!mplefunction so that we can focus on the process.
Ready? Then lets show these guys how cyber-commandos get it done!M
V style!
1. Study the original function.
deffactorial(n):
ifn < 2:
return1
returnn * factorial(n - 1)
Nothing scary here. Just one recursive call. We can do th!s!
2. Convert recursive calls into tail calls.
deffactorial1a(n, acc=1):
ifn < 2:
return1* acc
returnfactorial1a(n - 1, acc * n)
(If this step seemed confusing, see the Bonus Explainer at the end o
the article for the secret feature trick behind the step.)
3. Introduce a one-shot loop around the function body. You want
while True: body ; break .
deffactorial1b(n, acc=1):
whileTrue:
ifn < 2:
return1* acc
https://www.youtube.com/watch?v=0dofacvjRkc&feature=youtube_gdata_player -
7/23/2019 Tricks of the Trade - Recursion to Iteration
4/49
returnfactorial1b(n - 1, acc * n)
break
Yes, I know putting a break after a return is crazy, but do it anyway
Clean-up comes later. For now, were going by the numbers.
4. Replace all recursive tail calls f(x=x1, y=y1, ...) with
(x, y, ...) = (x1, y1, ...); continue . Be sure to update allargumein the assignment.
deffactorial1c(n, acc=1):
whileTrue:
ifn < 2:
return1* acc
(n, acc) = (n - 1, acc * n)
continue
break
For this step, I copy the original functions argument list, parenthes
and all, and paste it over the return keyword. Less chance to screw
something up that way. Its all about being mechanical.
5. Tidy the code and make it more idiomatic.
deffactorial1d(n, acc=1):
whilen > 1:
(n, acc) = (n - 1, acc * n)
returnacc
Okay. This step is less about mechanics and more about style. Elimi
nate the cruft. Simplify. Make it sparkle.
6. Thats it. Youre finished!
What have we gained?We just did five steps of work to convert our original, recursive factorial
unction into the equivalent, iterative factorial1d function. If our pro-
gramming language had supported tail-call elimination, we could have
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
5/49
topped at step two with factorial1a . But noooooooWe had to press on
all the way through step five, because were using Python. Its almost
enough to make a cyber-commando punch a kitten.
No, the work wasnt hard, but it was still work. So what did it buy us?
To seewhat it bought us, lets look inside the Python run-time environ-
ment. Well use the Online Python Tutors visualizer to observe the build
up of stack frames as factorial , factorial1a , and factorial1d each com
pute the factorial of 5.
This is very cool, so dont miss the link: Visualize It!(ProTip: Open it in a
new tab.)
Click the Forward button to step through the execution of the functionNotice what happens in the Frames column. When factorial is computi
he factorial of 5, five frames build up on the stack. Not a coincidence.
Same thing for our tail-recursive factorial1a . (Youre right. It !stragic.)
But not for our iterative wonder factorial1d . It uses just onestack frame
over and over, until its done. Thats economy!
Thats why we did the work. Economy. We converted O(n) stack use into
tack use. When ncould be large, that savings matters. It could be the di
erence between getting an answer and getting a segfault.
Not-so-simple casesOkay, so we tackled factorial . But that was an easy one. What if your fuion isnt so easy? Then its time for more advanced methods.
Thats our topic for next time.
Until then, keep your brain recursive and your Python code iterative.
http://www.pythontutor.com/visualize.html#code=%23+original+version%0A%0Adef+factorial(n)%3A%0A++++if+n+%3C+2%3A%0A++++++++return+1%0A++++return+n+*+factorial(n+-+1)%0A%0A%23+tail-call+version%0A%0Adef+factorial1a(n,+acc%3D1)%3A%0A+++++if+n+%3C+2%3A%0A+++++++++return+1+*+acc%0A+++++return+factorial1a(n+-+1,+acc+*+n)%0A%0A%23+iterative+version%0A%0Adef+factorial1d(n,+acc%3D1)%3A%0A++++while+n+%3E+1%3A%0A++++++++(n,+acc)+%3D+(n+-+1,+acc+*+n)%0A++++return+acc%0A%0Aprint+factorial(5)%0Aprint+factorial1a(5)%0Aprint+factorial1d(5)&mode=display&cumulative=false&heapPrimitives=false&drawParentPointers=false&textReferences=false&showOnlyOutputs=false&py=2 -
7/23/2019 Tricks of the Trade - Recursion to Iteration
6/49
Bonus Explainer: Using secret fea-tures for tail-call conversionn step 2 of The Simple Method, we converted the recursive call in this co
effactorial(n):
ifn < 2:
return1
returnn * factorial(n - 1) #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
7/49
t to do the multiplication for us.
Shh! Its a secretfeature, though, just for us. Nobody else.
n essence, every call to our extended function will not only compute a fa
orial but also(secretly) mult!ply that factor!al by whatever extra value we g!v
The variables that hold these extra values are often called accumulator
o I use the name acc here as a nod to tradition.
So heres our function, freshly extended:
effactorial(n, acc=1):
ifn < 2:
returnacc * 1
returnacc * n * factorial(n - 1)
See what I did to add the secret multiplication feature? Two things.
First, I took the original function and added an additional argument acc
he multiplier. Note that it defaults to 1 so that it has no effect until we
give it some other value (since ). Gotta keep the secret secret, a
all.
Second, I changed every s!ngle return statement from return {whatever}
return acc * {whatever} . Whenever our function would have returned x
now returns acc * x . And thats it. Our secret feature is done!Andits tr
al to prove correct. (In fact, we just did prove it correct! Re-read the seco
entence.)
Both changes were entirely mechanical, hard to screw up, and, together,
default to doing nothing. These are the properties you want when adding
ecret features to your functions.
Okay. Now we have a function that computes the factorial of n and, secr
y, multiplies it by acc .
Now lets return to that troublesome line, but in our newly extended fun
1 x= x
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
8/49
ion:
returnacc * n * factorial(n - 1)
t computes the factorial of n - 1 and then multiplies it by acc * n . But
wait! We dont need to do that multiplication ourselves. Not anymore. N
we can ask our extended factorial function to do it for us, us!ng the secret f
ure.
So we can rewrite the troublesome line as
returnfactorial(n - 1, acc * n)
And thats a tail call!
So our tail-call version of the factorial function is this:
effactorial(n, acc=1):
ifn < 2:
returnacc * 1
returnfactorial(n - 1, acc * n)
And now that all our recursive calls are tail calls there was only the one
his function is easy to convert into iterative form using The Simple Metdescribed in the main article.
Lets review the Secret Feature trick for making recursive calls into tail
alls. By the numbers:
1. Find a recursive call thats not a tail call.
2. Identify what work is being done between that call and itsreturn
statement.
3. Extend the function with a secret feature to do that work, as control
by a new accumulator argument with a default value that causes it to
nothing.
4. Use the secret feature to eliminate the old work.
5. Youve now got a tail call!
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
9/49
6. Repeat until all recursive calls are tail calls.
With a little practice, it becomes second nature. So
Exercise: Get your practice on!
You mission is to get rid of the recursion in the following function. Feel lyou can handle it? Then just fork the exercise repoand do your stuff to ex
ise1.py.
effind_val_or_next_smallest(bst, x):
"""Get the greatest value
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
10/49
Tom Moertels Blog Home About Arch
Tricks of the trade:
Recursion to Itera-tion, Part 2: Elimi-nating Recursion
with the Time-Trav-eling Secret FeatureTrickBy Tom Moertel
osted on May 14, 2013
Tags: programm!ng, recurs!on, !terat!on, python, google code jam, puzzles, recurs!on-to-!terat
er!es, ta!l calls
This is the second post in a series on converting recursive algorithms int
terative algorithms. If you havent read the previous post, you probablyhould. It introduces some terms and background that will be helpful.
Last time, if youll recall, we discussed The Simple Method of converting
ursive functions into iterative functions. The method, as its name impl
s straightforward and mostly mechanical. The only potential trouble is
hat, for the methodto work, you must convert all of the recursive calls
your function into tail calls.
http://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/posts/2013-05-11-recursive-to-iterative.htmlhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttps://plus.google.com/103957126301745359889?rel=authorhttp://blog.moertel.com/http://blog.moertel.com/http://blog.moertel.com/posts/2013-05-11-recursive-to-iterative.htmlhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/tail%20calls.htmlhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/puzzles.htmlhttp://blog.moertel.com/tags/google%20code%20jam.htmlhttp://blog.moertel.com/tags/python.htmlhttp://blog.moertel.com/tags/iteration.htmlhttp://blog.moertel.com/tags/recursion.htmlhttp://blog.moertel.com/tags/programming.htmlhttps://plus.google.com/103957126301745359889?rel=authorhttp://blog.moertel.com/archive.htmlhttp://blog.moertel.com/about.htmlhttp://blog.moertel.com/http://blog.moertel.com/ -
7/23/2019 Tricks of the Trade - Recursion to Iteration
11/49
This task can be tricky. So we also discussed the Secret Featuretrick for
putting recursive calls into tail-call form. That trick works well for simp
ecursive calls, but when your calls arent so simple, you need a beefier v
ion of the trick.
Thats the subject of this post: the T!me-Travel!ng Secret Featuretrick. It
ike sending a T-800back in time to terminate a functions recursivenbefore it can ever offer resistance in the present.
Yeah.
But well have to work up to it. So stick with the early examples to prepa
or the cybernetic brain augmentations to come.
Enough talk! Lets start with a practical example.
Update 2013-06-17:If you want another example, heres also a step-by-step
onvers!on of the F!bonacc!funct!on.
Computing binomial coefficientsThe binomial coefficient for integers and gives the number of wa
o choose items from a set of items. For this reason, its often pro-
nounced choose . Its very common in combinatorics and statistics
and often pops up in the analysis of algorithms. A whole chapter, in fact
dedicated to them in Graham, Knuth, and Patashniks Concrete Mathema
Math textbooks define the coefficient like this:
but that form causes all sorts of problems for computers. Fortunately, Co
rete Mathemat!cshelpfully points out a lifesaving absorption identity:
( )nk n kk n
n k
( ) = ,n
k
n!
k!(n k)!
http://blogs.msdn.com/b/oldnewthing/archive/2013/05/08/10416823.aspxhttp://en.wikipedia.org/wiki/Concrete_Mathematicshttp://en.wikipedia.org/wiki/Binomial_coefficient#Combinatorics_and_statisticshttp://en.wikipedia.org/wiki/Binomial_coefficienthttps://gist.github.com/tmoertel/5798134http://en.wikipedia.org/wiki/Terminator_(character) -
7/23/2019 Tricks of the Trade - Recursion to Iteration
12/49
and we know what that is, dont we? Thats a recursive function just wai
o happen!
And that identity, along with the base case , gives us todays run
ning example:
efbinomial(n, k):
ifk == 0:
return1
returnn * binomial(n - 1, k - 1) // k
Before going further, a cautionary point. Math math and computer math
not the same. In math math, , but in computer math theequation does not necessarily hold because of finite precision or, in our
ase, because were dealing with integer division that throws away rema
ders. (By the way, in Python, // is the division operator that throws awa
emainders.) For this reason, I have been careful to use the form
* binomial(n - 1, k - 1) // k
nstead of the more literal translation
n // k) * binomial(n - 1, k - 1)
which, if you try it, youll see often produces the wrong answer.
Okay, our challenge is set before us. Ready to de-recursivify binomial ?
Once again, the Secret Feature trickFirst, however, we must put the functions recursive call into tail-call fo
And you remember how to do that, right? With the Secret Feature trick, o
ourse! As a refresher from last time, heres the trick in a nutshell:
( ) = ( ),n
k
n
k
n 1
k 1
( ) = 1n
0
= x = nnx
k
n
k
x
k
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
13/49
1. Find a recursive call thats not a tail call.
2. Identify what work is being done between that call and its return
statement.
3. Extend the function with a secret feature to do that work, as control
by a new accumulator argument with a default value that causes it to
nothing.
4. Use the secret feature to eliminate the old work.
5. Youve now got a tail call!
6. Repeat until all recursive calls are tail calls.
Lets go through it, by the numbers.
1. F!nd a recurs!ve call thats not a ta!l call.
Well, theres only three lines of logic. Were not talking rocket scien
here.
defbinomial(n, k):
ifk == 0:
return1
returnn * binomial(n - 1, k - 1) // k
# ^^^^^^^^^^^^^^^^^^^^^^ # right here!
2. Ident!fy what work !s be!ng done between that call and !ts return statem
In our minds eye, lets lift the recursive call out of the return state
ment and replace it with the variable x .
x = binomial(n - 1, k - 1) returnn * x // k
Now we can visualize the additional work as a separate function ope
ing on x : multiplying it on the left by one number and dividing it on
the right by another:
defwork(x, lmul, rdiv):
returnlmul * x // rdiv
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
14/49
For functions this simple, you can just hold them in your head and i
line them into your code as needed. But for more-complicated func
tions, it really does help to break them out like this. For this exampl
Im going to pretend that our work function is more complicated, ju
to show how to do it.
3. Extend the funct!on w!th a secret feature to do that work,as controlled bynew accumulator argument in this case a pair ( lmul , rdiv ) with
default value that causes it to do nothing.
defwork(x, lmul, rdiv):
returnlmul * x // rdiv
defbinomial(n, k, lmul=1, rdiv=1):
ifk == 0: returnwork(1, lmul, rdiv)
returnwork(n * binomial(n - 1, k - 1) // k, lmul, rdiv)
Note that I just mechanically converted all return {whatever} state-
ments into return work({whatever}, lmul, rdiv) .
4. Use the secret feature to el!m!nate the old work.
Watch what happens to that final line.
defwork(x, lmul, rdiv):
returnlmul * x // rdiv
defbinomial(n, k, lmul=1, rdiv=1):
ifk == 0:
returnwork(1, lmul, rdiv)
returnbinomial(n - 1, k - 1, lmul * n, k * rdiv)
5. Youve now got a ta!l call!
Indeed, we do! All thats left is to inline the sole remaining work cal
defbinomial(n, k, lmul=1, rdiv=1):
ifk == 0:
returnlmul * 1// rdiv
returnbinomial(n - 1, k - 1, lmul * n, k * rdiv)
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
15/49
And to simplify away the needless multiplication by 1 in the return
statement:
defbinomial(n, k, lmul=1, rdiv=1):
ifk == 0:
returnlmul // rdiv
returnbinomial(n - 1, k - 1, lmul * n, k * rdiv)
6. Repeat unt!l all recurs!ve calls are ta!l calls.
There was only one, so were done! Yay!
With all the recursive calls (just the one) converted into tail calls, we can
easily convert the function into iterative form by applying the Simple
Method. Heres what we get after we load the function body into a one-s
oop and convert the tail call into an assignment-and- continue pair.
efbinomial_iter(n, k, lmul=1, rdiv=1):
whileTrue:
ifk == 0:
returnlmul // rdiv
(n, k, lmul, rdiv) = (n - 1, k - 1, lmul * n, k * rdiv)
continue
break
As a final touch, we can tidy up and in the process tuck the accumulators
ide the function body to keep them completely secret:
efbinomial_iter(n, k):
lmul = rdiv = 1
whilek > 0:
(n, k, lmul, rdiv) = (n - 1, k - 1, lmul * n, k * rdiv)
returnlmul // rdiv
And now we have an iterative version of our original binomial function!
A short, cautionary lessonThis next part is subtle but important. To understand whats going on, y
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
16/49
irst need to see it. So, once again, lets use the Online Python Tutors ex
ent Python-runtime visualizer. Open the link below in a new tab if you c
and then read on.
Visualize the execution of binomial(39, 9) .
Click the Forward button to advance through each step of the computati
When you get to step 22, where the recursive version has fully loaded thetack with its nested frames, click slowly. Watch the return value (in red
arefully as you advance. See how it gently climbs to the final answer of
211915132, never exceeding that value?
Now continue stepping through to the iterative version. Watch the value
lmul as you advance. See how it grows rapidly, finally reaching a whopp
76899763100160?
This difference matters. While both versions computed the correct answ
he original recursive version would do so without exceeding the capacit
2-bit signed integers.1The iterative version, however, needs a compara
ively hoggish 47 bits to faithfully arrive at the correct answer.
Pythons integers, lucky for us, grow as needed to hold their values, so w
need not fear overflow in this case, but not all languages for which we
might want to use our recursion-to-iteration techniques offer such prot
ions. Its something to keep in mind the next time youre taking the rec
ion out of an algorithm in C:
ypedefint32_tInt;
nt binomial(Int n, Int k) {
if(k == 0)
return1;
returnn * binomial(n - 1, k - 1) / k;
nt binomial_iter(Int n, Int k) {
Int lmul = 1, rdiv = 1;
http://blog.moertel.com/posts/2013-05-14-recursive-to-iterative-2.html#fn1http://www.pythontutor.com/visualize.html#code=def+binomial(n,+k)%3A%0A++++%22%22%22Compute+binomial+coefficient+C(n,+k)+%3D+n!+/+(k!+*+(n-k)!).%22%22%22%0A++++if+k+%3D%3D+0%3A%0A++++++++return+1%0A++++return+n+*+binomial(n+-+1,+k+-+1)+//+k%0A%0Adef+binomial_iter(n,+k)%3A%0A++++%22%22%22Compute+binomial+coefficient+C(n,+k)+%3D+n!+/+(k!+*+(n-k)!).%22%22%22%0A++++lmul+%3D+rdiv+%3D+1%0A++++while+k+%3E+0%3A%0A++++++++(n,+k,+lmul,+rdiv)+%3D+(n+-+1,+k+-+1,+lmul+*+n,+k+*+rdiv)%0A++++return+lmul+//+rdiv%0A%0Aprint+binomial(39,+9)%0Aprint+binomial_iter(39,+9)&mode=display&cumulative=false&heapPrimitives=false&drawParentPointers=false&textReferences=false&showOnlyOutputs=false&py=2&curInstr=0 -
7/23/2019 Tricks of the Trade - Recursion to Iteration
17/49
while(k > 0) {
lmul *= n; rdiv *= k; n -= 1; k -= 1;
}
returnlmul / rdiv;
ntmain(intargc, char* argv[]) {
printf("binomial(39, 9) = %ld\n", (long) binomial(39, 9));
printf("binomial_iter(39, 9) = %ld\n", (long) binomial_iter(39, 9));
/* Output with Int = int32_t:
binomial(39, 9) = 211915132
binomial_iter(39, 9) = -4481
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
18/49
until we hit the base case of . The series of expansions proceeds lik
o:
At every step (except for the base case) we perform a multiplication by
and then a division by . Its that division by that keeps intermediate r
ults from getting out of hand.
Now lets look at the iterative version. Its not so obvious whats going on
Score another point for recursion over iteration!) But we can puzzle it ouWe start out with both accumulators at and then loop over the decreas
values of , building up the accumulators until , at which point we
urn the quotient of the accumulators.
So the calculation is given by
Both the numerator and denominator grow and grow and grow until the
nal division. Not a good trend.
So why did the Secret Feature trick work great for factorial in our previ
article but fail us, albeit subtly, now? The answer is that in factorial the
extra work being done between each recursive call and its return statem
was multiplication and nothing more. And multiplication (of integers th
dont overflow) is commutative and associative. Meaning, ordering and
grouping dont matter.
The lesson, then, is this: Think carefully about whether ordering and
grouping matterbefore using the Secret Feature trick. If it matters, you
n ( )/kn 1
k 1
k= 0
( )= 5
( )/2 = 5
(4( )
/1)
/2 = 5
(4
(1)/1)/2 = 10.
5
2
4
1
3
0
n
k k
1
k k= 0
( )= = = 10.
5
2
lmul
rdiv
((1) 5) 4
((1) 2) 1
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
19/49
have two options: One, you can modify your algorithm so that ordering a
grouping dont matter and thenuse the Secret Feature trick. (But this op
s often intractable.) Two, you can use the T!me-Travel!ng Secret Featuretr
which preserves ordering and grouping. And thattrick is what weve bee
waiting for.
ts time.
The Time-Traveling Secret FeaturetrickWhats about to happen next is so mind-bendingly awesome that you
hould probably get a drink to prepare yourself. Its like combining a T-8rom The Term!nator, with the dreams-within-dreams layering from Ince
!on, with the momentary inversion of reality that occurs when the quan
um field surrounding an inductive proof collapses and everything snaps
nto place.
ts. Really. Cool.
Got your drink? Okay, lets do this.
Lets go back to our original, recursive binomial function:
efbinomial(n, k):
ifk == 0:
return1
returnn * binomial(n - 1, k - 1) // k
Our goal is to create an equivalent iterative version that exactlypreserves
he properties of the original. Well, how the hell are we going to do that?
Hold that thought for a sec.
Lets just stop for a moment and think about what that recursive functio
doing. We call it to compute some answer . But that answer depends ox
t
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
20/49
ome earl!eranswer that the function doesnt know, so it calls itself
get that answer. And so on. Until, finally, it finds one concrete answer
hat it actually knows and, from which, it can build every subsequent an
wer.
So, in truth, our answer is just the final element in a whole timeline o
needed earlier answers:
Well, so what? Why should we care?
Because weve watched The Term!natorabout six hundred times! We know
how to get rid of a problem in the present when weve seen its timeline:
end a Term!nator !nto the past to rewr!te that t!mel!ne so the problem never g
reated !n the"rst place.
And our little recursive problem with binomial here? Weve seen !ts t!mel!
Thats right. Its about to get real.
One more thing. Every s!ngle one of these steps preserves the or!g!nal funct!onbehav!or.You can run your unit tests after every step, and they should all
pass.
Lets begin.
1. Send a T-800 terminator unit into the functions timeline, back t
the time of .
Oh, we dont have a T-800 terminator unit? No problem.Assumewe
sent a T-800 terminator unit into the functions timeline, back to th
time of .
That was easy. Whats next?
2. Move the recursive call and surrounding work into a helper functi
xt1
x0
xt
, , .x0 x1 xt
x0
x0
http://www.imdb.com/title/tt0088247/ -
7/23/2019 Tricks of the Trade - Recursion to Iteration
21/49
This might seem like overkill but prevents mistakes. Do you make m
takes? Then just make the helper function already. Im calling it ste
for reasons that will shortly become obvious.
defstep(n, k):
returnn * binomial(n - 1, k - 1) // k
defbinomial(n, k): ifk == 0:
return1
returnstep(n, k)
3. Partition the helper function into its 3 fundamental parts.
They are:
1.the recursive call to get the previous answer ,
2. the work to compute the current answer from , and
3. a return statement applied to alone:
defstep(n, k):
previous_x = binomial(n - 1, k - 1) # part (1)
x = n * previous_x // k # part (2)
returnx # part (3)
defbinomial(n, k):
ifk == 0:
return1
returnstep(n, k)
4. Secretly prepare to receive communications from the T-800 term
nator unit.
You see, once the T-800 gets its cybernetic hands on earlier values ithe timeline, we can use its knowledge to eliminate the recursion. W
just need some way to get that knowledge from the T-800, now stuc
the past, to us, stuck in the present.
Fortunately, weve seen time-travel movies, so we know exactly how
this problem is solved. Well just use a dead drop! Thats a prearrang
xi1
xi xi1
xi
https://en.wikipedia.org/wiki/Dead_drop -
7/23/2019 Tricks of the Trade - Recursion to Iteration
22/49
secret location where the T-800 will drop values in the past and whe
we will check for them in the future.
So, per prior arrangement with the T-800, well extend our helper
function with a secret feature that checks the dead drop for a previo
value and, if ones there, uses it to muahahahaha! break the
cursive call chain:
defstep(n, k, previous_x=None): #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
23/49
secret arguments.
defstep(n, k, previous_x=None):
ifprevious_x is None:
previous_x = binomial(n - 1, k - 1)
x = n * previous_x // k
return(n, k, x) #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
24/49
And now we can reverse the#ow of t!me.
When we started out with our original recursive function, if we aske
for , the function had to go back in the timeline to get . And to
, it had to go back even further to get . And so on, chewing u
stack every backward step of the way to . It was heartbreaking,
watching it work like that.
But now, we can step the other way. If we have any ( we ca
get straight away, no recursion required. In goes , out comes
We can go forward.
Why, if we knew , , and , we could compute the entire series
withzerorecursion by starting at and working for
ward.
So lets get those values!
7. Determine the initial conditions at the start of the timeline.
For this simple example, most of you can probably determine the in
conditions by inspection. But Im going to go through the process anway. You never know when you might need it. So:
Whats the start of the timeline? Its when the recursive binomial
function calls itself so many times that it finally hits one of its base
cases, which defines the first entry in the timeline, anchoring the
timeline at time . The base cases in this example are easy to fin
since weve already split out the step logic; all thats left in binomiathe base-case logic. Its easy to see that there is only one base case,
its when :
defstep(n, k, previous_x=None):
ifprevious_x is None:
previous_x = binomial(n - 1, k - 1)
x = n * previous_x // k
return(n + 1, k + 1, x)
xt xt1
xt1 xt2
x0
, , )ni ki xi1xi xi1 x
n1 k1 x0
, , x0 x1 xt i = 0
i = 0
k= 0
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
25/49
defbinomial(n, k):
ifk == 0: #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
26/49
And when we know from the equation above that .
Finally, we can compute , which is reversesteps from
the only that we know so far:
And now we have our initial conditions:
So now our knowledge of the timeline looks like this:
time :
:
:
: ?
And with this knowledge, we can step forward through the timeline
from time to time , and so on, until finally, at the last stewhen we will have determined , our desired answer!
Lets do it!
8. In the main function, iterate the step helper times, starting from
the initial conditions. Then return .
defstep(n, k, previous_x=None):
ifprevious_x is None:
previous_x = binomial(n - 1, k - 1)
x = n * previous_x // k
return(n + 1, k + 1, x)
defbinomial(n, k):
ifk == 0:
return1
t = k #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
27/49
(n, k, previous_x) = (n - k + 1, 1, 1) #
for_i in xrange(1, t + 1): #
(n, k, previous_x) = step(n, k, previous_x) #
returnprevious_x # = x_t #
Boom! Thats 100% iterative. But theres more!
9. Remove the base cases from the original function.
Since our iterations start from a base case, the base cases are already
incorporated into our new, iterative logic.
defstep(n, k, previous_x=None):
ifprevious_x is None:
previous_x = binomial(n - 1, k - 1)
x = n * previous_x // k
return(n + 1, k + 1, x)
defbinomial(n, k):
# if k == 0: #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
28/49
for_i in xrange(1, t + 1):
(n, k, previous_x) = step(n, k, previous_x)
returnprevious_x # = x_t
11. Inline the step function.
Were on it!
defbinomial(n, k): t = k
(n, k, previous_x) = (n - k + 1, 1, 1)
for_i in xrange(1, t + 1):
x = n * previous_x // k
(n, k, previous_x) = (n + 1, k + 1, x)
returnprevious_x # = x_t
12. Apply finishing touches.
This example is pretty tight already, but we can substitute away the
variable.
And, because , we can replace the for loops induction variabl
_i with and correspondingly eliminate from the step logic.
And, while were making stuff better, theres an obvious optimizati
One property of binomial coefficients is that . One pro
erty of our code is that it runs for steps. So when , w
can reduce the number of steps by solving for instead. Lets a
a couple of lines at the start of the logic to claim this optimization.
And, of course, lets add a docstring were nice like that.
The final result:
defbinomial(n, k):
"""Compute binomial coefficient C(n, k) = n! / (k! * (n-k)!).""
ifk > n - k:
k = n - k # 'pute C(n, n-k) instead if it's easier
t = k
(n, previous_x) = (n - k + 1, 1)
fork in xrange(1, t + 1):
= ikiki ki
( ) =( )n
k
n
nk
t= k k> n k
( )n
nk
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
29/49
(n, previous_x) = (n + 1, n * previous_x // k)
returnprevious_x # = x_t
Lets review what we just did. Its mind blowing:
1. We sent an !mag!naryT-800 terminator unit back into the function
timeline.
2. Then we added a secret feature to our function that allowed the T-80to send us values from the t!mel!nes past.
3. Then we used those values to break the recursive chain, reverse the
backward flow of time, and compute the timeline forward and itera-
tively instead of backward and recursively.
4. The function then being fully iterative, we removed the secret featu
and the imaginary T-800 winked out of existence. (In an ironic twisfate, the T-800, via the secret-feature dead drop, had delivered into
our hands precisely the knowledge we needed to write both of them
of history!)
5. And now we have a fast, efficient, and property-preserving iterative
version of our original recursive function, and its about as good as
anything we could hope to conjure up from scratch. (See it in action
just one stack frame and easy on the ints, too.)
6. And most importantly we did it all using a mechanical, repeatab
process!
Thanks!Well, thats it for this installment. I hope you enjoyed reading it as much
did writing it. If you liked it (or didnt), or if you found a mistake, or esp
ially if you can think of any way to help make my explanations better, le
me know. Just post a comment or fire me a tweet at @tmoertel.
Until next time, keep your brain recursive and your Python code iterative
https://twitter.com/tmoertelhttp://www.pythontutor.com/visualize.html#code=def+binomial(n,+k)%3A%0A++++%22%22%22Compute+binomial+coefficient+C(n,+k)+%3D+n!+/+(k!+*+(n-k)!).%22%22%22%0A++++if+k+%3D%3D+0%3A%0A++++++++return+1%0A++++return+n+*+binomial(n+-+1,+k+-+1)+//+k%0A%0Adef+binomial_iter(n,+k)%3A%0A++++%22%22%22Compute+binomial+coefficient+C(n,+k)+%3D+n!+/+(k!+*+(n-k)!).%22%22%22%0A++++lmul+%3D+rdiv+%3D+1%0A++++while+k+%3E+0%3A%0A++++++++(n,+k,+lmul,+rdiv)+%3D+(n+-+1,+k+-+1,+lmul+*+n,+k+*+rdiv)%0A++++return+lmul+//+rdiv%0A%0Adef+binomial_iter_final(n,+k)%3A%0A++++%22%22%22Compute+binomial+coefficient+C(n,+k)+%3D+n!+/+(k!+*+(n-k)!).%22%22%22%0A++++if+k+%3D%3D+0%3A%0A++++++++return+1%0A++++t+%3D+k%0A++++(n,+k,+previous_x)+%3D+(n+-+k+%2B+1,+1,+1)%0A++++for+_+in+xrange(t)%3A%0A++++++++(n,+k,+previous_x)+%3D+(n+%2B+1,+k+%2B+1,+n+*+previous_x+//+k)%0A++++return+previous_x++%23+%3D+x_t%0A%0Aprint+binomial(39,+9)%0Aprint+binomial_iter(39,+9)%0Aprint+binomial_iter_final(39,+9)&mode=display&cumulative=false&heapPrimitives=false&drawParentPointers=false&textReferences=false&showOnlyOutputs=false&py=2&curInstr=0 -
7/23/2019 Tricks of the Trade - Recursion to Iteration
30/49
1. In the visualization, you cant actually see the largest integer produ
by the recursive versions computation. Its produced between steps
and 33, when the return value from step 32 is multiplied by step 33s
to arrive at an integer of bits,
which is then immediately divided by to produce the final an-
swer.!
Site proudly generated by H
n = 39 (39 48903492) 30. 8log2
k= 9
http://jaspervdj.be/hakyllhttp://blog.moertel.com/posts/2013-05-14-recursive-to-iterative-2.html#fnref1 -
7/23/2019 Tricks of the Trade - Recursion to Iteration
31/49
Tom Moertels Blog Home About Arch
Tricks of the trade:
Recursion to Itera-tion, Part 3: RecursivData StructuresBy Tom Moertel
osted on June 3, 2013
Tags: programm!ng, recurs!on, !terat!on, python, recurs!on-to-!terat!on ser!es, ta!l calls, data
tructures
This is the third article in a series on converting recursive algorithms int
terative algorithms. If any of what follows seems confusing, you may wao read the earlier articles first.
This is an extra article that I hadnt planned. Im writing it because in a
omment on the previous article a reader asked me to show a less mathe
matical example and suggested tree traversal. So thats the subject of th
article: Well take a binary tree and flatten it into a list, first recursively,
hen iteratively.
The challengeFirst, lets define a binary tree to be either empty or given by a node hav
hree parts: (1) a value, (2) a left subtree, and (3) a right subtree, where b
of the subtrees are themselves binary trees. In Haskell, we might define
http://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/data%20structures.htmlhttps://plus.google.com/103957126301745359889?rel=authorhttp://blog.moertel.com/http://blog.moertel.com/http://blog.moertel.com/about.htmlhttp://blog.moertel.com/archive.htmlhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/data%20structures.htmlhttp://blog.moertel.com/tags/tail%20calls.htmlhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/python.htmlhttp://blog.moertel.com/tags/iteration.htmlhttp://blog.moertel.com/tags/recursion.htmlhttp://blog.moertel.com/tags/programming.htmlhttps://plus.google.com/103957126301745359889?rel=authorhttp://blog.moertel.com/archive.htmlhttp://blog.moertel.com/about.htmlhttp://blog.moertel.com/http://blog.moertel.com/ -
7/23/2019 Tricks of the Trade - Recursion to Iteration
32/49
ike so:
ataBinaryTreea =Empty|Nodea (BinaryTreea) (BinaryTreea)
n Python, which well use for the rest of this article, well say that None
epresents an empty tree and that the following class represents a node:
mportcollectionsode = collections.namedtuple('Node', 'val left right')
# some sample trees having various node counts
ree0 = None # empty tree
ree1 = Node(5, None, None)
ree2 = Node(7, tree1, None)
ree3 = Node(7, tree1, Node(9, None, None))
ree4 = Node(2, None, tree3)
ree5 = Node(2, Node(1, None, None), tree3)
Let us now define a function to flatten a tree using an in-order traversal
The recursive definition is absurdly simple, the data type having only tw
ases to consider:
efflatten(bst):
# empty case
ifbst is None:return[]
# node case
returnflatten(bst.left) + [bst.val] + flatten(bst.right)
A few tests to check that it does what we expect:
efcheck_flattener(f):
assertf(tree0) == []
assertf(tree1) == [5]assertf(tree2) == [5, 7]
assertf(tree3) == [5, 7, 9]
assertf(tree4) == [2, 5, 7, 9]
assertf(tree5) == [1, 2, 5, 7, 9]
print'ok'
heck_flattener(flatten) # ok
Our challenge for today is to convert flatten into an iterative version. O
http://en.wikipedia.org/wiki/Tree_traversal#In-order -
7/23/2019 Tricks of the Trade - Recursion to Iteration
33/49
er than a new trick partial evaluation the transformation is straightf
ward, so Ill move quickly.
Lets do this!
Eliminating the first recursive callFirst, lets separate the base case from the incremental work:
efstep(bst):
returnflatten(bst.left) + [bst.val] + flatten(bst.right)
efflatten(bst):
ifbst is None:
return[]
returnstep(bst)
And lets break the incremental work into smaller pieces to see whats g
ng on.
efstep(bst):
left = flatten(bst.left)
left.append(bst.val)
right = flatten(bst.right)left.extend(right)
returnleft
efflatten(bst):
ifbst is None:
return[]
returnstep(bst)
Lets try to get rid of the first recursive call by assuming that somebody hpassed us its result via a secret argument left :
efstep(bst, left=None):
ifleft is None:
left = flatten(bst.left)
left.append(bst.val)
right = flatten(bst.right)
left.extend(right)
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
34/49
returnleft
efflatten(bst):
ifbst is None:
return[]
returnstep(bst)
And now well make step return values that parallel its input argument
efstep(bst, left=None):
ifleft is None:
left = flatten(bst.left)
left.append(bst.val)
right = flatten(bst.right)
left.extend(right)
returnbst, left #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
35/49
But this plan hits the same brick wall: If we add a new argument to accep
he parent, we must for parallelism add a new return value to emit the
ransformed parent, which is the parent of the parent. But we cant com
pute the parent of the parent because, as before, we have no way of impl
menting get_parent .
So we do what mathematicians do when their assumptions hit a brick wawe strengthen our assumption! Now we assume that someone has passe
us all of the parents, right up to the trees root. And that assumption give
what we need:
efstep(bst, parents, left=None):
ifleft is None:
left = flatten(bst.left)
left.append(bst.val)right = flatten(bst.right)
left.extend(right)
returnparents[-1], parents[:-1], left
Note that were using the Python stack convention for parents ; thus the
mmediate parent of bst is given by the final element parents[-1] .
As a simplification, we can eliminate the bst argument by considering ihe final parent pushed onto the stack:
efstep(parents, left=None):
bst = parents.pop() #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
36/49
parents = [bst]
returnstep(parents)[-1]
But we still havent eliminated the first recursive call. To do that, well n
o pass the step function a value for its left argument, which will cause
he recursive call to be skipped.
But we only know what that value should be for one case, the base case,when bst is None ; then left must be [] . To get to that case from the tr
oot, where bst is definitely not None , we must iteratively replicate the
normal recursive calls on bst.left until we hit the leftmost leaf node. A
hen, to compute the desired result, we must reverse the trip, iterating t
step function until we have returned to the trees root, where the paren
tack must be empty:
efflatten(bst):
# find initial conditions for secret-feature "left"
left = []
parents = []
whilebst is not None:
parents.append(bst)
bst = bst.left
# iterate to compute the result
whileparents:
parents, left = step(parents, left)
returnleft
And just like that, one of the recursive calls has been transformed into it
ation. Were halfway to the finish line!
Eliminating the second recursivecallBut we still have to eliminate that final recursive call to flatten , now se
questered in step . Lets take a closer look at that function after we make
left argument required since it always gets called with a value now:
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
37/49
efstep(parents, left):
bst = parents.pop()
left.append(bst.val)
right = flatten(bst.right)
left.extend(right)
returnparents, left
To get rid of the recursive call to flatten , were going to use a new trick:
partial evaluation. Basically, were going to replace the call to flatten whe function body of flatten , after we rename all its variables to preven
onflicts. So lets make a copy of flatten and suffix all its variables with
efflatten1(bst1):
left1 = []
parents1 = []
whilebst1 is not None:
parents1.append(bst1)bst1 = bst1.left
whileparents1:
parents1, left1 = step(parents1, left1)
returnleft1
And then lets make its arguments and return values explicit:
(bst1, ) = ARGUMENTS
left1 = []
parents1 = []
whilebst1 is not None:
parents1.append(bst1)
bst1 = bst1.left
whileparents1:
parents1, left1 = step(parents1, left1)
RETURNS = (left1, )
And then well drop this expansion into step :
efstep(parents, left):
bst = parents.pop()
left.append(bst.val)
# -- begin partial evaluation --
(bst1, ) = (bst.right, )
left1 = []
parents1 = []
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
38/49
whilebst1 is not None:
parents1.append(bst1)
bst1 = bst1.left
whileparents1:
parents1, left1 = step(parents1, left1)
(right, ) = (left1, )
# -- end partial evaluation --
left.extend(right)
returnparents, left
Now we can eliminate code by fusion across the partial-evaluation boun
ary.
First up: left1 . We can now see that this variable accumulates values th
n the end, get appended to left (via the return variable right ). But we
ust as well append those values to left directly, eliminating left1 with
he boundary and the call to left.extend(right) without:
efstep(parents, left):
bst = parents.pop()
left.append(bst.val)
# -- begin partial evaluation --
(bst1, ) = (bst.right, )
# left1 = [] #
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
39/49
(bst1, ) = (bst.right, )
parents1 = []
whilebst1 is not None:
parents1.append(bst1)
bst1 = bst1.left
whileparents1:
parents1, left = step(parents1, left)
# -- end partial evaluation --
returnparents, left
efflatten(bst):
left = []
parents = []
whilebst is not None:
parents.append(bst)
bst = bst.left
whileparents:
parents, left = step(parents, left)
returnleft
When flatten calls step and the code within the partially evaluated reg
executes, it builds up a stack of nodes parents1 and then calls step itera
ively to pop values off of that stack and process them. When its finishe
ontrol returns to step proper, which then returns to its caller, flatten
with the values ( parents , left ). But look at what flatten then does with
parents : it calls step iteratively to pop values off of that stack and proce
hem in exactly the same way.
So we can eliminate the while loop in step and the recursive call! by
urning not parents but parents + parents1 , which will make the while
oop in flatten do the exact same work.
efstep(parents, left):
bst = parents.pop()
left.append(bst.val)
# -- begin partial evaluation --
(bst1, ) = (bst.right, )
parents1 = []
whilebst1 is not None:
parents1.append(bst1)
bst1 = bst1.left
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
40/49
# while parents1: # parents + parents1
And then we can eliminate parents1 completely by taking the values we
would have appended to it and appending them directly to parents :
efstep(parents, left):bst = parents.pop()
left.append(bst.val)
# -- begin partial evaluation --
(bst1, ) = (bst.right, )
# parents1 = [] # parents
bst1 = bst1.left
# -- end partial evaluation --returnparents, left # parents + parents1 -> parents
And now, once we remove our partial-evaluation scaffolding, our step
unction is looking simple again:
efstep(parents, left):
bst = parents.pop()
left.append(bst.val)bst1 = bst.right
whilebst1 is not None:
parents.append(bst1)
bst1 = bst1.left
returnparents, left
For the final leg of our journey simplification lets inline the step lo
back into the base function:efflatten(bst):
left = []
parents = []
whilebst is not None:
parents.append(bst)
bst = bst.left
whileparents:
parents, left = parents, left
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
41/49
bst = parents.pop()
left.append(bst.val)
bst1 = bst.right
whilebst1 is not None:
parents.append(bst1)
bst1 = bst1.left
parents, left = parents, left
returnleft
Lets eliminate the trivial argument-binding and return-value assign-
ments:
efflatten(bst):
left = []
parents = []
whilebst is not None:
parents.append(bst)
bst = bst.leftwhileparents:
# parents, left = parents, left # = no-op
bst = parents.pop()
left.append(bst.val)
bst1 = bst.right
whilebst1 is not None:
parents.append(bst1)
bst1 = bst1.left
# parents, left = parents, left # = no-opreturnleft
And, finally, factor out the duplicated while loop into a local function:
efflatten(bst):
left = []
parents = []
defdescend_left(bst):
whilebst is not None:
parents.append(bst)
bst = bst.left
descend_left(bst)
whileparents:
bst = parents.pop()
left.append(bst.val)
descend_left(bst.right)
returnleft
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
42/49
And thats it! We now have a tight, efficient, and iterative version of our
original function. Further, the code is close to idiomatic.
Thats it for this time. If you have any questions or comments, just hit m
at @tmoertelor use the comment form below.
Thanks for reading!
Site proudly generated by H
http://jaspervdj.be/hakyllhttps://twitter.com/tmoertel -
7/23/2019 Tricks of the Trade - Recursion to Iteration
43/49
Tom Moertels Blog Home About Arch
Tricks of the trade:
Recursion to Itera-tion, Part 4: TheTrampolineBy Tom Moertel
osted on June 12, 2013
Tags: programm!ng, recurs!on, !terat!on, python, recurs!on-to-!terat!on ser!es, ta!l calls, data
tructures, trampol!nes
This is the fourth article in a series on converting recursive algorithms in
terative algorithms. If you havent read the earlier articles first, you mawant to do sobefore continuing.
n the first article of our series, we showed that if you can convert an alg
ithms recursive calls into tail calls, you can eliminate those tail calls to
reate an iterative version of the algorithm using The Simple Method. In
his article, well look at another way to eliminate tail calls: the trampol!n
The idea behind the trampoline is this: before making a tail call, manual
emove the current execution frame from the stack, eliminating stack
build-up.
Execution frames and the stack
http://blog.moertel.com/posts/2013-05-11-recursive-to-iterative.htmlhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/data%20structures.htmlhttp://blog.moertel.com/tags/data%20structures.htmlhttp://blog.moertel.com/tags/trampolines.htmlhttp://blog.moertel.com/tags/trampolines.htmlhttp://blog.moertel.com/tags/trampolines.htmlhttps://plus.google.com/103957126301745359889?rel=authorhttp://blog.moertel.com/posts/2013-05-11-recursive-to-iterative.htmlhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/trampolines.htmlhttp://blog.moertel.com/tags/data%20structures.htmlhttp://blog.moertel.com/tags/tail%20calls.htmlhttp://blog.moertel.com/tags/recursion-to-iteration%20series.htmlhttp://blog.moertel.com/tags/python.htmlhttp://blog.moertel.com/tags/iteration.htmlhttp://blog.moertel.com/tags/recursion.htmlhttp://blog.moertel.com/tags/programming.htmlhttps://plus.google.com/103957126301745359889?rel=authorhttp://blog.moertel.com/archive.htmlhttp://blog.moertel.com/about.htmlhttp://blog.moertel.com/http://blog.moertel.com/ -
7/23/2019 Tricks of the Trade - Recursion to Iteration
44/49
To understand why we might want to manually remove an execution fram
ets think about what happens when we call a function. The language ru
ime needs some place to store housekeeping information and any local
variables the function may use, so it allocates a new execution frame on
tack. Then it turns control over to the function. When the function is do
t executes a return statement. This statement tells the runtime to remo
he execution frame from the stack and to give control (and any result) b
o the caller.
But what if the function doesnt return right away? What if it makes ano
er function call instead? In that case, the runtime must create a new exe
ion frame for thatcall and push it onto the stack, on top of the current
rame. If the function ends up calling itself many times recursively, each
all will add another frame to the stack, and pretty soon we will have eat
up a lot of stack space.
Eliminating stack build-upTo avoid this problem, some programming languages guarantee that the
will recycle the current execution frame whenever a function makes a ta
all. That is, if the function calls some other function (or itself recursive
and just returns that functions result verbatim, thats a tail call. In that
ase, the runtime will recycle the current functions execution frame bef
ransferring control to the other function, making it so that the other fu
ion will return its result directly to the original functions caller. This
process is called ta!l-call el!m!nat!on.
But in languages like Python that dont offer tail-call elimination, every
all, even if its a tail call, pushes a new frame onto the stack. So if we wa
o prevent stack build-up, we must somehow eliminate the current fram
rom the stack ourselves, before making a tail call.
But how? The only obvious way to eliminate the current frame is to retu
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
45/49
o our caller. If were to make this work, then, the caller must be willing
help us out. Thats where the trampoline comes in. Its our co-conspirat
n the plot to eliminate stack build-up.
The trampolineHeres what the trampoline does:
1. It calls our function f , making itself the current caller.
2. When f wants to make a recursive tail call to itself, it returns the in
struction call(f)(*args, **kwds) . The language runtime dutifully re
moves the current execution frame from the stack and returns contr
to the trampoline, passing it the instruction.3. The trampoline interprets the instruction and calls f back, giving it
the supplied arguments, and again making itself the caller.
4. This process repeats until f wants to return a final result z ; then it
turns the new instruction result(z) instead. As before, the runtime
moves the current execution frame from the stack and returns contr
to the trampoline.
5. But now when the trampoline interprets the new instruction it will
turn z to !tscaller, ending the trampoline dance.
Now you can see how the trampoline got its name. When our function us
a return statement to remove its own execution frame from the stack, t
rampoline bounces control back to it with new arguments.
Heres a simple implementation. First, we will encode our instructions t
he trampoline as triples. Well let call(f)(*args, **kwds) be the triple
(f, args, kwds) , and result(z) be the triple (None, z, None) :
efcall(f):
"""Instruct trampoline to call f with the args that follow."""
defg(*args, **kwds):
returnf, args, kwds
-
7/23/2019 Tricks of the Trade - Recursion to Iteration
46/49
returng
efresult(value):
"""Instruct trampoline to stop iterating and return a value."""
returnNone, value, None
Now well create a decorator to wrap a function with a trampoline that w
nterpret the instructions that the function returns:
mportfunctools
efwith_trampoline(f):
"""Wrap a trampoline around a function that expects a trampoline."""
@functools.wraps(f)
defg(*args, **kwds):
h = f
# the trampoline
whileh is not None:h, args, kwds = h(*args, **kwds)
returnargs
returng
Note that the trampoline boils down to three lines:
hileh is not None:
h, args, kwds = h(*args, **kwds)
eturnargs
Basically, the trampoline keeps calling whatever function is in h until th
unction returns a result(z) instruction, at which time the loop exits an
s returned. The original recursive tail calls have been boiled down to a
while loop. Recursion has become iteration.
Example: factorialTo see how we might use this implementation, lets return to the factori
example from the first article in our series:
effactorial(n):
ifn < 2:
return1
http://blog.moertel.com/posts/2013-05-11-recursive-to-iterative.html -
7/23/2019 Tricks of the Trade - Recursion to Iteration
47/49
returnn * factorial(n - 1)
Step one, as before, is to tail-convert the lone recursive call:
deffactorial(n, acc=1):
ifn < 2:
returnacc
returnfactorial(n - 1, acc * n)
Now we can create an equivalent function that uses trampoline idioms:
eftrampoline_factorial(n, acc=1):
ifn < 2:
returnresult(acc)
returncall(trampoline_factorial)(n - 1, n * acc)
Note how the return statements have been transformed.
Finally, we can wrap this function with a trampoline to get a callable ver
ion that we can use just like the original:
actorial = with_trampoline(trampoline_factorial)
Lets take it for a spin:
>> factorial(5)20
To really see whats going on, be sure to use the Online Python Tutors vi
alizer to step through the original, tail-recursive, and trampoline versio
of the function. Just open this link: Visualize the execution. (ProTip: use
new tab.)
Why use the trampoline?As I mentioned at the beginning of this article, if you can convert a func-
ions recursive calls into tail calls which you must do to use a trampol
you can also use the Simple Method to convert the functions recursion
nto iteration, eliminating the calls altogether. For example, heres what
http://www.pythontutor.com/visualize.html#code=%23+our+trampoline+library%0A%0Aimport+functools%0A%0Adef+call(f)%3A%0A++++%22%22%22Instruct+trampoline+to+call+f+with+the+args+that+follow.%22%22%22%0A++++def+g(*args,+**kwds)%3A%0A++++++++return+f,+args,+kwds%0A++++return+g%0A%0Adef+result(value)%3A%0A++++%22%22%22Instruct+trampoline+to+stop+iterating+and+return+a+value.%22%22%22%0A++++return+None,+value,+None%0A%0Adef+with_trampoline(f)%3A%0A++++%22%22%22Wrap+a+trampoline+around+a+function+that+expects+a+trampoline.%22%22%22%0A++++%40functools.wraps(f)%0A++++def+g(*args,+**kwds)%3A%0A++++++++h+%3D+f%0A++++++++%23+the+trampoline%0A++++++++while+h+is+not+None%3A%0A++++++++++++h,+args,+kwds+%3D+h(*args,+**kwds)%0A++++++++return+args%0A++++return+g%0A%0A%0A%23+original+recursive+version+of+factorial+function%0A%0Adef+factorial(n)%3A%0A++++if+n+%3C+2%3A%0A++++++++return+1%0A++++return+n+*+factorial(n+-+1)%0A%0Aprint+factorial(5)%0A%0A%0A%23+tail-call+recursive+version%0A%0Adef+factorial(n,+acc%3D1)%3A%0A+++++if+n+%3C+2%3A%0A+++++++++return+acc%0A+++++return+factorial(n+-+1,+acc+*+n)%0A%0Aprint+factorial(5)%0A%0A%0A%23+trampoline-based+tail-call+version+(%3D+iterative)%0A%0Adef+trampoline_factorial(n,+acc%3D1)%3A%0A++++if+n+%3C+2%3A%0A++++++++return+result(acc)%0A++++return+call(trampoline_factorial)(n+-+1,+n+*+acc)%0A%0Afactorial+%3D+with_trampoline(trampoline_factorial)%0A%0Aprint+factorial(5)%0A&mode=display&cumulative=false&heapPrimitives=false&drawParentPointers=false&textReferences=false&showOnlyOutputs=false&py=2&curInstr=0 -
7/23/2019 Tricks of the Trade - Recursion to Iteration
48/49
he Simple Method does to our original factorial function:
effactorial(n, acc=1):
whilen > 1:
(n, acc) = (n - 1, acc * n)
returnacc
This version is simpler and more efficient than the trampoline version. S
why not use the Simple Method always?
The answer is that the Simple Method is tricky to apply to functions that
make tail calls from within loops. Recall that it introduces a loop around
unctions body and replaces recursive tail calls with continue statemen
But if the function already has its own loops, replacing a tail call within o
of them with a continue statement will restart that inner loop instead of
he whole-body loop, as desired. In that case, you must add condition fla
o make sure the right loop gets restarted, and that gets old fast. Then, u
ng a trampoline may be a win.
That said, I almost never use trampolines. Getting a function into tail-ca
orm is nine tenths of the battle. If Ive gone that far already, Ill usually
he rest of the way to get a tight, iterative version.
Why, then, did we make this effort to understand the trampoline? Two r
ons. First, its semi-common in programming lore, so its best to know
about it. Second, its a stepping stone to a more-general, more-powerfu
echnique: cont!nuat!on-pass!ng-style express!ons. Thats our subject for n
ime.
n the meantime, if you want another take on trampolines in Python, Ky
Miller wrote a nice article on the subject: Tail call recursion in Python.
Thanks for reading! As always, if you have questions or comments, pleas
eave a comment on the blog or hit me at @tmoertel.
https://twitter.com/tmoertelhttp://web.mit.edu/kmill/www/programming/tailcall.html -
7/23/2019 Tricks of the Trade - Recursion to Iteration
49/49
Site proudly generated by H
http://jaspervdj.be/hakyll