Recurrences
Oscar Civit [email protected] Giribet [email protected]
What are recurrences and where do we find them?
Recurrences are functions defined on an integer domain that include themselves in their own definition.
Ex: T(n) = T(n/2) + n
They appear when trying to calculate the cost of recursive algorithms.
What are recurrences and where do we find them? (II)
MERGE-SORT sorting algorithm cost can be written as:
T(n) = (1) if n=12T(n/2) + (n) if n>1
Resolution
When analyzing the cost of an algorithm, we aim to find a closed formula that computes the asymptotic cost with the input size as the only variable. This asymptotic cost allows us to compare the efficiency of different algorithms.
There are 3 main methods to solve this kind of recurrences, which we will explain in detail later: Substitution method Iteration method Master method
Technicalities
Integer arguments
We usually ignore that the input size is an integer, omiting the use of floor/ceil operators in the formulas.
T(n) = T(floor(n/2)) + T(ceil(n/2)) + n = 2T(n/2) + n
Boundary conditions: T(n_0) = T_0
We usually omit boundary conditions when solving the recurrence, as most of the times they only affect the result by some constant which does not change asymptotic cost.
Substitution Method
Basic idea is making a reasonable guess of the solution to the recurrence and prove it by induction.
Ex: T(n) = 2T(n/2) + n
Guess:T(n) = O(n log n)
To be proved:T(n) 0
Substitution Method (II)
General case
We assume that it works for n/2, lets try for n:T(n)
Substitution Method (III)
Base case
Proving the base case is proving that the inequality holds for the boundary conditions of the recurrence. Due to the asymptotic notation, it is sufficient to prove it for some n_0.
In the example, for n_0 = 3:T(3) = 5T(3) = 2
How to become a Guess Star
Similar recurrences usually have similar solutions.If a recurrence is similar to another one they may have thesame asymptotic behavior.Ex: T_1(n) = 2T_1(n/2) + n = O(n log n)
T_2(n) = 2T_2(n/2 + K) + n = O(n log n) Sometimes a change of variable can be performed to
transform an unknown recurrence into a known one:T(n) = 2T(sqrt(n)) + log n
we change variable: m = log nT(2^m) = 2T(2^(m/2)) + mS(m) = T(2^m) = 2S(m/2) + m
which has a known result: S(m) = O(m log m)and changing back the variable:
T(n) = O(log n log( log n ))
How to become a Guess Star (II)
Prove loose upper and lower bounds and try to make them converge. Ex:We can start proving:
T(n) = (n)T(n) = O(n^2)
If we succeed we can raise the lower limit and lower the higher limit to finally prove:
T(n) = O(n log n)
T(n) = (n log n) And therefore:
T(n) = (n log n)
Iteration Method
1) Expand the recurrence formula iterating it until it can be expressed as a summation of terms dependent only on n and the initial conditions (boundary).
2) Use standard techniques for bounding summations to get the asymptotic cost.
Ex:T(n) = 3T(n/4) + nT(n) = n + 3T(n/4)
= n + 3( (n/4) + 3T(n/16) )= n + 3( (n/4) + 3( (n/16) + 3T(n/64) ) )= n + 3(n/4) + 9(n/16) + 27T(n/64)
Iteration Method (II)
T(n)
Master Method
Recipe for success, but only when the recurrence can be cast into the form:
T(n) = aT(n/b) + f(n)where
a >= 1b > 1f(n) is asymptotically positive
Master Method (II)
The Master Theorem is divided in 3 cases: a) If f(n) = O(n^log_b(a - e)) for some e > 0
then T(n) = (n^log_b a)
b) If f(n) = (n^log_b a) then T(n) = ( (n^log_b a) log n )
c) If f(n) = ((n^log_b(a + e)) for some e > 0 and if a f(n/b)
Example: the Hanoi Towers
Classic example of moving a tower of n blocks from one pleace to another under some restrictions.
Blocks cannot be placed on top of smaller ones.
There is a starting block, an auxiliary place and a final destination.
The Hanoi Towers: solution
A simple and intuitive solution to the problem can befound by induction
Case where n=1 is trivial, block is simply moved to proper place (base case) in one step
The Hanoi Towers: solution
To solve case where n>1, a smaller by one subset of blocks is taken (n-1)
The subset is smaller and we can suppose it solved by induction
Instead of placing it at destination, it is placed on intermediate area
Tasks left: Lowest block is yet unsolved and in original place Place n-1 subset on final destination
The Hanoi Towers: solution
Lowest block is trivially moved to proper place
By induction, smaller subset left is placed on the finaldestination (over just-moved lowest block)
The Hanoi Towers: solution
Base case solved
Method of moving from smaller subset of problem solved(by hyphotesis) to full completion
Subsets decrease to empty consistently
Solved! But what is the cost O() of the algorithm?
Cost can be described by a recurrence:T(n) = T(n-1)+1+T(n-1) = 2T(n-1)+1, with T(1)=1
The Hanoi Towers: cost
Recurrence is not in asymptotic form
The iteration method of solving can be attempted to get the asymptotic bound
Expanding (iterating) the sequence to depend only on n leads to:
T(n) = 2( 2( 2T(n-3)+1 )+1 )+1
The Hanoi Towers: cost
Further applying the accumulated multiply factors to the constants leads to:T(n) = 2^n-1 * T(1) + ... + 2^2 + 2 + 1
In this case, the series is well-known and easy to solve without resorting to more convoluted algebra, though by iterating the recurrence it is definitely far easier to come up with the solution
The solution in this case is trivial enough: T(n) = 2^n - 1
Top Related