ADVANCED ALGORITHMS

ADVANCED ALGORITHMS1REVIEW OF ANALYSIS TECHNIQUES(UNIT-1)21. ALGORITHM : is a sequence of unambiguous instructions for solving a problem in a finite amount of time. Ex-1 : Write an algorithm to find the factorial of a given number n. Ex-2 : Write an algorithm for find the value of m power n.3Ex-3 : Write an algorithm for finding the GCD of (m,n).Ex-4 : Find the GCD of 60, 24 using the method of common prime factors.Ex-5 : Find the minimum distance between any two elements in the array of size n. ASYMPTOTIC NOTATION 1.1. What is the meaning of asymptotic.Consider a curve such as y = 1 / xAs x increases, y decreases. The positive x-axis is an asymptote of the curve. As you continually increase x towards infinity, the value of y gets closer and closer to zero. Hence the curve gets closer and closer to the x-axis but never reaches it. We say the curve is asymptotic45 1.2 Asymptotic Complexity :is a way of expressing the main component of the cost of an algorithm, using idealized units of computational work.

Ex-6 : Sorting the deck of cards.Cost Formula : 52 + 51 + 50 + + 2So, CF = N(N+1) / 2 - 1 = (N2) + (N) 1Here, the term N2 is dominating the CF.So, the behavior is said to be O(N2) .61.3 Big-O Notation :

f(n) - the cost function of an algorithm.g(n) - the cost function of another algorithm.

Here, if f(n) 0 such that for all integers n >= no 0 no f(n) >= c g(n), then f(n) is Big- of g(n).

This is denoted as : f(n) = (g(n))Ex-8 : Let f(n) = 3n + 2,g(n) = n, c = 3Show that g(n) is Lower Bound of f(n). 91.5 Theta Notation : Def : For non-negative functions f(n) and g(n), if there exists an integer n0, and two constants c1,c2 > 0 such that for all integers n >= no c1 g(n) = no 0 0 such that for all integers n > no 0 0, and n, we havee) a = blogb af) logc(ab) = logca + logcb

g) logban = n log bah) logba = logca / logcb

i) logb(1/a) = - logbaj) logba = 1 / logab

k) alogbc = clogbaA simple series expansion is : Ln(1+x) = x (x2/2) + (x3/3) - (x4/4) + (x5/5) -

Here |x| < 1.

202.7 Factorials : The notation n! is defined for integers n 0 asn! = 1 if n = 0 = n. (n-1)! if n > 0Thus, n! = 1.2.3n2.8 Functional Iteration :

We use the notation f(i)(n) to denote the function f(n) iteratively applied i times to an initial value of n.

Formally, let f(n) be a function over the reals.21For non-negative integers i, we recursively definef(i)(n) = n if i = 0= f ( f(i-1)(n) )if i > 0Ex-18 : Let f(n) = 2n. g(n) = 2n+3 Then find the values of f(i)(n), g(i)(n) ? 2.9 The Iterated Logarithmic Function :The notation lg*n (read as log star of n) to denote The iterated logarithm, defined as :

Let lg(i)n be the logarithm function applied i times in succession, starting with argument n.22i.e., log(0)n = nlog(1)n = log nlog(2)n = log (log n)log(3)n = log (log (log n))log(k+1)n = log (log(k)n)Thus we define the iterated logarithm function as :

Log *n : = 0 , if n 1.lg*2 = 1,lg*4 = 2

lg*16 = 3, lg*65536 = 4

lg*(265536) = 5232.10 Fibonacci Numbers :

Def : F0 = 0,F1 = 1,Fi = Fi-1 + Fi-2 for i 2

Thus, each fibonacci number is the sum of its previous two numbers. Ex : 0,1,1,2,3,5,8,13,21,34,

Find the next five fibonacci numbers. The table for x, log star x is as follows :

X: (-,1](1,2](2,4](4,16](16,65536](65536, 265536]

Lg* x: 0 1 2 3 4 5

24Golden Ratio : Fibonacci Numbers are related to thegolden Ratio 1 and its conjugate 2, which are the two roots of the equation :

x2 = x + 1 The values of 1 and 2 are :

1 = (1 + 5) /2 = 1.618 (Golden Ratio) 2 = (1 - 5) /2 = - 0.618Let x = 2, y = (1 + 5) = (1 + 2.236) = 3.236Draw a rectangle for (x,y), (x,y-x), 25

The relationship between FN and GR : In the above diagram, we can see that y/x = x / (y-x)..(1) In Fibonacci Sequence : For large values of n, we haveFn / Fn-1 = Fn-1 / Fn-2 ..(2)26But, Fn = Fn-1 + Fn-2 Fn-2 = Fn Fn-1So, Fn / Fn-1 = Fn-1 / Fn-2 Fn / Fn-1 = Fn-1 / (Fn - Fn-1)If we replace, Fn with y and Fn-1 with x,we have y / x = x / (y x) which is Golden Ratio. Compute any number in Fibonacci Sequence :

Fn = n / 5

where F0 = 0, F1 = 1, F2 = 1, F3 = 2,..

*****273. Recurrences & Solutions of Recurrence Equations :3.1 Divide-and-Conquer : Divide : the problem into a number of subproblemsthat are smaller instances of the same problem.

Conquer : the subproblems by solving them recursively

Combine : the solutions to the subproblems into thesolution for the original problem.

283.2 Recurrences : A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs.Ex-19 : The recurrence equation for Fibonacci is : F0 = 0, F1 = 1 Fi = Fi-1 + Fi-2 for i 2 Ex-20 : Find the value of the recurrence equation g(0) = 0 g(n) = g(n-1) + 2n 1 using iteration. 293.3 Recurrence Equation :

A homogeneous recurrence equation is written as : a0tn + a1tn-1 + . . . . + aktn-k = 0.Ex-21 : tn 3tn-1 4tn-2 = 0, for n >= 2.{Initial condition: t0=0, t1=1}

Characteristic equation: xn 3x(n-1) 4x(n-2) = 0,or, x(n-2) [x2 3x 4] = 0,or, x2 3x 4 = 0,or, x2 + x 4x 4 = 0,or, x(x+1) 4(x+1) = 0,or, (x+1)(x-4) = 0,Therefore, roots are, x = -1, 4.30So, the general solution of the given recurrence equation is:tn = c1*(-1)n + c2*(4n)Use t0 = c1 + c2 = 0, and t1 = -c1 + 4c2 = 1.

Solve for c1 and c2,c1 = -(1/5), c2 = (1/5).So, the particular solution is:tn = (1/5)[4n (-1)n]= (4n)NOTE : The general solution for the original recurrence equation is:

tn = i=1kcirin

31Inhomogeneous recurrence equation

a0tn + a1tn-1 + . . . . + aktn-k = bnp(n), where b is a constant and p(n) is a polynomial of order n.

Note : Homogenize the given equation to an equivalent homogeneous recurrence equation form. Ex-22 : Solve tn 2tn-1 = 3n.

The concerned Homogeneous Equation is : tn+1 5tn + 6tn-1 = 0

Characteristic equation: x2 5 x + 6 = 0.

General solution of the given RE is: tn = c1*(2n) + c2*(3n) = (3n)32 Ex-23 : Solve tn 2tn-1 = n..(1)

Replace n with n+1tn+1 2tn = n+1.(2)

Subtract (1) from (2) :

tn+1 3tn + 2tn-1 = 1 .(3)Replace n with n+1 tn+2 3tn+1 + 2tn = 1 ..(4)

Subtract (3) from (4).tn+2 4tn+1 + 5tn - 2tn-1 = 0 ..(5)

Now it is a homogeneous recurrence equation

333.4 Analysis of Insertion Sort :

Input: A sequence of n numbers _a1, a2, . . . , an_.

Output: A permutation (reordering) a1, a2, . . . , an of the input sequence such that a1< a2 < . . . < an

Ex-23: 5 2 4 6 1 3 2 5 4 6 1 3 2 4 5 6 1 3

2 4 5 6 1 3 1 2 4 5 6 3 1 2 3 4 5 6INSERTION-SORT(A) 4) i j - 1

1) for j 2 to n 5) While i> 0 and A[i] > key2) do key A[ j ] 6) do A[i+1] A[i]

3)/* Insert A[ j ] into the sorted 7) i i - 1 sequence A[1 . . j 1]. */ 8) A[i+1] key34 Statement NoCost Times

1C1 n2C2 n-130 n 14C4 n - 1

5 C5 tjj=2 to n

6 C6 (tj-1)j = 2 to n 7 C7 (tj-1)j = 2 to n

8 C8n - 135The running time of the algorithm is

(cost of statement) (no. of times statement is for all statementsexecuted)Let T (n) = running time of INSERTION-SORT.

T (n) = c1 n + c2 (n 1) + c4 (n 1) +

C5 tj + C6 (tj - 1 ) + C7 (tj - 1 ) + j=2 to n j=2 to n j=2 to n c8 (n 1).

36Best case: The array is already sorted.

Always find that A[i ] key upon the first time the while loop test is run (when i = j 1). All t j are 1. Running time is T (n) = c1n + c2(n 1) + c4(n 1) + c5(n 1) + c8(n 1)

= (c1 + c2 + c4 + c5 + c8)n (c2 + c4 + c5 + c8) Can express T (n) as an +b for constants a and b (that depend on the statement costs ci )

T (n) is a linear function of n.

37Worst case: The array is in reverse sorted order.

Always find that A[i ] > key in while loop test.

Have to compare key with all elements to the left of the j th position compare with j 1 elements. Since the while loop exits because i reaches 0, there the j 1 tests tj = j . tj = j and (tj -1) = (j - 1) j=2 to n j=2 to n j=2 to n j=2 to n Here j = [n(n+1)/2] - 1 j=2 to n38Letting k = j 1, we see that

(j 1) = k = n(n-1)/2 j=2 to n k=1 to n-1So, Running Time is : T (n) = c1n + c2(n 1) + c4(n 1) +C5 [n(n+1)/2 - 1] + C6 [n(n-1)/2] + C7 [n(n-1)/2] + c8 (n 1)

= [c5/2 + c6/2 + c7/2] n2 + [c1 + c2 + c4 + C5/2 c6/2 c7/2 + c8] n [c2+c4+ c5+c8]Can express T (n) as an2 + bn + c for constants a, b, c (that again depend on statement costs)

T (n) is a quadratic function of n. 393.5 Recurrence Relations :A recurrence relation for the sequence {an} is an equation that expresses an in terms of one or moreof the previous terms of the sequence, namely :a0, a1, a2, , an-1, for all integers n with n n0, where no is a nonnegative integer. Ex-24 : Is an = 3n, the solution of the recurrence relation, wherean = 2. an-1 an-2. for n = 2,3,4Ex-25 : Is an = 5, the solution of the recurrence relation, wherean = 2. an-1 an-2. for n = 2,3,4

40Ex-26 : Some one deposits Rs. 10,000 in a savingsaccount at a bank yielding 5% per year withinterest compounded annually.

How much money will be in the account after 30 years.Ex-27 : Let an denote the number of bit strings of length n that do not have two consecutive 0s (valid strings).

Find a recurrence relation and give initialconditions for the sequence {an}.

41Ex-28 : What is the solution of the recurrence relation for the following.

an = an-1 + 2an-2 with a0 = 2 and a1 = 7.

Find its Characteristic Equation.Ex-29 : Prove that the following is explicit formula for Fibonacci Numbers.

fn= (1/5)[(1+5)/2]n - (1/5)[(1-5)/2]n

Find its Characteristic Equation. 42 Ex-30 : Find the solution of recurrence relationan = 6an-1 - 9an-2 with a0 = 1 and a1 = 6.The Characteristic Equation is : r 2 - 6 r + 9 = 0The only root is 3.

Hence the solution to the RR is : an = c1 3n + c2 n 3nThe Value of C1 = 1 and C2 = 1

Solution : an = 3n + n 3n 433.6 Solving Recurrences :

Ex-31 : Find number of calls for a plain recursive Fibonacci To compute the Fibonacci numbers fn,

f0 = 0, f1 = 1, and fn = fn1 + fn2, for n > 1. Denote by cn = #calls to compute fn.

c2 = 2, c3 = 4 = 22, c4 = 8 = 23 Following the recursion: cn = cn1 + cn2 + 2 = .

So, the value of cn is O(2n).44 a) The Substitution Method

The substitution method for solving recurrences consists of two steps:

1 Guess the form of the solution.

2 Use mathematical induction to find constants in the form and show that the solution works.We substitute the guessed solution for the function when applying the inductive hypothesis to smaller values. Hence, the name substitution method.

We can use the substitution method to establisheither upper or lower bounds on a recurrence.45Ex-32 : Solve T(n) = 2T(n/2) + n

Solution : Guess T(n) cn log n for some constant c (that is, T(n) = O(n log n))Base case: we need to show that our guess holds for some base case (not necessarily n = 1, some small n is ok). Ok, since function constant for small constant n.

Assume holds for n/2: T(n/2) c. (n/2) log(n/2) Prove that holds for n: T(n) cn log n , if c >=1.46Ex-33 : Solve T(n) = 2 1 = 3Prove that T(n) = O(n log n) for all n >= m.Here we have to show that T(n)