National University Of Computer & Emerging Sciences

Divide & Conquer

Design and Analysis of Algorithms (Lecture 14)

Divide And Conquer

Fall-29

National University Of Computer & Emerging Sciences

Divide & Conquer

• General techniques that often yield effective algorithms for solving large classes of problems.

• Some of the more important techniques are (i) Greedy (ii) Dynamic Programming and (iii) Divide & Conquer

• Before using any technique, better ask about the quality of the solution obtained.

• Remember that for NP-Complete problems, these or any other known techniques will NOT yield efficient solutions.

Introduction

National University Of Computer & Emerging Sciences

Divide & Conquer

• Breaking up large problems into smaller units that are easier to handle

• Example: Cleaning the house, implementing a word processor

• Top-down methodology and object-oriented methodology are based on the principle of divide and conquer

Divide and Conquer

National University Of Computer & Emerging Sciences

Divide & Conquer

Divide and Conquer

Easy ProblemEasy Problem Easy ProblemEasy Problem

Hard ProblemHard ProblemEasy ProblemEasy Problem Easy ProblemEasy Problem

Hard ProblemHard Problem

National University Of Computer & Emerging Sciences

Divide & Conquer

Divide & Conquer• Divide & conquer paradigm involves 3

steps:– Divide the problem into sub-problems– Conquer the sub-problems– Combine the solutions to the sub-problems.

• This nature of divide & conquer algorithms automatically lends to recursion.

• Classic examples: – Merge sort : T(n) = 2 T(n/2) + O(n)– Binary search: T(n) = T(n/2) + 1

• Proof: usually by induction method.

P

P1 P2

S1 S2

S

National University Of Computer & Emerging Sciences

Divide & Conquer

Divide & Conquer (Constructing Tennis Tournament)

• Round robin tournament of n = 2k players.

• Each player must play every other player.

• Each player must play one match per day for n – 1 days.

• The tournament schedule is thus an n row by n-1 column table, whose entry is row i and column j is that player i must play with which player on jth day.

• Work by finding schedule for one half of players, keep on doing it till we get down to two players.

National University Of Computer & Emerging Sciences

Divide & Conquer

Divide & Conquer (Constructing Tennis Tournament)

• Divide the matrix into four parts

• The upper left corner comes from the smallest schedule, recursively.

• The bottom left corner is obtained by adding n/2 to each player code of the top left corner

• The top right corner is obtained by pitting the high order players against low order players, by cyclically permuting the orders.

• The bottom right corner is similarly obtained by pitting the low order players against high order players, by cyclically permuting the orders.

National University Of Computer & Emerging Sciences

Divide & Conquer

Divide & Conquer (Constructing Tennis Tournament)

Player

Will play player

National University Of Computer & Emerging Sciences

Divide & Conquer

Divide & Conquer (Constructing Tennis Tournament)

National University Of Computer & Emerging Sciences

Divide & Conquer

Example: Min-Max Problems

• Problem: Find the largest (heaviest) and smallest (lightest) elements in a set of n elements (e.g., gold nuggets, counterfeit coins)

• Simple Algorithm: Make n-1 comparisons to find largest, n-2 more comparisons to find smallest for a total of 2n-3 comparisons.

National University Of Computer & Emerging Sciences

Divide & Conquer

Min-Max Cont’d

• Divide-and-Conquer:– For n 2, make 1 comparison– For large n, divide set into two smaller sets and determine

largest/smallest element for each set– Compare largest/smallest from two subsets to determine

smallest/largest of combined sets– Do recursively

• Let n =8• How many comparisons?

National University Of Computer & Emerging Sciences

Divide & Conquer

Min-Max Cont’d

• Recursive solution: recurrence equation

1, n=2

• c(n) =

2* c(n/2) + 2, n =2k

• Solve using substitutionc(n) = (3n/2) - 2, n=2k

National University Of Computer & Emerging Sciences

Divide & Conquer

Misc• Divide-and-Conquer methodology naturally leads

to recursive algorithms• In general, implement algorithm as recursive

program– Few instances where non-recursive is better

• Rule of thumb: Balance sub-problems, i.e., try and make sub-problems of equal size– See merge sort

National University Of Computer & Emerging Sciences

Divide & Conquer

MULTIPLICATION OF TWO n-BIT NUMBERS• For this problem, the basic operations should be

bit operations.• Let X and Y be two n-bit numbers, the traditional

method requires O(n2) bit operations, say

National University Of Computer & Emerging Sciences

Divide & Conquer

MULTIPLICATION OF TWO n-BIT NUMBERS (cont.)

• Assume n=2m, we treat X and Y as two n-bit strings and partition them into two halves as X = A * B, Y= C * D, i.e., concatenation of A, B and C, D. As binary numbers, we can express

XY = (A2n/2 + B) (C2n/2 + D)

= AC2n + (AD + BC) 2n/2 + BD ............ (*)• The above computation of XY reduces to four multiplications

of (n/2)-bit numbers plus some additions and shifts (multiplications by powers of 2).

National University Of Computer & Emerging Sciences

Divide & Conquer

Two n-bits Number Analysis

• Let T(n) be the number of bit-operations required to multiply two n-bit numbers.

National University Of Computer & Emerging Sciences

Divide & Conquer

Two n-bits Number Analysis (cont.)

National University Of Computer & Emerging Sciences

Divide & Conquer

Two n-bits Number Analysis (cont.)

• Thus T(n) depends very much the number of subproblems and also the size of the subproblems.

• If the number of subproblems were 1, 3, 4, 8, then the algorithms would be of order n, nlog3, n2, n3 respectively.

National University Of Computer & Emerging Sciences

Divide & Conquer

A better Algorithm

• So, the multiplication of two n-bit numbers remains O(n2) if the number of half-size subproblems remains 4

• In order to reduce the time complexity, we attempt to reduce the number of subproblems (i.e., multiplications ) by a trick which uses more additions and shifts, specifically, by reducing one multiplication with many additions and shifts.

• This trick pays off asymptotically, i.e., when n is large.

National University Of Computer & Emerging Sciences

Divide & Conquer

A better Algorithm (cont.)U = (A + B)(C + D)V = ACW = BDZ = U - V - W = AD + BC

So, XY can be written as V2n+ Z2 n/2 + W (from Eq (*)).

• This approach requires only three multiplications of two (n/2)-bit numbers, plus some additions and shifts, to multiply two n-bit numbers.

XY= AC2n + (AD + BC) 2n/2 + BD ............ (*)

National University Of Computer & Emerging Sciences

Divide & Conquer

Analysis

• One can use the multiplication routine recursively to evaluate the products u, v, and w. The additions and shifts require O(n) time. Thus the time complexity of multiplying two n-bit numbers is bounded from above by

• Where k is a constant reflecting the additions and shifts. From the above discussion, the solution to the recurrence is then O(n log3) = O(n 1.59).

National University Of Computer & Emerging Sciences

Divide & Conquer

Example

This divide and conquer technique for multiplication of two n-bit number can also applied to the multiplication problem of two n-digit integers and n-degree polynomial.

National University Of Computer & Emerging Sciences

Divide & Conquer

Is it really better than O(n2)?

If this technique is so good, why don’t we teach it in school?

1. The technique is quite complex to understand. Most likely the students will never learn multiplication.

2. The constant ignored is very large i.e. 500. Useful for large multiplications.

3. Usually don’t multiply numbers 500 bit numbers by paper pencil technique in schools.