Chapter 6 Divide and Conquer Introduction Binary Search Mergesort The Divide and Conquer Paradigm Quicksort Multiplication of Large Integers Matrix Multiplication The Closest Pair Problem

6.1 Introduction

Main idea: A divide-and-conquer algorithm divides the problem instance into a number of subinstances, recursively solves each subinstance separately, and then combines the solutions to the subinstances to obtain the solution to the original problem instance.

6.1 Introduction

1.X A[1]; y A[1]2.For I 2 to n3.If A[i]>y then y A[i]4.If A[i]<x then x A[i]5.End for6.Return (x,y)

E.g.: The problem of finding both the minimum and maximum in an array of integers A[1..n] and assume for simplicity that n is a power of 2. A straightforward algorithm might look line the one below.

6.1 Introduction Algorithm 6.1 MINMAX

Input: An array A[1..n] of n integers, where n is a power of 2.Output: (x,y): the minimum and maximum integers in A. 1.minmax(1,n)Procedure minmax(low,high) 1. if high-low=1 then 2. if A[low]<A[high] then return(A[low],A[high]) 3. else return(A[high],A[low]) 4. end if 5. else 6. mid 7. (x1,y1) minmax(low,mid) 8. (x2,y2) minmax(mid+1,high) 9. x min{x1,x2} 10. y max{y1,y2} 11. return (x, y) 12. end if

high)/2(low

6.1 Introduction Theorem 6.1

Given an array A[1..n] of n elements, where n is a power of 2, it is possible to find both the minimum and maximum of the elements in A using only (3n/2)-2 element comparisons.

6.2 Binary Search

Binary search algorithm is one of the divide-and-conquer algorithm.

Algorithm 6.2 BINARYSEARCHREC

Input: An array A[1..n] of n elements sorted in nondecreasing order and an element x.

Output: j if x=A[j],1<=j<=n, and 0 otherwise. 1.binarysearch(1,n)Procedure binarysearch(low,high) 1. if low>high then return 0 2. else 3. mid 4. if x=A[mid] then return mid 5. else if x<A[mid] then return binarysearch(low,mid-1) 6. else return binarysearch(mid+1,high) 7. end if

high)/2(low

6.2

Theorem 6.2

The number of element comparisons performed by Algorithm

BINARYSEARCHREC to search for an element in an array of n elements is at most +1. Thus, the time complexity of Algorithm BINARYSEARCHREC is O(log n).

n log

6.2

6.3 MergesortAlgorithm 6.3 MERGESORTInput: An array A[1..n] of n elements.Output: A[1..n] sorted in nondecreasing order. 1.mergesort(A,1,n)Procedure mergesort(low,high) 1. if low<high then 2. mid 3. mergesort(A,low,mid) 4. mergesort(A,mid+1,high) 5. MERGE(A,low,mid,high) 6. end if

high)/2(low

Theorem 6.3

Algorithm MERGESORT

sorts an array of n elements in

time (n log n) and space (n).

6.3

6.4 The Divide-and-conquer Paradigm

In general, a divide-and-conquer algorithm has the following format.

(1)If the size of the instance I is “small”, then solve the problem using a straightforward method and return the answer. Otherwise, continue to the next step.

(2) Divide the instance I into p subinstances I1,I2,…,Ip of approximately the same size.

(3) Recursively call the algorithm on each subinstance Ij, 1<=j<=p, to obtain p partial solutions.

(4) Combine the results of the p partial solutions to obtain the solution to the original instance I. Return the solution of instance I.

6.5 Selection: Finding the Median and the kth Smallest Element Problem: the median of a sequence of n sorted

numbers A[1..n] is the “middle” element. If n is odd, then the middle element is the (n+1)/2 th element in the sequence. If n is even, then there are two middle elements occurring at positions n/2 and n/2+1. in this cases, we will choose the n/2th smallest element. Thus, in both cases, the median is the n/2 th smallest element.

Algorithm 6.4 SELECTInput:An array A[1..n] of n elements and an integer k, 1 k n.Output: The kth smallest element in A. 1. select(A,1,n,k)Procedure select(A,low,high,k) 1. p high-low+1 2. if p<44 then sort A and return (A[k]) 3. Let q= . Divide A into q groups of 5 elements each. If 5 does not di

vide p, then discard the remaining elements 4. Sort each of the q groups individually and exact its median. Let the

set of medians be M. 5. mm select(M,1,q, ). {mm is the median of medians} 6. Partition A[low..high] into three arrays ： A1={a|a<mm} A2={a|a=mm} A3={a|a>mm} 7.case |A1| k: return select(A1,1,|A1|,k) |A1|+|A2| k: return mm |A1|+|A2|<k: return select(A3,1,|A3|,k-|A1|-|A2|) 8. end case

p/5

2/q

6.5

Theorem 6.4 The kth smallest element in a set of

n elements drawn from a linearly ordered set can be found in (n) time. In particular, the median of n elements can be found in (n) time.

6.5

6.6 QuicksortAlgorithm 6.5 SPLIT

Input: An array of elements A[low..high].Output: (1) A with its elements rearranged, if necessary, as described

above. (2) w,the new position of the splitting element A[low].

1. i=low 2. x=A[low] 3. for j=low+1 to high 4. if A[j]<=x then 5. i=i+1 6. if i/=j then interchange A[i] and A[j] 7. end if 8. end for 9. interchange A[low] and A[i] 10. w=i 11. Return A and w

6.6 QuicksortAlgorithm 6.6 QUICKSORTInput: An array A[1..n] of n elements.Output: The elements in A sorted in nondecreasing order. 1.quicksort(A,1,n)Procedure quicksort(A,low,high) 1.if low<high then 2. SPLIT(A[low..high],w) {w is the new position of A[low]} 3. quicksort(A,low,w-1) 4. quicksort(A,w+1,high) 5. end if

Theorem 6.5

The running time of Algorithm QUICKSORT is in the worst case. If, however, the median is always chosen as the pivot, then its time Complexity is (n log n).

2n

6.6

Theorem 6.6

The average number of comparisons performed by Algorithm QUICKSORT to sort an array of n elements is (n log n).

6.6

Multiplication of Large Integers

Let u and v be two n-bit integers. The traditional multiplication algorithm requires (n2) digit multiplications to compute the product of u and v. Using the divide and conquer technique, the bound can be reduced significantly.

6.7

6.7

xwu n 2/2

zyv n 2/2

W X

Y Z

xzxywzwyzyxwuv nnnn 2/2/2/ 2)(2)2)(2(

1)2/(4

1)(

nifbnnT

nifdnT

)()( 2nnT

6.7

xzwyzyxwxywz ))((

xzxzwyzyxwwyuv nn 2/2)))(((2

1)2/(3

1)(

nifbnnT

nifdnT

)()()( 59.13log nnnT

Matrix Multiplication

Let A and B to be two n*n matrices. We with to compute their product C=AB.

6.8.1 The traditional algorithm:

6.8

n

k

jkBkiAjiC1

),(),(),(

Matrix Multiplication

6.8.2 Recursive versionAssume n=2k, k>=0.

6.8

22

12

21

11

A

A

A

AA

22

12

21

11

C

C

C

CC

22

12

21

11

B

B

B

BB

22221221

22121211

21221121

21121111

BABA

BABA

BABA

BABAC

2)2/(4)2/(8

1)( 2 nifannT

nifmnT

Matrix Multiplication

6.8.3 Strassen’s algorithmAssume n=2k, k>=0.

6.8

22

12

21

11

a

a

a

aA

22

12

21

11

b

b

b

bB

2221

1211

2221

1211

22

12

21

11

bb

bb

aa

aa

c

c

c

cC

Matrix Multiplication

6.8.3 Strassen’s algorithm

6.8

6231

53

42

7541

dddd

dd

dd

ddddC

))((7

))((6

)(5

)(4

)(3

)(2

))((1

22212212

12111121

221211

112122

221211

112221

22112211

bbaad

bbaad

baad

bbad

bbad

baad

bbaad

Matrix Multiplication

6.8.3 Strassen’s algorithm

6.8

2)2/(18)2/(7

1)( 2 nifannT

nifmnT

)()()( 81.27log nnnT

6.9 The Closest Pair Problem Let S be a set of n points in the plane. In

this section, we consider the problem of finding a pair of points p and q in S whose mutual distance is minimum. In other words, we want to find two points p1=(x1,y1) and p2=(x2,y2) in S with the property that the distance between them defined by

Is minimum among all pairs of points in S.

22 )21()21()2,1( yyxxppd

Algorithm 6.7 CLOSESTPAIR

Input: A set S of n points in the plane.Output: The minimum separation realized by two

points in S.

1.Sort The points in S in nondecreasing order of their x-coordinates.

2.Y The points in S sorted in nondecreasing order of their y-coordinates.

3. cp(1,n)

6.9

Procedure cp(low,high)1. If high-low+1 3 then compute by a straightforward

method.2. else 3. mid 4. x(S[mid])5. cp(low,mid)6. cp(mid+1,high)7. min{ , }8. k 09. for i 1 to n {Extract T from Y}10. if |x(Y[i])- | then11. k k+112. T[k] Y[i]13. end if14. end for {k is the size of T}15. 2 {Initialize to any number greater than }16. for i 1 to k-1 {compute }17. for j i+1 to min{i+7,k}18. if d(T[i],T[j])< then d(T[i],T[j])19. end for20. end for21. min{ , }22. End if23. Return

high)/2(low

0x

l

r

0x

l

r

6.9

Observation 6.5

Each point in T needs to be compared with at most seven points in T.

6.9

6.9 Theorem 6.7 Given a set S of n points in the plane, Algorithm CLOSESTPAIR finds a pair of points in S with minimum separation in ⊙ (n log n) time.

3)()2/(2

33

21

)(

nifnnT

nif

nif

nT