5 -1

Chapter 5

The Divide-and-Conquer Strategy

5 -2

A simple example finding the maximum of a set S of n

numbers

5 -3

Time complexity:

Calculation of T(n): Assume n = 2k,

T(n) = 2T(n/2)+1= 2(2T(n/4)+1)+1= 4T(n/4)+2+1

:=2k-1T(2)+2k-2+…+4+2+1=2k-1+2k-2+…+4+2+1=2k-1 = n-1

T(n)= 2T(n/2)+1 1 , n>2 , n2

Time complexity

5 -4

Another simple exampleFind both the minimum and maximum

in an array of integers A[1..n] and assume that n is a power of 2.

The straightforward method:1. xA[1]; yA[1]2. for i2 to n3. if A[i]<x then xA[i]4. if A[i]>y then yA[i]5. end for6. return (x,y)

The number of element comparisons: 2n-2

5 -5

Algorithm MINMAXInput: An array A[1..n] of n integers, where n is a power of 2Output: (x,y), the minimum and maximum integers in A 1. minmax(1,n)

Procedure minmax(low,high) 1. if high-low=1 then 2. if A[low]<A[high] then return (A[low],A[high]) 3. else return (A[high],A[low]) 4. end if 5. else 6. mid(low+high)/2 7. (x1,y1)minmax(low,mid) 8. (x2,y2)minimax(mid+1,high) 9. xmin{x1,x2} 10. ymax{y1,y2} 11. return (x,y) 12. endif

The number of element comparisons:

(3n/2)-2

5 -6

A general divide-and-conquer algorithm

Step 1: If the problem size is small, solve this problem directly; otherwise, split the original problem into 2 sub-problems with equal sizes.

Step 2: Recursively solve these 2 sub-problems by applying this algorithm.

Step 3: Merge the solutions of the 2 sub- problems into a solution of the original problem.

5 -7

Time complexity of the general algorithm

Time complexity:

where S(n) : time for splitting M(n) : time for merging b : a constant

c : a constant

T(n)= 2T(n/2)+S(n)+M(n) b

, n c , n < c

5 -8

A partitioning algorithmSplitting/partitioning around x: The action of rearranging the

elements so that all elements less than or equal to x precede x which in turn precedes all elements all elements greater than x. x is called the pivot or splitting element.

Definition : We say that an element A[j] is in its proper position or correct position if it is neither smaller than the elements in A[low..j-1] nor larger than the elements in A[j+1..high]

Observation : After partitioning an array A using xA as a pivot, x will be in its correct position.

If we sort the elements in A in nondecreasing order after they have been rearranged, then we will still have A[w]=x

5 -9

Algorithm SPLITInput: An array of elements A[low..high]Output: 1) A with its elements rearranged, if necessary, as

described above 2) w, the new position of the splitting element A[low] 1. ilow 2. xA[low] 3. for jlow+1 to high 4. if A[j]x then 5. ii+1 6. if ij then interchange A[i] and A[j] 7. end if 8. end for 9. interchange A[low] and A[i] 10. wi 11. return A and w

5 -10

Throughout the execution of the algorithm, we maintain two pointers i and j that are initially set to low and low+1, respectively. These two pointers move from left to right so that after each iteration of the for loop, we have

1) A[low]=x2) A[k]x for all k, lowki3) A[k]>x for all k, i<kj

Observation : The number of element comparisons performed by Algorithm SPLIT is exactly n-1. Thus the time complexity is (n)

The only extra space used by Algorithm SPLIT is that needed to hold its local variables. Thus the space complexity is (1)

5 -11

QuicksortAlgorithm QUICKSORTInput: An array A[1..n] of n elementsOutput: The elements in A sorted in nondecreasing order 1. quicksort(A,1,n)Procedure quicksort(A,low,high) 1. if low<high then 2. SPLIT(A[low..high],w) {w is the new position of

A[low]) 3. quicksort(A,low,w-1) 4. quicksort(A,w+1,high) 5. end if

5 -12

Analysis of the quicksort algorithm

Theorem : The running time of Algorithm QUICKSORT is (n^2) in the worst case. If, however, the median is always chosen as the pivot, then its time complexity is (nlogn)

Theorem : The average number of comparisons performed by Algorithm QUICKSORT to sort an array of n elements is (nlogn)

5 -13

2-D maxima finding problem

Def : A point (x1, y1) dominates (x2, y2) if x1 > x2 and y1 > y2. A point is called a maxima if no other point dominates it

Straightforward method : Compare every pair of points.

Time complexity: O(n2)

5 -14

Divide-and-conquer for maxima finding

The maximal points of SL and SR

5 -15

The algorithm: Input: A set of n planar points. Output: The maximal points of S.Step 1: If S contains only one point, return it

as the maxima. Otherwise, find a line L perpendicular to the X-axis which separates the set of points into two subsets SLand SR , each of which consisting of n/2 points.

Step 2: Recursively find the maximal points of SL and SR .

Step 3: Find the largest y-value of SR. Project the maximal points of SL onto L. Discard each of the maximal points of SL if its y-value is less than the largest y-value of SR .

5 -16

Time complexity: T(n) Step 1: O(n) Step 2: 2T(n/2) Step 3: O(n)

Assume n = 2k

T(n) = O(n log n)

T(n)= 2T(n/2)+O(n)+O(n) 1

, n > 1 , n = 1

5 -17

The closest pair problem Given a set S of n points, find a pair of

points which are closest together. 1-D version :

Solved by sortingTime complexity :

O(n log n)

2-D version

5 -18

at most 6 points in area A:

5 -19

The algorithm: Input: A set of n planar points. Output: The distance between two closest

points. Step 1: Sort points in S according to their y-

values and x-values.Step 2: If S contains only one point, return

infinity as their distance.Step 3: Find a median line L perpendicular to

the X-axis to divide S into two subsets, with equal sizes, SL and SR.

Step 4: Recursively apply Step 2 and Step 3 to solve the closest pair problems of SL and SR. Let dL(dR) denote the distance between the closest pair in SL (SR). Let d = min(dL, dR).

5 -20

Step 5: For a point P in the half-slab bounded by L-d and L, let its y-value by denoted as yP . For each such P, find all points in the half-slab bounded by L and L+d whose y-value fall within yP +d and yP -d. If the distance d between P and a point in the other half-slab is less than d, let d=d . The final value of d is the answer.

Time complexity: O(n log n) Step 1: O(n log n) Steps 2~5:

T(n) = O(n log n)

T(n)= 2T(n/2)+O(n)+O(n) 1

, n > 1 , n = 1

5 -21

The convex hull problem

The convex hull of a set of planar points is the smallest convex polygon containing all of the points.

concave polygon:

convex polygon:

5 -22

The divide-and-conquer strategy to solve the problem:

Calculate the tangentOf this angle

5 -23

The merging procedure: 1. Select an interior point p.2. There are 3 sequences of points which have

increasing polar angles with respect to p.(1) g, h, i, j, k(2) a, b, c, d(3) f, e

3. Merge these 3 sequences into 1 sequence:g, h, a, b, f, c, e, d, i, j, k.

4. Apply Graham scan to examine the points one by one and eliminate the points which cause reflexive angles.

(See the example on the next page.)

5 -24

e.g. points b and f need to be deleted.

Final result:

(X2-x1)(y3-y1)-(y2-y1)(x3-x1)Zero: co-linearPositive: left turnOtherwise: right turn

5 -25

Divide-and-conquer for convex hull

Input : A set S of planar points Output : A convex hull for SStep 1: If S contains no more than five points, use

exhaustive searching to find the convex hull and return.

Step 2: Find a median line perpendicular to the X-axis which divides S into SL and SR ; SL lies to the left of SR .

Step 3: Recursively construct convex hulls for SL and SR. Denote these convex hulls by Hull(SL) and Hull(SR) respectively.

5 -26

Step 4: Apply the merging procedure to merge Hull(SL) and Hull(SR) together to form a convex hull.

Time complexity: T(n) = 2T(n/2) + O(n) = O(n log n)

5 -27

Multiplication of large integers

Let u and v be two n-bit integers. The traditional multiplication algorithm requires (n^2) digit multiplications to compute the product of u and v.

Assume n is a power of 2.Each integer is divided into 2 parts of n/2 bits

each: u=w*2^(n/2)+x v=y*2^(n/2)+zThen u*v=w*y*2^n+(w*z+x*y)*2^(n/2)+x*zBut T(n)=(n^2)!

5 -28

Multiplication of large integers

Let u and v be two n-bit integers. The traditional multiplication algorithm requires (n^2) digit multiplications to compute the product of u and v.

Assume n is a power of 2.Each integer is divided into 2 parts of n/2 bits each: u=w*2^(n/2)+x v=y*2^(n/2)+zThen u*v=w*y*2^n+(w*z+x*y)*2^(n/2)+xzBecause w*z+x*y=(w+x)(y+z)-w*y-x*zThen

u*v=w*y*2^n+((w+x)(y+z)-wy-xz)*2^(n/2)+xzAnd T(n)=(n^log3)=O(n^1.59)!

课堂练习 循环赛日程安排设有 n=2k个运动员要进行网球循环赛。现要求设计一个满足如下要求的比赛日程表：

1. 每个选手必须与其他 n-1个选手各赛一场；

2. 每个选手一天只能赛一次；

3. 循环赛共进行 n-1天。

5 -29

课堂练习

5 -30

A) 特殊棋盘 B) 不同形态的L型骨牌

在一个 2k*2k个方格组成的棋盘中，若恰有一个方格与其他方格不同，则称该方格为一个特殊方格，且称该棋盘为一个特殊棋盘，如图 A 所示。设计一个程序，用图B所示的 4 种不同形态的 L型骨牌覆盖一个除特殊棋盘上特殊方格以外的所有方格，且任何 2 个 L 型骨牌不得重叠覆盖。

5 -31

Matrix multiplication

Let A, B and C be n n matrices C = AB C(i, j) = A(i, k)B(k, j)

The straightforward method to perform a matrix multiplication requires O(n3) time.

1 k n

5 -32

Recursive versionAssume that n=2^k, k0. If n2, then A, B and C can be

partitioned as

The divide-and-conquer version consists of computing C as defined by the equation

Observation : The recursive version requires n^3 multiplications and n^3-n^2 additions. These are exactly the figures stated above for the traditional method

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2222122121221121

2212121121121111

BABABABA

BABABABAC

5 -33

Strassen’s algorithm

623142

537541

222122127

121111216

2212115

1121224

2212113

1122212

221122111

2221

1211

2221

1211

2221

1211

2221

1211

2221

1211

equation thefrom C compute Then

))((

))((

)(

)(

)(

)(

))((

products following theCompute

Let

dddddd

ddddddC

bbaad

bbaad

baad

bbad

bbad

baad

bbaad

bb

bb

aa

aa

cc

ccC

bb

bbB

aa

aaA

)()( is timerunning The

66

2 if)2/(18)2/(7

1 if)(

81.27log

27log7log

2

nOn

ananmn

nannT

nmnT

5 -34

Fast Fourier transform (FFT)

Fourier transform A(f) = a(t)e2iftdt

Inverse Fourier transform a(t) = A(f)e-2iftdf Discrete Fourier transform(DFT)

Given a0, a1, …, an-1

Aj = ake2ijk/n ,0 j n-1

Inverse DFT ak = Aje

-2ijk/n ,0 k n-1

1

2

0 1 k n

1

0 1n j n

5 -35

DFT and waveform Any periodic waveform can be

decomposed into the linear sum of sinusoid functions (sine or cosine).

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-4

-3

-2

-1

0

1

2

3

4

7 15 48 56f( )頻 率

))15(2cos(3))7(2cos()( tttf See the left part:

5 -36

An application of the FFT polynomial multiplication

Polynomial multiplication:

The straightforward product requires O(n2) time. DFT notations:

.},,,{ of DFTinverse theis },,,{

10 ,

...

.},,,{ of DFT theis },,,{

1 ,10 ,Let

...

110110

1

11

2210

110110

11

2210

nn

knk

nn

nn

njj

nn

bbbaaa

njwha

xbxbxbbxh

aaabbb

wnjwfb

xaxaxaaxf

xgxfxhxcxgxaxfn

k

kk

n

j

jj

,1

0

1

0

5 -37

Fast polynomial multiplication

Step 1: Let N be the smallest integer that N=2q and N2n-1. Step 2: Compute FFT of

Step 3: Compute FFT of

Step 4:Step 5:

Time complexity: O(NlogN)=O(nlogn), N<4n.

Nijj ewNjwgwf /2 ,10 ),( Compute

.}0,,0,0,,,,{ 110 N

naaa

.}0,,0,0,,,,{ 110 N

nccc

.)( of tscoefficien the

are numbers of sequence resulting The

)}.(,),(),({ of DFTinverse Compute

)()(Let 110

xh

whwhwh

wgwfwhN

jjj

5 -38

FFT algorithm A straightforward method to calculate DFT

requires O(n2) time. DFT can be solved by the divide-and-conquer

strategy (FFT). Let w=e2i/n. i.e. wn=1, wn/2=-1.

Aj = ake2ijk/n , 0 j n-1

= akwjk

e.g. n = 8, let e2i/8= w= ei/4

0 1 k n

0 1 k n

sincos iei

5 -39

FFT algorithm when n=4

n =4, let e2i/4=w = ei/2 (w4 = 1, w2 = -1) A0 = a0 + a1 + a2 + a3

A1 = a0 + a1w + a2w2 + a3w

3

A2 = a0 + a1w2 + a2w

(2)(2) + a3w(2)(3)

= a0 + a1w2 + a2w

4 + a3w6

A3 = a0 + a1w3 + a2w

(3)(2) + a3w(3)(3)

= a0 + a1w3 + a2w

6 + a3w9

5 -40

Rewrite as:

A0 = a0 + a2 + (a1 + a3)

A2 = a0 + a2w4 + (a1w

2 + a3w6)

= a0 + a2 - (a1 + a3) When we calculate A0 , we shall calculate (a0 +

a2) and (a1 + a3). Later, A2 can be easily found. Similarly,

A1 = (a0 + a2w2) + a1w + a3w

3

A3 = (a0 + a2w6) + (a1w

3 + a3w9)

= (a0 + a2w2) - (a1w + a3w

3)

5 -41

n = 8, let e2i/8= w = ei/4 (w8 = 1, w4 = -1)A0 = a0 + a1 + a2 + a3 + a4 + a5 + a6 + a7

A1= a0+a1w+a2w2+a3w

3+a4w4+a5w

5+a6w6+a7w

7

A2 = a0+a1w2+a2w

4+a3w6+a4w

8+a5w10+a6w

12+a7w14

A3 = a0+a1w3+a2w

6+a3w9+a4w

12+a5w15+a6w

18+a7w21

A4 = a0+a1w4+a2w

8+a3w12+a4w

16+a5w20+a6w

24+a7w28

A5 = a0+a1w5+a2w

10+a3w15+a4w

20+a5w25+a6w

30+a7w35

A6 = a0+a1w6+a2w

12+a3w18+a4w

24+a5w30+a6w

36+a7w42

A7 = a0+a1w7+a2w

14+a3w21+a4w

28+a5w35+a6w

42+a7w49

FFT algorithm when n=8

5 -42

After reordering, we haveA0=(a0+a2+a4+a6)+(a1+a3 +a5 +a7)

A4=(a0+a2+a4+a6)-(a1+a3 +a5 +a7)

A1=(a0+a2w2+a4w

4+a6w6)+w(a1+a3w

2+a5w4+a7w

6)

A5=(a0+a2w2+a4w

4+a6w6)-w(a1+a3w

2+a5w4+a7w

6)

A2=(a0+a2w4+a4w

8+a6w12)+w2(a1+a3w

4+a5w8+a7w

12)

A6=(a0+a2w4+a4w

8+a6w12)-w2(a1+a3w

4+a5w8+a7w

12)

A3=(a0+a2w6+a4w

12+a6w18)+w3(a1+a3w

6+a5w9+a7w

18)

A7=(a0+a2w6+a4w

12+a6w18)-w3(a1+a3w

6+a5w9+a7w

18) Rewrite as:

A0 = B0 + C0 A4 = B0 － C0

A1 = B1 + C1 A5 = B1 － C1

A2 = B2 + C2 A6 = B2 － C2

A3 = B3 + C3 A7 = B3 － C3

5 -43

let x=w2 =e2i/4= ei/2 (x4 = 1, x2 = -1) We can recursively apply the same method to calculate

Bi’s and Ci’s.

B0 = a0+a2+a4+a6

B1 = a0+a2w2+a4w

4+a6w6

= a0+a2x+a4x2+a6x

3

B2 = a0+a2w4+a4w

8+a6w12

= a0+a2x2+a4x

4+a6x6

B3 = a0+a2w6+a4w

12+a6w18

= a0+a2x3+a4x

6+a6x9

Thus, {B0, B1, B2, B3} is the DFT of {a0, a1, a2, a3}.

5 -44

General FFT In general, let w = e2i/n (assume n is even)

(wn = 1, wn/2 = -1)

Aj = a0+a1wj+a2w

2j+…+an-1w(n-1)j

={a0+a2w2j+a4w

4j+…+an-2w(n-2)j} +

{a1wj+a3w

3j+…+an-1w(n-1)j }

= Bj + Cj

Aj+n/2 = a0+a1wj+n/2+a2w

2j+n+a3w3j+3n/2+…+

an-1w(n-1)j+(n(n-1)/2)

=a0-a1wj+a2w

2j-a3w3j+…+an-2w

(n-2)j-an-1w(n-1)j

= Bj - Cj

5 -45

Divide-and-conquer (FFT) Input : a0 , a1 , …, an-1, n = 2k Output : Aj, j=0, 1, 2, …, n-1,

where Aj = 0kn-1ake2ijk/n

Step 1: If n=2, compute

A0 = a0 + a1,

A1 = a0 - a1, and return.

Step 2: Divide each aj, 0 j n/2 - 1 into two sequences: Oj and Ej, where Oj(Ej), consists of odd-numbered (even-numbered) terms of Aj.

5 -46

Step 3: Recursively calculate the sums of terms in Oj and Ej . Denote the sum of terms of Oj and Ej by Bj and Cj, respectively.

Step 4: Compute Aj by the following formual :

Aj = Bj + Cj for 0 j n/2 - 1

Aj+n/2 = Bj - Cj for 0 j n/2 - 1.

Time complexity :T(n) = 2T(n/2) + O(n)

= O(n log n)

5 -47

Comparisons of the algorithms

5 -48

The Voronoi diagram problem

e.g. The Voronoi diagram for three points

Each Lij is the perpendicular bisector of the line.

5 -49

Definition of Voronoi diagrams

Def : Given two points Pi, Pj S, let H(Pi,Pj) denote the half plane containing Pi. The Voronoi polygon associated with Pi is defined as

jiji PPHiV

),()(

5 -50

Given a set of n points, the Voronoi diagram consists of all the Voronoi polygons of these points.

The vertices of the Voronoi diagram are called Voronoi points and its segments are called Voronoi edges.

5 -51

Delaunay triangulation

5 -52

Example for constructing Voronoi diagrams

Divide the points into two parts.

5 -53

Merging two Voronoi diagrams

Merging along the piecewise linear hyperplane

5 -54

After merging

The final Voronoi diagram

5 -55

Divide-and-conquer for Voronoi diagram

Input: A set S of n planar points. Output: The Voronoi diagram of S.Step 1: If S contains only one point, return.Step 2: Find a median line L perpendicular

to the X-axis which divides S into SL and SR such that SL (SR) lies to the left(right) of L and the sizes of SL and SR are equal.

5 -56

Step 3: Construct Voronoi diagrams of SL and SR recursively. Denote these Voronoi diagrams by VD(SL) and VD(SR).

Step 4: Construct a dividing piece-wise linear hyperplane HP which is the locus of points simultaneously closest to a point in SL and a point in SR. Discard all segments of VD(SL) which lie to the right of HP and all segments of VD(SR) that lie to the left of HP. The resulting graph is the Voronoi diagram of S.

(See details on the next page.)

5 -57

Mergeing Two Voronoi Diagrams into One Voronoi Diagram

Input: (a) SL and SR where SL and SR are divided by a perpendicular line L.

(b) VD(SL ) and VD(SR ).

Output: VD(S) where S = SL ∩SR

Step 1: Find the convex hulls of SL and SR, denoted as Hull(SL) and Hull(SR), respectively. (A special algorithm for finding a convex hull in this case will by given later.)

5 -58

Step 2: Find segments and which join HULL(SL ) and HULL(SR ) into a convex hull (Pa and Pc belong to SL and Pb and Pd belong to SR) Assume that lies above . Let x = a, y = b, SG= and HP = .

Step 3: Find the perpendicular bisector of SG. Denote it by BS. Let HP = HP∪{BS}. If SG = , go to Step 5; otherwise, go to Step 4.

dcPPba PP

baPP dcPP

yxPP

dcPP

5 -59

Step 4: The ray from VD(SL ) and VD(SR) which BS first intersects with must be a perpendicular bisector of either or for some z. If this ray is the perpendicular bisector of , then let SG = ; otherwise, let SG = . Go to Step 3.

Step 5: Discard the edges of VD(SL) which extend to the right of HP and discard the edges of VD(SR) which extend to the left of HP. The resulting graph is the Voronoi diagram of S = SL∪SR.

zxPP zy PP

zy PP zxPP

yz PP

5 -60

Properties of Voronoi Diagrams Def : Given a point P and a set S of

points, the distance between P and S is the distance between P and Pi which is the nearest neighbor of P in S.

The HP obtained from the above algorithm is the locus of points which keep equal distances to SL and SR .

The HP is monotonic in y.

5 -61

# of edges of a Voronoi diagram 3n - 6, where n is # of points.

Reasoning:i. # of edges of a planar graph with n

vertices 3n - 6.ii. A Delaunay triangulation is a planar

graph.iii. Edges in Delaunay triangulation edges in Voronoi diagram.

1 1

# of Voronoi edges

5 -62

# of Voronoi vertices

# of Voronoi vertices 2n - 4. Reasoning:

i. Let F, E and V denote # of face, edges and vertices in a planar graph.

Euler’s relation: F = E - V + 2.ii. In a Delaunay triangulation,

V = n, E 3n – 6 F = E - V + 2 3n - 6 - n + 2 = 2n -

4.

5 -63

Construct a convex hull from a Voronoi diagram

After a Voronoi diagram is constructed, a convex hull can by found in O(n) time.

5 -64

Construct Convex Hull from Voronoi diagram

Step 1: Find an infinite ray by examining all Voronoi edges.

Step 2: Let Pi be the point to the left of the infinite ray. Pi is a convex hull vertex. Examine the Voronoi polygon of Pi to find the next infinite ray.

Step 3: Repeat Step 2 until we return to the Starting ray.

5 -65

Time complexity Time complexity for merging 2 Voronoi

diagrams: Total: O(n)

Step 1: O(n) Step 2: O(n) Step 3 ~ Step 5: O(n)

(at most 3n - 6 edges in VD(SL) and VD(SR) and at most n segments in HP)

Time complexity for constructing a Voronoi diagram: O(n log n)because T(n) = 2T(n/2) + O(n)=O(n log

n)

5 -66

Lower bound The lower bound of the Voronoi

diagram problem is (n log n). sorting Voronoi diagram problem

The Voronoi diagram for a set of points on a straight line

5 -67

Applications of Voronoi diagrams

The Euclidean nearest neighbor searching problem.

The Euclidean all nearest neighbor problem.