Dinive conquer algorithm

Divide and Conquer

Divide and Conquer

• divide the problem into a number of subproblems

• conquer the subproblems (solve them)• combine the subproblem solutions to get the

solution to the original problem

• Note: often the “conquer” step is done recursively

Divide-and-Conquer

A general methodology for using recursion to design efficient algorithms

It solves a problem by:– Diving the data into parts– Finding sub solutions for each of the parts– Constructing the final answer from the sub

solutions

Divide and Conquer

• Based on dividing problem into subproblems

• Approach1. Divide problem into smaller subproblems

Subproblems must be of same type

Subproblems do not need to overlap

2. Solve each subproblem recursively

3. Combine solutions to solve original problem

• Usually contains two or more recursive calls

Divide-and-conquer technique

subproblem 2 of size n/2

subproblem 1 of size n/2

a solution to subproblem 1

a solution tothe original problem

a solution to subproblem 2

a problem of size n

Divide and Conquer Algorithms

• Based on dividing problem into subproblems– Divide problem into sub-problems

Subproblems must be of same type

Subproblems do not need to overlap

– Conquer by solving sub-problems recursively. If the sub-problems are small enough, solve them in brute force fashion

– Combine the solutions of sub-problems into a solution of the original problem (tricky part)

D-A-C

• For Divide-and-Conquer algorithms the running time is mainly affected by 3 criteria:

• The number of sub-instances into which a problem is split.

• The ratio of initial problem size to sub-problem size.

• The number of steps required to divide the initial instance and to combine sub-solutions.

Algorithm for General Divide and Conquer Sorting

• Algorithm for General Divide and Conquer Sorting

• Begin Algorithm

Start Sort(L) If L has length greater than 1 then Begin

Partition the list into two lists, high and low Start Sort(high) Start Sort(low) Combine high and low

End

• End Algorithm

Analyzing Divide-and-Conquer Algorithms

• When an algorithm contains a recursive call to itself, its running time can often be described by a recurrence equation which describes the overall running time on a problem of size n in terms of the running time on smaller inputs.

• For divide-and-conquer algorithms, we get recurrences that look like:

• (1) if n < c• T(n) = aT(n/b) +D(n) +C(n) otherwise

Analyzing Divide-and-Conquer Algorithms (cont.)

• where

• a = the number of subproblems we break the problem into

• n/b = the size of the subproblems (in terms of n)

• D(n) is the time to divide the problem of size n into the subproblems

• C(n) is the time to combine the subproblem solutions to get the answer for the problem of size n

The algorithm

• Lets assume the following array

• We divide the values into pairs

• We sort each pair

• Get the first pair (both lowest values!)

2 6 7 3 5 6 9 2 4 1

2 6 7 3 5 6 9 2 4 1

2 6 3 7 5 6 2 9 1 4

The algorithm (2)

• We compare these values (2 and 6) with the values of the next pair (3 and 7)

– Lowest 2,3

• The next one (5 and 6)– Lowest 2,3



2 6 3 7 5 6 2 9 1 4

Example: Divide and Conquer

• Binary Search • Heap Construction• Tower of Hanoi • Exponentiation

– Fibonnacci Sequence

• Quick Sort• Merge Sort• Multiplying large Integers • Matrix Multiplications• Closest Pairs

Quicksort

Design Follows the divide-and-conquer paradigm. Divide: Partition (separate) the array A[p..r] into two

(possibly nonempty) subarrays A[p..q–1] and A[q+1..r].Each element in A[p..q–1] A[q].A[q] each element in A[q+1..r].Index q is computed as part of the partitioning

procedure. Conquer: Sort the two subarrays A[p..q–1] &

A[q+1..r] by recursive calls to quicksort.

Combine: Since the subarrays are sorted in place – no work is needed to combine them.

How do the divide and combine steps of quicksort compare with those of merge sort?

Pseudocode

Quicksort(A, p, r)if p < r then

q := Partition(A, p, r);Quicksort(A, p, q – 1);Quicksort(A, q + 1, r)

fi

Quicksort(A, p, r)if p < r then

q := Partition(A, p, r);Quicksort(A, p, q – 1);Quicksort(A, q + 1, r)

fi

Partition(A, p, r)x:= A[r],

i :=p – 1;for j := p to r – 1 do

if A[j] x theni := i + 1;

A[i] A[j]fi

od;A[i + 1] A[r];return i + 1

Partition(A, p, r)x:= A[r],

i :=p – 1;for j := p to r – 1 do


A[i] A[j]fi


5

A[p..r]

A[p..q – 1] A[q+1..r]

5 5

Partition 5

Example

p rinitially: 2 5 8 3 9 4 1 7 10 6 note: pivot (x) = 6 i j

next iteration: 2 5 8 3 9 4 1 7 10 6 i j




Partition(A, p, r)x, i := A[r], p – 1;for j := p to r – 1 do


A[i] A[j]fi




A[i] A[j]fi


Example (Continued)next iteration: 2 5 3 8 9 4 1 7 10 6 i j






after final swap: 2 5 3 4 1 6 9 7 10 8 i j



A[i] A[j]fi




A[i] A[j]fi


Partitioning Select the last element A[r] in the subarray A[p..r] as

the pivot – the element around which to partition. As the procedure executes, the array is partitioned

into four (possibly empty) regions.1. A[p..i] — All entries in this region are pivot.

2. A[i+1..j – 1] — All entries in this region are > pivot.

3. A[r] = pivot.

4. A[j..r – 1] — Not known how they compare to pivot. The above hold before each iteration of the for loop,

and constitute a loop invariant. (4 is not part of the LI.)

Correctness of Partition

Use loop invariant.Initialization:

– Before first iteration• A[p..i] and A[i+1..j – 1] are empty – Conds. 1 and 2 are satisfied

(trivially).• r is the index of the pivot – Cond. 3 is satisfied.

Maintenance:– Case 1: A[j] > x

• Increment j only.• LI is maintained.



A[i] A[j]fi




A[i] A[j]fi



>x x

p i j r

x > x

x

p i j r

x > x

Case 1:


x x

p i j r

x > x

• Case 2: A[j] x– Increment i

– Swap A[i] and A[j]• Condition 1 is maintained.

– Increment j• Condition 2 is maintained.

– A[r] is unaltered.• Condition 3 is maintained.

x > x

x

p i j r

Correctness of PartitionTermination:

– When the loop terminates, j = r, so all elements in A are partitioned into one of the three cases:

• A[p..i] pivot

• A[i+1..j – 1] > pivot

• A[r] = pivot

The last two lines swap A[i+1] and A[r].– Pivot moves from the end of the array to between the

two subarrays.– Thus, procedure partition correctly performs the divide

step.

Complexity of Partition

• PartitionTime(n) is given by the number of iterations in the for loop.

(n) : n = r – p + 1. Partition(A, p, r)x, i := A[r], p – 1;for j := p to r – 1 do


A[i] A[j]fi




A[i] A[j]fi


Algorithm Performance• Running time of quicksort depends on whether the partitioning

is balanced or not.

• Worst-Case Partitioning (Unbalanced Partitions):– Occurs when every call to partition results in the most unbalanced

partition.

– Partition is most unbalanced when

• Subproblem 1 is of size n – 1, and subproblem 2 is of size 0 or vice versa.

• pivot every element in A[p..r – 1] or pivot < every element in A[p..r – 1].

– Every call to partition is most unbalanced when

• Array A[1..n] is sorted or reverse sorted!

•

Worst-case Partition Analysis

• Running time for worst-case partitions at each recursive level:

• T(n) = T(n – 1) + T(0) + PartitionTime(n)• = T(n – 1) + (n)• = k=1 to n(k)• = (k=1 to n k )• = (n2)•

n

n – 1

n – 2

n – 3

2

1

n

Recursion tree forworst-case partition

Best-case Partitioning

• Size of each subproblem n/2.– One of the subproblems is of size n/2– The other is of size n/2 1.

• Recurrence for running time– T(n) 2T(n/2) + PartitionTime(n)

= 2T(n/2) + (n)

• T(n) = (n lg n)

Recursion Tree for Best-case Partition

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

c c c cc c

lg n

cn

cn

cn

Total : O(n lg n)

cn

Conclusion

• • Divide and conquer is just one of several

• powerful techniques for algorithm design.

• • Divide-and-conquer algorithms can be analyzed using recurrences and the master method (so practice this math).

• • Can lead to more efficient algorithms

Divide and Conquer (Merge Sort)Divide and Conquer (Merge Sort)

Divide and Conquer

• Recursive in structure – Divide the problem into sub-problems that

are similar to the original but smaller in size– Conquer the sub-problems by solving them

recursively. If they are small enough, just solve them in a straightforward manner.

– Combine the solutions to create a solution to the original problem

An Example: Merge Sort

• Sorting Problem: Sort a sequence of n elements into non-decreasing order.

• Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each

• Conquer: Sort the two subsequences recursively using merge sort.

• Combine: Merge the two sorted subsequences to produce the sorted answer.

Merge Sort – Example

18 26 32 6 43 15 9 1

18 26 32 6 43 15 9 1

18 26 32 6 43 15 9 1

2618 6 32 1543 1 9

18 26 32 6 43 15 9 1

18 26 32 6 43 15 9 1

18 26 32 6 15 43 1 9

6 18 26 32 1 9 15 43

1 6 9 15 18 26 32 43

18 26

18 26

18 26

32

32

6

6

32 6

18 26 32 6

43

43

15

15

43 15

9

9

1

1

9 1

43 15 9 1

18 26 32 6 43 15 9 1

18 26 6 32

6 26 3218

1543 1 9

1 9 15 43

1 6 9 1518 26 32 43

Original Sequence Sorted Sequence

Merge-Sort (A, p, r)

• INPUT: a sequence of n numbers stored in array A• OUTPUT: an ordered sequence of n numbersMergeSort (A, p, r) // sort A[p..r] by divide & conquer

1 if p < r2 then q (p+r)/23 MergeSort (A, p, q)4 MergeSort (A, q+1, r)5 Merge (A, p, q, r) // merges A[p..q] with A[q+1..r]

Initial Call: MergeSort(A, 1, n)

Procedure Merge• Merge(A, p, q, r)• 1 n1 q – p + 1• 2 n2 r – q for i 1 to n1 do L[i] A[p + i – 1] for j 1 to n2 do R[j] A[q + j] L[n1+1] R[n2+1] i 1 j 1 for k p to r do if L[i] R[j] then A[k] L[i] i i + 1 else A[k] R[j] j j + 1

•

Sentinels, to avoid having tocheck if either subarray isfully copied at each step.

Input: Array containing sorted subarrays A[p..q] and A[q+1..r].

Output: Merged sorted subarray in A[p..r].

j

Merge – Example

6 8 26 32 1 9 42 43 … …A

k

6 8 26 32 1 9 42 43

k k k k k k k

i i i i

i j j j j

6 8 26 32 1 9 42 43

1 6 8 9 26 32 42 43

k

L R

Correctness of Merge• Merge(A, p, q, r)• 1 n1 q – p + 1• 2 n2 r – q for i 1 to n1 do L[i] A[p + i – 1] for j 1 to n2 do R[j] A[q + j] L[n1+1] R[n2+1] i 1 j 1 for k p to r do if L[i] R[j] then A[k] L[i] i i + 1 else A[k] R[j] j j + 1

•

Loop Invariant for the for loopAt the start of each iteration of the for loop: Subarray A[p..k – 1] contains the k – p smallest elementsof L and R in sorted order. L[i] and R[j] are the smallest elements of L and R that have not been copied back into A.

Initialization:Before the first iteration: •A[p..k – 1] is empty.•i = j = 1.•L[1] and R[1] are the smallest elements of L and R not copied to A.

Correctness of Merge• Merge(A, p, q, r)• 1 n1 q – p + 1• 2 n2 r – q for i 1 to n1 do L[i] A[p + i – 1] for j 1 to n2 do R[j] A[q + j] L[n1+1] R[n2+1] i 1 j 1 for k p to r do if L[i] R[j] then A[k] L[i] i i + 1 else A[k] R[j] j j + 1

•

Maintenance:Case 1: L[i] R[j]•By LI, A contains p – k smallest elements of L and R in sorted order.•By LI, L[i] and R[j] are the smallest elements of L and R not yet copied into A.•Line 13 results in A containing p – k + 1 smallest elements (again in sorted order).Incrementing i and k reestablishes the LI for the next iteration.Similarly for L[i] > R[j].

Termination:•On termination, k = r + 1.•By LI, A contains r – p + 1 smallest elements of L and R in sorted order.•L and R together contain r – p + 3 elements. All but the two sentinels have been copied back into A.

Analysis of Merge Sort

• Running time T(n) of Merge Sort:• Divide: computing the middle takes (1) • Conquer: solving 2 subproblems takes 2T(n/2) • Combine: merging n elements takes (n)

• Total:T(n) = (1) if n = 1T(n) = 2T(n/2) + (n) if n > 1

T(n) = (n lg n) (CLRS, Chapter 4)

Dinive conquer algorithm

Technology

Transcript of Dinive conquer algorithm