1

Divide and Conquer Binary Search Mergesort Recurrence Relations

CSE 30331CSE 30331Lecture 4 – Algorithms IILecture 4 – Algorithms II

2

Divide & Conquer

Basic approach is …

break problem in parts

separately solve each part

rejoin resulting partial solutions into whole

3

Binary Search Algorithm

Case 1: A match occurs. The search is complete and mid is the index that locates the target.

if (midValue == target) // found match return mid;

m idfirs t

target

C as e 1: target = m id valueS earc h is d o ne

las t-1 las t

4

Binary Search Algorithm

Case 2: The value of target is less than midvalue and the search must continue in the lower sublist.

if (target < midvalue) // search lower sublist

<reposition last to mid>

<search sublist arr[first]…arr[mid-1]

las t-1firs t

target

C as e 2: target < m id valueS earc h lo w er s ub lis t

m id -1 las t

5

Binary Search Algorithm

Case 3: The value of target is greater than midvalue and the search must continue in the upper sublist .

if (target > midvalue)// search upper sublist

<reposition first to mid+1>

<search sublist arr[mid+1]…arr[last-1]>

C as e 3: target > m id valueS earc h up p er s ub lis t

las t-1new firs t = m id + 1

firs t

target

las t

6

Successful Binary Search

Search for target = 23

Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.

Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

7

Successful Binary Search

Step 2:

Indices first = 5, last = 9, mid = (5+9)/2 = 7.

Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

8

Successful Binary Search

Step 3:

Indices first = 5, last = 7, mid = (5+7)/2 = 6.

Since target = midvalue = 23, a match is found at index mid = 6.

m id

-7 3 5 8 12 16arr0 1 2 3 4 5

23 33 55

6 7 8 9

9

Unsuccessful Binary Search

Search for target = 4.

Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.

10

Unsuccessful Binary Search

Step 2:

Indices first = 0, last = 4, mid = (0+4)/2 = 2.

Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.

m id

-7 3 5 8 12 16arr0 1 2 3 4 5

23 33 556 7 8

11

Unsuccessful Binary Search

Step 3: Indices first = 0, last = 2, mid = (0+2)/2 = 1.

Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index last = 9.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

12

Binary Search Implementationint binSearch (const int arr[], int first, int last,

int target)

{

int origLast = last; // original value of last

while (first < last) {

int mid = (first+last)/2;

if (target == arr[mid])

return mid; // a match so return mid

else if (target < arr[mid])

last = mid; // search lower sublist

else

first = mid+1; // search upper sublist

}

return origLast; // target not found

}

13

Binary Search Analysis

Each comparison divides the problem size in half

Assume for simplicity that n = 2k

So, worst case … How many times do we divide n in half before

there are no more values to check? T(n) = 1+ k = 1 + lg n Logarithmic Θ(lg n)

14

Merge Sort

Recursively divide vector in half …

… until sub-vectors reach size 1

Merge results into larger sorted sub-vectors while backing out of recursion

15

Mergesort()

// sort v in range [first,last)

template <typename T>

void mergeSort(vector<T>& v, int first, int last)

{

if (first+1 < last) // more than 1 element

{

int mid = (first + last) / 2;

mergesort(v, first, mid); // sort 1st half

mergesort(v, mid, last); // sort 2nd half

merge(v, first, mid, last); // merge halves

}

}

16

Partitioning and Merging of Sublists in mergeSort()

(2 5 1 0 7 1 9 3 4 8 1 2 1 7 5 6 3 0 2 1 )

(2 5 1 0 7 1 9 3 )

(7 1 9 3 )(2 5 1 0 )

[3 7 1 0 1 9 2 5 ]

(4 8 1 2 1 7 5 6 3 0 2 1 )

(5 6 3 0 2 1 )(4 8 1 2 1 7 )

[1 2 1 7 2 1 3 0 4 8 5 6 ]

[3 7 1 0 1 2 1 7 1 9 2 1 2 5 3 0 4 8 5 6 ]

(1 0 )

[1 0 2 5 ]

(2 5 ) (1 9 3 )(7 )

[3 7 1 9 ]

(5 6 )

[2 1 3 0 5 6 ]

(1 2 1 7 )(4 8 )

[1 2 1 7 4 8 ]

(2 1 )(3 )

[3 1 9 ]

(1 9 )

(3 0 2 1 )

[2 1 3 0 ]

(3 0 )(1 7 )

[1 2 1 7 ]

(1 2 )

17

Merge Algorithm

Use a temporary vector Scan both sub-vectors left to right

If value in vector A is smaller Append value to temp vector and advance in vector A

Else value in vector B is smaller Append value to temp vector and advance in vector B

Now one of vectors (A or B) is empty, so copy remaining values from non-empty vector to temp

Now copy all values back from temp to area originally occupied by vectors A and B

18

Merge()template <typename T>void merge(vector<T>& v, int first, int mid, int last) { vector<T> temp; // temporary vector needed int iA = first, iB = mid; while (iA < mid && iB < last) // values in both halves if (v[iA] < v[iB]) // smallest in 1st half temp.push_back(v[iA++]); else // smallest in 2nd half temp.push_back(v[iB++]); while (iA < mid) // still values in 1st half temp.push_back(v[iA++]); while (iB < last) // still values in 2nd half temp.push_back(v[iB++]); iA = first; for (int iV=0; iV<temp.size();iV++) // copy from temp v[iA++] = temp[iV];}

19

Merge Algorithm Example

7 1 0 1 9 2 5 1 2 1 7 2 1 3 0 4 8

s u b lis t Bs u b lis t A

firs t las tm id

The merge algorithm takes a sequence of elements in a vector v having index range [first, last). The sequence consists of two ordered sublists separated by an intermediate index, called mid.

20

Merge Algorithm Example …

7 1 0 1 9 2 5 1 2 1 7 2 1 3 0 4 8

7

s u b lis t Bs u b lis t A

in d exA in d exB

t em p Vect o r

7 1 0

t em p Vect o r

7 1 0 1 9 2 5 1 2 1 7 2 1 3 0 4 8

s u b lis t Bs u b lis t A

in d exA in d exrB

21

Merge Algorithm Example …

7 1 0 1 9 2 5 1 2 1 7 2 1 3 0 4 8

7 1 0 1 2

s u b lis t Bs u b lis t A

in d exA in d exB

t em p Vect o r

7 1 0 1 9 2 5 1 2 1 7 2 1 3 0 4 8

7 1 0 1 2 1 7 1 9 2 1 2 5

s u b lis t Bs u b lis t A

in d exA in d exB

t em p Vect o r

22

Merge Algorithm Example …

7 1 0 1 9 2 5 1 2 1 7 2 1 3 0 4 8

7 1 0 1 2 1 7 1 9 2 1 2 5 3 0 4 8

s u b lis t Bs u b lis t A

in d exA in d exB

t em p Vect o r

las t

7 10 12 17 19 21 25 30 48

t e m p Ve c t o r

7 10 12 17 19 21 25 30 48

firs t la s t

23

Function calls in mergeSort() C all m s o rt () - recu s iv e call1 t o m s o rt () - recu s iv e call2 t o m s o rt () - call1 m erge()

m s o rt () : n /2 m s o rt (): n /2

m s o rt (): n /4

m s o rt (): n /8 m s o rt (): n /8

m s o rt (): n /4

m s o rt (): n /8 m s o rt (): n /8

m s o rt (): n /4

m s o rt (): n /8 m s o rt (): n /8

m so r t (): n /4

m s o rt (): n /8 m s o rt (): n /8

Lev el 0 :

Lev el 3 :

Lev el 2 :

Lev el 1 :

Lev el i:

. . .

24

Mergesort Efficiency (Informal) Assume n = 2k Calls continue until subarrays have 1 element

Levels in call tree (1 + k = 1 + lg n) Elements at each level

Level 0 (1 vector of n elements) Level 1 (2 vectors of n/2 elements) Level 2 (4 vectors of n/4 elements) …. Level k (2k vectors of n/2k elements)

Each level has n elements & merges across entire level requires Θ(n) effort

Mergesort is always logarithmic Θ(n lg n)

25

Mergesort Efficiency (Formal)

For any divide and conquer algorithm … Cost is expressed by the recurrence

a is the number of subproblems n/b is the size of each subproblem D(n) is the cost to divide into subproblems C(n) is the cost to combine the results of the

subproblems

otherwisenCnDbnaT

cnifnT

)()()/(

)1()(

26

Mergesort Efficiency …

Divide: compute middle index – Θ(1) Conquer: solve 2 subproblems of size n/2 Combine: merge results – Θ(n)

Putting it together … … and noting that Θ(n) + Θ(1) is still only Θ(n) we get the specific recurrence …

otherwisennT

cnifnT

)()2/(2

)1()(

27

Mergesort Efficiency

Rewriting the recurrence as …

... and assuming n = 2k

We can create a cost tree by expanding the recurrence at each level

otherwisecnnT

cnifcnT

)2/(2)(

T(n/2)

T(n) cncn

T(n/4)

cn/2T(n/2) cn/2

T(n/4) T(n/4) T(n/4)

28

Mergesort Efficiency

Total cost at each level of the tree is … 2c(n/2), 4c(n/4), 8c(n/8), … or in general … 2ic(n/2i) = cn

At the bottom level there are n subarrays of 1 element each and the cost there is … n*T(1) = cn

Depth of tree for sort of n = 2k elements is … k = lg n, so the tree has … 1 + k = 1 + lg n levels, but only n lg n merges

Therefore, the total cost of the algorithm is … cn lg n + cn, which is logarithmic Θ(n lg n)

29

Asymptotic Growth Notation

Big Theta A function f(n) is Θ(g(n))

if c1g(n) ≤ f(n) ≤ c2g(n), for some c1, c2 and all n ≥ n0

Example: 3n2 + 2n is Θ(n2), Let c1=3 and c2=5. 3n2 ≤ 3n2 + 2n ≤ 5n2 for all n ≥ 1

Essentially f(n) is bounded by scalar multiples of g(n) for

sufficiently large values of n So, g(n) is an asymptotically tight bound for f(n)

30

Asymptotic Growth Notation

Big O A function f(n) is O(g(n))

if f(n) ≤ cg(n), for some c and all n ≥ n0

Example: 3n2 + 2n is O(n2), Let c=5. 3n2 + 2n ≤ 5n2 for all n ≥ 1

Essentially f(n) is upper bounded by a scalar multiple of g(n)

for sufficiently large values of n So, g(n) is an asymptotic upper bound for f(n)

31

Asymptotic Growth Notation

Big Omega A function f(n) is Ω(g(n))

if cg(n) ≤ f(n), for some c and all n ≥ n0

Example: 3n2 + 2n is Ω(n2), Let c=3. 3n2 ≤ 3n2 + 2n for all n ≥ 1

Essentially f(n) is lower bounded by a scalar multiple of g(n) for

sufficiently large values of n So, g(n) is an asymptotic lower bound for f(n)