LEVEL II, TERM IICSE – 243

MD. MONJUR-UL-HASANLECTURER

DEPT OF CSE, CUETEMAIL: MAIL@MONJUR-UL-HASAN.INFO

Sorting

Bubble Sort

bubble1(data, n) {

for j=1 to nfor i=1 to n - j

if data[i-1]>data[i]swap(data,i-1,i);

}

The internal loop runs n-1 times, then n-2 times, then n-3 times and so on…

What is the complexity?

2

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Analysis of Running Time

Let’s assume the swapping procedure along with the if that surrounds it run in at most k time units.

Total running time is then at most:

What is the lower bound?

)(222

)1(

)1(...21

22

1

1

nOn

kn

knn

k

ikknkkn

i

3

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

4

One Pass

Array after

Completion

of Each Pass

Bubble Sort

bubble1(data, n) {

for j=1 to nfor i=1 to n - j

if data[i-1]>data[i]swap(data,i-1,i);

}Question: Is this a stable sort?Question: What happens if we start this on a sorted array?Question: What happens in the following case?

a) 1 4 6 20 10b) 9 8 6 5 4 3

Calculate the number of swap for each case.

5

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Modified Bubble Sort

bubble1(data, n) {

for j=1 to nboolean movedSomething = false; //added thisfor i=1 to n - j

if data[i-1]>data[i] movedSomething = true; //added this

swap(data,i-1,i);if(!movedSomething) break; //added this

}

We detect when the array is sorted and stop in the middle.

6

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Bubble Sort

Again Answer the following

Question: Is this a stable sort?Question: What happens if we start this on a sorted array?Question: What happens in the following case?

a) 1 4 6 20 10b) 9 8 6 5 4 3

Calculate the number of swap for each case.

7

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Modified Bubble Sort

Before the modification, running time was always nearly the same but now what?

What is the best case? What is the worst case?What is the running time in each case?

Which version of bubble sort is better?

8

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Complexity of Modified Bubble Sort.

What is the worst case running time? O(n2) - The array is sorted in reverse – we don’t exit early.

What is the best case running time? Ω(n) – in the best case we start with a sorted array. Go over it once and see that nothing was moved.

9

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Selection Sort

Pseudo Code:

For i=0,1,…,n-2 Run over the array from i onwards.

Find j- the index of the minimal item. Swap the values in indexes i,j.

10

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

11

Running Time of selection sort

What is the worst case?

What is the best case?

Can we say something about the average case?

What is the running time in each one?

12

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Running Time of selection sort

Note first that the running time is always pretty much the same.

Finding the minimal value always requires us to go over the entire sub-array. There is no shortcut, and we never stop early.

First, we have an array of size n-1 to scan, then size n-2, then size n-2… We’ve already analyzed this pattern: Running time is O(n2). (HOW ?)

13

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Selection Sort: Analysis

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

14

Number of comparisons:(n-1) + (n-2) + ... + 3 + 2 + 1

=n * (n-1)/2

=(n² - n )/2

O(n²)

Number of exchanges (worst case):n – 1

O(n)

Overall (worst case) O(n) + O(n²) = O(n²) (‘quadratic sort‘)

Finding the smallest difference

The naïve way:

minimalDif(numbers,length){

diff = MAX_VALUE;for i=0 to length

for j=i+1 to lengthcurrentDiff = abs(numbers[i]-numbers[j]);if(currentDiff<diff)

diff = currentDiff;return diff;

}

Try to find a better algorithm to do this.

Insertion Sorting

insertSort(data, length){for i=1 to length//assume before index i the array is sorted

insert(data,i); //insert i into array 0...(i-1)}

insert(data,index) {value = data[index];I = index-1

while data[i]>value and i>=0data[i+1]=data[i]i=i-1

data[i+1] = value;}

16

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

17

Insertion Sort:

4 passes

Pass 3

Complexity of Insertion Sort

What is the worst case?

What is the best case?

What is the complexity in each one?

18

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Insertion Sort: Analysis

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

19

Number of comparisons (worst case): (n-1) + (n-2) + ... + 3 + 2 + 1 O(n²)

Number of comparisons (best case): n –1 O(n)

Number of exchanges (worst case): (n-1) + (n-2) + ... + 3 + 2 + 1 O(n²)

Number of exchanges (best case): 0 O(1)Overall worst case: O(n²) + O(n²) = O(n²)

Marge Sort

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

20

Divide & Conquer AlgorithmFaster than Babble sort , Insertion sort and

selection sort.Merge Sort Tree:

Take a binary tree T Each node of T represents a recursive call of the merge

sort algorithm. We associate with each node v of T a the set of input

passed to the invocation v represents. The external nodes are associated with individual elements

of S, upon which no recursion is called.

CIS 068

Merge Sort: Example

9 12 19 16 1 25 4 3

9 12 19 16 1 25 4 3

9 12 19 16 1 25 4 3

9 12 16 19 1 3 4 25

1 3 4 9 12 16 19 25

9 12 16 19 1 25 3 4

9 12 19 16 1 25 4 3

split

merge

split

Marge Sort

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

22

Mergesort(a, from, to) {if (from == to)

return;int mid = (from + to) / 2;

merge_sort(a, from, mid);merge_sort(a, mid+1, to);merge(a, from, to);

}

Marge sort (Cont)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

23

Merge(a,from,to){

mid = (from+to)/2h = I = low;j = mid+1while( j<= mid and j<=mid)

if( a[h] <=a[j])b[i] = a[h] // b is an auxiliary arrayh = h+1

else b[i] = a[j]j = j+ 1

i = i +1if(h>j)

for k = j to tob[i] = a[k] i = i+1;

else for k = h to mid

b[i] = a[k]i = i+1

for k= from to toa[k] = b[k]

}

Complexity and Complexity

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

24

The complexity is O(n * Log(n))

The idea is: We need Log(n) merging steps

Each merging step has complexity O(n)

Problem: Merge Sort needs extra memory !

Heap Sort

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

25

Knowledge Required Binary Tree Mathematics Complete Binary Tree Array Representation of Binary Tree

Some Basic A complete Binary tree have n node then it will have

ceil(n/2) leaf and floor(n/2) non leaf If we put all the node in a row level wise then each

non leaf at position I will have left child at 2*i and right child at 2*i+1 position.

If a root have only left child then all subsequent child will be leaf.

Heap

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

26

Heap two types: Max Heap and Min Heap

Heap Sort: Pseudo Code

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

27

Max-Heapify(A,i,heapsize){

l = 2*ir = l+1largest = iif l< heapsize and A[l]>A[i]

largest = lif r < heapsize and a[r]>a[largest]

largest = rif largest ≠ I

exchange a[i]a[largest]Max-Heapify(a,largest,heapsize)

}

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

28

Building a Heap

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

29

Build Heap The above list is not a heap. Let’s see how to build it into a heap

function Build-Max-Heap(a, start, heapsize) {for i = floor(heapsize/2) downto 1

Max-Heapify(A,i,heapsize) }

Building a Heap (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

30

Building a Heap (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

31

Building a Heap (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

32

Building a Heap (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

33

Building a Heap (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

34

Now, the list becomes a heap

Heap Sort: Pseudo Code (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

35

function heapSort(a,length) {heapsize = lengthBuild-Max-Heap(A, heapsize)for i = length downto 2

exchange a[1] a[i]heapsize = heapsize-1Max-Heapify(A,1,heapsize)

}

Heap Sort: Pseudo Code (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

36

Lower Bound for Sorting

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

37

Theorem : any sorting algorithm based on only comparisons takes (N log N) comparisons in the worst case (worse-case input) to sort N elements.

Prove:Suppose we want to sort N distinct elementsHow many possible orderings do we have for

N elements?

Lower Bound for Sorting (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

38

We can have N! possible orderings e.g., the sorted output for a,b,c can be

a b c b a c a c b c a b c b a b c a

Lower Bound for Sorting (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

39

Any comparison-based sorting process can be represented as a binary decision tree. Each node represents a set of possible orderings,

consistent with all the comparisons that have been made

The tree edges are results of the comparisons

Lower Bound for Sorting (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

40

Lower Bound for Sorting (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

41

A different algorithm would have a different decision tree

Decision tree for Insertion Sort on 3 elements:

Lower Bound for Sorting (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

42

The worst-case number of comparisons used by the sorting algorithm is equal to the depth of the deepest leaf The average number of comparisons used is equal to the

average depth of the leaves

A decision tree to sort N elements must have N! leaves a binary tree of depth d has at most 2d leaves the tree must have depth at least log2 (N!)

Therefore, any sorting algorithm based on only comparisons between elements requires at least

log2(N!) comparisons in the worst case.

Lower Bound for Sorting (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

43

Any sorting algorithm based on comparisons between elements requires (N log N) comparisons.

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

44

Linear time sorting

Linear time sorting

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

45

Can we do better (linear time algorithm) if the input has special structure (e.g., uniformly distributed, every numbers can be represented by d digits)?

Yes.

Counting sort, radix sort

Counting Sort

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

46

for i 1 to k ⊳ k : highest number in Ado C[i] 0

for j 1 to ndo C[A[ j]] C[A[ j]] + 1

for i 2 to kdo C[i] C[i] + C[i–1]

for j n downto 1do B[C[A[ j]]] A[ j]

C[A[ j]] C[A[ j]] – 1

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

47

A: 4 1 3 4 3

B:

1 2 3 4 5

C:

1 2 3 4

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

48

for i 1 to kdo C[i] 0

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 0 0 0 0

1 2 3 4

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

49

for j 1 to ndo C[A[ j]] C[A[ j]] + 1

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 0 0 0 1

1 2 3 4

j= 1 A[j] => a[1] => 4C[A[j]] =>C[4]=> 0C[A[j]] +1=> = > 0+1

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

50

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 1 0 0 1

1 2 3 4

for j 1 to ndo C[A[ j]] C[A[ j]] + 1j= 2

A[j] => a[2] => 1C[A[j]] =>C[1]=> 0C[A[j]] +1=> = > 0+1

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

51

for j 1 to ndo C[A[ j]] C[A[ j]] + 1

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 1 0 1 1

1 2 3 4

j= 3 A[j] => a[3] => 3C[A[j]] =>C[3]=> 0C[A[j]] +1=> = > 0+1

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

52

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 1 0 1 2

1 2 3 4

for j 1 to ndo C[A[ j]] C[A[ j]] + 1

j= 4 A[j] => a[4] => 4C[A[j]] =>C[4]=> 1C[A[j]] +1=> = > 1+1

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

53

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 1 0 2 2

1 2 3 4

for j 1 to ndo C[A[ j]] C[A[ j]] + 1

j= 5 A[j] => a[5] => 3C[A[j]] =>C[3]=> 1C[A[j]] +1=> = > 1+1

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

54

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 1 0 2 2

1 2 3 4

C': 1 1 2 2

for i 2 to kdo C[i] C[i] + C[i–1]

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

55

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 1 0 2 2

1 2 3 4

C': 1 1 3 2

for i 2 to kdo C[i] C[i] + C[i–1]

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

56

A: 4 1 3 4 3

B:

1 2 3 4 5

C: 1 0 2 2

1 2 3 4

C': 1 1 3 5

for i 2 to kdo C[i] C[i] + C[i–1]

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

57

A: 4 1 3 4 3

B: 3

1 2 3 4 5

C: 1 1 3 5

1 2 3 4

C': 1 1 2 5

for j n downto 1do B[C[A[ j]]] A[ j]

C[A[ j]] C[A[ j]] – 1

j= n =>j= 5A[j] => a[5] => 3C[A[j]] =>C[3]=> 3B[C[A[j]] ] = > B[3]

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

58

A: 4 1 3 4 3

B: 3 4

1 2 3 4 5

C: 1 1 2 5

1 2 3 4

C': 1 1 2 4

for j n downto 1do B[C[A[ j]]] A[ j]

C[A[ j]] C[A[ j]] – 1

j= n =>j= 4A[j] => a[4] => 4C[A[j]] =>C[4]=> 5B[C[A[j]] ] = > B[5]

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

59

A: 4 1 3 4 3

B: 3 3 4

1 2 3 4 5

C: 1 1 2 4

1 2 3 4

C': 1 1 1 4

for j n downto 1do B[C[A[ j]]] A[ j]

C[A[ j]] C[A[ j]] – 1

j= n =>j= 3A[j] => a[3] => 3C[A[j]] =>C[3]=> 3B[C[A[j]] ] = > B[3]

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

60

A: 4 1 3 4 3

B: 1 3 3 4

1 2 3 4 5

C: 1 1 1 4

1 2 3 4

C': 0 1 1 4

for j n downto 1do B[C[A[ j]]] A[ j]

C[A[ j]] C[A[ j]] – 1

j= n =>j= 2A[j] => a[2] => 1C[A[j]] =>C[1]=> 1B[C[A[j]] ] = > B[1]

Counting Sort (Continue)

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

61

A: 4 1 3 4 3

B: 1 3 3 4 4

1 2 3 4 5

C: 0 1 1 4

1 2 3 4

C': 0 1 1 3

for j n downto 1do B[C[A[ j]]] A[ j]

C[A[ j]] C[A[ j]] – 1

j= n =>j= 1A[j] => a[1] => 4C[A[j]] =>C[4]=> 4B[C[A[j]] ] = > B[4]

Complexity: Counting Sort

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

62

But it needs k Extra memory

(n)

(k)

(n)

(k)

for j 1 to ndo C[A[ j]] C[A[ j]] + 1for i 2 to kdo C[i] C[i] + C[i–1]

for j n downto 1do B[C[A[ j]]] A[ j]

C[A[ j]] C[A[ j]] – 1(n + k)

Complexity: Counting Sort: Stable?

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

63

A: 4 1 3 4 3

B: 1 3 3 4 4