SEQUENCES

SORTING

Sorting

Input: (unsorted) sequence of n elements a1, a2, …, an

Output: the same sequence in increasing order (or in nondecreasing order if there are equal elements)

Sorting

What do we measure ?- Comparisons and swaps whichever dominates - However, comparisons dominate most of the time

Sorting: A Different View

sorting can be seen as a permutation problem: determine a permutation of indices i1, i2,

…, in such that a[i1]<=a[i2]<=…<=a[in] n! possible permutations

A Few Assumptions

we “extend” dictionary ADT to include a sort method sort the keys of the key-element pairs

(k, e) implemented by an array-based

sequence, unless specifically stated

DICTIONARY METHODS

size()

isEmpty()findElement(k)

findAllElements(k)

return no. of items in D

test if D is emptyreturn element of

item with key equal to k, else NO_SUCH_KEY

return an enumeration of all elements with key k

Sorting

Focus on the processMethodology might be useful in

generating solutions to other problems (i.e. brainstorming)

Sorting Algorithms

O(n2): Bubble, Selection, Insertion Sort

O(n log n): Merge, Quick, Heap Sort*O(n)[input is qualified]: Bucket Sort

Bubble Sort

view the sequence to be in a vertical instead of a horizontal position (easier to visualize)

the items are like bubbles in a water tank with weights according to their keys

then, each pass results in the bubble rising to its proper level of weight

Bubble Sort - Pseudocode

Algorithm BubbleSortfor i 0 to n-1

for j 0 to n-iif (s[j].key > s[j+1].key) then

swap (s[j],s[j+1])

Insertion Sort

Repeatedly insert element in its proper place

Algorithm InsertionSort for i 1 to n-1 do

x S[i]insert x in the proper place in S[0]…S[i]

Insertion Sort Example

44 55 12 42 94 18 6 6744 55 12 42 94 18 6 6712 44 55 42 94 18 6 6712 42 44 55 94 18 6 6712 42 44 55 94 18 6 6712 18 42 44 55 94 6 676 12 18 42 44 55 94 676 12 18 42 44 55 67 94

Insertion Sort TC

array implementation: at most O(n) data moves and O(n) comparisons per pass [O(log n) if binary search is used]

linked-list implementation: O(1) data moves but still O(n) comparisons per pass (binary search is not possible)

insertion sort is on the average n*(n-1)/4 or O(n2)

Simple Sorting Algorithms

Bubble SortSelection SortInsertion SortAll have running times of O(n2)

Advanced Sorting Algorithms

Merge - Query

Given two sorted arrays A and B, arrange the elements to a third array C (in sorted order)

A[23,47,81,95]B[7,14,39,55,62,74]

Merge - Operations

Step Comparison Copy1 Compare 23 and 7 Copy 7 from B to C2 Compare 23 and 14 Copy 14 from B to C3 Compare 23 and 39 Copy 23 from A to C4 Compare 39 and 47 Copy 39 from B to C5 Compare 55 and 47 Copy 47 from A to C6 Compare 55 and 81 Copy 55 from B to C7 Compare 62 and 81 Copy 62 from B to C8 Compare 74 and 81 Copy 74 from B to C9 Copy 81 from A to C10 Copy 95 from A to C

Merged List

A[23,47,81,95]B[7,14,39,55,62,74]C[7,14,23,39,47,55,62,74,81,95]

Merge Sort - General

Divide into two parts

Sort left side, Sort right side

Merge left and right side

Merge Sort - Mathematical

Divide divide S into 2 equal sequences S1 and

S2

Recurse recursively sort sequences S1 and S2

Conquer merge sorted sequences S1 and S2 back

into S

Merge Sort Example

6 2 8 5 10 9 12 12 6 5 8 10 9 12 12 6 5 8 10 9 12 12 5 6 8 10 9 12 12 5 6 8 9 10 12 12 5 6 8 9 10 1 122 5 6 8 1 9 10 121 2 5 6 8 9 10 12

Closer Look At Merging

S1: 24-45-63-85S2: 17-31-50-96S:

S1: 24-45-63-85S2: 31-50-96S: 17

S1: 45-63-85S2: 31-50-96S: 17-24

S1: 45-63-85S2: 50-96S: 17-24-31

S1: 63-85S2: 50-96S: 17-24-31-45

S1: 63-85S2: 96S: 17-24-31-45-50

S1: 85S2: 96S: 17-24-31-45-50-

63

S1:S2: 96S: 17-24-31-45-50-63-85

S1:S2:S: 17-24-31-45-50-

63-85-96

Merge Sort TC

t(n) = time to merge sort n elementst(n/2) = time to merge sort n/2 elementst(n) = 2t(n/2) + O(n), a recurrence

relation

==> merge sort is O(n log n) {look at diagram}

Note: merge sort requires extra space

Divide-and-Conquer

Divide if input size < threshold, solve directly else, divide data into 2 or more subsets

Recurse recursively solve subproblems associated

with subsetsConquer

take solutions to subproblems combine into solution to original problem

Advanced Sorting

O(n log n) vs O(n2) : significantFor 10,000 records, n2 = 100,000,000

n log n = 40,000if sorting using merge sort takes 40

seconds, insertion sort will take 28 hrsThere’s another sorting algorithm

(same level) which does not require additional space

Partitioning - Query

Consider this problem, Make an algorithm, that will arrange List L so that all items less than 50 will be to the left of 50 while all items greater than 50 will be at the right of 50 in one pass.List L [85,24,63,45,17,31,96,50].

Partitioning

Create pointer at both ends of the list If a[0]<50, then it is in the right position,

else (wrong position) stop and wait for the right side to find a wrong entry If a[n]>50 then it is in the right position else (wrong position) stop and wait for the right side to find a wrong entry If both wrong entries, swap

Partitioning Example

partitioning around pivot 50

85 24 63 45 17 31 96 5031 24 63 45 17 85 96 5031 24 17 45 63 85 96 5031 24 17 45 50 85 96 63

Quick SortDivide: if S has at least 2 elements,

select x from S called the pivot. Put elements of S into 3 subsequences: L, all elements in S less than x E, all elements in S equal to x G, all elements in S greater than x

Recurse: recursively sort L and GConquer: Put back elements into S in

order by concatenating L, E, and G

Quick Sort Example

new pivot is selected each line

85 24 63 45 17 31 96 5031 24 17 45 50 85 96 6331 24 17 45 50 85 96 6317 24 31 45 50 85 96 6317 24 31 45 50 85 96 6317 24 31 45 50 63 96 8517 24 31 45 50 63 85 96

Quick Sort

Pivot can be selected at random Usually, it’s the rightmost or the

leftmost entry

On the average, it will be in the middle portion of the list

Quick Sort TC

time complexity depends on pivot choice

average O(n log n)worst O(n2) - data is skewed, reverse

order

O(n log n) Algorithms

Algorithm Time Space

Merge O(n log n) Not in-place

Quick O(n log n) ave.O(n2) worst

in-place

Heap* O(n log n) In-place

Bucket Sort

What if you had 10,000 unsorted records and each record had a value between 1 to 10 which you had to arrange in increasing order (1,1,1,1,2,2,2,etc.)?

Can we beat O(n log n) ?

Bucket Sortinput restriction: sequence S of n

items with keys in the range [0,N-1]

can sort S in O(n+N) timeif N is O(n), then sorting time is

O(n)note: NOT based on comparison

Pseudocode

Algorithm BucketSortInput: Sequence S with integer keys in range [0,N-1]Output: S sortedlet B=array of N initially empty sequencesfor each item x in S do

k=key of xremove x from S and insert at end of sequence B[k]

for i 0 to N-1 dofor each item x in sequence B[i] do

remove x from B[i] and insert at end of S

Other Sorting Algorithms

Shell SortImprovement of Insertion Sort-Sorts widely spaced elements (i.e. every 4th)-Diminishing gaps (364,121,40,1)- Insertion sort the last pass- Running time (estimated) between O(n7/6) to O(n3/2){more realistic)

Sorting - Summary

Looked at various sorting algorithmsAnalyzed their running timeComparisons and swaps/copiesImplementation described in the

book Review recursion*