1 TCSS 342, Winter 2005 Lecture Notes Sorting Weiss Ch. 8, pp. 283-321.
-
date post
20-Dec-2015 -
Category
Documents
-
view
216 -
download
2
Transcript of 1 TCSS 342, Winter 2005 Lecture Notes Sorting Weiss Ch. 8, pp. 283-321.
2
Sorting
Sorting = putting objects in order in array/list
one of the fundamental problems in computer science
comparison-based sorting: must determine order through comparison operations on the input data:
<, >, compareTo, …
15 2 8 1 17 10 12 5
0 1 2 3 4 5 6 7
3
1. Bogo sort
Bogo sort orders a list of values by repetitively shuffling them and checking if they are sorted
More specifically: scan the list, seeing if it is sorted if not, shuffle the values in the list and repeat
This sorting algorithm has terrible performance! Can we deduce its runtime?
4
Bogo sort codepublic static void bogoSort(int[] a) { while (!isSorted(a)) shuffle(a);}
// Returns true if array a's elements// are in sorted order.public static boolean isSorted(int[] a) { for (int i = 0; i < a.length - 1; i++) if (a[i] > a[i+1]) return false; return true;}
5
Bogo sort code, helpers// Shuffles an array of ints by randomly swapping each// element with an element ahead of it in the array.public static void shuffle(int[] a) { for (int i = 0; i < a.length - 1; i++) { // pick random number in [i+1, a.length-1] inclusive int range = a.length-1 - (i+1) + 1; int j = (int)(Math.random()*range + (i+1)); swap(a, i, j); }}
// Swaps a[i] with a[j].private static void swap(int[] a, int i, int j) { if (i == j) return;
int temp = a[i]; a[i] = a[j]; a[j] = temp;}
6
Bogo sort runtime How long should we expect bogo sort to take?
related to probability of shuffling into sorted order assuming shuffling code is fair, probability equals
1 / (number of permutations of n elements)
bogo sort takes roughly factorial time to run note that if array is initially sorted, bogo finishes
quickly! it should be clear that this is not satisfactory...
!nPnn
7
2. Bubble sort
Bubble sort orders a list of values by repetitively comparing neighboring elements and swapping their positions if necessary
More specifically: scan the list, exchanging adjacent elements if
they are not in relative order; this bubbles the highest value to the top
scan the list again, bubbling up the second highest value
repeat until all elements have been placed in their proper order
8
Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using
pair-wise comparisons and swapping
512354277 101
1 2 3 4 5 6
Swap42 77
"Bubbling" largest element
9
"Bubbling" largest element
Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using
pair-wise comparisons and swapping
512357742 101
1 2 3 4 5 6
Swap35 77
10
"Bubbling" largest element
Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using
pair-wise comparisons and swapping
512773542 101
1 2 3 4 5 6
Swap12 77
11
"Bubbling" largest element
Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using
pair-wise comparisons and swapping
577123542 101
1 2 3 4 5 6
No need to swap
12
"Bubbling" largest element
Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using
pair-wise comparisons and swapping
577123542 101
1 2 3 4 5 6
Swap5 101
13
"Bubbling" largest element
Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using
pair-wise comparisons and swapping
77123542 5
1 2 3 4 5 6
101
Largest value correctly placed
14
Bubble sort codepublic static void bubbleSort(int[] a) { for (int i = 0; i < a.length; i++) for (int j = 1; j < a.length - i; j++) if (a[j-1] > a[j]) swap(a, j-1, j);}
private static void swap(int[] a, int i, int j) { if (i == j) return;
int temp = a[i]; a[i] = a[j]; a[j] = temp;}
15
Bubble sort runtime Running time (# comparisons) for input size
n:
number of actual swaps performed depends on the data; out-of-order data performs many swaps
)(O
2
)1(
)(1
2
1
0
1
0
1
0 1
n
nn
i
in
n
i
n
i
n
i
in
j
16
3. Selection sort
Selection sort orders a list of values by repetitively putting a particular value into its final position
More specifically: find the smallest value in the list switch it with the value in the first position find the next smallest value in the list switch it with the value in the second position repeat until all values are in their proper
places
18
Index 0 1 2 3 4 5 6 7
Value 27 63 1 72 64 58 14 9
1st pass 1 63 27 72 64 58 14 9
2nd pass 1 9 27 72 64 58 14 63
3rd pass 1 9 14 72 64 58 27 63
…
Selection sort example 2
19
Selection sort codepublic static void selectionSort(int[] a) { for (int i = 0; i < a.length; i++) { // find index of smallest element int min = i; for (int j = i + 1; j < a.length; j++) if (a[j] < a[min]) min = j; // O(1)
// swap smallest element with a[i] swap(a, i, min); // O(1) }}
20
Selection sort runtime
Running time for input size n: in practice, a bit faster than bubble sort.
Why?
)(O
2
)1(
)(
)1)1((1
2
1
0
1
0
1
0
1
0 1
n
nn
i
in
in
n
i
n
i
n
i
n
i
n
ij
21
4. Insertion sort Insertion sort orders a list of values by
repetitively inserting a particular value into a sorted subset of the list
More specifically: consider the first item to be a sorted sublist
of length 1 insert the second item into the sorted sublist,
shifting the first item if needed insert the third item into the sorted sublist,
shifting the other items as needed repeat until all values have been inserted into
their proper positions
22
Insertion sort
Simple sorting algorithm. n-1 passes over the array At the end of pass i, the elements that
occupied A[0]…A[i] originally are still in those spots and in sorted order.
2 8 15 1 17 10 12 5
0 1 2 3 4 5 6 7
1 2 8 15 17 10 12 5
0 1 2 3 4 5 6 7
afterpass 2
afterpass 3
28
Insertion sort codepublic static void insertionSort(int[] a) { for (int i = 1; i < a.length; i++) { int temp = a[i];
// slide elements down to make room for a[i] int j; for (j = i; j > 0 && a[j - 1] > temp; j--) a[j] = a[j - 1];
a[j] = temp; }}
29
Insertion sort runtime worst case: reverse-ordered elements in array.
best case: array is in sorted ascending order.
average case: each element is about halfway in order.
)(
2
)1()1(...321
2
1
1
nO
nnni
n
i
)(111
1
nOnn
i
)(
4
)1())1(...321(
2
1
22
1
1
nO
nnn
in
i
30
Comparing sorts We've seen "simple" sorting algos. so far, such
as: selection sort insertion sort
They all use nested loops and perform approximately n2 comparisons
They are relatively inefficient
comparisons swaps
selection n2/2 n
insertionworst: n2/2best: n
worst: n2/2best: n
31
Average case analysis
Given an array A of elements, an inversion is an ordered pair (i, j) such that i < j, but A[i] > A[j]. (out of order elements)
Assume no duplicate elements. Theorem: The average number of
inversions in an array of n distinct elements is n (n - 1) / 4.
Corollary: Any algorithm that sorts by exchanging adjacent elements requires O(n2) time on average.
32
Merge sort Merge sort orders a list of values by
recursively dividing the list in half until each sub-list has one element, then recombining
More specifically: divide the list into two roughly equal parts recursively divide each part in half, continuing
until a part contains only one element merge the two parts into one sorted list continue to merge parts as the recursion unfolds
a "divide and conquer" algorithm
33
Merge sort idea: Divide the array into two halves. Recursively sort the two halves (using merge sort). Use merge to combine the two arrays.
sort sortmerge(0, n/2, n-1)
mergeSort(0, n/2-1) mergeSort(n/2, n-1)
Merge sort
51
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676
23 98 4514
14 23 45 98
52
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676
Merge
23 98 4514
14 23 45 98
53
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676
6
Merge
23 98 4514
14 23 45 98
54
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676
67
Merge
23 98 4514 6
14 23 45 98
55
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
23 98 4514 676
14 23 45 98
56
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676
14 23 45 98
57
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
3323 98 4514 676
14 23 45 98
58
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
4223 98 4514 676 33
14 23 45 98
59
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 23 45 98
60
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 6 4233
14 23 45 98 6
67
61
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 6 33
14 23 45 98 6 33
67 42
62
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 6 4233
14 23 45 98 6 33 42
67
63
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 23 45 98 6 33 42 67
64
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
23 45 98 33 42 6714 6
65
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
23 45 98 6 42 67
6
14 33
66
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 45 98 6 42 67
6 14
23 33
67
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 23 98 6 42 67
6 14 23
45 33
68
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 23 98 6 33 67
6 14 23 33
45 42
69
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 23 98 6 33 42
6 14 23 33 42
45 67
70
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 23 45 6 33 42
6 14 23 33 42 45
98 67
71
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67
72
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
Merge
23 98 4514 676 4233
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67 98
73
674523 14 6 3398 42
674523 14 6 3398 42
4523 1498
2398 45 14
676 33 42
676 33 42
23 98 4514 676 4233
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67 98
75
13 6 21 18 9 4 8 20
0 7
4 6 8 9 13 18 20 21
0 7
13 6 21 18
0 3
9 4 8 20
4 7
6 13 18 21
0 3
4 8 9 20
4 7
13 6
0 1
21 18
2 3
9 4
4 5
8 20
6 7
6 13
0 1
18 21
2 3
4 9
4 5
8 20
6 7
13
0
6
1
21
2
18
3
9
4
4
5
8
6
20
7
Merge sort illustrated
76
merge operation: Given two sorted arrays, merge operation produces
a sorted array with all the elements of the two arrays
A 6 13 18 21 B 4 8 9 20
C 4 6 8 9 13 18 20 21
Running time of merge: O(n), where n is the number of elements in the merged array.
Merging two sorted arrays
77
Merge sort codepublic static void mergeSort(int[] a) { int[] temp = new int[a.length]; mergeSort(a, temp, 0, a.length - 1);}
private static void mergeSort(int[] a, int[] tmp, int left, int right) { if (left >= right) return;
// sort the two halves int mid = (left + right) / 2; mergeSort(a, tmp, left, mid); mergeSort(a, tmp, mid + 1, right);
// merge the sorted halves into a sorted whole merge(a, tmp, left, mid, right);}
78
Merge codeprivate static void merge(int[] a, int[] temp, int left, int leftEnd, int rightEnd) { int count = rightEnd - left + 1; int right = leftEnd + 1; // main loop to copy the halves into the temp array for (int i = 0, l = left, r = right; i < count; i++) if (r > rightEnd) temp[i] = a[l++]; else if (l > leftEnd) temp[i] = a[r++]; else if (a[l] < a[r]) temp[i] = a[l++]; else temp[i] = a[r++];
// copy sorted temp array back into main array for (int i = 0; i < count; i++) a[left + i] = temp[i];}
79
Merge sort runtime
let T(n) be runtime of merge sort on n items T(0) = 1 T(1) = 2*T(0) + 1 T(2) = 2*T(1) + 2 T(4) = 2*T(2) + 4 T(8) = 2*T(4) + 8 ... T(n/2) = 2*T(n/4) + n/2 T(n) = 2*T(n/2) + n
80
Merge sort runtime T(n) = 2*T(n/2) + n T(n/2) = 2*T(n/4) + n/2
T(n) = 2*(2*T(n/4) + n/2) + n T(n) = 4*T(n/4) + 2n T(n) = 8*T(n/8) + 3n ... T(n) = 2k T(n/2k) + kn
To get to a base case, let's set k = log2 n. T(n) = 2log n T(n/2log n) + (log n) n T(n) = n * T(n/n) + n log n T(n) = n * T(1) + n log n T(n) = n * 1 + n log n T(n) = n + n log n T(n) = O(n log n)
81
Quick sort
Quick sort orders a list of values by partitioning the list around one element called a pivot, then sorting each partition
More specifically: choose one element in the list to be the pivot
(= partition element) organize the elements so that all elements
less than the pivot are to the left and all greater are to the right
apply the quick sort algorithm (recursively) to both partitions
82
Quick sort, continued
For correctness, can choose any pivot. For efficiency, one of following is best
case, the other worst case: pivot partitions the list roughly in half pivot is greatest or least element in list
Which case above is best? We will divide the work into two methods:
quickSort – performs the recursive algorithm partition – rearranges the elements into
two partitions
83
Quick sort pseudo-code
Let S be the input set.1. If |S| = 0 or |S| = 1, then return.2. Pick an element v in S. Call v the pivot.3. Partition S – {v} into two disjoint groups:
• S1 = {x S – {v} | x v}
• S2 = {x S – {v} | x v}
4. Return { quicksort(S1), v, quicksort(S2) }
84
40
10
1832
235
37
176
12
pick a pivot
610 12 2
1718
40 3732 35
partition
quicksort quicksort
1810 12 1762 40373532
combine
1810 12 1762 40373532
Quick sort illustrated
85
How to choose a pivot first element
bad if input is sorted or in reverse sorted order bad if input is nearly sorted variation: particular element (e.g. middle
element)
random element even a malicious agent cannot arrange a bad
input
median of three elements choose the median of the left, right, and center
elements
86
Partitioning algorithm
Basic idea:1. Move the pivot to the rightmost position.2. Starting from the left, find an element
pivot. Call the position i.3. Starting from the right, find an element
pivot. Call the position j.4. Swap S[i] and S[j].
8 1 4 9 0 3 5 2 7 6
0 9
87
Quick sort codeprivate static void quickSort(int[] a, int min, int max) { if (min >= max) // base case; no need to sort return;
// choose pivot (we'll use the middle element) int mid = (min + max) / 2; int pivot = a[mid]; swap(a, mid, max); // move pivot to end
// partition the two sides of the array int i = partition(a, min, max - 1, pivot);
// restore the pivot to its proper location swap(a, max, i);
// recursively sort the left and right partitions quickSort(a, min, i - 1); quickSort(a, i + 1, max);}
88
Quick sort code, cont'd.// partitions a with elements < pivot on left and// elements > pivot on rightprivate static int partition(int[] a, int i, int j, int pivot) {
i--; j++; // kludge because the loops pre-increment while (true) { // move index markers i,j toward center // until we find a pair of mis-partitioned elements do { i++; } while (i < j && a[i] < pivot); do { j--; } while (i < j && a[j] > pivot);
if (i >= j) break;
swap(a, i, j); }
return i;}
89
9 17 3 12 8 7 21 1
0 7
1 17 3 12 8 7 21 9
0 7
1 7 3 12 8 17 21 9
0 7
1 7 3 8 12 17 21 9
0 7
1 7 3 8 9 17 21 12
0 7
pick pivot
i
i
j
j
j i swap S[i]with S[right]
"Median of three" pivot
90
Special cases
What happens when the array contains many duplicate elements?
What happens when the array is already sorted (or nearly sorted) to begin with?
Small arrays Quicksort is slower than insertion sort when is
N is small (say, N 20). Optimization: Make |A| 20 the base case
and call insertion sort.
91
Quick sort runtime Worst case: pivot is the smallest (or largest)
element all the time.T(n) = T(n-1) + cnT(n-1) = T(n-2) + c(n-1)T(n-2) = T(n-3) + c(n-2)…T(2) = T(1) + 2c
Best case: pivot is the medianT(n) = 2 T(n/2) + cn
T(n) = cn log n + n = O(n log n)
)O(N i c T(1)T(N) 2N
2i
92
Quick sort runtime, cont'd.
Assume each of the sizes for S1 are equally likely. 0 |S1| N-1.
22N
0i
21N
0i
1N
0i
1N
0i
1)-c(N T(i)21)T(N 1)(N
cN T(i)2 T(N) N
cNT(i)N
2
cN 1)]-i-T(N T(i)[N
1 T(N)
93
2cN 1)T(N 1)(NT(N) N
1N
2c
N
1)T(N
1N
T(N)
N
2c
1-N
2)T(N
N
1)-T(N
1N
2c
2-N
3)T(N
1N
2)-T(N
…
3
2c
2
)1T(
3
T(2)
1N
3i i
12c
2
T(1)
1N
T(N)
N) log O(NT(N) loge (N+1) – 3/2
divideequationby N(N+1)
Quick sort runtime, cont'd.
94
Quick sort runtime summary
O(n log n) on average. O(n2) worst case.
comparisons
merge O(n log n)
quickaverage: O(n log n)worst: O(n2)
95
Selection problem, quickSelect
Recall: the Selection problem Find the kth smallest element in an array a
quickSelect(a, k):1. If a.length = 1, then k=1 and return the element.2. Pick a pivot v a.3. Partition a – {v} into a1 (left side) and a2 (right side).
• if k a1.length, then the kth smallest element must be in a1. So return quickSelect(a1, k).
• else if k = 1 + a1.length, return the pivot v.
• Otherwise, the kth smallest element is in a2. Return quickSelect(a2, k - a1.length - 1).