Sorting and Element Selection
Transcript of Sorting and Element Selection
![Page 1: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/1.jpg)
Sorting and Element SelectionThomas Schwarz, SJ
![Page 2: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/2.jpg)
Permutations• A permutation of the set is a reordering of the
numbers where each number between 1 and n appears exactly once.
{1,2,…, n}
![Page 3: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/3.jpg)
Permutations• How many permutations are there?
• Use recurrence!
• In a permutation of , where is the located?
• There are other numbers.
• This gives us gaps and spots before and after
{1,2,…, n} n
n − 1
n − 2
a1 a2 a3 a4 a5
![Page 4: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/4.jpg)
Permutations• Let be the number of permutations of elements
• This gives us the recurrence
•
• which can be unfolded very simply
•
n! n
n! = n ⋅ (n − 1)!
n! =n
∏i=1
i
![Page 5: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/5.jpg)
PermutationsHow do we determine its asymptotic growth?
Use Logarithms!
n! =n
∏i=1
i
![Page 6: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/6.jpg)
Permutations• Approximation of the factorial
Use
Use an integral!
log n! =n
∑i=1
log(i)
![Page 7: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/7.jpg)
Permutations
0 5 10 15 20i0.0
0.5
1.0
1.5
2.0
2.5
3.0
log(i)
n−1
∑i=1
log(i) < ∫n
1log(x)dx <
n
∑i=1
log i
![Page 8: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/8.jpg)
Permutations
log(n!) =n
∑i=1
log(i)
≈ ∫n
i=1log(x)dx
= [x log x − x]n1
= n log(n) − n + 1
![Page 9: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/9.jpg)
PermutationsTherefore
n! ≈ exp(n log(n) − n − 1)
= exp(log(nn) − n + 1)
= nn ⋅ e−n ⋅ e
= e ⋅ ( ne )
n
![Page 10: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/10.jpg)
PermutationsAn analysis of the error substituting the Riemann sum for an integral gives Stirling’s formula (invented by de Moivre)
2πnn+ 12 e−n ≤ n! ≤ enn+ 1
2 e−n
![Page 11: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/11.jpg)
Sorting by Comparison
• Many sorting algorithms use comparisons
• An algorithm needs to be able to sort with all orders of inputs, i.e. distinguish between arrangements of the input by order
• assuming all elements are different
n!
![Page 12: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/12.jpg)
Sorting by Comparison• Sorting algorithm makes a comparison, then decides on
what to do
• Can be represented as a binary tree
![Page 13: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/13.jpg)
Sorting by Comparisona1 < a2
a1 < a3a2 < a3
a1 < a2 < a3
yes no
yes
a1 < a3
no
a1 < a3 < a2
yes
a3 < a1 < a2
no
a2 < a1 < a3
yes
a2 < a3
no
a2 < a3 < a1
yes
a3 < a2 < a1
no
A fictitious algorithm for sorting three elements as a Decision Tree
![Page 14: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/14.jpg)
Sorting by Comparison• Represent any comparison based algorithm by such a
tree
• Any run of the algorithm represents a path from the root to a leaf node
• Leaf nodes represent an algorithm finishing,
• So they need to have an ordering, i.e. a permutation of the input array
![Page 15: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/15.jpg)
Sorting by Comparison• How many leaves does a tree with leaves have?
• A tree of height has how many leaves?
• Height 0: only root, one leaf
• Height 1: only root plus one or two leaves:
• Height 2: at most two nodes at height one have at most leaves
• Induction: Height has at most leaves
N
h
≤ 2
≤ 22
h 2h
![Page 16: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/16.jpg)
Sorting by Comparison• Relationship between height of decision tree and number of
elements to be sorted:
• Need to have at least leaves:
•
• which implies
•
•
•
n!
2h ≥ n!
h ≥ log2(n!) =1
log(2)log(n!)
≈1
log(2)n log(n) − n + 1
= Θ(n log(n))
![Page 17: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/17.jpg)
Sorting by Comparison• Since the height of the decision tree is the worst time
runtime, we have
• The runtime of a comparison based sorting algorithm is Ω(n log(n))
![Page 18: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/18.jpg)
Linear Time Sorting• Counting sort
• Assume we want to sort numbers in
• Create a dictionary with keys in
• E.g. as an array Int(1:k)
• Walk through the array, updating the count
• Once the count is done, go through the dictionary in order of the keys, emitting as many keys as the count
{1,2,…, k − 1,k}
{1,2,…, k − 1,k}
![Page 19: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/19.jpg)
Linear Time Sorting• Counting sort:
•
• create a counting array:
•
• Walk through the array and calculate counts
•
• Emit keys according to count
• 1 2 2 2 3 3 3 4 4 5 5 7 8 9 10 10 10 12
10 12 443 3 28 9 55 2 10 1 2 710
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13:
1: 1 2: 3 3: 3 4: 2 5: 2 6: 0 7: 1 8: 1 9: 1 10: 3 11: 0 12: 1 13: 0
![Page 20: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/20.jpg)
Linear Time Sorting• If there are elements in the array, then counting sort
uses
• to create and evaluate the counting array
• to update the counting array
• Therefore: counting sort run-time is
n
∼ k
∼ n
Θ(n + k)
![Page 21: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/21.jpg)
Linear Time Sorting• Radix Sort
• Imagine sorting punch cards with by ID in the first columns
![Page 22: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/22.jpg)
Linear Time Sorting• Simple Method:
• Create heaps of cards based on the first digit
• Then recursively sort the heaps
![Page 23: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/23.jpg)
Linear Time Sorting• Better method:
• Sort according to the last digit
• Then use a stable sort to sort after the second-last digit
• Then use a stable sort to sort after the third-last digit
![Page 24: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/24.jpg)
Linear Time Sorting• Stable sort:
• Leave order of elements with the same key during sorting
• Insertion sort, merge sort, bubble sort, counting sort are all stable
• Heap sort, selection sort, shell sort, and quick sort are not
![Page 25: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/25.jpg)
Linear Time Sorting• Radix sort:
• for i in range(length(key), 0, -1): stable_sort on digit i of key
![Page 26: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/26.jpg)
Linear Time Sorting135
242
122
023
220
144
321
221
203
302 135
242
122
023
220
144
321
221
203
302
135
242
122
023
220
144
321
221
203
302
135
242
122
023
220
144
321
221
203
302
![Page 27: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/27.jpg)
Linear Time Sorting• Radix sort correctness
• What would be a loop invariant?
![Page 28: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/28.jpg)
Linear Time Sorting• Assume keys of digits in
• Use counting sort to sort in time
• Radix sort then takes time
n d {0,1,…, r − 1}
Θ(n + r)
Θ(d(n + r))
![Page 29: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/29.jpg)
Linear Time Sorting• Given numbers of bits each
• Assume
• Choose .
• Divide the -bit numbers into “digits” of length
• Thus, each round of radix sort takes time
• There are rounds
• So, radix sort takes time!
n b
b = O(log(n))
r = ⌊log2(n)⌋
b r
Θ(n + 2r)
⌈br
⌉
Θ(br
(n + 2r)) = Θ(br
(n + n)) = Θ(n)
![Page 30: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/30.jpg)
Selection
![Page 31: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/31.jpg)
Selection Problems• Given an unordered array:
• Find the -largest (-smallest) element in an unordered array
• Naïve Solution:
• Sort (usually in time )
• Pick element or of the sorted array
k
Θ(n log n)
n − k k
![Page 32: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/32.jpg)
Selection Problem• Finding the maximum
• Finding the maximum and minimum at the same time
• Finding the th largest element
• Finding the median
k
![Page 33: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/33.jpg)
Maximum• Obvious algorithm:
• comparisonsn − 1
def max(array): result = array[0] for i in range(1, len(array)): if array[i]>result: result = array[i]
![Page 34: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/34.jpg)
Maximum• Toy algorithm:
• Partition array into pairs.
• (There might be an additional element).
• Use one comparison in order to select the largest of each pair (plus the odd one out if exists)
• These form an array of length
• Recursively call the toy algorithm
⌊n/2⌋
⌊n/2⌋ + 1
![Page 35: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/35.jpg)
Maximum• What is the recurrence relation?
![Page 36: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/36.jpg)
Maximum•
• T(2) = 1
• Now use substitution to get an idea of solving the recurrence
T(n) = T(n − ⌊n/2⌋) + ⌊n/2⌋
![Page 37: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/37.jpg)
Maximum• Assume is a power of 2n
![Page 38: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/38.jpg)
Maximum• Recurrence then becomes
•
•
•
•
•
•
T(n) = T(n/2) + n/2, T(2) = 1
= T(n/4) + n/4 + n/2
= T(n/8) + n/8 + n/4 + n/2
…
= T(2) + 2 + 4 + 8 + … + n/8 + n/4 + n/2
= n − 1
![Page 39: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/39.jpg)
Maximum• Now prove by induction for all
•
•
n ∈ ℕ
T(n) = T(n − ⌊n/2⌋) + ⌊n/2⌋
T(2) = 1
![Page 40: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/40.jpg)
Maximum• Induction Hypothesis: if .
•
•
•
•
T(m) = m − 1 m < n
T(n)
= T(n − ⌊n/2⌋) + ⌊n/2⌋
= n − ⌊n/2⌋ − 1 + ⌊n/2⌋
= n − 1
![Page 41: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/41.jpg)
Maximum• In fact:
• Theorem: Finding the maximum of an array of length costs at least comparisons
• Proof: Place all elements into three buckets:
• One for not-looked at
• One for won all comparisons
• One for lost all comparisons
nn − 1
![Page 42: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/42.jpg)
Maximum
• A single comparison can involves 6 cases
• X-X: move two elements from X, one into W, one into L
• X-W: move one element from X into W or move one element from X into W and one from W into L
• X-L: move one element from X into W or one into L
• W-W: move one element from W to L
• W-L: nothing or move one element from W to L
• L-L: nothing
W LX
![Page 43: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/43.jpg)
Maximum• To have finished the algorithm:
• No elements left in X
• Only one element left in W
• Otherwise, can construct counterexample
W LX
![Page 44: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/44.jpg)
Maximum• One left in X: could be the maximum
• Two (or more) left in W:
• Which one is the maximum?
W LX
W LX
![Page 45: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/45.jpg)
Maximum• Each comparison sends at most one element to
• At best, comparisons
L
n − 1
![Page 46: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/46.jpg)
Combined Maximum and Minimum
• Combined Maximum and Minimum
• Naïve algorithm:
• Calculate the max, then the min (can exclude the max)
• comparisonsm − 1 + m − 2 = 2m − 3
![Page 47: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/47.jpg)
Combined Maximum and Minimum
• A better algorithm
• Divide the array into pairs
• Compare the values of each pair
• Place the winner of each pair in one array, the looser of each array in a second array
• (Or use swapping so that the winners are in even position and the losers are in odd positions)
• Now use maximum and minimum on the two sub-arrays
![Page 48: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/48.jpg)
Combined Maximum and Minimum
• Case 1: is even
• There are pairs or comparisons
• Run maximum on even indexed array elements
• This gives us comparisons
• Same for minimum
• Total is comparisons
n
n/2 n/2
n/2 − 1
n/2 + n/2 − 1 + n/2 − 1 =3n2
− 2
compare and swap
compare and swap
compare and swap
compare and swap
compare and swap
compare and swap
compare and swap
compare and swap
compare and swap
compare and swap
compare and swap
compare and swap
![Page 49: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/49.jpg)
Combined Maximum and Minimum
• Case: is odd
• Run algorithm on the first elements
• comparisons
• Then add two comparisons to see whether the last element is either minimum or maximum
• Total of comparisons
n
n − 13n − 3
2− 2
3n − 32
![Page 50: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/50.jpg)
Combined Maximum and Minimum
• Can we do better?
• Use a more sophisticated bin method
• X - not looked at, W - won every comparison, L - lost every comparison, Q - at least one win and at least one loss
W LX Q
![Page 51: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/51.jpg)
Combined Maximum and Minimum
• To be successful, need to move everything out of X and have only one element in W and L
• Otherwise can have a counter-example
W LX Q
![Page 52: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/52.jpg)
Combined Maximum and Minimum
• Just counting the moves is not sufficient
• Example:
• We compare an element with an element
• Possibly:
• And we move both elements to the bucket
• So, possible to move all elements out of into in comparisons and elements out of into
in comparisons
• Only gives moves!
w ∈ W l ∈ L
w < l
Q
n X W ∪ Ln/2 n − 2 W ∪ L Q
n/2 − 1
n − 1
![Page 53: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/53.jpg)
Combined Maximum and Minimum
• Use an adversary argument
• Algorithm can only depend on the knowledge of the previous comparisons when making a decision
• An adversary is allowed to change all values as long as the results of the comparisons stay the same
• If and , then the only thing the algorithm knows is that has won all of its comparisons and has lost all of its comparisons
• Adversary therefore is allowed to change the value of downward
• Adversary guarantees that .
w ∈ W l ∈ Lw l
l
w > l
![Page 54: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/54.jpg)
Combined Maximum and Minimum
• With the help of the adversary who substitutes values when needed
• Potential:
• Calculate net changes for comparisons between buckets
32
|X | + |W | + |L |
![Page 55: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/55.jpg)
Combined Maximum and Minimum
• Compare X with X
• Net change (-2, 1, 1, 0)
• Potential change: 1
![Page 56: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/56.jpg)
Combined Maximum and Minimum
• Compare X with W
• Case 1: Net change (-1,0,1,0)
• Case 2: Net change(-1,0,0,1)
• The adversary can prevent Case 2 by decreasing
• Possible because this is the first time that we look at
• Potential changes by
x ∈ X, w ∈ W, x < w
x ∈ X, w ∈ W, x > w
x
x12
![Page 57: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/57.jpg)
Combined Maximum and Minimum
• Compare with
• similar as before
X L
![Page 58: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/58.jpg)
Combined Maximum and Minimum
• Compare with
• The element in changes to either or
• Net change (-1, 1, 0, 0) or (-1, 0, 1, 0 )
• Potential change
X Q
X W L
12
![Page 59: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/59.jpg)
Combined Maximum and Minimum
• Compare W with W
• One element looses
• Net change (0, -1, 0, 1)
• Potential change 1
![Page 60: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/60.jpg)
Combined Maximum and Minimum
• Compare with
• Adversary guarantees that the element in wins by making all of them bigger
• This works because each element in has only seen wins and that does not change if the elements are made bigger.
• No change
W L
W
W
![Page 61: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/61.jpg)
Combined Maximum and Minimum
• Compare with
• Since the elements in have always won, the adversary can make them larger
• No net change
W Q
W
![Page 62: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/62.jpg)
Combined Maximum and Minimum
• Comparisons with are the same as with
• Comparisons within are useless, but make no changes
L W
Q
![Page 63: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/63.jpg)
Combined Maximum and Minimum
• With the help of the adversary
• Potential changes by at most 1
• Initial Potential:
• Final Potential:
• Need at least comparisons
32
n
23n − 4
2
![Page 64: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/64.jpg)
Selection• Find the th largest element
• Algorithm 1: Use the idea of quicksort
• Find a random pivot and partition around it
• Now use recursion:
• If find the th largest element in
• If , select
• If , find the largest element in
k
k ≤ len(A>p) k A>p
k = len(A>p) + 1 p
k > len(A>p) k− len(A<p) − 1 A<p
A<p A>p
![Page 65: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/65.jpg)
Selection• Worst case behavior:
• Pivot is always the maximum
• Search in array of length one less
• Partitioning an array of length takes time
• Worst time:
•
•
Θ(n)
∼ n + (n − 1) + (n − 2) + … + 2 + 1
=n(n + 1)
2= Θ(n2)
![Page 66: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/66.jpg)
Selection• Expected behavior:
• Let be the expected run-time on input array
• How does the pivot fall in an array?
T(n) n
![Page 67: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/67.jpg)
Selection
• Call either or or are done
• Bad luck assumption:
• its always the one for the larger array
• All positions of the pivot are equally probable
T(k) T(l) = T(n − k − 1)
P
k l
![Page 68: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/68.jpg)
Selection• Gives a recurrence
•
• where is the costs of partitioning
• Now assume that
T(n) ≤ 2n−1
∑i=⌊n/2⌋
1n
T(i) + dn
dn
T(n) ≤ cn
![Page 69: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/69.jpg)
SelectionThen:
T(n) ≤2n
n−1
∑i=⌊n/2⌋
1n
T(i) + dn
≤2cn (
n−1
∑i=1
i −⌊n/2⌋
∑i=1
i) + dn
=2cn ( (n − 1)n
2−
(⌊n/2⌋ − 1)⌊n/2⌋2 ) + dn
≤2cn ( (n − 1)n
2−
(n/2 − 2)(n/2 − 1)2 ) + dn
![Page 70: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/70.jpg)
Selection
≤2cn ( (n − 1)n
2−
(n/2 − 2)(n/2 − 1)2 ) + dn
=2cn ( n2 − n
2−
n2/4 − 3n/2 + 22 ) + dn
=cn ( 3n2
4+
n2
− 2) + dn
= c(3n4
+12
−2n
) + dn
![Page 71: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/71.jpg)
Selection
which is cn if and only if
= c(3n4
+12
−2n
) + dn
= cn − ( cn4
−c2
− dn)≤
![Page 72: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/72.jpg)
Selection
If we assume , then the right side is at most
Thus, if then the previous calculation goes through
cn4
−c2
− dn ≥ 0
⟺ cn ≥ 2c + 4dn
⟺ c ≥ 2c/n + 4d
n ≥ 4c2
+ 4d
c > 8d
![Page 73: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/73.jpg)
Selection• We have shown
• if and
• Make C larger if necessary to obtain
•
• Then: Induction base works and Induction hypothesis works.
• So: expected runtime is linear
• But: we can do better
T(n) < Cn n ≥ 4 C ≥ 8d
T(1) ≤ C, T(2) ≤ 2C, T(3) ≤ 3C, T(4) ≤ 4C
![Page 74: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/74.jpg)
Selection• Linear worst case selection
• Idea: Improve the selection of the pivot!
• Need to take at most linear time for the pivot selection
![Page 75: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/75.jpg)
Selection• Divide the elements of the input array into
groups of five elements and possibly one additional group
• In each group, choose the median (middle element)
• In the last one, you might need to break a tie
• Then select the median of the medians by recurrence
n ⌊n/5⌋
![Page 76: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/76.jpg)
Selection• Show that the median of medians divides the array fairly
well
• Show that adding up the costs, we still are linear
![Page 77: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/77.jpg)
Selection• About half the medians are below the median of medians
• About half the medians are atop of the median of medians
• This allows us to guarantee that a certain number of elements is below and a certain number of elements is above the median of medians
![Page 78: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/78.jpg)
Selection
below
above
A number of elements are below and above the median of medians for sure.
![Page 79: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/79.jpg)
Selection• At least half of the medians are greater or equal than the
median of medians
• At least half of the contributes at least three elements that are larger
• Discard the group that is smaller and the group with the median of median
• The number of elements larger than the median of medians is at least
•
⌈n/5⌉
3 (⌈12
⌈n5
⌉⌉ − 2)
![Page 80: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/80.jpg)
Selection
• larger than the median of
medians
• smaller than the median
of medians
3 (⌈12
⌈n5
⌉⌉ − 2) ≥3n10
− 6
3 (⌈12
⌈n5
⌉⌉ − 2) ≥3n10
− 6
![Page 81: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/81.jpg)
Selection• run time of the algorithm
• Division into groups of five:
• Determination of the medians: because there are groups and we sort them in constant time to get the median
• Determination of the median of median by recurrence
• Partitioning around the median of medians
• Recursive call on at most = elements
T(n)
Θ(n)
Θ(n) Θ(n)
T(⌈n5
⌉)
Θ(n)
n −3n10
− 67n10
+ 6
![Page 82: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/82.jpg)
Selection• Total runtime:
•
• Show that this is linear using induction / substitution
• Again: induction step only needs to work for large enough
T(n) ≤ T(⌈n5
⌉) + T(0.7n + 6) + an
n
![Page 83: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/83.jpg)
Selection
This is at most if and only if .
Since , we assume
so that needs to be larger than .
T(n) ≤ c(n5
+ 1) + c(7n10
+ 6) + an
= 0.9cn + 7c + an
cn 7c + an ≤ 0.1cn
7c + an ≤ 0.1cn ⟺70n
c + 10a ≤ c
n > 140 c 20a
![Page 84: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/84.jpg)
Selection• We also need to make larger than , , ,
• Then we have an induction base on 140 values
• And an induction step that works
• So
c T(1) T(2)/2 …T(140)/140
T(n) ≤ cn
![Page 85: Sorting and Element Selection](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3911a7b741a34e3bd4/html5/thumbnails/85.jpg)
Selection• This algorithm makes no assumptions on the input
• Unless our results on linear sorting