MA 515: Introduction to Algorithms & MA353 : Design and ...
Transcript of MA 515: Introduction to Algorithms & MA353 : Design and ...
![Page 1: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/1.jpg)
MA 515: Introduction to Algorithms &MA353 : Design and Analysis of Algorithms
[3-0-0-6] Lecture 9
http://www.iitg.ernet.in/psm/indexing_ma353/y09/index.html
Partha Sarathi [email protected]
Dept. of Mathematics, IIT Guwahati
Mon 10:00-10:55 Tue 11:00-11:55 Fri 9:00-9:55
Class Room : 2101
![Page 2: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/2.jpg)
Sorting in linear time
Counting sort: No comparisons between elements.
• Input: A[1 . . n], where A[j]∈{1, 2, …, k}.
• Output: B[1 . . n], sorted.
• Auxiliary storage: C[1 . . k].
![Page 3: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/3.jpg)
Counting sort
![Page 4: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/4.jpg)
Counting-sort example
![Page 5: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/5.jpg)
Counting-sort exampleLoop 1
![Page 6: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/6.jpg)
Counting-sort example Loop 2
![Page 7: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/7.jpg)
Counting-sort exampleLoop 2
![Page 8: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/8.jpg)
Counting-sort exampleLoop 2
![Page 9: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/9.jpg)
Counting-sort exampleLoop 2
![Page 10: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/10.jpg)
Counting-sort example Loop 2
![Page 11: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/11.jpg)
Counting-sort exampleLoop 3
![Page 12: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/12.jpg)
Counting-sort exampleLoop 3
![Page 13: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/13.jpg)
Counting-sort example Loop 3
![Page 14: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/14.jpg)
Counting-sort example Loop 4
![Page 15: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/15.jpg)
Counting-sort example Loop 4
![Page 16: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/16.jpg)
Counting-sort example Loop 4
![Page 17: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/17.jpg)
Counting-sort example Loop 4
![Page 18: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/18.jpg)
Counting-sort example Loop 4
![Page 19: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/19.jpg)
Analysis
![Page 20: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/20.jpg)
Running time
If k = O(n), then counting sort takes Θ(n) time.
• But, sorting takes Ω(nlg n) time!
• Where’s the fallacy?
Answer:
Comparison sorting takes Ω(nlg n) time.
• Counting sort is not a comparison sort.
• In fact, not a single comparison between elements occurs!
![Page 21: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/21.jpg)
Stable sorting
• Counting sort is a stable sort: it preserves the input order among equal elements.
• Exercise: What other sorts have this property?
![Page 22: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/22.jpg)
Radix sort
• Origin: Herman Hollerith’s card-sorting machine for the 1890 U.S. Census.
• Digit-by-digit sort.
• Hollerith’s original (bad) idea: sort on most-significant digit first.
• Good idea: Sort on least-significant digit first with auxiliary stable sort.
![Page 23: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/23.jpg)
Operation of radix sort
![Page 24: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/24.jpg)
Operation of radix sort
![Page 25: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/25.jpg)
Pseudocode for radix sort
RADIX-SORT(A, d)
for i ← 1 to d
do use a stable sort to sort array A on digit i.
![Page 26: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/26.jpg)
Correctness of radix sort
Induction on digit position
• Assume that the numbers are sorted by their low-order t – 1 digits.
• Sort on digit t
![Page 27: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/27.jpg)
Correctness of radix sort
Induction on digit position
• Assume that the numbers are sorted by their low-order t – 1 digits.
• Sort on digit t
– Two numbers that differ in digit tare correctly sorted.
![Page 28: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/28.jpg)
Correctness of radix sort
Induction on digit position
• Assume that the numbers are sorted by their low-order t – 1 digits.
• Sort on digit t
– Two numbers that differ in digit tare correctly sorted.
– Two numbers equal in digit tare put in the same order as the input ⇒ correct order.
![Page 29: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/29.jpg)
Analysis of radix sort
• Assume counting sort is the auxiliary stable sort.
• Sort n computer words of b bits each.
• Each word can be viewed as having b/r base-2r
digits.
Example: 32-bit word
r = 8 ⇒ b/r = 4 passes of counting sort on base-28 digits; or r = 16 ⇒ b/r = 2 passes of counting sort on base-216 digits.
![Page 30: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/30.jpg)
Analysis (cont..)
Recall: Counting sort takes Θ(n + k) time to sort nnumbers in the range from 0 to k –1.If each b-bit word is broken into r-bit pieces, each pass of counting sort takes Θ(n + 2r) time. Since there are b/r passes, we have
• Choose r to minimize T(n,b):• Increasing r means fewer passes, but as r >> lg n,
the time grows exponentially.
![Page 31: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/31.jpg)
Choosing r
Minimize T(n,b) by differentiating and setting to 0.
Or, just observe that we don’t want 2r >> n, and there’s no harm asymptotically in choosing r as large as possible subject to this constraint.
Choosing r = lg n implies T(n,b) = Θ(bn/lg n).
• For numbers in the range from 0 to nd–1, we have b = d lg n ⇒ radix sort runs in Θ(dn) time.
![Page 32: MA 515: Introduction to Algorithms & MA353 : Design and ...](https://reader031.fdocuments.us/reader031/viewer/2022012103/616a0c3811a7b741a34e3bca/html5/thumbnails/32.jpg)
Comparison
Is radix sort preferable to a comparison-based sorting algorithm, such as quick-sort ?
• If b = O(lg n), as is often the case, and we choose r ≈ lg n, then radix sort's running time is Θ(n), which appears to be better than quicksort's average-case time of Θ(n lg n).
• Although radix sort may make fewer passes than quicksort over the n keys, each pass of radix sort may take significantly longer.