mergesort counting inversions closest pair of points ... Divide problem of size n into two...

Lecture slides by Kevin WayneCopyright © 2005 Pearson-Addison Wesley

http://www.cs.princeton.edu/~wayne/kleinberg-tardos

Last updated on 3/12/18 9:21 AM

5. DIVIDE AND CONQUER I

‣ mergesort

‣ counting inversions

‣ randomized quicksort

‣ median and selection

‣ closest pair of points

Divide-and-conquer paradigm

Divide-and-conquer.

独Divide up problem into several subproblems (of the same kind).

独Solve (conquer) each subproblem recursively.

独Combine solutions to subproblems into overall solution.

Most common usage.

独Divide problem of size n into two subproblems of size n / 2.

独Solve (conquer) two subproblems recursively.

独Combine two solutions into overall solution.

Consequence.

独Brute force: Θ(n2).

独Divide-and-conquer: O(n log n).

2attributed to Julius Caesar

O(n) time

O(n) time

5. DIVIDE AND CONQUER

‣ mergesort





SECTIONS 5.1–5.2

Sorting problem

Problem. Given a list L of n elements from a totally ordered universe,

rearrange them in ascending order.

4

Sorting applications

Obvious applications.

独Organize an MP3 library.

独Display Google PageRank results.

独List RSS news items in reverse chronological order.

Some problems become easier once elements are sorted.

独Identify statistical outliers.

独Binary search in a database.

独Remove duplicates in a mailing list.

Non-obvious applications.

独Convex hull.

独Closest pair of points.

独Interval scheduling / interval partitioning.

独Scheduling to minimize maximum lateness.

独Minimum spanning trees (Kruskal’s algorithm).

独...5

Mergesort

独Recursively sort left half.

独Recursively sort right half.

独Merge two halves to make sorted whole.

6

A G H I L M O R S T

merge results

A L G O R I T H M S

input

I T H M SA G L O R

sort left half

H I M S T

sort right half

A G L O R

Merging

Goal. Combine two sorted lists A and B into a sorted whole C.

独Scan A and B from left to right.

独Compare ai and bj.

独If ai ≤ bj, append ai to C (no larger than any remaining element in B).

独If ai > bj, append bj to C (smaller than every remaining element in A).

7

sorted list A

2 3 7 10 11

merge to form sorted list C

2 11 bj 20 233 7 10 ai 18

sorted list B

Mergesort implementation

Input. List L of n elements from a totally ordered universe.

Output. The n elements in ascending order.

8

MERGE-SORT(L) _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IF (list L has one element)

RETURN L.

Divide the list into two halves A and B.

A ← MERGE-SORT(A).

B ← MERGE-SORT(B).

L ← MERGE(A, B).

RETURN L._________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

T (n / 2)

T (n / 2)

Θ(n)

A useful recurrence relation

Def. T (n) = max number of compares to mergesort a list of length n.

Recurrence.

Solution. T (n) is O(n log2 n). Assorted proofs. We describe several ways to solve this recurrence. Initially we assume n is a power of 2 and replace ≤ with = in the recurrence.

9

T (n) �

��

�0 B7 n = 1

T (�n/2�) + T (�n/2�) + n B7 n > 1<latexit sha1_base64="G9BiiiqVnrc2NLAK0QbOMus1fE8=">AAACy3icbVFdb9MwFHUyPkr56sYjL1d0oCKkLJmQ2mkCTeKFJzSklk2qo8pxbjprjhPZDmoJfeSP8K/4NzhZJtGO++Kjc67P/UpKKYwNwz+ev3fv/oOHvUf9x0+ePns+2D/4ZopKc5zxQhb6MmEGpVA4s8JKvCw1sjyReJFcf2r0i++ojSjU1K5LjHO2VCITnFlHLQa/pyP1FugpUInN06cJLoWqufM0mz49DeENUIsrW4PI4FDBB4gOYQOUzsNgjHnscqYjKjNZFBrU0TFQ3eLG9Z1zbESOQnZaA28ltev98da7T1GlXROLwTAMwjbgLog6MCRdnC/2vQOaFrzKUVkumTHzKCxtXDNtBZfopqoMloxfsyXOHVQsRxPX7TI38NoxKWRumKxQFlr23x81y41Z54nLzJm9MrtaQ/5Pm1c2m8S1UGVlUfGbQlklwRbQXAZSoZFbuXaAcS1cr8CvmGbcuvttVWm9S+Rbk9SrSglepLjDSruymjVbjHZ3dhfMjoOTIPr6fng26dbZIy/JKzIiERmTM/KZnJMZ4V7PC7yxN/G/+Nb/4f+8SfW97s8LshX+r7/FTNWb</latexit><latexit sha1_base64="G9BiiiqVnrc2NLAK0QbOMus1fE8=">AAACy3icbVFdb9MwFHUyPkr56sYjL1d0oCKkLJmQ2mkCTeKFJzSklk2qo8pxbjprjhPZDmoJfeSP8K/4NzhZJtGO++Kjc67P/UpKKYwNwz+ev3fv/oOHvUf9x0+ePns+2D/4ZopKc5zxQhb6MmEGpVA4s8JKvCw1sjyReJFcf2r0i++ojSjU1K5LjHO2VCITnFlHLQa/pyP1FugpUInN06cJLoWqufM0mz49DeENUIsrW4PI4FDBB4gOYQOUzsNgjHnscqYjKjNZFBrU0TFQ3eLG9Z1zbESOQnZaA28ltev98da7T1GlXROLwTAMwjbgLog6MCRdnC/2vQOaFrzKUVkumTHzKCxtXDNtBZfopqoMloxfsyXOHVQsRxPX7TI38NoxKWRumKxQFlr23x81y41Z54nLzJm9MrtaQ/5Pm1c2m8S1UGVlUfGbQlklwRbQXAZSoZFbuXaAcS1cr8CvmGbcuvttVWm9S+Rbk9SrSglepLjDSruymjVbjHZ3dhfMjoOTIPr6fng26dbZIy/JKzIiERmTM/KZnJMZ4V7PC7yxN/G/+Nb/4f+8SfW97s8LshX+r7/FTNWb</latexit><latexit sha1_base64="G9BiiiqVnrc2NLAK0QbOMus1fE8=">AAACy3icbVFdb9MwFHUyPkr56sYjL1d0oCKkLJmQ2mkCTeKFJzSklk2qo8pxbjprjhPZDmoJfeSP8K/4NzhZJtGO++Kjc67P/UpKKYwNwz+ev3fv/oOHvUf9x0+ePns+2D/4ZopKc5zxQhb6MmEGpVA4s8JKvCw1sjyReJFcf2r0i++ojSjU1K5LjHO2VCITnFlHLQa/pyP1FugpUInN06cJLoWqufM0mz49DeENUIsrW4PI4FDBB4gOYQOUzsNgjHnscqYjKjNZFBrU0TFQ3eLG9Z1zbESOQnZaA28ltev98da7T1GlXROLwTAMwjbgLog6MCRdnC/2vQOaFrzKUVkumTHzKCxtXDNtBZfopqoMloxfsyXOHVQsRxPX7TI38NoxKWRumKxQFlr23x81y41Z54nLzJm9MrtaQ/5Pm1c2m8S1UGVlUfGbQlklwRbQXAZSoZFbuXaAcS1cr8CvmGbcuvttVWm9S+Rbk9SrSglepLjDSruymjVbjHZ3dhfMjoOTIPr6fng26dbZIy/JKzIiERmTM/KZnJMZ4V7PC7yxN/G/+Nb/4f+8SfW97s8LshX+r7/FTNWb</latexit><latexit sha1_base64="G9BiiiqVnrc2NLAK0QbOMus1fE8=">AAACy3icbVFdb9MwFHUyPkr56sYjL1d0oCKkLJmQ2mkCTeKFJzSklk2qo8pxbjprjhPZDmoJfeSP8K/4NzhZJtGO++Kjc67P/UpKKYwNwz+ev3fv/oOHvUf9x0+ePns+2D/4ZopKc5zxQhb6MmEGpVA4s8JKvCw1sjyReJFcf2r0i++ojSjU1K5LjHO2VCITnFlHLQa/pyP1FugpUInN06cJLoWqufM0mz49DeENUIsrW4PI4FDBB4gOYQOUzsNgjHnscqYjKjNZFBrU0TFQ3eLG9Z1zbESOQnZaA28ltev98da7T1GlXROLwTAMwjbgLog6MCRdnC/2vQOaFrzKUVkumTHzKCxtXDNtBZfopqoMloxfsyXOHVQsRxPX7TI38NoxKWRumKxQFlr23x81y41Z54nLzJm9MrtaQ/5Pm1c2m8S1UGVlUfGbQlklwRbQXAZSoZFbuXaAcS1cr8CvmGbcuvttVWm9S+Rbk9SrSglepLjDSruymjVbjHZ3dhfMjoOTIPr6fng26dbZIy/JKzIiERmTM/KZnJMZ4V7PC7yxN/G/+Nb/4f+8SfW97s8LshX+r7/FTNWb</latexit>

between ⎣n / 2⎦ and n − 1 compares

Divide-and-conquer recurrence: recursion tree

Proposition. If T (n) satisfies the following recurrence, then T (n) = n log2 n.

10

log 2 n

T(n) = n log2 n

n = n

2 (n/2) = n

8 (n/8) = n

⋮

T (n)

4 (n/4) = n

T (n / 2) T (n / 2)

T (n / 8) T (n / 8)T (n / 8) T (n / 8) T (n / 8) T (n / 8)T (n / 8) T (n / 8)

T (n / 4) T (n / 4) T (n / 4) T (n / 4)

⋮

assuming n is a power of 2T (n) =

��

�0 B7 n = 1

2T (n/2) + n B7 n > 1<latexit sha1_base64="Z3k0jP42WP//6lnqi9e2pn3k2fw=">AAACo3icbVFNa9tAEF0pbZq4H3HSYy5DnYaUFkcyhaSElEAvPfSQFrsJWMKsViNnyWoldkfBrvAP7S0/pStHhdrpwC6PNzNvZt8mpZKWguC35288ebr5bGu78/zFy1c73d29n7aojMCRKFRhrhNuUUmNI5Kk8Lo0yPNE4VVy+6XJX92hsbLQQ5qXGOd8qmUmBSdHTbq/hkf6HURncN5cnSjBqdS1cIp20YnOAjiEiHBGNcgMDrQrCw9gAVE0DvonmMeuZgDRB3Ayx4NG6L2T0etdn/92dSLUaSs/6faCfrAMeAzCFvRYG5eTXW8vSgtR5ahJKG7tOAxKimtuSAqFbt/KYsnFLZ/i2EHNc7RxvTRpAW8dk0JWGHc0wZL9t6PmubXzPHGVOacbu55ryP/lxhVlp3EtdVkRavEwKKsUUAGN45BKg4LU3AEujHS7grjhhgty/7IyZaldolh5ST2rtBRFimusohkZ3rgYrnv2GIwG/U/98PvH3sVpa+cW22dv2BEL2Qm7YF/ZJRsxwe69TW/H6/qH/jf/hz98KPW9tuc1Wwk//gPFucbd</latexit><latexit sha1_base64="Z3k0jP42WP//6lnqi9e2pn3k2fw=">AAACo3icbVFNa9tAEF0pbZq4H3HSYy5DnYaUFkcyhaSElEAvPfSQFrsJWMKsViNnyWoldkfBrvAP7S0/pStHhdrpwC6PNzNvZt8mpZKWguC35288ebr5bGu78/zFy1c73d29n7aojMCRKFRhrhNuUUmNI5Kk8Lo0yPNE4VVy+6XJX92hsbLQQ5qXGOd8qmUmBSdHTbq/hkf6HURncN5cnSjBqdS1cIp20YnOAjiEiHBGNcgMDrQrCw9gAVE0DvonmMeuZgDRB3Ayx4NG6L2T0etdn/92dSLUaSs/6faCfrAMeAzCFvRYG5eTXW8vSgtR5ahJKG7tOAxKimtuSAqFbt/KYsnFLZ/i2EHNc7RxvTRpAW8dk0JWGHc0wZL9t6PmubXzPHGVOacbu55ryP/lxhVlp3EtdVkRavEwKKsUUAGN45BKg4LU3AEujHS7grjhhgty/7IyZaldolh5ST2rtBRFimusohkZ3rgYrnv2GIwG/U/98PvH3sVpa+cW22dv2BEL2Qm7YF/ZJRsxwe69TW/H6/qH/jf/hz98KPW9tuc1Wwk//gPFucbd</latexit><latexit sha1_base64="Z3k0jP42WP//6lnqi9e2pn3k2fw=">AAACo3icbVFNa9tAEF0pbZq4H3HSYy5DnYaUFkcyhaSElEAvPfSQFrsJWMKsViNnyWoldkfBrvAP7S0/pStHhdrpwC6PNzNvZt8mpZKWguC35288ebr5bGu78/zFy1c73d29n7aojMCRKFRhrhNuUUmNI5Kk8Lo0yPNE4VVy+6XJX92hsbLQQ5qXGOd8qmUmBSdHTbq/hkf6HURncN5cnSjBqdS1cIp20YnOAjiEiHBGNcgMDrQrCw9gAVE0DvonmMeuZgDRB3Ayx4NG6L2T0etdn/92dSLUaSs/6faCfrAMeAzCFvRYG5eTXW8vSgtR5ahJKG7tOAxKimtuSAqFbt/KYsnFLZ/i2EHNc7RxvTRpAW8dk0JWGHc0wZL9t6PmubXzPHGVOacbu55ryP/lxhVlp3EtdVkRavEwKKsUUAGN45BKg4LU3AEujHS7grjhhgty/7IyZaldolh5ST2rtBRFimusohkZ3rgYrnv2GIwG/U/98PvH3sVpa+cW22dv2BEL2Qm7YF/ZJRsxwe69TW/H6/qH/jf/hz98KPW9tuc1Wwk//gPFucbd</latexit><latexit sha1_base64="Z3k0jP42WP//6lnqi9e2pn3k2fw=">AAACo3icbVFNa9tAEF0pbZq4H3HSYy5DnYaUFkcyhaSElEAvPfSQFrsJWMKsViNnyWoldkfBrvAP7S0/pStHhdrpwC6PNzNvZt8mpZKWguC35288ebr5bGu78/zFy1c73d29n7aojMCRKFRhrhNuUUmNI5Kk8Lo0yPNE4VVy+6XJX92hsbLQQ5qXGOd8qmUmBSdHTbq/hkf6HURncN5cnSjBqdS1cIp20YnOAjiEiHBGNcgMDrQrCw9gAVE0DvonmMeuZgDRB3Ayx4NG6L2T0etdn/92dSLUaSs/6faCfrAMeAzCFvRYG5eTXW8vSgtR5ahJKG7tOAxKimtuSAqFbt/KYsnFLZ/i2EHNc7RxvTRpAW8dk0JWGHc0wZL9t6PmubXzPHGVOacbu55ryP/lxhVlp3EtdVkRavEwKKsUUAGN45BKg4LU3AEujHS7grjhhgty/7IyZaldolh5ST2rtBRFimusohkZ3rgYrnv2GIwG/U/98PvH3sVpa+cW22dv2BEL2Qm7YF/ZJRsxwe69TW/H6/qH/jf/hz98KPW9tuc1Wwk//gPFucbd</latexit>

Proof by induction

Proposition. If T (n) satisfies the following recurrence, then T (n) = n log2 n.

Pf. [ by induction on n ]

独Base case: when n = 1, T(1) = 0 = n log2 n.

独Inductive hypothesis: assume T(n) = n log2 n.

独Goal: show that T(2n) = 2n log2 (2n).

11

T(2n) = 2 T(n) + 2n

= 2 n log2 n + 2n

= 2 n (log2 (2n) – 1) + 2n

= 2 n log2 (2n). ▪

recurrence

inductive hypothesis

assuming n is a power of 2T (n) =

��

�0 B7 n = 1

2T (n/2) + n B7 n > 1<latexit sha1_base64="Z3k0jP42WP//6lnqi9e2pn3k2fw=">AAACo3icbVFNa9tAEF0pbZq4H3HSYy5DnYaUFkcyhaSElEAvPfSQFrsJWMKsViNnyWoldkfBrvAP7S0/pStHhdrpwC6PNzNvZt8mpZKWguC35288ebr5bGu78/zFy1c73d29n7aojMCRKFRhrhNuUUmNI5Kk8Lo0yPNE4VVy+6XJX92hsbLQQ5qXGOd8qmUmBSdHTbq/hkf6HURncN5cnSjBqdS1cIp20YnOAjiEiHBGNcgMDrQrCw9gAVE0DvonmMeuZgDRB3Ayx4NG6L2T0etdn/92dSLUaSs/6faCfrAMeAzCFvRYG5eTXW8vSgtR5ahJKG7tOAxKimtuSAqFbt/KYsnFLZ/i2EHNc7RxvTRpAW8dk0JWGHc0wZL9t6PmubXzPHGVOacbu55ryP/lxhVlp3EtdVkRavEwKKsUUAGN45BKg4LU3AEujHS7grjhhgty/7IyZaldolh5ST2rtBRFimusohkZ3rgYrnv2GIwG/U/98PvH3sVpa+cW22dv2BEL2Qm7YF/ZJRsxwe69TW/H6/qH/jf/hz98KPW9tuc1Wwk//gPFucbd</latexit><latexit sha1_base64="Z3k0jP42WP//6lnqi9e2pn3k2fw=">AAACo3icbVFNa9tAEF0pbZq4H3HSYy5DnYaUFkcyhaSElEAvPfSQFrsJWMKsViNnyWoldkfBrvAP7S0/pStHhdrpwC6PNzNvZt8mpZKWguC35288ebr5bGu78/zFy1c73d29n7aojMCRKFRhrhNuUUmNI5Kk8Lo0yPNE4VVy+6XJX92hsbLQQ5qXGOd8qmUmBSdHTbq/hkf6HURncN5cnSjBqdS1cIp20YnOAjiEiHBGNcgMDrQrCw9gAVE0DvonmMeuZgDRB3Ayx4NG6L2T0etdn/92dSLUaSs/6faCfrAMeAzCFvRYG5eTXW8vSgtR5ahJKG7tOAxKimtuSAqFbt/KYsnFLZ/i2EHNc7RxvTRpAW8dk0JWGHc0wZL9t6PmubXzPHGVOacbu55ryP/lxhVlp3EtdVkRavEwKKsUUAGN45BKg4LU3AEujHS7grjhhgty/7IyZaldolh5ST2rtBRFimusohkZ3rgYrnv2GIwG/U/98PvH3sVpa+cW22dv2BEL2Qm7YF/ZJRsxwe69TW/H6/qH/jf/hz98KPW9tuc1Wwk//gPFucbd</latexit><latexit sha1_base64="Z3k0jP42WP//6lnqi9e2pn3k2fw=">AAACo3icbVFNa9tAEF0pbZq4H3HSYy5DnYaUFkcyhaSElEAvPfSQFrsJWMKsViNnyWoldkfBrvAP7S0/pStHhdrpwC6PNzNvZt8mpZKWguC35288ebr5bGu78/zFy1c73d29n7aojMCRKFRhrhNuUUmNI5Kk8Lo0yPNE4VVy+6XJX92hsbLQQ5qXGOd8qmUmBSdHTbq/hkf6HURncN5cnSjBqdS1cIp20YnOAjiEiHBGNcgMDrQrCw9gAVE0DvonmMeuZgDRB3Ayx4NG6L2T0etdn/92dSLUaSs/6faCfrAMeAzCFvRYG5eTXW8vSgtR5ahJKG7tOAxKimtuSAqFbt/KYsnFLZ/i2EHNc7RxvTRpAW8dk0JWGHc0wZL9t6PmubXzPHGVOacbu55ryP/lxhVlp3EtdVkRavEwKKsUUAGN45BKg4LU3AEujHS7grjhhgty/7IyZaldolh5ST2rtBRFimusohkZ3rgYrnv2GIwG/U/98PvH3sVpa+cW22dv2BEL2Qm7YF/ZJRsxwe69TW/H6/qH/jf/hz98KPW9tuc1Wwk//gPFucbd</latexit><latexit sha1_base64="Z3k0jP42WP//6lnqi9e2pn3k2fw=">AAACo3icbVFNa9tAEF0pbZq4H3HSYy5DnYaUFkcyhaSElEAvPfSQFrsJWMKsViNnyWoldkfBrvAP7S0/pStHhdrpwC6PNzNvZt8mpZKWguC35288ebr5bGu78/zFy1c73d29n7aojMCRKFRhrhNuUUmNI5Kk8Lo0yPNE4VVy+6XJX92hsbLQQ5qXGOd8qmUmBSdHTbq/hkf6HURncN5cnSjBqdS1cIp20YnOAjiEiHBGNcgMDrQrCw9gAVE0DvonmMeuZgDRB3Ayx4NG6L2T0etdn/92dSLUaSs/6faCfrAMeAzCFvRYG5eTXW8vSgtR5ahJKG7tOAxKimtuSAqFbt/KYsnFLZ/i2EHNc7RxvTRpAW8dk0JWGHc0wZL9t6PmubXzPHGVOacbu55ryP/lxhVlp3EtdVkRavEwKKsUUAGN45BKg4LU3AEujHS7grjhhgty/7IyZaldolh5ST2rtBRFimusohkZ3rgYrnv2GIwG/U/98PvH3sVpa+cW22dv2BEL2Qm7YF/ZJRsxwe69TW/H6/qH/jf/hz98KPW9tuc1Wwk//gPFucbd</latexit>

Which is the exact solution of the following recurrence?

A. T(n) = n ⎣log2 n⎦

B. T(n) = n ⎡log2 n⎤

C. T(n) = n ⎣log2 n⎦ + 2⎣log2 n⎦ − 1

D. T(n) = n ⎡log2 n⎤ − 2⎡log2 n⎤ + 1

E. Not even Knuth knows.

12

Divide-and-conquer: quiz 1

=n�

k=1

�log2 k�<latexit sha1_base64="OIDlLBbcsVZR990sJ0LjHkzac4E=">AAACWXicbVDLSgMxFE3HV62vVpdugkVwUcqMCCoiCG5cVrC20KlDJr2toZlkSO6IZehv+DVu9R8EP8b0sbDVC0lOzrk3N/fEqRQWff+r4K2srq1vFDdLW9s7u3vlyv6j1Znh0ORaatOOmQUpFDRRoIR2aoAlsYRWPLyd6K0XMFZo9YCjFLoJGyjRF5yho6KyH17Ra+q20GZJlA+vg/GTomEtrNFQchDSHXoQndIhDc3kHpWrft2fBv0Lgjmoknk0okphP+xpniWgkEtmbSfwU+zmzKDgEsalMLOQMj5kA+g4qFgCtptPRxvTY8f0aF8btxTSKfu7ImeJtaMkdpkJw2e7rE3I/7ROhv2Lbi5UmiEoPmvUzyRFTSc+0Z4wwFGOHGDcCPdXyp+ZYRydmwtdpm+nwBcmyV8zJbjuwRIr8RUNGzsXg2XP/oLmaf2y7t+fVW8u5nYWySE5IickIOfkhtyRBmkSTt7IO/kgn4Vvz/OKXmmW6hXmNQdkIbyDH4oEtJc=</latexit><latexit sha1_base64="OIDlLBbcsVZR990sJ0LjHkzac4E=">AAACWXicbVDLSgMxFE3HV62vVpdugkVwUcqMCCoiCG5cVrC20KlDJr2toZlkSO6IZehv+DVu9R8EP8b0sbDVC0lOzrk3N/fEqRQWff+r4K2srq1vFDdLW9s7u3vlyv6j1Znh0ORaatOOmQUpFDRRoIR2aoAlsYRWPLyd6K0XMFZo9YCjFLoJGyjRF5yho6KyH17Ra+q20GZJlA+vg/GTomEtrNFQchDSHXoQndIhDc3kHpWrft2fBv0Lgjmoknk0okphP+xpniWgkEtmbSfwU+zmzKDgEsalMLOQMj5kA+g4qFgCtptPRxvTY8f0aF8btxTSKfu7ImeJtaMkdpkJw2e7rE3I/7ROhv2Lbi5UmiEoPmvUzyRFTSc+0Z4wwFGOHGDcCPdXyp+ZYRydmwtdpm+nwBcmyV8zJbjuwRIr8RUNGzsXg2XP/oLmaf2y7t+fVW8u5nYWySE5IickIOfkhtyRBmkSTt7IO/kgn4Vvz/OKXmmW6hXmNQdkIbyDH4oEtJc=</latexit><latexit sha1_base64="OIDlLBbcsVZR990sJ0LjHkzac4E=">AAACWXicbVDLSgMxFE3HV62vVpdugkVwUcqMCCoiCG5cVrC20KlDJr2toZlkSO6IZehv+DVu9R8EP8b0sbDVC0lOzrk3N/fEqRQWff+r4K2srq1vFDdLW9s7u3vlyv6j1Znh0ORaatOOmQUpFDRRoIR2aoAlsYRWPLyd6K0XMFZo9YCjFLoJGyjRF5yho6KyH17Ra+q20GZJlA+vg/GTomEtrNFQchDSHXoQndIhDc3kHpWrft2fBv0Lgjmoknk0okphP+xpniWgkEtmbSfwU+zmzKDgEsalMLOQMj5kA+g4qFgCtptPRxvTY8f0aF8btxTSKfu7ImeJtaMkdpkJw2e7rE3I/7ROhv2Lbi5UmiEoPmvUzyRFTSc+0Z4wwFGOHGDcCPdXyp+ZYRydmwtdpm+nwBcmyV8zJbjuwRIr8RUNGzsXg2XP/oLmaf2y7t+fVW8u5nYWySE5IickIOfkhtyRBmkSTt7IO/kgn4Vvz/OKXmmW6hXmNQdkIbyDH4oEtJc=</latexit>

T (n) =

��

�0 B7 n = 1

T (�n/2�) + T (�n/2�) + n � 1 B7 n > 1<latexit sha1_base64="VRAYMk/LbRQG3UoGOQDaFFRdfHc=">AAACxXicbVFda9swFJW9ry77SrvHvVyWbqSMZXYYtCNsFPbSxw6StRCZIMvXqagsGUkuCSbsj+yP7d9Mdl1Y0t2nwzn389y0lMK6KPoThA8ePnr8ZO9p79nzFy9f9fcPflpdGY4zrqU2lymzKIXCmRNO4mVpkBWpxIv0+nujX9ygsUKrqVuXmBRsqUQuOHOeWvR/T4fqCL5Cj6a4FKrmvpfd9OgkgvdAHa5cDSKHQ+Vz4kPYAKXzaHSMReJzpkMqc6m1AfVpDNS0+Ajo5AOdQCNyFLLTGngnqY/xbvdvd917FFXWrbHoD6JR1AbcB3EHBqSL88V+cEAzzasCleOSWTuPo9IlNTNOcIn+rspiyfg1W+LcQ8UKtEnd2riBd57JIPfn5Fo5aNl/K2pWWLsuUp9ZMHdld7WG/J82r1x+ktRClZVDxW8H5ZUEp6H5CWTCIHdy7QHjRvhdgV8xw7jzn9ua0vYukW9dUq8qJbjOcIeVbuUMa1yMdz27D2bj0ZdR/OPz4PSks3OPvCFvyZDE5JickjNyTmaEB2EwDOJgHJ6FKnThzW1qGHQ1r8lWhL/+Aje6018=</latexit><latexit sha1_base64="VRAYMk/LbRQG3UoGOQDaFFRdfHc=">AAACxXicbVFda9swFJW9ry77SrvHvVyWbqSMZXYYtCNsFPbSxw6StRCZIMvXqagsGUkuCSbsj+yP7d9Mdl1Y0t2nwzn389y0lMK6KPoThA8ePnr8ZO9p79nzFy9f9fcPflpdGY4zrqU2lymzKIXCmRNO4mVpkBWpxIv0+nujX9ygsUKrqVuXmBRsqUQuOHOeWvR/T4fqCL5Cj6a4FKrmvpfd9OgkgvdAHa5cDSKHQ+Vz4kPYAKXzaHSMReJzpkMqc6m1AfVpDNS0+Ajo5AOdQCNyFLLTGngnqY/xbvdvd917FFXWrbHoD6JR1AbcB3EHBqSL88V+cEAzzasCleOSWTuPo9IlNTNOcIn+rspiyfg1W+LcQ8UKtEnd2riBd57JIPfn5Fo5aNl/K2pWWLsuUp9ZMHdld7WG/J82r1x+ktRClZVDxW8H5ZUEp6H5CWTCIHdy7QHjRvhdgV8xw7jzn9ua0vYukW9dUq8qJbjOcIeVbuUMa1yMdz27D2bj0ZdR/OPz4PSks3OPvCFvyZDE5JickjNyTmaEB2EwDOJgHJ6FKnThzW1qGHQ1r8lWhL/+Aje6018=</latexit><latexit sha1_base64="VRAYMk/LbRQG3UoGOQDaFFRdfHc=">AAACxXicbVFda9swFJW9ry77SrvHvVyWbqSMZXYYtCNsFPbSxw6StRCZIMvXqagsGUkuCSbsj+yP7d9Mdl1Y0t2nwzn389y0lMK6KPoThA8ePnr8ZO9p79nzFy9f9fcPflpdGY4zrqU2lymzKIXCmRNO4mVpkBWpxIv0+nujX9ygsUKrqVuXmBRsqUQuOHOeWvR/T4fqCL5Cj6a4FKrmvpfd9OgkgvdAHa5cDSKHQ+Vz4kPYAKXzaHSMReJzpkMqc6m1AfVpDNS0+Ajo5AOdQCNyFLLTGngnqY/xbvdvd917FFXWrbHoD6JR1AbcB3EHBqSL88V+cEAzzasCleOSWTuPo9IlNTNOcIn+rspiyfg1W+LcQ8UKtEnd2riBd57JIPfn5Fo5aNl/K2pWWLsuUp9ZMHdld7WG/J82r1x+ktRClZVDxW8H5ZUEp6H5CWTCIHdy7QHjRvhdgV8xw7jzn9ua0vYukW9dUq8qJbjOcIeVbuUMa1yMdz27D2bj0ZdR/OPz4PSks3OPvCFvyZDE5JickjNyTmaEB2EwDOJgHJ6FKnThzW1qGHQ1r8lWhL/+Aje6018=</latexit><latexit sha1_base64="VRAYMk/LbRQG3UoGOQDaFFRdfHc=">AAACxXicbVFda9swFJW9ry77SrvHvVyWbqSMZXYYtCNsFPbSxw6StRCZIMvXqagsGUkuCSbsj+yP7d9Mdl1Y0t2nwzn389y0lMK6KPoThA8ePnr8ZO9p79nzFy9f9fcPflpdGY4zrqU2lymzKIXCmRNO4mVpkBWpxIv0+nujX9ygsUKrqVuXmBRsqUQuOHOeWvR/T4fqCL5Cj6a4FKrmvpfd9OgkgvdAHa5cDSKHQ+Vz4kPYAKXzaHSMReJzpkMqc6m1AfVpDNS0+Ajo5AOdQCNyFLLTGngnqY/xbvdvd917FFXWrbHoD6JR1AbcB3EHBqSL88V+cEAzzasCleOSWTuPo9IlNTNOcIn+rspiyfg1W+LcQ8UKtEnd2riBd57JIPfn5Fo5aNl/K2pWWLsuUp9ZMHdld7WG/J82r1x+ktRClZVDxW8H5ZUEp6H5CWTCIHdy7QHjRvhdgV8xw7jzn9ua0vYukW9dUq8qJbjOcIeVbuUMa1yMdz27D2bj0ZdR/OPz4PSks3OPvCFvyZDE5JickjNyTmaEB2EwDOJgHJ6FKnThzW1qGHQ1r8lWhL/+Aje6018=</latexit>

no longer assuming n is a power of 2

Proposition. If T(n) satisfies the following recurrence, then T(n) ≤ n ⎡log2 n⎤. Pf. [ by strong induction on n ]

独Base case: n = 1.

独Define n1 = ⎣n / 2⎦ and n2 = ⎡n / 2⎤ and note that n = n1 + n2.

独Induction step: assume true for 1, 2, ... , n – 1.

Analysis of mergesort recurrence

T(n) ≤ T(n1) + T(n2) + n

≤ n1 ⎡log2 n1⎤ + n2 ⎡log2 n2⎤ + n

≤ n1 ⎡log2 n2⎤ + n2 ⎡log2 n2⎤ + n

= n ⎡log2 n⎤. ▪

= n ⎡log2 n2⎤ + n

≤ n (⎡log2 n⎤ – 1) + n

n2 = dn/2e

l2dlog2 ne / 2

m

= 2dlog2 ne / 2

dlog2 n2e dlog2 ne � 1

n2 = dn/2e

l2dlog2 ne / 2

m

= 2dlog2 ne / 2


n2 = dn/2e

l2dlog2 ne / 2

m

= 2dlog2 ne / 2


n2 = dn/2e

l2dlog2 ne / 2

m

= 2dlog2 ne / 2

log2 n2 dlog2 ne � 1


T (n) �

��

�0 B7 n = 1

T (�n/2�) + T (�n/2�) + n B7 n > 1<latexit sha1_base64="PlaAp4PYCkUIob8GuIy/Wl4iXIM=">AAACynicbVFdb9MwFHUyYKN8deORlys6UBGiJBOiQxNoEi+8IA2pZZPqqHKcm86aY0e2M7WK+sYf4Wfxb3CyTKId98VH51yf+5WWUlgXRX+CcOfe/Qe7ew97jx4/efqsv3/w0+rKcJxyLbW5SJlFKRROnXASL0qDrEglnqdXXxv9/BqNFVpN3KrEpGALJXLBmfPUvP97MlRvgJ4Aldg8PZriQqiae0+77tGTCF4Ddbh0NYgcDhV8hvgQ1kDpbDTGIvEpkyGVudTagHp/BNS0uDF96w0bkaOQndbAW0ltW3+5te5RVFnXw7w/iEZRG3AXxB0YkC7O5vvBAc00rwpUjktm7SyOSpfUzDjBJfqhKosl41dsgTMPFSvQJnW7yzW88kwGuR8m18pBy/77o2aFtasi9ZkFc5d2W2vI/2mzyuXHSS1UWTlU/KZQXklwGprDQCYMcidXHjBuhO8V+CUzjDt/vo0qrXeJfGOSelkpwXWGW6x0S2dYs8V4e2d3wfRo9GkU//gwOD3u1rlHXpCXZEhiMian5Bs5I1PCg93gXfAxGIffQxuuwvomNQy6P8/JRoS//gIlc9Vh</latexit><latexit sha1_base64="PlaAp4PYCkUIob8GuIy/Wl4iXIM=">AAACynicbVFdb9MwFHUyYKN8deORlys6UBGiJBOiQxNoEi+8IA2pZZPqqHKcm86aY0e2M7WK+sYf4Wfxb3CyTKId98VH51yf+5WWUlgXRX+CcOfe/Qe7ew97jx4/efqsv3/w0+rKcJxyLbW5SJlFKRROnXASL0qDrEglnqdXXxv9/BqNFVpN3KrEpGALJXLBmfPUvP97MlRvgJ4Aldg8PZriQqiae0+77tGTCF4Ddbh0NYgcDhV8hvgQ1kDpbDTGIvEpkyGVudTagHp/BNS0uDF96w0bkaOQndbAW0ltW3+5te5RVFnXw7w/iEZRG3AXxB0YkC7O5vvBAc00rwpUjktm7SyOSpfUzDjBJfqhKosl41dsgTMPFSvQJnW7yzW88kwGuR8m18pBy/77o2aFtasi9ZkFc5d2W2vI/2mzyuXHSS1UWTlU/KZQXklwGprDQCYMcidXHjBuhO8V+CUzjDt/vo0qrXeJfGOSelkpwXWGW6x0S2dYs8V4e2d3wfRo9GkU//gwOD3u1rlHXpCXZEhiMian5Bs5I1PCg93gXfAxGIffQxuuwvomNQy6P8/JRoS//gIlc9Vh</latexit><latexit sha1_base64="PlaAp4PYCkUIob8GuIy/Wl4iXIM=">AAACynicbVFdb9MwFHUyYKN8deORlys6UBGiJBOiQxNoEi+8IA2pZZPqqHKcm86aY0e2M7WK+sYf4Wfxb3CyTKId98VH51yf+5WWUlgXRX+CcOfe/Qe7ew97jx4/efqsv3/w0+rKcJxyLbW5SJlFKRROnXASL0qDrEglnqdXXxv9/BqNFVpN3KrEpGALJXLBmfPUvP97MlRvgJ4Aldg8PZriQqiae0+77tGTCF4Ddbh0NYgcDhV8hvgQ1kDpbDTGIvEpkyGVudTagHp/BNS0uDF96w0bkaOQndbAW0ltW3+5te5RVFnXw7w/iEZRG3AXxB0YkC7O5vvBAc00rwpUjktm7SyOSpfUzDjBJfqhKosl41dsgTMPFSvQJnW7yzW88kwGuR8m18pBy/77o2aFtasi9ZkFc5d2W2vI/2mzyuXHSS1UWTlU/KZQXklwGprDQCYMcidXHjBuhO8V+CUzjDt/vo0qrXeJfGOSelkpwXWGW6x0S2dYs8V4e2d3wfRo9GkU//gwOD3u1rlHXpCXZEhiMian5Bs5I1PCg93gXfAxGIffQxuuwvomNQy6P8/JRoS//gIlc9Vh</latexit><latexit sha1_base64="PlaAp4PYCkUIob8GuIy/Wl4iXIM=">AAACynicbVFdb9MwFHUyYKN8deORlys6UBGiJBOiQxNoEi+8IA2pZZPqqHKcm86aY0e2M7WK+sYf4Wfxb3CyTKId98VH51yf+5WWUlgXRX+CcOfe/Qe7ew97jx4/efqsv3/w0+rKcJxyLbW5SJlFKRROnXASL0qDrEglnqdXXxv9/BqNFVpN3KrEpGALJXLBmfPUvP97MlRvgJ4Aldg8PZriQqiae0+77tGTCF4Ddbh0NYgcDhV8hvgQ1kDpbDTGIvEpkyGVudTagHp/BNS0uDF96w0bkaOQndbAW0ltW3+5te5RVFnXw7w/iEZRG3AXxB0YkC7O5vvBAc00rwpUjktm7SyOSpfUzDjBJfqhKosl41dsgTMPFSvQJnW7yzW88kwGuR8m18pBy/77o2aFtasi9ZkFc5d2W2vI/2mzyuXHSS1UWTlU/KZQXklwGprDQCYMcidXHjBuhO8V+CUzjDt/vo0qrXeJfGOSelkpwXWGW6x0S2dYs8V4e2d3wfRo9GkU//gwOD3u1rlHXpCXZEhiMian5Bs5I1PCg93gXfAxGIffQxuuwvomNQy6P8/JRoS//gIlc9Vh</latexit>


an integer

no longer assuming n is a power of 2

Comparable[] a = ...; ... Arrays.sort(a);

can access elements only via calls to compareTo()

Digression: sorting lower bound

Challenge. How to prove a lower bound for all conceivable algorithms?

Model of computation. Comparison trees.

独Can access the elements only through pairwise comparisons.

独All other operations (control, data movement, etc.) are free.

Cost model. Number of compares.

Q. Realistic model?

A1. Yes. Java, Python, C++, …

A2. Yes. Mergesort, insertion sort, quicksort, heapsort, …

A3. No. Bucket sort, radix sorts, …

14

Comparison tree (for 3 distinct keys a, b, and c)

15

b < c

yes no

a < c

yes no

a < c

yes no

a c b c a b

b a ca b c b < c

yes no

b c a c b a

height of pruned tree =worst-case number

of compares

a < b

yes no

code between compares(e.g., sequence of exchanges)

each reachable leaf corresponds to one (and only one) ordering;

exactly one reachable leaf for each possible ordering

Sorting lower bound

Theorem. Any deterministic compare-based sorting algorithm must make

Ω(n log n) compares in the worst-case.

Pf. [ information theoretic ]

独Assume array consists of n distinct values a1 through an.

独Worst-case number of compares = height h of pruned comparison tree.

独Binary tree of height h has ≤ 2h leaves.

独n! different orderings ⇒ n! reachable leaves.

16

h

≤ 2h leavesn! leaves

Sorting lower bound

Theorem. Any deterministic compare-based sorting algorithm must make

Ω(n log n) compares in the worst-case.

Pf. [ information theoretic ]

独Assume array consists of n distinct values a1 through an.

独Worst-case number of compares = height h of pruned comparison tree.

独Binary tree of height h has ≤ 2h leaves.

独n! different orderings ⇒ n! reachable leaves.

Note. Lower bound can be extended to include randomized algorithms.

17

2h ≥ # leaves ≥ n !

⇒ h ≥ log2 (n!)

≥ n log2 n − n / ln 2 ▪

Stirling’s formula

SHUFFLING A LINKED LIST

Problem. Given a singly linked list, rearrange its nodes uniformly at random.

Assumption. Access to a perfect random-number generator.

Performance. O(n log n) time, O(log n) extra space.

18

L

3♣ 4♣ 5♣ 6♣2♣ 7♣ null

L

6♣ 2♣ 7♣ 3♣5♣ 4♣ null

input

shuffled

all n! permutations equally likely


‣ mergesort





SECTION 5.3

Counting inversions

Music site tries to match your song preferences with others.

独You rank n songs.

独Music site consults database to find people with similar tastes.

Similarity metric: number of inversions between two rankings.

独My rank: 1, 2, …, n.

独Your rank: a1, a2, …, an.

独Songs i and j are inverted if i < j, but ai > aj.

Brute force: check all Θ(n2) pairs.

20

A B C D E

me 1 2 3 4 5

you 1 3 4 2 5

2 inversions: 3-2, 4-2

Counting inversions: applications

独Voting theory.

独Collaborative filtering.

独Measuring the “sortedness” of an array.

独Sensitivity analysis of Google’s ranking function.

独Rank aggregation for meta-searching on the Web.

独Nonparametric statistics (e.g., Kendall’s tau distance).

21

Rank Aggregation Methods for the Web

Cynthia Dwork Ravi Kumar Moni Naor D. Sivakumar

ABSTRACT

1. INTRODUCTION

Copyright is held by the author/owner.WWW10, May 1-5, 2001, Hong Kong.ACM 1-58113-348-0/01/0005.

1.1 Motivation

613

Rank Aggregation Methods for the Web

Cynthia Dwork Ravi Kumar Moni Naor D. Sivakumar

ABSTRACT

1. INTRODUCTION

Copyright is held by the author/owner.WWW10, May 1-5, 2001, Hong Kong.ACM 1-58113-348-0/01/0005.

1.1 Motivation

613

Counting inversions: divide-and-conquer

独Divide: separate list into two halves A and B.

独Conquer: recursively count inversions in each list.

独Combine: count inversions (a, b) with a ∈ A and b ∈ B.

独Return sum of three counts.

22

1 5 4 8 10 2 6 9 3 7

input

output 1 + 3 + 13 = 17

count inversions in left half A

5-4

1 5 4 8 10 2 6 9 3 7

6-3 9-3 9-7

count inversions in right half B

count inversions (a, b) with a ∈ A and b ∈ B

4-2 4-3 5-2 5-3 8-2 8-3 8-6 8-7 10-2 10-3 10-6 10-7 10-9

2 6 9 3 71 5 4 8 10

Counting inversions: how to combine two subproblems?

Q. How to count inversions (a, b) with a ∈ A and b ∈ B?

A. Easy if A and B are sorted!

Warmup algorithm.

独Sort A and B.

独For each element b ∈ B,

- binary search in A to find how elements in A are greater than b.

23

2 11 16 20 23

sort A

3 7 10 14 18

sort B

binary search to count inversions (a, b) with a ∈ A and b ∈ B

5 2 1 0 0

2 11 16 20 233 7 10 14 18

20 23 2 11 16

list A

7 10 18 3 14

list B

Counting inversions: how to combine two subproblems?

Count inversions (a, b) with a ∈ A and b ∈ B, assuming A and B are sorted.

独Scan A and B from left to right.

独Compare ai and bj.

独If ai < bj, then ai is not inverted with any element left in B.

独If ai > bj, then bj is inverted with every element left in A.

独Append smaller element to sorted list C.

24

count inversions (a, b) with a ∈ A and b ∈ B

5 2

2 3 7 10 11

merge to form sorted list C

2 11 bj 20 233 7 10 ai 18

Counting inversions: divide-and-conquer algorithm implementation

Input. List L.

Output. Number of inversions in L and L in sorted order.

25

SORT-AND-COUNT(L) _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IF (list L has one element)

RETURN (0, L).

Divide the list into two halves A and B.

(rA, A) ← SORT-AND-COUNT(A).

(rB, B) ← SORT-AND-COUNT(B).

(rAB, L) ← MERGE-AND-COUNT(A, B).

RETURN (rA + rB + rAB, L)._________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

T (n / 2)

T (n / 2)

Θ(n)

Counting inversions: divide-and-conquer algorithm analysis

Proposition. The sort-and-count algorithm counts the number of inversions

in a permutation of size n in O(n log n) time.

Pf. The worst-case running time T(n) satisfies the recurrence:

26

T (n) =

��

��(1) B7 n = 1

T (�n/2�) + T (�n/2�) + �(n) B7 n > 1<latexit sha1_base64="3pXfe56Q7VvCgldQ8ZGmWT26i7U=">AAAC2HicbVHLattAFB0pfaTuy0npqpuhTotDwZFCIQmmJdAWukzBbgIaYUbjK3vIaCRmroqN8KKr0m1/pN/Tv+lIUaC2exfD4Zw799xHUihpMQj+eP7Onbv37u8+6Dx89PjJ0+7e/lebl0bAWOQqN1cJt6CkhjFKVHBVGOBZouAyuf5Q65ffwFiZ6xEuC4gzPtMylYKjoybd36O+PqRsSN/VT4clMJO6Eq6iXXXYkI3mgLwfHtLXlCEssKIypQfapYcHdEUZiwYnkMUuddRnKlV5bqg+OqbMNLgu/cYVrkUBUrVaDW+l1kJvWby/tegw0NO2p0m3FwyCJug2CFvQI21cTPa8fTbNRZmBRqG4tVEYFBhX3KAUCtyQpYWCi2s+g8hBzTOwcdVsdkVfOWZKUzdUmmukDfvvj4pn1i6zxGVmHOd2U6vJ/2lRielpXEldlAha3BilpaKY0/pMdCoNCFRLB7gw0vVKxZwbLtAdc82lqV2AWJukWpRainwKG6zCBRpebzHc3Nk2GB8Pzgbhl7e989N2nbvkBXlJ+iQkJ+ScfCYXZEyE99wbeh+9T37kf/d/+D9vUn2v/fOMrIX/6y+lidpE</latexit><latexit sha1_base64="3pXfe56Q7VvCgldQ8ZGmWT26i7U=">AAAC2HicbVHLattAFB0pfaTuy0npqpuhTotDwZFCIQmmJdAWukzBbgIaYUbjK3vIaCRmroqN8KKr0m1/pN/Tv+lIUaC2exfD4Zw799xHUihpMQj+eP7Onbv37u8+6Dx89PjJ0+7e/lebl0bAWOQqN1cJt6CkhjFKVHBVGOBZouAyuf5Q65ffwFiZ6xEuC4gzPtMylYKjoybd36O+PqRsSN/VT4clMJO6Eq6iXXXYkI3mgLwfHtLXlCEssKIypQfapYcHdEUZiwYnkMUuddRnKlV5bqg+OqbMNLgu/cYVrkUBUrVaDW+l1kJvWby/tegw0NO2p0m3FwyCJug2CFvQI21cTPa8fTbNRZmBRqG4tVEYFBhX3KAUCtyQpYWCi2s+g8hBzTOwcdVsdkVfOWZKUzdUmmukDfvvj4pn1i6zxGVmHOd2U6vJ/2lRielpXEldlAha3BilpaKY0/pMdCoNCFRLB7gw0vVKxZwbLtAdc82lqV2AWJukWpRainwKG6zCBRpebzHc3Nk2GB8Pzgbhl7e989N2nbvkBXlJ+iQkJ+ScfCYXZEyE99wbeh+9T37kf/d/+D9vUn2v/fOMrIX/6y+lidpE</latexit><latexit sha1_base64="3pXfe56Q7VvCgldQ8ZGmWT26i7U=">AAAC2HicbVHLattAFB0pfaTuy0npqpuhTotDwZFCIQmmJdAWukzBbgIaYUbjK3vIaCRmroqN8KKr0m1/pN/Tv+lIUaC2exfD4Zw799xHUihpMQj+eP7Onbv37u8+6Dx89PjJ0+7e/lebl0bAWOQqN1cJt6CkhjFKVHBVGOBZouAyuf5Q65ffwFiZ6xEuC4gzPtMylYKjoybd36O+PqRsSN/VT4clMJO6Eq6iXXXYkI3mgLwfHtLXlCEssKIypQfapYcHdEUZiwYnkMUuddRnKlV5bqg+OqbMNLgu/cYVrkUBUrVaDW+l1kJvWby/tegw0NO2p0m3FwyCJug2CFvQI21cTPa8fTbNRZmBRqG4tVEYFBhX3KAUCtyQpYWCi2s+g8hBzTOwcdVsdkVfOWZKUzdUmmukDfvvj4pn1i6zxGVmHOd2U6vJ/2lRielpXEldlAha3BilpaKY0/pMdCoNCFRLB7gw0vVKxZwbLtAdc82lqV2AWJukWpRainwKG6zCBRpebzHc3Nk2GB8Pzgbhl7e989N2nbvkBXlJ+iQkJ+ScfCYXZEyE99wbeh+9T37kf/d/+D9vUn2v/fOMrIX/6y+lidpE</latexit><latexit sha1_base64="3pXfe56Q7VvCgldQ8ZGmWT26i7U=">AAAC2HicbVHLattAFB0pfaTuy0npqpuhTotDwZFCIQmmJdAWukzBbgIaYUbjK3vIaCRmroqN8KKr0m1/pN/Tv+lIUaC2exfD4Zw799xHUihpMQj+eP7Onbv37u8+6Dx89PjJ0+7e/lebl0bAWOQqN1cJt6CkhjFKVHBVGOBZouAyuf5Q65ffwFiZ6xEuC4gzPtMylYKjoybd36O+PqRsSN/VT4clMJO6Eq6iXXXYkI3mgLwfHtLXlCEssKIypQfapYcHdEUZiwYnkMUuddRnKlV5bqg+OqbMNLgu/cYVrkUBUrVaDW+l1kJvWby/tegw0NO2p0m3FwyCJug2CFvQI21cTPa8fTbNRZmBRqG4tVEYFBhX3KAUCtyQpYWCi2s+g8hBzTOwcdVsdkVfOWZKUzdUmmukDfvvj4pn1i6zxGVmHOd2U6vJ/2lRielpXEldlAha3BilpaKY0/pMdCoNCFRLB7gw0vVKxZwbLtAdc82lqV2AWJukWpRainwKG6zCBRpebzHc3Nk2GB8Pzgbhl7e989N2nbvkBXlJ+iQkJ+ScfCYXZEyE99wbeh+9T37kf/d/+D9vUn2v/fOMrIX/6y+lidpE</latexit>


‣ mergesort





SECTION 7.1–7.3

3-WAY PARTITIONING

Goal. Given an array A and pivot element p, partition array so that:

独Smaller elements in left subarray L.

独Equal elements in middle subarray M.

独Larger elements in right subarray R.

Challenge. O(n) time and O(1) space.

28

7 6 12 3 11 8 9 1 4 10 2

3 1 4 2 6 7 12 11 8 9 10

p

the array A

the partitioned array A

RL M

独Pick a random pivot element p ∈ A.

独3-way partition the array into L, M, and R.

独Recursively sort both L and R.

7 6 12 3 11 8 9 1 4 10 2

Randomized quicksort

29

p

7 6 12 3 11 8 9 1 4 10 2the array A

3 1 4 2 6 7 12 11 8 9 10partition A

1 2 3 4 6 7 12 11 8 9 10sort L

1 2 3 4 6 7 8 9 10 11 12sort R

1 2 3 4 6 7 8 9 10 11 12the sorted array A

Randomized quicksort



独Recursively sort both L and R.

30

RANDOMIZED-QUICKSORT(A) _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IF (array A has zero or one element)

RETURN.

Pick pivot p ∈ A uniformly at random.

(L, M, R) ← PARTITION-3-WAY(A, p).

RANDOMIZED-QUICKSORT(L).

RANDOMIZED-QUICKSORT(R)._______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

T (i)

T (n − i − 1)

Θ(n)

new analysis required (i is a random variable—depends on p)

Proposition. The expected number of compares to quicksort an array of n distinct elements a1 < a2 < � < an is O(n log n). Pf. Consider BST representation of pivot elements.

a7 a6 a12 a3 a11 a8 a9 a1 a4 a10 a2 a13 a5

Analysis of randomized quicksort

31

the original array of elements A

first pivot in left subarray

first pivot (chosen uniformly at random)

a9

a11a3

a8 a10 a13

a12

a6

a7a5

a4

a1

a2


Proposition. The expected number of compares to quicksort an array of n distinct elements a1 < a2 < � < an is O(n log n).

Pf. Consider BST representation of pivot elements.

独ai and aj are compared once iff one is an ancestor of the other.

32

a3 and a6 are compared(when a3 is pivot)first pivot

(chosen uniformly at random)

a9

a11a3

a8 a10 a13

a12

a6

a7a5

a4

a1

a2



Proposition. The expected number of compares to quicksort an array of n distinct elements a1 < a2 < � < an is O(n log n).



33

a2 and a8 are not compared(because a3 partitions them)

a9

a11a3

a8 a10 a13

a12

a6

a7a5

a4

a1

a2


first pivot (chosen uniformly at random)

Given an array of n ≥ 8 distinct elements a1 < a2 < � < an, what is the probability that a7 and a8 are compared during randomized quicksort?

A. 0

B. 1 / n

C. 2 / n

D. 1

34


a9

a11a3

a8 a10 a13

a12

a6

a7a5

a4

a1

a2

if two elements are adjacent,

one must be an ancestor of the other

Given an array of n ≥ 2 distinct elements a1 < a2 < � < an, what is the probability that a1 and an are compared during randomized quicksort?

A. 0

B. 1 / n

C. 2 / n

D. 1

35


a9

a11a3

a8 a10 a13

a12

a6

a7a5

a4

a1

a2

a1 and an are compared iff first pivot is either a1 or an

a8

a2


Proposition. The expected number of compares to quicksort an array of n

distinct elements a1 < a2 < � < an is O(n log n).



独Pr [ ai and aj are compared ] = 2 / ( j - i + 1), where i < j.

36

Pr[a2 and a8 compared] = 2/7compared iff either a2 or a8 is chosen

as pivot before any of { a3, a4, a5, a6, a7}a9

a11a3

a8 a10 a13

a12

a6

a7a5

a4

a1

a2


Proposition. The expected number of compares to quicksort an array of n

distinct elements a1 < a2 < � < an is O(n log n). Pf. Consider BST representation of pivot elements.


独Pr [ ai and aj are compared ] = 2 / ( j - i + 1), where i < j.

独Expected number of compares =

Remark. Number of compares only decreases if equal elements.

37

n�

i=1

n�

j=i+1

2

j � i + 1= 2

n�

i=1

n�i+1�

j=2

1

j

� 2nn�

j=1

1

j

� 2n (ln n + 1)<latexit sha1_base64="TSQczVbIEUgfpzVbrGceKD3sGfA=">AAADBHicjZFNb9NAEIbXLh8lfKXlyGVEBCoqRLZViVZRpSIuHItEaKVsiNabcbrteu3urlEjy2d+DSfElf+B+DOs7YCSFCFGsvzuMzPv2jNxLoWxQfDD8zdu3Lx1e/NO5+69+w8edre2P5is0ByHPJOZPo2ZQSkUDq2wEk9zjSyNJZ7EF2/q/Mkn1EZk6r2d5zhO2UyJRHBmHZp0f1JTpJNSHIbVx1JVQF9AS87hEMSuo6phiWa8jKry/GUN4RnQAR24kuYVAfzLJqpZ2/fHKnRW7kg7v62oxFZApJZ6l/z+p28AO1QqULAL4fNJtxf0gybguggXokcWcTzZ8rbpNONFispyyYwZhUFuxyXTVnCJVYcWBnPGL9gMR04qlqIZl80WKnjqyBSSTLtHWWjockfJUmPmaewqU2bPzHquhn/LjQqb7I9LofLCouLtRUkhwWZQrxSmQiO3cu4E41q4bwV+xty0rFv8yi2Nd4585U/Kq0IJnk1xjUp7ZTWr3BTD9ZldF8Oof9AP3+31jvYX49wkj8kTskNC8oockbfkmAwJ9157My/3Lv3P/hf/q/+tLfW9Rc8jshL+91/5zen5</latexit><latexit sha1_base64="TSQczVbIEUgfpzVbrGceKD3sGfA=">AAADBHicjZFNb9NAEIbXLh8lfKXlyGVEBCoqRLZViVZRpSIuHItEaKVsiNabcbrteu3urlEjy2d+DSfElf+B+DOs7YCSFCFGsvzuMzPv2jNxLoWxQfDD8zdu3Lx1e/NO5+69+w8edre2P5is0ByHPJOZPo2ZQSkUDq2wEk9zjSyNJZ7EF2/q/Mkn1EZk6r2d5zhO2UyJRHBmHZp0f1JTpJNSHIbVx1JVQF9AS87hEMSuo6phiWa8jKry/GUN4RnQAR24kuYVAfzLJqpZ2/fHKnRW7kg7v62oxFZApJZ6l/z+p28AO1QqULAL4fNJtxf0gybguggXokcWcTzZ8rbpNONFispyyYwZhUFuxyXTVnCJVYcWBnPGL9gMR04qlqIZl80WKnjqyBSSTLtHWWjockfJUmPmaewqU2bPzHquhn/LjQqb7I9LofLCouLtRUkhwWZQrxSmQiO3cu4E41q4bwV+xty0rFv8yi2Nd4585U/Kq0IJnk1xjUp7ZTWr3BTD9ZldF8Oof9AP3+31jvYX49wkj8kTskNC8oockbfkmAwJ9157My/3Lv3P/hf/q/+tLfW9Rc8jshL+91/5zen5</latexit><latexit sha1_base64="TSQczVbIEUgfpzVbrGceKD3sGfA=">AAADBHicjZFNb9NAEIbXLh8lfKXlyGVEBCoqRLZViVZRpSIuHItEaKVsiNabcbrteu3urlEjy2d+DSfElf+B+DOs7YCSFCFGsvzuMzPv2jNxLoWxQfDD8zdu3Lx1e/NO5+69+w8edre2P5is0ByHPJOZPo2ZQSkUDq2wEk9zjSyNJZ7EF2/q/Mkn1EZk6r2d5zhO2UyJRHBmHZp0f1JTpJNSHIbVx1JVQF9AS87hEMSuo6phiWa8jKry/GUN4RnQAR24kuYVAfzLJqpZ2/fHKnRW7kg7v62oxFZApJZ6l/z+p28AO1QqULAL4fNJtxf0gybguggXokcWcTzZ8rbpNONFispyyYwZhUFuxyXTVnCJVYcWBnPGL9gMR04qlqIZl80WKnjqyBSSTLtHWWjockfJUmPmaewqU2bPzHquhn/LjQqb7I9LofLCouLtRUkhwWZQrxSmQiO3cu4E41q4bwV+xty0rFv8yi2Nd4585U/Kq0IJnk1xjUp7ZTWr3BTD9ZldF8Oof9AP3+31jvYX49wkj8kTskNC8oockbfkmAwJ9157My/3Lv3P/hf/q/+tLfW9Rc8jshL+91/5zen5</latexit><latexit sha1_base64="TSQczVbIEUgfpzVbrGceKD3sGfA=">AAADBHicjZFNb9NAEIbXLh8lfKXlyGVEBCoqRLZViVZRpSIuHItEaKVsiNabcbrteu3urlEjy2d+DSfElf+B+DOs7YCSFCFGsvzuMzPv2jNxLoWxQfDD8zdu3Lx1e/NO5+69+w8edre2P5is0ByHPJOZPo2ZQSkUDq2wEk9zjSyNJZ7EF2/q/Mkn1EZk6r2d5zhO2UyJRHBmHZp0f1JTpJNSHIbVx1JVQF9AS87hEMSuo6phiWa8jKry/GUN4RnQAR24kuYVAfzLJqpZ2/fHKnRW7kg7v62oxFZApJZ6l/z+p28AO1QqULAL4fNJtxf0gybguggXokcWcTzZ8rbpNONFispyyYwZhUFuxyXTVnCJVYcWBnPGL9gMR04qlqIZl80WKnjqyBSSTLtHWWjockfJUmPmaewqU2bPzHquhn/LjQqb7I9LofLCouLtRUkhwWZQrxSmQiO3cu4E41q4bwV+xty0rFv8yi2Nd4585U/Kq0IJnk1xjUp7ZTWr3BTD9ZldF8Oof9AP3+31jvYX49wkj8kTskNC8oockbfkmAwJ9157My/3Lv3P/hf/q/+tLfW9Rc8jshL+91/5zen5</latexit>

all pairs i and j

n�

i=1

n�

j=i+1

2

j � i � 1= 2

n�

i=1

n�i+1�

j=2

1

j

� 2nn�

j=1

1

j

� 2n

� n

x=1

1

xdx

= 2n ln n

n�

i=1

n�

j=i+1

2

j � i � 1= 2

n�

i=1

n�i+1�

j=2

1

j

� 2nn�

j=1

1

j

� 2n

� n

x=1

1

xdx

= 2n ln n

n�

i=1

n�

j=i+1

2

j � i � 1= 2

n�

i=1

n�i+1�

j=2

1

j

� 2nn�

j=1

1

j

� 2n (ln n + 1)<latexit sha1_base64="GjZqOMHt99byYg9EpBf0uRtS8Xc=">AAADBHicjZFdb9MwFIad8DXKx7pxyc0RFWhotIojJDZVk4a44XJIlE2qS+W4TufNcYLtTKusXO/X7Apxy/9A/BmctIi2Q4gjRXn9nHNeJ+ckhRTGRtGPILx1+87dexv3Ww8ePnq82d7a/mTyUjM+YLnM9UlCDZdC8YEVVvKTQnOaJZIfJ+fv6vzxBddG5OqjnRV8lNGpEqlg1Ho0bv8kpszGThzg6rNTFZBXMCdncABi11PVsFRT5uLKnXVFF1fwAkif9H1J84oB/mUT16xbm/2xwt7KH0nrtxWRfC4gVku9S37/09eHHSIVKNgF/HLc7kS9qAm4KfBCdNAijsZbwTaZ5KzMuLJMUmOGOCrsyFFtBZO8apHS8IKyczrlQy8VzbgZuWYLFTz3ZAJprv2jLDR0ucPRzJhZlvjKjNpTs56r4d9yw9KmeyMnVFFartj8orSUYHOoVwoToTmzcuYFZVr4bwV2Sv20rF/8yi2Nd8HZyp+4y1IJlk/4GpX20mpa+Sni9ZndFIO4t9/DH153DvcW49xAT9EztIMweoMO0Xt0hAaIBW+DaVAEX8Kr8Dr8Gn6bl4bBoucJWonw+y/+pen7</latexit><latexit sha1_base64="GjZqOMHt99byYg9EpBf0uRtS8Xc=">AAADBHicjZFdb9MwFIad8DXKx7pxyc0RFWhotIojJDZVk4a44XJIlE2qS+W4TufNcYLtTKusXO/X7Apxy/9A/BmctIi2Q4gjRXn9nHNeJ+ckhRTGRtGPILx1+87dexv3Ww8ePnq82d7a/mTyUjM+YLnM9UlCDZdC8YEVVvKTQnOaJZIfJ+fv6vzxBddG5OqjnRV8lNGpEqlg1Ho0bv8kpszGThzg6rNTFZBXMCdncABi11PVsFRT5uLKnXVFF1fwAkif9H1J84oB/mUT16xbm/2xwt7KH0nrtxWRfC4gVku9S37/09eHHSIVKNgF/HLc7kS9qAm4KfBCdNAijsZbwTaZ5KzMuLJMUmOGOCrsyFFtBZO8apHS8IKyczrlQy8VzbgZuWYLFTz3ZAJprv2jLDR0ucPRzJhZlvjKjNpTs56r4d9yw9KmeyMnVFFartj8orSUYHOoVwoToTmzcuYFZVr4bwV2Sv20rF/8yi2Nd8HZyp+4y1IJlk/4GpX20mpa+Sni9ZndFIO4t9/DH153DvcW49xAT9EztIMweoMO0Xt0hAaIBW+DaVAEX8Kr8Dr8Gn6bl4bBoucJWonw+y/+pen7</latexit><latexit sha1_base64="GjZqOMHt99byYg9EpBf0uRtS8Xc=">AAADBHicjZFdb9MwFIad8DXKx7pxyc0RFWhotIojJDZVk4a44XJIlE2qS+W4TufNcYLtTKusXO/X7Apxy/9A/BmctIi2Q4gjRXn9nHNeJ+ckhRTGRtGPILx1+87dexv3Ww8ePnq82d7a/mTyUjM+YLnM9UlCDZdC8YEVVvKTQnOaJZIfJ+fv6vzxBddG5OqjnRV8lNGpEqlg1Ho0bv8kpszGThzg6rNTFZBXMCdncABi11PVsFRT5uLKnXVFF1fwAkif9H1J84oB/mUT16xbm/2xwt7KH0nrtxWRfC4gVku9S37/09eHHSIVKNgF/HLc7kS9qAm4KfBCdNAijsZbwTaZ5KzMuLJMUmOGOCrsyFFtBZO8apHS8IKyczrlQy8VzbgZuWYLFTz3ZAJprv2jLDR0ucPRzJhZlvjKjNpTs56r4d9yw9KmeyMnVFFartj8orSUYHOoVwoToTmzcuYFZVr4bwV2Sv20rF/8yi2Nd8HZyp+4y1IJlk/4GpX20mpa+Sni9ZndFIO4t9/DH153DvcW49xAT9EztIMweoMO0Xt0hAaIBW+DaVAEX8Kr8Dr8Gn6bl4bBoucJWonw+y/+pen7</latexit><latexit sha1_base64="GjZqOMHt99byYg9EpBf0uRtS8Xc=">AAADBHicjZFdb9MwFIad8DXKx7pxyc0RFWhotIojJDZVk4a44XJIlE2qS+W4TufNcYLtTKusXO/X7Apxy/9A/BmctIi2Q4gjRXn9nHNeJ+ckhRTGRtGPILx1+87dexv3Ww8ePnq82d7a/mTyUjM+YLnM9UlCDZdC8YEVVvKTQnOaJZIfJ+fv6vzxBddG5OqjnRV8lNGpEqlg1Ho0bv8kpszGThzg6rNTFZBXMCdncABi11PVsFRT5uLKnXVFF1fwAkif9H1J84oB/mUT16xbm/2xwt7KH0nrtxWRfC4gVku9S37/09eHHSIVKNgF/HLc7kS9qAm4KfBCdNAijsZbwTaZ5KzMuLJMUmOGOCrsyFFtBZO8apHS8IKyczrlQy8VzbgZuWYLFTz3ZAJprv2jLDR0ucPRzJhZlvjKjNpTs56r4d9yw9KmeyMnVFFartj8orSUYHOoVwoToTmzcuYFZVr4bwV2Sv20rF/8yi2Nd8HZyp+4y1IJlk/4GpX20mpa+Sni9ZndFIO4t9/DH153DvcW49xAT9EztIMweoMO0Xt0hAaIBW+DaVAEX8Kr8Dr8Gn6bl4bBoucJWonw+y/+pen7</latexit>

harmonic sum

▪

Tony Hoare

独Invented quicksort to translate Russian into English.

独[ but couldn’t explain his algorithm or implement it! ]

独Learned Algol 60 (and recursion).

独Implemented quicksort.

38

Tony Hoare1980 Turing Award

4

A L G O R I T H M 61 P R O C E D U R E S F O R R A N G E A R I T H M E T I C ALLAN GIBB* U n i v e r s i t y of A l b e r t a , C a l g a r y , A l b e r t a , C a n a d a

b e g i n p r o c e d u r e RANGESUM (a, b, c, d, e, f);

rea l a , b , c , d , e , f ; c o m m e n t The term "range number" was used by P. S. Dwyer, Linear Computations (Wiley, 1951). Machine procedures for range ari thmetic were developed about 1958 by Ramon Moore, "Automatic Error Analysis in Digital Computa t ion ," LMSD Report 48421, 28 Jan. 1959, Lockheed Missiles and Space Divi- sion, Palo Alto, California, 59 pp. If a _< x -< b and c ~ y ~ d, then RANGESUM yields an interval [e, f] such tha t e =< (x + y)

f. Because of machine operation (truncation or rounding) the machine sums a -4- c and b -4- d may not provide safe end-points of the output interval. Thus RANGESUM requires a non-local real procedure ADJUSTSUM which will compensate for the machine ari thmetic. The body of ADJUSTSUM will be de- pendent upon the type of machine for which it is wri t ten and so is not given here. (An example, however, appears below.) I t is assumed tha t ADJUSTSUM has as parameters real v and w, and integer i, and is accompanied by a non-local real procedure CORRECTION which gives an upper bound to the magnitude of the error involved in the machine representat ion of a number. The output ADJUSTSUM provides the left end-point of the output interval of RANGESUM when ADJUSTSUM is called with i = --1, and the right end-point when called with i = 1 The procedures RANGESUB, RANGEMPY, and RANGEDVD provide for the remaining fundamental operations in range ari thmetic. RANGESQR gives an interval within which the square of a range nmnber must lie. RNGSUMC, RNGSUBC, RNGMPYC and RNGDVDC provide for range ari thmetic with complex range arguments, i.e. the real and imaginary parts are range numbers~ b e g i n

e := ADJUSTSUM (a, c, - 1 ) ; f : = ADJUSTSUM (b, d, 1)

end RANGESUM; p r o c e d u r e RANGESUB (a, b, c, d, e, f) ;

real a, b ,c , d ,e , f; c o m m e n t RANGESUM is a non-local procedure; b e g i n

RANGESUM (a, b, - d , --c, e, f) e n d RANGESUB ; p r o c e d u r e RANGEMPY (a, b, c, d, e, f);

real a, b, c, d, e, f; c o m m e n t ADJUSTPROD, which appears at the end of this procedure, is analogous to ADJUSTSUM above and is a non- local real procedure. MAX and MIN find the maximum and minimum of a set of real numbers and are non-local; b e g i n

rea l v, w; i f a < 0 A c => 0 t h e n

1: b e g i n v : = c ; c : = a ; a : = v ; w : = d ; d : = b ; b : = w

end 1; i f a => O t h e n

2: b e g i n i f c >= 0 t h e n

3 :beg in e : = a X e ; f := b X d ; g o t o 8

en d 3 ; e : = b X c ; i f d ~ 0 t h e n

4: b e g i n f : = b X d ; g o t o 8

end 4; f : = a X d ; g o t o 8

5: en d 2; i f b > 0 t h e n

6: b e g i n i f d > 0 t h e n b e g i n

e := MIN(a X d, b X c); f : = MAX(a X c , b X d); go t o 8

e n d 6; e : = b X c; f : = a X c; go t o 8

end 5; f : = a X c ; i f d _-< O t h e n

7: b e g i n e : = b X d ; g o t o 8

end 7 ; e : = a X d ;

8: e : = ADJUSTPROD (e, - 1 ) ; f := ADJUSTPROD (f, 1)

en d RANGEMPY; p r o c e d u r e RANGEDVD (a, b, c, d, e, f) ;

real a, b, c, d, e, f; c o m m e n t If the range divisor includes zero the program exists to a non-local label "zerodvsr" . RANGEDVD assumes a non-local real procedure ADJUSTQUOT which is analogous (possibly identical) to ADJUSTPROD; b e g i n

i f c =< 0 A d ~ 0 t h e n go to zer0dvsr; i f c < 0 t h e n

1: b e g i n i f b > 0 t h e n

2: b e g i n e : = b /d ; go t o 3

e n d 2; e : = b /c ;

3: i f a -->_ 0 t h e n 4: b e g i n

f : = a /c ; go to 8 e n d 4; f : = a /d ; go to 8

e nd 1 ; i f a < 0 t h e n

5: b e g i n e : = a/c; go t o 6

e nd 5 ; e : = a /d ;

6: i f b > 0 t h e n 7: b e g i n

f : = b/c ; go t o 8 e n d 7 ; f : = b /d ;

8: e := ADJUSTQUOT (e, - 1 ) ; f : = ADJUSTQUOT (f,1) end RANGEDVD ; p r o c e d u r e RANGESQR (a, b, e, f);

rea l a, b, e, f; c o m m e n t ADJUSTPROD is a non-10cal procedure; b e g i n

i f a < 0 t h e n

C o m m u n i c a t i o n s o f t h e &CM 319

n u m b e r ) . 9.9 X 10 45 is u sed to r e p r e s e n t inf in i ty . I m a g i n a r y v a l u e s of x m a y no t be n e g a t i v e a n d reM v a l u e s of x m a y n o t be s m a l l e r t h a n 1.

Va lues of Qd~'(x) m a y be ca l cu l a t ed eas i ly by h y p e r g e o m e t r i c ser ies if x is n o t too sma l l no r (n - m) too large. Q~m(x) can be c o m p u t e d f rom an a p p r o p r i a t e se t of v a l u e s of Pnm(X) if X is nea r 1.0 or ix is nea r 0. Loss of s ign i f i can t d ig i t s occurs for x as sma l l as 1.1 if n is l a rge r t h a n 10. Loss of s ign i f i can t d ig i t s is a m a j o r diffi- cu l t y in u s i n g finite p o l y n o m i M r e p r e s e n t a t i o n s also if n is l a rge r t h a n m. Howeve r , Q L E G h a s been t e s t e d in reg ions of x a n d n b o t h large a n d smal l ; p r o c e d u r e Q L E G ( m , n m a x , x, ri, R, Q); v a l u e In, n m a x , x, ri ;

r e a l In, m n a x , x, ri ; r e a l a r r a y R , Q; b e g i n r e a l t , i, n, q0, s ;

n : = 20; i f n m a x > 13 t h e n

n : = n m a x + 7 ; i f ri = 0 t h e n

b e g i n i f m = 0 t h e n Q[0] : = 0.5 X 10g((x + 1 ) / (x - 1)) e l s e

b e g i n t : = - - 1 . 0 / s q r t ( x X x - - 1); q0 : = 0; Q[O] : = t ; fo r i : = 1 s t e p 1 u n t i l m d o

b e g i n s : = ( x + x ) X ( i - 1 ) X t ×Q [ 0 ] + ( 3 i - i× i - 2 )×q 0 ; q0 : = Q[0]; Q[0] : = s e n d e n d ;

i f x = 1 t h e n Q[0] : = 9.9 I" 45;

R[n + 1] : = x - s q r t ( x X x - 1); for i : = n s t e p --1 u n t i l 1 d o

R[i] : = (i + m ) / ( ( i + i + 1) X x + ( m - i - 1) X R [ i + l ] ) ;

go to t h e e n d ; i f m = 0 t h e n

b e g i n i f x < 0.5 t b e n Q[0] : = a r c t a n ( x ) - 1.5707963 e l s e Q[0] : = - a r e t a n ( 1 / x ) e n d e l s e

b e g i n t : = 1 / s q r t ( x X x + 1); q0 : = 0; q[0] := t; f o r i : = 2 s t e p 1 u n t i l m do

b e g i n s : = (x + x) X (i -- 1) X t X Q[0I + ( 3 i + i X i -- 2) × q0; qO : = Q[0]; Q[0] := s e n d e n d ;

R[n + 1] : = x - s q r t ( x × x + 1); for i : = n s t e p - 1 u n t i l 1 do

R[i] : = (i + m ) / ( ( i -- m + 1) × R[i + 1] - - ( i + i + 1) X x);

f o r i : = 1 s t e p 2 u n t i l n m a x do Ril l : = -- Ri l l ;

t h e : f o r i : = 1 s t e p 1 u n t i l n n m x d o Q[i] : = Q[i - 1] X R[i]

e n d Q L E G ;

* T h i s p r o c e d u r e was deve loped in p a r t u n d e r t he s p o n s o r s h i p of t he Air Force C a m b r i d g e R e s e a r c h C en t e r .

ALGORITHM 63 PARTITION C. A. R. HOARE Elliott Brothers Ltd., Borehamwood, Hertfordshire, Eng. p r o c e d u r e p a r t i t i o n ( A , M , N , I , J ) ; v a l u e M , N ;

a r r a y A; i n t e g e r M , N , 1 , J ;

c o n u n e n t I and J are o u t p u t va r i ab le s , a n d A is t h e a r r a y (wi th s u b s c r i p t b o u n d s M : N ) wh i ch is o p e r a t e d u p o n by th i s p rocedure . P a r t i t i o n t a k e s t h e va l ue X of a r a n d o m e l e m e n t of the a r r a y A, a n d r e a r r a n g e s t he va lue s of t he e l e m e n t s of t he a r r a y in s u c h a way t h a t t he r e ex is t i n t ege r s I a n d J w i t h t he fo l lowing p ro p e r t i e s :

M _-< J < I =< N p r o v i d e d M < N A[R] =< X f o r M =< R _-< J A[R] = X f o r J < R < I A[R] ~ X f o r I =< R ~ N

T h e p roce du re uses an in tege r p rocedu re r a n d o m (M,N) wh ich chooses e q u i p r o b a b l y a r a n d o m in t ege r F b e t w e e n M an d N, a n d also a p roc e du re exchange , wh ich e x c h a n g e s t he v a lu e s of i t s two p a r a m e t e r s ; b e g i n r e a l X ; i n t e g e r F;

F : = r a n d o m ( M , N ) ; X : = A[F]; I : = M ; J : = N ;

up : for I : = I s t e p 1 u n t i l N d o i f X < A [I] t h e n g o to do wn ;

I : = N ; down: f o r J : = J s t e p --1 u n t i l M d o

i f A [ J ] < X t h e n g o t o c h a n g e ; J : = M ;

c hange : i f I < J t h e n b e g i n e x c h a n g e (A[IL A[J]) ; I : = I + 1 ; J : = J - 1; g o to up

e n d e l s e i f [ < F t h e n b e g i n e x c h a n g e (A[IL A[F]) i

I : = I + l e n d

e l s e i f F < J t l l e n b e g i n e x c h a n g e (A[F], A[J]) ; J : = J - 1

e n d ; e n d p a r t i t i o n

ALGORITHM 64 QUICKSORT C. A. R. HOARE Elliott Brothers Ltd., Borehamwood, Hertfordshire, Eng.

p r o c e d u r e q u i c k s o r t ( A , M , N ) ; v a l u e M , N ; a r r a y A; i n t e g e r M , N ;

c o m m e n t Q u i c k s o r t is a v e r y f a s t a n d c o n v e n i e n t m e t h o d of s o r t i n g an a r r a y in t he r a n d o m - a c c e s s s tore of a c o m p u t e r . T h e en t i r e c o n t e n t s of t he s tore m a y be so r t ed , s ince no e x t r a space is r equ i red . T h e a ve rage n u m b e r of c o m p a r i s o n s m a d e is 2 ( M - - N ) In ( N - - M ) , a n d t he ave r age n m n b e r of e x c h a n g e s is one s ix th th i s a m o u n t . Su i t ab le r e f inemen t s of th i s m e t h o d will be des i rab le for i t s i m p l e m e n t a t i o n on any ac tua l c o m p u t e r ; b e g i n i n t e g e r 1,J ;

i f M < N t h e n b e g i n p a r t i t i o n ( A , M , N , I , J ) ; q u i c k s o r t (A,M,J ) ; q u i c k s o r t (A, I, N)

e n d e n d q u i e k s o r t

ALGORITHM 65 FIND C. A. R. HOARE Elliott Brothers Ltd., Borehamwood, Hertfordshire, Eng. p r o c e d u r e f ind ( A , M , N , K ) ; v a l u e M , N , K ;

a r r a y A; i n t e g e r M , N , K ; c o m m e n t F i n d will a s s ign to A [K] t he va l u e wh ich it would h a v e if t he a r r a y A [M:N] h a d been sor ted . T h e a r r a y A will be p a r t l y so r t ed , a n d s u b s e q u e n t en t r i e s will be f a s t e r t h a n t h e f i rs t ;

C o m m u n i c a t i o n s o f t h e A C M 321

Communications of the ACM (July 1961)

NUTS AND BOLTS

Problem. A disorganized carpenter has a mixed pile of n nuts and n bolts.

独The goal is to find the corresponding pairs of nuts and bolts.

独Each nut fits exactly one bolt and each bolt fits exactly one nut.

独By fitting a nut and a bolt together, the carpenter can see which one is

bigger (but cannot directly compare either two nuts or two bolts).

Brute-force solution. Compare each bolt to each nut—Θ (n2) compares.

Challenge. Design an algorithm that makes O(n log n) compares.39


‣ mergesort





SECTION 9.3

Median and selection problems

Selection. Given n elements from a totally ordered universe, find kth smallest.

独Minimum: k = 1; maximum: k = n.

独Median: k = ⎣(n + 1) / 2⎦.

独O(n) compares for min or max.

独O(n log n) compares by sorting.

独O(n log k) compares with a binary heap.

Applications. Order statistics; find the “top k”; bottleneck paths, …

Q. Can we do it with O(n) compares?

A. Yes! Selection is easier than sorting.

43

max heap with k smallest

Randomized quickselect



独Recur in one subarray—the one containing the kth smallest element.

44

QUICK-SELECT(A, k) _____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Pick pivot p ∈ A uniformly at random.


IF (k ≤ | L |) RETURN QUICK-SELECT(L, k).

ELSE IF (k > | L | + | M |) RETURN QUICK-SELECT(R, k – | L | – | M |)

ELSE IF (k = | L |) RETURN p._____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

T (i)

T (n − i − 1)

Θ(n)

Randomized quickselect analysis

Intuition. Split candy bar uniformly ⇒ expected size of larger piece is ¾.

Def. T(n, k) = expected # compares to select kth smallest in array of length ≤ n.

Def. T(n) = maxk T(n, k).

Proposition. T(n) ≤ 4 n. Pf. [ by strong induction on n ]

独Assume true for 1, 2, …, n – 1.

独T(n) satisfies the following recurrence:

45

T(n) ≤ T(3 n / 4) + n ⇒ T(n) ≤ 4 n

T(n) ≤ n + 1 / n [ 2T(n / 2) + … + 2T(n – 3) + 2T(n – 2) + 2T(n – 1) ]

≤ n + 1 / n [ 8(n / 2) + … + 8(n – 3) + 8(n – 2) + 8(n – 1) ]

≤ n + 1 / n (3n2)

= 4 n. ▪

can assume we always recur of larger of two subarrays since T(n)

is monotone non-decreasing

tiny cheat: sum should start at T(⎣n/2⎦)


not rigorous: can’t assume E[T(i)] ≤ T(E[i])

Selection in worst-case linear time

Goal. Find pivot element p that divides list of n elements into two pieces so

that each piece is guaranteed to have ≤ 7/10 n elements.

Q. How to find approximate median in linear time?

A. Recursively compute median of sample of ≤ 2/10 n elements.

46

Θ(1) if n = 1T (7/10 n) + T (2/10 n) + Θ(n) otherwiseT(n) =

two subproblems of different sizes!

⇒ T(n) = Θ(n)

we’ll need to show this

Choosing the pivot element

独Divide n elements into ⎣n / 5⎦ groups of 5 elements each (plus extra).

47

1029 3738 2 1855 24 3534 36

4422 1152 53 1312 43 420 27

2328 266 40 119 46 4931 8

914 35 54 4830 47 5132 21

3945 1550 25 4116 17 722

n = 54



独Find median of each group (except extra).

1029 3738 2 1855 24 3534 36

4422 1152 53 1312 43 420 27

2328 266 40 119 46 4931 8

914 35 54 4830 47 5132 21

3945 1550 25 4116 17 722

38 18 35

43

2328 40 19 31

15

48

medians

n = 54



独Find median of each group (except extra).

独Find median of ⎣n / 5⎦ medians recursively.

独Use median-of-medians as pivot element.

1029 3738 2 1855 24 3534 36

4422 1152 53 1312 43 420 27

2328 266 40 119 46 4931 8

914 35 54 4830 47 5132 21

3945 1550 25 4116 17 722

medians

49

median of medians 38 18 35

43

2328 40 19 31

15

28

n = 54

Median-of-medians selection algorithm

50

MOM-SELECT(A, k) _____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

n ← | A |.

IF (n < 50)

RETURN kth smallest of element of A via mergesort.

Group A into ⎣n / 5⎦ groups of 5 elements each (ignore leftovers).

B ← median of each group of 5.

p ← MOM-SELECT(B, ⎣n / 10⎦)


IF (k ≤ | L |) RETURN MOM-SELECT(L, k).

ELSE IF (k > | L | + | M |) RETURN MOM-SELECT(R, k – | L | – | M |)

ELSE RETURN p._____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

median of medians

38 35

43

40 31

Analysis of median-of-medians selection algorithm

独At least half of 5-element medians ≤ p.

51

1029 37 2 1855 24 34 36

4422 1152 53 1312 420 27

2328 266 119 46 49 8

914 35 54 4830 47 5132 21

3945 1550 25 4116 17 722

median of medians p 38 18 35

43

2328 40 19 31

15

28

n = 54

medians



独At least ⎣⎣n / 5⎦ / 2⎦ = ⎣n / 10⎦ medians ≤ p.

52

1029 3738 2 1855 24 3534 36

4422 1152 53 1312 43 420 27

2328 266 40 119 46 4931 8

914 35 54 4830 47 5132 21

3945 1550 25 4116 17 722

18

2328 19

15

28

n = 54

median of medians p

medians ≤ p



独At least ⎣⎣n / 5⎦ / 2⎦ = ⎣n / 10⎦ medians ≤ p.

独At least 3 ⎣n / 10⎦ elements ≤ p.

53n = 54

1029 3738 2 1855 24 3534 36

4422 1152 53 1312 43 420 27

2328 266 40 119 46 4931 8

914 35 54 4830 47 5132 21

3945 1550 25 4116 17 722

18

2328 19

15

28

14

16

3

10

11 13

9

1

1222

median of medians p

medians ≤ p

独At least half of 5-element medians ≥ p.

18

23 19

15

18

23 19

15


54

1029 3738 2 55 24 3534 36

4422 1152 53 1312 43 420 27

28 266 40 1 46 4931 8

914 35 54 4830 47 5132 21

3945 50 25 4116 17 722

38 35

43

28 40 3128

n = 54

median of medians p

medians



独At least ⎣⎣n / 5⎦ / 2⎦ = ⎣n / 10⎦ medians ≥ p.

55

1029 3738 2 1855 24 3534 36

4422 1152 53 1312 43 420 27

2328 266 40 119 46 4931 8

914 35 54 4830 47 5132 21

3945 1550 25 4116 17 722

38 35

43

28 40 3128

n = 54

median of medians p

medians ≥ p



独At least ⎣⎣n / 5⎦ / 2⎦ = ⎣n / 10⎦ medians ≥ p.

独At least 3 ⎣n / 10⎦ elements ≥ p.

56

1029 3738 2 1855 24 3534 36

4422 1152 53 1312 43 420 27

2328 266 40 119 46 4931 8

914 35 54 4830 47 5132 21

3945 1550 25 4116 17 722

38 35

43

28 40 3128

5147

52

29 34

46 49

53

54

45 50

32

n = 54

median of medians p

medians ≥ p

18

23 19

15

Median-of-medians selection algorithm recurrence

Median-of-medians selection algorithm recurrence.

独Select called recursively with ⎣n / 5⎦ elements to compute MOM p.



独Select called recursively with at most n – 3 ⎣n / 10⎦ elements.

Def. C(n) = max # compares on any array of n elements.

Intuition.

独C(n) is going to be at least linear in n ⇒ C(n) is super-additive.

独Ignoring floors, this implies that C(n) ≤ C(n / 5 + n − 3n / 10) + 11/5 n = C(9n / 10) + 11/5 n

⇒ C(n) ≤ 22n.57

median of medians

recursive select

computing median of 5(≤ 6 compares per group)

partitioning(≤ n compares)

C(n) � C (�n/5�) + C (n � 3�n/10�) + 115 n


Median-of-medians selection algorithm recurrence.

独Select called recursively with ⎣n / 5⎦ elements to compute MOM p.



独Select called recursively with at most n – 3 ⎣n / 10⎦ elements.

Def. C(n) = max # compares on any array of n elements.

Now, let’s solve given recurrence.

独Assume n is both a power of 5 and a power of 10 ?

独Prove that C(n) is monotone non-decreasing.

58

median of medians

recursive select

computing median of 5(≤ 6 compares per group)

partitioning(≤ n compares)

C(n) � C (�n/5�) + C (n � 3�n/10�) + 115 n

Consider the following recurrenceIs C(n) monotone non-decreasing?

A. Yes, obviously.

B. Yes, but proof is tedious.

C. Yes, but proof is hard.

D. No.

59


C(19) = 165 C(20) = 134.2

C(n) =

��

�

0 B7 n � 1

C(�n/5�) + C(n � 3�n/10�) + 115 n B7 n > 1

<latexit sha1_base64="Y96HaCEvN4aOrD6hbv5tphcOY0Y=">AAAC0nicbVHLbtNAFB2bVwmPpmXJ5ooIlAoRbCCiKKJUyoZlkTCtlLGi8fg6HXU8tmbGKJGVBWLLj/BJ/A1j1xJNwl0dnXOf5yalFMYGwR/Pv3X7zt17e/d7Dx4+erzfPzj8ZopKc4x4IQt9kTCDUiiMrLASL0qNLE8knidX00Y//47aiEJ9tasS45wtlMgEZ9ZR8/7v6VAdAZ18pJMeTXAhVM1dO7Pu0UkAL4BaXFqoQWSwBgVUIoRA6SzEPHYp0yGVmSwKDer1GKhu8RG8BNcWXsFb+CeHwU2dZprxOgzX9bjpuzPopB3To6jSbqF5fxCMgjZgF4QdGJAuzuYH3iFNC17lqCyXzJhZGJQ2rpm2gkt0F1YGS8av2AJnDiqWo4nr1tM1PHdMCplbPSuUhZa9WVGz3JhVnrjMnNlLs6015P+0WWWz47gWqqwsKn49KKsk2AKaB0EqNHIrVw4wroXbFfglc25Z98aNKW3vEvnGJfWyUoIXKW6x0i6tZo2L4bZnuyB6M/owCr+8G5wed3bukafkGRmSkLwnp+QzOSMR4d6+N/ZOvE9+5Nf+D//ndarvdTVPyEb4v/4Ck0DZRw==</latexit><latexit sha1_base64="Y96HaCEvN4aOrD6hbv5tphcOY0Y=">AAAC0nicbVHLbtNAFB2bVwmPpmXJ5ooIlAoRbCCiKKJUyoZlkTCtlLGi8fg6HXU8tmbGKJGVBWLLj/BJ/A1j1xJNwl0dnXOf5yalFMYGwR/Pv3X7zt17e/d7Dx4+erzfPzj8ZopKc4x4IQt9kTCDUiiMrLASL0qNLE8knidX00Y//47aiEJ9tasS45wtlMgEZ9ZR8/7v6VAdAZ18pJMeTXAhVM1dO7Pu0UkAL4BaXFqoQWSwBgVUIoRA6SzEPHYp0yGVmSwKDer1GKhu8RG8BNcWXsFb+CeHwU2dZprxOgzX9bjpuzPopB3To6jSbqF5fxCMgjZgF4QdGJAuzuYH3iFNC17lqCyXzJhZGJQ2rpm2gkt0F1YGS8av2AJnDiqWo4nr1tM1PHdMCplbPSuUhZa9WVGz3JhVnrjMnNlLs6015P+0WWWz47gWqqwsKn49KKsk2AKaB0EqNHIrVw4wroXbFfglc25Z98aNKW3vEvnGJfWyUoIXKW6x0i6tZo2L4bZnuyB6M/owCr+8G5wed3bukafkGRmSkLwnp+QzOSMR4d6+N/ZOvE9+5Nf+D//ndarvdTVPyEb4v/4Ck0DZRw==</latexit><latexit sha1_base64="Y96HaCEvN4aOrD6hbv5tphcOY0Y=">AAAC0nicbVHLbtNAFB2bVwmPpmXJ5ooIlAoRbCCiKKJUyoZlkTCtlLGi8fg6HXU8tmbGKJGVBWLLj/BJ/A1j1xJNwl0dnXOf5yalFMYGwR/Pv3X7zt17e/d7Dx4+erzfPzj8ZopKc4x4IQt9kTCDUiiMrLASL0qNLE8knidX00Y//47aiEJ9tasS45wtlMgEZ9ZR8/7v6VAdAZ18pJMeTXAhVM1dO7Pu0UkAL4BaXFqoQWSwBgVUIoRA6SzEPHYp0yGVmSwKDer1GKhu8RG8BNcWXsFb+CeHwU2dZprxOgzX9bjpuzPopB3To6jSbqF5fxCMgjZgF4QdGJAuzuYH3iFNC17lqCyXzJhZGJQ2rpm2gkt0F1YGS8av2AJnDiqWo4nr1tM1PHdMCplbPSuUhZa9WVGz3JhVnrjMnNlLs6015P+0WWWz47gWqqwsKn49KKsk2AKaB0EqNHIrVw4wroXbFfglc25Z98aNKW3vEvnGJfWyUoIXKW6x0i6tZo2L4bZnuyB6M/owCr+8G5wed3bukafkGRmSkLwnp+QzOSMR4d6+N/ZOvE9+5Nf+D//ndarvdTVPyEb4v/4Ck0DZRw==</latexit><latexit sha1_base64="Y96HaCEvN4aOrD6hbv5tphcOY0Y=">AAAC0nicbVHLbtNAFB2bVwmPpmXJ5ooIlAoRbCCiKKJUyoZlkTCtlLGi8fg6HXU8tmbGKJGVBWLLj/BJ/A1j1xJNwl0dnXOf5yalFMYGwR/Pv3X7zt17e/d7Dx4+erzfPzj8ZopKc4x4IQt9kTCDUiiMrLASL0qNLE8knidX00Y//47aiEJ9tasS45wtlMgEZ9ZR8/7v6VAdAZ18pJMeTXAhVM1dO7Pu0UkAL4BaXFqoQWSwBgVUIoRA6SzEPHYp0yGVmSwKDer1GKhu8RG8BNcWXsFb+CeHwU2dZprxOgzX9bjpuzPopB3To6jSbqF5fxCMgjZgF4QdGJAuzuYH3iFNC17lqCyXzJhZGJQ2rpm2gkt0F1YGS8av2AJnDiqWo4nr1tM1PHdMCplbPSuUhZa9WVGz3JhVnrjMnNlLs6015P+0WWWz47gWqqwsKn49KKsk2AKaB0EqNHIrVw4wroXbFfglc25Z98aNKW3vEvnGJfWyUoIXKW6x0i6tZo2L4bZnuyB6M/owCr+8G5wed3bukafkGRmSkLwnp+QzOSMR4d6+N/ZOvE9+5Nf+D//ndarvdTVPyEb4v/4Ck0DZRw==</latexit>


Analysis of selection algorithm recurrence.

独T(n) = max # compares on any array of ≤ n elements.

独T(n) is monotone non-decreasing, but C(n) is not!

Claim. T(n) ≤ 44 n.

Pf. [ by strong induction ]

独Base case: T(n) ≤ 6 n for n < 50 (mergesort).

独Inductive hypothesis: assume true for 1, 2, …, n – 1.

独Induction step: for n ≥ 50, we have either T(n) ≤ T(n – 1) ≤ 44 n or

60

T(n) ≤ T(⎣n / 5⎦) + T(n – 3 ⎣n / 10⎦) + 11/5 n

≤ 44 (⎣n / 5⎦) + 44 (n – 3 ⎣n / 10⎦) + 11/5 n

= 44 n. ▪≤ 44 (n / 5) + 44 n – 44 (n / 4) + 11/5 n for n ≥ 50, 3 ⎣n / 10⎦ ≥ n / 4


T (n) �

��

�

6n B7 n < 50

max{ T (n � 1), T (�n/5�) + T (n � 3�n/10�) + 115 n) } B7 n � 50

<latexit sha1_base64="uCnTptmZk5k2uvfF1msqYvcMsiY=">AAAC9HicbVFNj9MwEHXC11K+usuRy4gK1Aq2xEBhUTmsxIXjIrV0pboqjjvpWus4ke1Aq6g/hRPiyh/hF/BvcNJKbFvm9PTeeObNc5wraV0U/QnCa9dv3Lx1cLtx5+69+w+ah0efbVYYgUORqcycx9yikhqHTjqF57lBnsYKR/Hlh0offUVjZaYHbpnjJOVzLRMpuPPUtPl70NYdYH2mkPUbLMa51KXwA+2qwfpvNDwF5nDhoASZwAo0vIdeBIyNKaYT3wIs5QtgpZ8BftYx7Tz3cNBmKlFZZkC/6AEzNe7As6oFjuEV/JNpdFVnieGipHRV9vyyyhmw1b4JNse1jQZDPdsYnjZbUTeqC/YB3YAW2dTZ9DA4YrNMFClqJxS3dkyj3E1KbpwUCn0ChcWci0s+x7GHmqdoJ2Wd+gqeeGYGiT8iybSDmr36ouSptcs09p0pdxd2V6vI/2njwiUnk1LqvHCoxXpRUihwGVRfCDNpUDi19IALI71XEBfc5+b8R29tqWfnKLYuKReFliKb4Q6r3MIZXqVIdzPbB8OX3Xdd+ul16/RkE+cBeUQekzah5C05JR/JGRkSEdBgFHwJePgt/B7+CH+uW8Ng8+Yh2arw118pF+RQ</latexit><latexit sha1_base64="uCnTptmZk5k2uvfF1msqYvcMsiY=">AAAC9HicbVFNj9MwEHXC11K+usuRy4gK1Aq2xEBhUTmsxIXjIrV0pboqjjvpWus4ke1Aq6g/hRPiyh/hF/BvcNJKbFvm9PTeeObNc5wraV0U/QnCa9dv3Lx1cLtx5+69+w+ah0efbVYYgUORqcycx9yikhqHTjqF57lBnsYKR/Hlh0offUVjZaYHbpnjJOVzLRMpuPPUtPl70NYdYH2mkPUbLMa51KXwA+2qwfpvNDwF5nDhoASZwAo0vIdeBIyNKaYT3wIs5QtgpZ8BftYx7Tz3cNBmKlFZZkC/6AEzNe7As6oFjuEV/JNpdFVnieGipHRV9vyyyhmw1b4JNse1jQZDPdsYnjZbUTeqC/YB3YAW2dTZ9DA4YrNMFClqJxS3dkyj3E1KbpwUCn0ChcWci0s+x7GHmqdoJ2Wd+gqeeGYGiT8iybSDmr36ouSptcs09p0pdxd2V6vI/2njwiUnk1LqvHCoxXpRUihwGVRfCDNpUDi19IALI71XEBfc5+b8R29tqWfnKLYuKReFliKb4Q6r3MIZXqVIdzPbB8OX3Xdd+ul16/RkE+cBeUQekzah5C05JR/JGRkSEdBgFHwJePgt/B7+CH+uW8Ng8+Yh2arw118pF+RQ</latexit><latexit sha1_base64="uCnTptmZk5k2uvfF1msqYvcMsiY=">AAAC9HicbVFNj9MwEHXC11K+usuRy4gK1Aq2xEBhUTmsxIXjIrV0pboqjjvpWus4ke1Aq6g/hRPiyh/hF/BvcNJKbFvm9PTeeObNc5wraV0U/QnCa9dv3Lx1cLtx5+69+w+ah0efbVYYgUORqcycx9yikhqHTjqF57lBnsYKR/Hlh0offUVjZaYHbpnjJOVzLRMpuPPUtPl70NYdYH2mkPUbLMa51KXwA+2qwfpvNDwF5nDhoASZwAo0vIdeBIyNKaYT3wIs5QtgpZ8BftYx7Tz3cNBmKlFZZkC/6AEzNe7As6oFjuEV/JNpdFVnieGipHRV9vyyyhmw1b4JNse1jQZDPdsYnjZbUTeqC/YB3YAW2dTZ9DA4YrNMFClqJxS3dkyj3E1KbpwUCn0ChcWci0s+x7GHmqdoJ2Wd+gqeeGYGiT8iybSDmr36ouSptcs09p0pdxd2V6vI/2njwiUnk1LqvHCoxXpRUihwGVRfCDNpUDi19IALI71XEBfc5+b8R29tqWfnKLYuKReFliKb4Q6r3MIZXqVIdzPbB8OX3Xdd+ul16/RkE+cBeUQekzah5C05JR/JGRkSEdBgFHwJePgt/B7+CH+uW8Ng8+Yh2arw118pF+RQ</latexit><latexit sha1_base64="uCnTptmZk5k2uvfF1msqYvcMsiY=">AAAC9HicbVFNj9MwEHXC11K+usuRy4gK1Aq2xEBhUTmsxIXjIrV0pboqjjvpWus4ke1Aq6g/hRPiyh/hF/BvcNJKbFvm9PTeeObNc5wraV0U/QnCa9dv3Lx1cLtx5+69+w+ah0efbVYYgUORqcycx9yikhqHTjqF57lBnsYKR/Hlh0offUVjZaYHbpnjJOVzLRMpuPPUtPl70NYdYH2mkPUbLMa51KXwA+2qwfpvNDwF5nDhoASZwAo0vIdeBIyNKaYT3wIs5QtgpZ8BftYx7Tz3cNBmKlFZZkC/6AEzNe7As6oFjuEV/JNpdFVnieGipHRV9vyyyhmw1b4JNse1jQZDPdsYnjZbUTeqC/YB3YAW2dTZ9DA4YrNMFClqJxS3dkyj3E1KbpwUCn0ChcWci0s+x7GHmqdoJ2Wd+gqeeGYGiT8iybSDmr36ouSptcs09p0pdxd2V6vI/2njwiUnk1LqvHCoxXpRUihwGVRfCDNpUDi19IALI71XEBfc5+b8R29tqWfnKLYuKReFliKb4Q6r3MIZXqVIdzPbB8OX3Xdd+ul16/RkE+cBeUQekzah5C05JR/JGRkSEdBgFHwJePgt/B7+CH+uW8Ng8+Yh2arw118pF+RQ</latexit>

Suppose that we divide n elements into ⎣n / r⎦ groups of r elements each, and use the median-of-medians of these ⎣n / r⎦ groups as the pivot.For which r is the worst-case running time of select O(n) ?

A. r = 3

B. r = 7

C. Both A and B.

D. Neither A nor B.

61


T(n) = T(n / 3) + T(n − 2n / 6) + Θ(n) ⇒ T(n) = Θ(n log n)

T(n) = T(n / 7) + T(n − 4n / 14) + Θ(n) ⇒ T(n) = Θ(n)

Linear-time selection retrospective

Proposition. [Blum–Floyd–Pratt–Rivest–Tarjan 1973] There exists a

compare-based selection algorithm whose worst-case running time is O(n). Theory.

独Optimized version of BFPRT: ≤ 5.4305 n compares.

独Upper bound: [Dor–Zwick 1995] ≤ 2.95 n compares.

独Lower bound: [Dor–Zwick 1999] ≥ (2 + 2−80) n compares.

Practice. Constants too large to be useful.

62

JOURNAL OF COMPUTER AND SYSTEM SCIENCES 7, 4 4 8 - 4 6 1 (1973)

Time Bounds for Selection*

MANUEL BLUM, ROBERT W. FLOYD, VAUGHAN PRATT, RONALD L. RIVEST, AND ROBERT E. TARJAN

Department of Computer Science, Stanford University, Stanford, California 94305

Received November 14, 1972

The number of comparisons required to select the i-th smallest of n numbers is shown to be at most a linear function of n by analysis of a new selection algori thm--PICK. Specifically, no more than 5.4305 n comparisons are ever required. This bound is improved for extreme values of i, and a new lower bound on the requisite number of comparisons is also proved.

1. INTRODUCTION

In this paper we present a new selection algorithm, PICK, and derive by an analysis of its efficiency the (surprising) result that the cost of selection is at most a linear function of the number of input items. In addition, we prove a new lower bound for the cost of selection.

The selection problem is perhaps best exemplified by the computation of medians. In general, we may wish to select the i-th smallest of a set of n distinct numbers, or the element ranking closest to a given percentile level.

Interest in this problem may be traced to the realm of sports and the design of (traditionally, tennis) tournaments to select the first- and second-best players. In 1883, Lewis Carroll published an article [1] denouncing the unfair method by which the second-best player is usually determined in a "knockout tournament" -- the loser of the final match is often not the second-best! (Any of the players who lost only to the best player may be second-best.) Around 1930, Hugo Steinhaus brought the problem into the realm of algorithmic complexity by asking for the minimum number of matches required to (correctly) select both the first- and second-best players from a field of n contestants. In 1932, J. Schreier [8] showed that no more than n + [logg(n)]- 2 matches are required, and in 1964, S. S. Kislitsin [6] proved this number to be necessary as well. Schreier's method uses a knockout tournament to determine the winner, followed by a second knockout tournament among the

* This work was supported by the National Science Foundation under grant GJ-992.

448 Copyright © I973 by Academic Press, Inc. All rights of reproduction in any form reserved.


‣ mergesort





SECTION 5.4

Closest pair of points

Closest pair problem. Given n points in the plane, find a pair of points with the smallest Euclidean distance between them.

Fundamental geometric primitive.

独Graphics, computer vision, geographic information systems,molecular modeling, air traffic control.

独Special case of nearest neighbor, Euclidean MST, Voronoi.

64

fast closest pair inspired fast algorithms for these problems

Closest pair of points

Closest pair problem. Given n points in the plane, find a pair of points with the smallest Euclidean distance between them.

Brute force. Check all pairs with Θ(n2) distance calculations.

1D version. Easy O(n log n) algorithm if points are on a line.

Non-degeneracy assumption. No two points have the same x-coordinate.

65

Closest pair of points: first attempt

Sorting solution.

独Sort by x-coordinate and consider nearby points.

独Sort by y-coordinate and consider nearby points.

66

Closest pair of points: first attempt

Sorting solution.

独Sort by x-coordinate and consider nearby points.

独Sort by y-coordinate and consider nearby points.

67

8

Closest pair of points: second attempt

Divide. Subdivide region into 4 quadrants.

68

Closest pair of points: second attempt

Divide. Subdivide region into 4 quadrants.

Obstacle. Impossible to ensure n / 4 points in each piece.

69

Closest pair of points: divide-and-conquer algorithm

独Divide: draw vertical line L so that n / 2 points on each side.

独Conquer: find closest pair in each side recursively.

独Combine: find closest pair with one point in each side.

独Return best of 3 solutions.

70

seems like Θ(n2)

12

218

L

How to find closest pair with one point in each side?

Find closest pair with one point in each side, assuming that distance < δ.

独Observation: suffices to consider only those points within δ of line L.

71δ

δ = min(12, 21)12

21

L


Find closest pair with one point in each side, assuming that distance < δ.

独Observation: suffices to consider only those points within δ of line L.

独Sort points in 2 δ-strip by their y-coordinate.

独Check distances of only those points within 7 positions in sorted list!

72δ

2

3

45

6

7

why?

1

δ = min(12, 21)

21

12

L

Def. Let si be the point in the 2 δ-strip, with the ith smallest y-coordinate.

Claim. If ⎜j – i⎟ > 7, then the distance betweensi and sj is at least δ. Pf.

独Consider the 2δ-by-δ rectangle R in stripwhose min y-coordinate is y-coordinate of si.

独Distance between si and any point sjabove R is ≥ δ.

独Subdivide R into 8 squares.

独At most 1 point per square.

独At most 7 other points can be in R. ▪


73

½δ

½δ

si

δ

sj

R

constant can be improved with morerefined geometric packing argument

diameter is� /

�2 < �

<latexit sha1_base64="Pfe5G3M0P8ASzqZ7Xob9I+XpZXk=">AAACT3icdVDBTuMwEHXKsgsFllKOXKytWO0pJKWlRewBicseuxIFpKaqHGcKFo6TtSeoVdQ/4Gu4Ll/BjT/hhNbJFokieJKtp/dmPJ4XplIY9LxHp7L0afnzl5XV6tr6xtfN2lb9zCSZ5tDniUz0RcgMSKGgjwIlXKQaWBxKOA+vTwr//Aa0EYk6xWkKw5hdKjEWnKGVRrXvQQQSGQ2O6F5xBeaPxrw5K/jPUij9Ua3hue2D/U63RT3XP2zvN5sFabW7nRb1Xa9Eg8zRG2059SBKeBaDQi6ZMQPfS3GYM42CS5hVg8xAyvg1u4SBpYrFYIZ5udCM7lolouNE26OQlurrjpzFxkzj0FbGDK/MW68Q3/MGGY67w1yoNENQ/P+gcSYpJrRIh0ZCA0c5tYRxLexfKb9imnG0GS5MKd9OgS9skk8yJXgSwRtV4gQ1m9kUX6KiH5N+0z10/d+txnF3HucK2SHfyA/ikw45Jr9Ij/QJJ7fkjvwl986D8+Q8V+alFWdOtskCKqv/AOSCstc=</latexit><latexit sha1_base64="Pfe5G3M0P8ASzqZ7Xob9I+XpZXk=">AAACT3icdVDBTuMwEHXKsgsFllKOXKytWO0pJKWlRewBicseuxIFpKaqHGcKFo6TtSeoVdQ/4Gu4Ll/BjT/hhNbJFokieJKtp/dmPJ4XplIY9LxHp7L0afnzl5XV6tr6xtfN2lb9zCSZ5tDniUz0RcgMSKGgjwIlXKQaWBxKOA+vTwr//Aa0EYk6xWkKw5hdKjEWnKGVRrXvQQQSGQ2O6F5xBeaPxrw5K/jPUij9Ua3hue2D/U63RT3XP2zvN5sFabW7nRb1Xa9Eg8zRG2059SBKeBaDQi6ZMQPfS3GYM42CS5hVg8xAyvg1u4SBpYrFYIZ5udCM7lolouNE26OQlurrjpzFxkzj0FbGDK/MW68Q3/MGGY67w1yoNENQ/P+gcSYpJrRIh0ZCA0c5tYRxLexfKb9imnG0GS5MKd9OgS9skk8yJXgSwRtV4gQ1m9kUX6KiH5N+0z10/d+txnF3HucK2SHfyA/ikw45Jr9Ij/QJJ7fkjvwl986D8+Q8V+alFWdOtskCKqv/AOSCstc=</latexit><latexit sha1_base64="Pfe5G3M0P8ASzqZ7Xob9I+XpZXk=">AAACT3icdVDBTuMwEHXKsgsFllKOXKytWO0pJKWlRewBicseuxIFpKaqHGcKFo6TtSeoVdQ/4Gu4Ll/BjT/hhNbJFokieJKtp/dmPJ4XplIY9LxHp7L0afnzl5XV6tr6xtfN2lb9zCSZ5tDniUz0RcgMSKGgjwIlXKQaWBxKOA+vTwr//Aa0EYk6xWkKw5hdKjEWnKGVRrXvQQQSGQ2O6F5xBeaPxrw5K/jPUij9Ua3hue2D/U63RT3XP2zvN5sFabW7nRb1Xa9Eg8zRG2059SBKeBaDQi6ZMQPfS3GYM42CS5hVg8xAyvg1u4SBpYrFYIZ5udCM7lolouNE26OQlurrjpzFxkzj0FbGDK/MW68Q3/MGGY67w1yoNENQ/P+gcSYpJrRIh0ZCA0c5tYRxLexfKb9imnG0GS5MKd9OgS9skk8yJXgSwRtV4gQ1m9kUX6KiH5N+0z10/d+txnF3HucK2SHfyA/ikw45Jr9Ij/QJJ7fkjvwl986D8+Q8V+alFWdOtskCKqv/AOSCstc=</latexit><latexit sha1_base64="Pfe5G3M0P8ASzqZ7Xob9I+XpZXk=">AAACT3icdVDBTuMwEHXKsgsFllKOXKytWO0pJKWlRewBicseuxIFpKaqHGcKFo6TtSeoVdQ/4Gu4Ll/BjT/hhNbJFokieJKtp/dmPJ4XplIY9LxHp7L0afnzl5XV6tr6xtfN2lb9zCSZ5tDniUz0RcgMSKGgjwIlXKQaWBxKOA+vTwr//Aa0EYk6xWkKw5hdKjEWnKGVRrXvQQQSGQ2O6F5xBeaPxrw5K/jPUij9Ua3hue2D/U63RT3XP2zvN5sFabW7nRb1Xa9Eg8zRG2059SBKeBaDQi6ZMQPfS3GYM42CS5hVg8xAyvg1u4SBpYrFYIZ5udCM7lolouNE26OQlurrjpzFxkzj0FbGDK/MW68Q3/MGGY67w1yoNENQ/P+gcSYpJrRIh0ZCA0c5tYRxLexfKb9imnG0GS5MKd9OgS9skk8yJXgSwRtV4gQ1m9kUX6KiH5N+0z10/d+txnF3HucK2SHfyA/ikw45Jr9Ij/QJJ7fkjvwl986D8+Q8V+alFWdOtskCKqv/AOSCstc=</latexit>

2δ

L

Closest pair of points: divide-and-conquer algorithm

74

O(n)

T(n / 2)

O(n)

O(n log n)

O(n)

CLOSEST-PAIR(p1, p2, …, pn) _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Compute vertical line L such that half the points are on each side of the line.

δ1 ← CLOSEST-PAIR(points in left half).

δ2 ← CLOSEST-PAIR(points in right half).

δ ← min { δ1 , δ2 }.

Delete all points further than δ from line L.

Sort remaining points by y-coordinate.

Scan points in y-order and compare distance between each point and next 7 neighbors. If any of these distances is less than δ, update δ.

RETURN δ._________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

T(n / 2)

What is the solution to the following recurrence?

A. T(n) = Θ(n).

B. T(n) = Θ(n log n).

C. T(n) = Θ(n log2 n).

D. T(n) = Θ(n2).

75


T (n) =

��

��(1) B7 n = 1

T (�n/2�) + T (�n/2�) + �(n log n) B7 n > 1<latexit sha1_base64="dpqSXns9eMyDq5iq+L5bY9kUO1I=">AAAC2nicbVFNixNBEO0Zv9b4lV0PHrwUZpUEIc4swq6ElQVBPa6QuAvpIfR0apJme7qH7h5JGHLyJF79I/4c/4092VkwiXV6vFdVrz7SQgrrouhPEN66fefuvb37rQcPHz1+0t4/+Gp1aTiOuJbaXKbMohQKR044iZeFQZanEi/Sqw+1fvENjRVaDd2ywCRnMyUywZnz1KT9e9hVPTiFFk1xJlTFfS+7atEBHc7RsW7cg1dAHS5cBSKDQ+Vz40NYAaXjqH+MeeJzh10qM6m1AfXmCKhZ4x7QwWs6gFrkKGSj1fBGajwUUKlnoHas3t9YtSiqaTPbpN2J+tE6YBfEDeiQJs4n+8EBnWpe5qgcl8zacRwVLqmYcYJL9MuWFgvGr9gMxx4qlqNNqvVtV/DSM1PI/G6ZVg7W7L8VFcutXeapz8yZm9ttrSb/p41Ll50klVBF6VDxa6OslOA01I+CqTDInVx6wLgRflbgc2YYd/6dGy7r3gXyjU2qRakE11PcYqVbOMPqK8bbN9sFo6P+u3785W3n7KQ55x55Tl6QLonJMTkjn8k5GREePAtOg4/BpzAJv4c/wp/XqWHQ1DwlGxH++gt4httm</latexit><latexit sha1_base64="dpqSXns9eMyDq5iq+L5bY9kUO1I=">AAAC2nicbVFNixNBEO0Zv9b4lV0PHrwUZpUEIc4swq6ElQVBPa6QuAvpIfR0apJme7qH7h5JGHLyJF79I/4c/4092VkwiXV6vFdVrz7SQgrrouhPEN66fefuvb37rQcPHz1+0t4/+Gp1aTiOuJbaXKbMohQKR044iZeFQZanEi/Sqw+1fvENjRVaDd2ywCRnMyUywZnz1KT9e9hVPTiFFk1xJlTFfS+7atEBHc7RsW7cg1dAHS5cBSKDQ+Vz40NYAaXjqH+MeeJzh10qM6m1AfXmCKhZ4x7QwWs6gFrkKGSj1fBGajwUUKlnoHas3t9YtSiqaTPbpN2J+tE6YBfEDeiQJs4n+8EBnWpe5qgcl8zacRwVLqmYcYJL9MuWFgvGr9gMxx4qlqNNqvVtV/DSM1PI/G6ZVg7W7L8VFcutXeapz8yZm9ttrSb/p41Ll50klVBF6VDxa6OslOA01I+CqTDInVx6wLgRflbgc2YYd/6dGy7r3gXyjU2qRakE11PcYqVbOMPqK8bbN9sFo6P+u3785W3n7KQ55x55Tl6QLonJMTkjn8k5GREePAtOg4/BpzAJv4c/wp/XqWHQ1DwlGxH++gt4httm</latexit><latexit sha1_base64="dpqSXns9eMyDq5iq+L5bY9kUO1I=">AAAC2nicbVFNixNBEO0Zv9b4lV0PHrwUZpUEIc4swq6ElQVBPa6QuAvpIfR0apJme7qH7h5JGHLyJF79I/4c/4092VkwiXV6vFdVrz7SQgrrouhPEN66fefuvb37rQcPHz1+0t4/+Gp1aTiOuJbaXKbMohQKR044iZeFQZanEi/Sqw+1fvENjRVaDd2ywCRnMyUywZnz1KT9e9hVPTiFFk1xJlTFfS+7atEBHc7RsW7cg1dAHS5cBSKDQ+Vz40NYAaXjqH+MeeJzh10qM6m1AfXmCKhZ4x7QwWs6gFrkKGSj1fBGajwUUKlnoHas3t9YtSiqaTPbpN2J+tE6YBfEDeiQJs4n+8EBnWpe5qgcl8zacRwVLqmYcYJL9MuWFgvGr9gMxx4qlqNNqvVtV/DSM1PI/G6ZVg7W7L8VFcutXeapz8yZm9ttrSb/p41Ll50klVBF6VDxa6OslOA01I+CqTDInVx6wLgRflbgc2YYd/6dGy7r3gXyjU2qRakE11PcYqVbOMPqK8bbN9sFo6P+u3785W3n7KQ55x55Tl6QLonJMTkjn8k5GREePAtOg4/BpzAJv4c/wp/XqWHQ1DwlGxH++gt4httm</latexit><latexit sha1_base64="dpqSXns9eMyDq5iq+L5bY9kUO1I=">AAAC2nicbVFNixNBEO0Zv9b4lV0PHrwUZpUEIc4swq6ElQVBPa6QuAvpIfR0apJme7qH7h5JGHLyJF79I/4c/4092VkwiXV6vFdVrz7SQgrrouhPEN66fefuvb37rQcPHz1+0t4/+Gp1aTiOuJbaXKbMohQKR044iZeFQZanEi/Sqw+1fvENjRVaDd2ywCRnMyUywZnz1KT9e9hVPTiFFk1xJlTFfS+7atEBHc7RsW7cg1dAHS5cBSKDQ+Vz40NYAaXjqH+MeeJzh10qM6m1AfXmCKhZ4x7QwWs6gFrkKGSj1fBGajwUUKlnoHas3t9YtSiqaTPbpN2J+tE6YBfEDeiQJs4n+8EBnWpe5qgcl8zacRwVLqmYcYJL9MuWFgvGr9gMxx4qlqNNqvVtV/DSM1PI/G6ZVg7W7L8VFcutXeapz8yZm9ttrSb/p41Ll50klVBF6VDxa6OslOA01I+CqTDInVx6wLgRflbgc2YYd/6dGy7r3gXyjU2qRakE11PcYqVbOMPqK8bbN9sFo6P+u3785W3n7KQ55x55Tl6QLonJMTkjn8k5GREePAtOg4/BpzAJv4c/wp/XqWHQ1DwlGxH++gt4httm</latexit>

Refined version of closest-pair algorithm

Q. How to improve to O(n log n) ? A. Don’t sort points in strip from scratch each time.

独Each recursive call returns two lists: all points sorted by x-coordinate, and all points sorted by y-coordinate.

独Sort by merging two pre-sorted lists.

Theorem. [Shamos 1975] The divide-and-conquer algorithm for finding a

closest pair of points in the plane can be implemented in O(n log n) time.

Pf.

76

T (n) =

��

��(1) B7 n = 1

T (�n/2�) + T (�n/2�) + �(n) B7 n > 1<latexit sha1_base64="3pQz3or04sCHqNIWE5a09TyF+wA=">AAAC03icbVFNb9NAEF27fJTwlZYjlxEpKBFSaleIFkVAJC4ci5S0lbJWtN6Mk1XXa2t3XSUyuSCu/BH+Ef+GtetKJGFOT+/NzJuPOJfC2CD44/l79+4/eLj/qPX4ydNnz9sHhxcmKzTHMc9kpq9iZlAKhWMrrMSrXCNLY4mX8fWXSr+8QW1EpkZ2lWOUsrkSieDMOmra/j3qqh58hBaNcS5UyV0vs27RAR0t0LJu2IM3QC0ubQkigSPlcsMjWAOlk6B/imnkckddKhOZZRrU8QlQXeMe0MFbOoBK5Chko1XwTmo81I7HpzuPFkU1a4aatjtBP6gDdkHYgA5p4nx64B3SWcaLFJXlkhkzCYPcRiXTVnCJbsvCYM74NZvjxEHFUjRRWR91Da8dM4PELZVkykLN/ltRstSYVRq7zJTZhdnWKvJ/2qSwyVlUCpUXFhW/NUoKCTaD6kMwExq5lSsHGNfCzQp8wTTj1v1xw6XunSPf2KRcFkrwbIZbrLRLq1l1xXD7ZrtgfNL/0A+/vesMz5pz7pOX5BXpkpCckiH5Ss7JmHCv7b33PntD/8L/7v/wf96m+l5T84JshP/rL+PW2NQ=</latexit><latexit sha1_base64="3pQz3or04sCHqNIWE5a09TyF+wA=">AAAC03icbVFNb9NAEF27fJTwlZYjlxEpKBFSaleIFkVAJC4ci5S0lbJWtN6Mk1XXa2t3XSUyuSCu/BH+Ef+GtetKJGFOT+/NzJuPOJfC2CD44/l79+4/eLj/qPX4ydNnz9sHhxcmKzTHMc9kpq9iZlAKhWMrrMSrXCNLY4mX8fWXSr+8QW1EpkZ2lWOUsrkSieDMOmra/j3qqh58hBaNcS5UyV0vs27RAR0t0LJu2IM3QC0ubQkigSPlcsMjWAOlk6B/imnkckddKhOZZRrU8QlQXeMe0MFbOoBK5Chko1XwTmo81I7HpzuPFkU1a4aatjtBP6gDdkHYgA5p4nx64B3SWcaLFJXlkhkzCYPcRiXTVnCJbsvCYM74NZvjxEHFUjRRWR91Da8dM4PELZVkykLN/ltRstSYVRq7zJTZhdnWKvJ/2qSwyVlUCpUXFhW/NUoKCTaD6kMwExq5lSsHGNfCzQp8wTTj1v1xw6XunSPf2KRcFkrwbIZbrLRLq1l1xXD7ZrtgfNL/0A+/vesMz5pz7pOX5BXpkpCckiH5Ss7JmHCv7b33PntD/8L/7v/wf96m+l5T84JshP/rL+PW2NQ=</latexit><latexit sha1_base64="3pQz3or04sCHqNIWE5a09TyF+wA=">AAAC03icbVFNb9NAEF27fJTwlZYjlxEpKBFSaleIFkVAJC4ci5S0lbJWtN6Mk1XXa2t3XSUyuSCu/BH+Ef+GtetKJGFOT+/NzJuPOJfC2CD44/l79+4/eLj/qPX4ydNnz9sHhxcmKzTHMc9kpq9iZlAKhWMrrMSrXCNLY4mX8fWXSr+8QW1EpkZ2lWOUsrkSieDMOmra/j3qqh58hBaNcS5UyV0vs27RAR0t0LJu2IM3QC0ubQkigSPlcsMjWAOlk6B/imnkckddKhOZZRrU8QlQXeMe0MFbOoBK5Chko1XwTmo81I7HpzuPFkU1a4aatjtBP6gDdkHYgA5p4nx64B3SWcaLFJXlkhkzCYPcRiXTVnCJbsvCYM74NZvjxEHFUjRRWR91Da8dM4PELZVkykLN/ltRstSYVRq7zJTZhdnWKvJ/2qSwyVlUCpUXFhW/NUoKCTaD6kMwExq5lSsHGNfCzQp8wTTj1v1xw6XunSPf2KRcFkrwbIZbrLRLq1l1xXD7ZrtgfNL/0A+/vesMz5pz7pOX5BXpkpCckiH5Ss7JmHCv7b33PntD/8L/7v/wf96m+l5T84JshP/rL+PW2NQ=</latexit><latexit sha1_base64="3pQz3or04sCHqNIWE5a09TyF+wA=">AAAC03icbVFNb9NAEF27fJTwlZYjlxEpKBFSaleIFkVAJC4ci5S0lbJWtN6Mk1XXa2t3XSUyuSCu/BH+Ef+GtetKJGFOT+/NzJuPOJfC2CD44/l79+4/eLj/qPX4ydNnz9sHhxcmKzTHMc9kpq9iZlAKhWMrrMSrXCNLY4mX8fWXSr+8QW1EpkZ2lWOUsrkSieDMOmra/j3qqh58hBaNcS5UyV0vs27RAR0t0LJu2IM3QC0ubQkigSPlcsMjWAOlk6B/imnkckddKhOZZRrU8QlQXeMe0MFbOoBK5Chko1XwTmo81I7HpzuPFkU1a4aatjtBP6gDdkHYgA5p4nx64B3SWcaLFJXlkhkzCYPcRiXTVnCJbsvCYM74NZvjxEHFUjRRWR91Da8dM4PELZVkykLN/ltRstSYVRq7zJTZhdnWKvJ/2qSwyVlUCpUXFhW/NUoKCTaD6kMwExq5lSsHGNfCzQp8wTTj1v1xw6XunSPf2KRcFkrwbIZbrLRLq1l1xXD7ZrtgfNL/0A+/vesMz5pz7pOX5BXpkpCckiH5Ss7JmHCv7b33PntD/8L/7v/wf96m+l5T84JshP/rL+PW2NQ=</latexit>

What is the complexity of the 2D closest pair problem?

A. Θ(n).

B. Θ(n log* n).

C. Θ(n log log n).

D. Θ(n log n).

E. Not even Tarjan knows.

77


Computational complexity of closest-pair problem

Theorem. [Ben-Or 1983, Yao 1989] In quadratic decision tree model, any

algorithm for closest pair (even in 1D) requires Ω(n log n) quadratic tests.

Theorem. [Rabin 1976] There exists an algorithm to find the closest pair of

points in the plane whose expected running time is O(n).

78

not subject to Ω(n log n) lower bound

because it uses the floor function

(x1 - x2)2 + (y1 - y2)2

Volume 8, number 1 IhFCRMATION PROCESSING LE,TTERS January 1979

A NOTE ON RABIN’S NEAREST-NEIGHBOR ALGORITHM *

Steve FORTUNE and John HOPCROFT Department of Computer Science, Cornell University, Ithaca, NY, U.S.A.

Received 20 July 1978, revised version received 21 August 1978

Probabiktic algorithms, nearest neighbor, hashing

9. introduction

One notion that has received some attention recently is that of a probabilistic algorithm. An algorithm is probabilistic if at certain steps it chooses a number randomly to determine the next step and at all other steps it is deterministic. The expected running time of such an algorithm on a given input is obtained by averaging over all possible random choices of the algorithm on that input. The running time of the algorithm as a whole can be measured either by averaging the expected running time over all inputs or by taking the worst case of the expected running time. In [l] the former analysis is used; in [4] and [6] the latter analysis is used.

Rabin [4] has proposed a probabilistic algorithm for finding the closest pair of a set of points in Euclidean space. The running time of his algorithm using the worst-case expected-time measure is linear. This compares with time O(n log II) for the best of the known determinis?ic algorithms [or this problem j2,5,7].

Rabin’s algc-rithm works by randomly choosing a subset of the points and recursively using the algorithm to find the distance between the nearest neighbors of the subset. Ra.bin is able to show that, with very high probability, tht; distance between the nearest neighbors of the subset is a very good ;Ipproxima- * Research was wpported under grant number ONR N00014,

76-C-lrO18.

20

tion to the distance between nearest neighbors in the whole set. Using this approximation the distance between the nearest neighbor in the whole set can be found in expected linear time. The algorithm’s use of randomness in choosing the subset is crucial.

Rabin’s algorithm also assumes constant time arithmetic operations. In partictilar, he assumes that a special operation, described below, which is similar to hashing can be performed in constant expected time. It follows from [3] that the special operation can indeed be implemented by a probabilistic algorithm to run in constant expected time given that evaluating a hash function takes constant time.

The fast algorithms in [ 11, [4], and [6] all have the property that for some sequence of random choices an incorrect answer could result. Rabin’s algorithm is apparently the only known example other than hashing of an error-free probabilistic algorithm which runs faster than the deterministic equivalent.

In this paper we present an algorithm to find the nearest neighbor which runs in time O(n loglog n). The algorithm does not make any random choices; however it does assume that the special operation uses only constant time. The conclusion we reach is that mclst of the speedup of Rabin’s algorithm is due to the hashing, and not to the probabilistic nature of the basic algorithm. It would be interesting to find examples of probabilistic algorithms that are faster than their deterministic counterparts and for which the speed does noi come from the capability to hash.

Digression: computational geometry

Ingenious divide-and-conquer algorithms for core geometric problems.

Note. 3D and higher dimensions test limits of our ingenuity.

79

running time to solve a 2D problem with n points

problem brute clever

closest pair O(n2) O(n log n)

farthest pair O(n2) O(n log n)

convex hull O(n2) O(n log n)

Delaunay/Voronoi O(n4) O(n log n)

Euclidean MST O(n2) O(n log n)

Convex hull

The convex hull of a set of n points is the smallest perimeter fence enclosing the points.

Equivalent definitions.

独Smallest area convex polygon enclosing the points.

独Intersection of all convex set containing all the points.

80

Farthest pair

Given n points in the plane, find a pair of points with the largest Euclidean

distance between them.

Fact. Points in farthest pair are extreme points on convex hull.

81

The Delaunay triangulation is a triangulation of n points in the plane such that no point is inside the circumcircle of any triangle.

Some useful properties.

独No edges cross.

独Among all triangulations, it maximizes the minimum angle.

独Contains an edge between each point and its nearest neighbor.

Delaunay triangulation

82

Delaunay triangulation of 19 points

no point in the set isinside the circumcircle

point inside circumcircleof 3 points

Given n points in the plane, find MST connecting them.[distances between point pairs are Euclidean distances]

Fact. Euclidean MST is subgraph of Delaunay triangulation.

Implication. Can compute Euclidean MST in O(n log n) time.

独Compute Delaunay triangulation.

独Compute MST of Delaunay triangulation.

Euclidean MST

83

it’s planar (≤ 3n edges)

Computational geometry applications

Applications.

独Robotics.

独VLSI design.

独Data mining.

独Medical imaging.

独Computer vision.

独Scientific computing.

独Finite-element meshing.

独Astronomical simulation.

独Models of physical world.

独Geographic information systems.

独Computer graphics (movies, games, virtual reality).

84

http://www.ics.uci.edu/~eppstein/geom.html

airflow around an aircraft wing

mergesort counting inversions closest pair of points ... Divide problem of size n into two...

Documents

Transcript of mergesort counting inversions closest pair of points ... Divide problem of size n into two...