1 Dr. Scott Schaefer Data Structures and Algorithms CSCE 221H.
Analysis & Design of Algorithms (CSCE 321)
description
Transcript of Analysis & Design of Algorithms (CSCE 321)
Prof. Amr Goneid, AUC 1
Analysis & Design of Analysis & Design of AlgorithmsAlgorithms(CSCE 321)(CSCE 321)
Prof. Amr GoneidDepartment of Computer Science, AUC
Part 7. Divide & Conquer Algorithms
Prof. Amr Goneid, AUC 2
Divide & Conquer AlgorithmsDivide & Conquer Algorithms
Prof. Amr Goneid, AUC 3
Divide & Conquer AlgorithmsDivide & Conquer Algorithms
The General Method Time Complexity Common Recurrence relations A General Divide & Conquer Recurrence Merge Sort Algorithm Quicksort Algorithm An Application of Quicksort: Convex Hull Selection by Partitioning Closest Pair Problem
Prof. Amr Goneid, AUC 4
1. The General Method1. The General MethodTo solve a problem using Divide and
Conquer:
• Divide it into smaller problems
• Solve the smaller problems recursively
• Combine their solutions into a solution
for the big problem
Prof. Amr Goneid, AUC 5
The General MethodThe General Method
The base case for recursion is a sub-problem of constant (and small) size.
Analysis of D&Q algorithms is done by constructing a recurrence relation
We can obtain T(n) in a closed form by solving the recurrence relation
Prof. Amr Goneid, AUC 6
The General MethodThe General Method For problem p, small(p) = true if p is simple. In this
case S(p) is the solution. Otherwise, Divide, Conquer, then Combine.
Divide & Conquer Algorithm
DQ(p)
{ if small(p) return S(p);
else {
1. Divide p into smaller instances p1,p2,..,pk
2. Apply DQ to each pi
3. Return Combine(DQ(p1),DQ(p2),…,DQ(pk))}
}
Prof. Amr Goneid, AUC 7
2. Time Complexity2. Time Complexity
Let g(n) = time for S(p) when p is small
f(n) = time to divide p and combine
solutions
then
for p small T(n) = g(n)
otherwise
)()()(1
nfnTnTk
ii
Prof. Amr Goneid, AUC 8
3. Common Recurrence Relations3. Common Recurrence Relations
A recursive algorithm that halves the input in one step then processes the two halves:
T(n) = 2T(n/2) + 1 for n >1 with T(1) given
A recursive algorithm that halves the input in one step then processes one half and ignores the other:
T(n) = T(n/2) + 1 for n >1 with T(1) given
Prof. Amr Goneid, AUC 9
Common Recurrence RelationsCommon Recurrence Relations A recursive algorithm that examines every element
before dividing and then ignores one half:
T(n) = T(n/2) + n for n >1 with T(1) given
A recursive algorithm that makes a linear pass through the input, before, during or after it is split into two halves then processes the two halves:
T(n) = 2 T(n/2) + n for n >1 with T(1) given
Prof. Amr Goneid, AUC 10
Example: Recursive Binary Search Example: Recursive Binary Search AlgorithmAlgorithmBinSearch (a , s , e , x)
{ if (s == e) // if small (p), i.e. one element{ if (x == as) return s ; else return -1 ;} // return S(p)
else { // two or more elements mid = (s + e)/2 ; // Divide if (x == amid) return mid ; else if (x < amid) // Conquer return BinSearch (a , s , mid-1 , x) ; // n/2 or else return BinSearch (a , mid+1 , e , x) ; } // n/2
}// No Combine
Prof. Amr Goneid, AUC 11
http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/BSearch.html
Binary Search DemoBinary Search Demo
Prof. Amr Goneid, AUC 12
Worst Case of Binary SearchWorst Case of Binary Search
Let T(n) be the worst case number of comparisons
used on an array of n items. Hence
Now we seek a closed form solution for T(n)
11122 )(Twithnfor)/n(T)n(T
Prof. Amr Goneid, AUC 13
Worst Case of Binary SearchWorst Case of Binary Search
)n(logOnlog)n(Tand
m)m(THence
)(Twithmfor)m(T)m(T
mn,mn,nlogm
/n,.....,,,m,n
)(Twithnfor)/n(T)n(T
m
i
mm
12
2121
10021
recurrencesecondary aobtain We
0101 Hence,
222102 Assume
11122
1
1
Prof. Amr Goneid, AUC 14
4. A General D&Q Recurrence4. A General D&Q Recurrence
given. is F(1) that meansit given is T(c) if
Similarly, given. is F(0) that meansit given is T(1) If
1
RecurrenceSecondary
101
10
2,3,...c constants,areca,where
given.with or given 1
recurrence a have algorithms Q&DMany
1
1
)c(f)m(aF)m(F
:
)c(f)c(aT)c(THence
)m()cn(,)m()n(andcc/n
nlogmHence,...,m,cnLet
)c(Tcn)(Twithcnfor
,)n(f)c/n(aT)n(T
m
mmm
m
c
m
Prof. Amr Goneid, AUC 15
A General SolutionA General Solution
The secondary recurrence is solved as a first order linear recurrence. For T(1) i.e. F(0) given, we have as shown before:
given. T(c)for Also
given. T(1)for 1
0
0
0
2
0
1
1
0
1
0
1
1
1
11
)c/n(faa)c(T)n(T
)c/n(faa)(T)n(THence
)c(fa)(Fa
)c(fa)(Fa
)c(fa)c(fa)(F)m(F
im
i
im
im
i
im
imm
i
im
im
i
imm
mm
ij
m
i
im
i
Prof. Amr Goneid, AUC 16
Very Common: Very Common: f(n) = b nf(n) = b nxx
and :arise cases Two Now,
given. T(c)for
given. T(1)for 1
:become solutions The
and
hence
2
0
1
1
0
xx
im
ix
xm
im
ix
xm
i
x
x
i
i
ix
x
i
x
caca
)c
a(bn)c(Ta)n(T
)c
a(bn)(Ta)n(T
)c
a(bn)
c
n(fa
c
nb)
c
n(fbn)n(f
Prof. Amr Goneid, AUC 17
Case(a): Case(a): f(n) = b nf(n) = b nxx, a = c, a = cxx
)1(log)()(
log)1()1()(
1
nbncTanT
nbnTabmnTanT
cxm
cxmxm
:given T(c)For
:given T(1)For
Prof. Amr Goneid, AUC 18
SummarySummary
given T(c)For
given T(1)For
:solutions the has Case Special
given. T(c)for
given. T(1)for
:become solutions The
:Recurrence Q&D Common
)1(log)()(
log)1()1()(
)()()(
)()1()(
)/()(
1
2
0
1
1
0
nbncTanT
nbnTabmnTanT
ca
c
abncTanT
c
abnTanT
bncnaTnT
cxm
cxmxm
x
im
ix
xm
im
ix
xm
x
Prof. Amr Goneid, AUC 19
Case(a): Case(a): f(n) = b nf(n) = b nxx, a = c, a = cxx
12
11
: andgiven T(1)For
0221
1122
:Search Binary :exampleFor
nlog)n(T
nlogbn)(Tabmn)(Ta)n(T
nlogm
cahence,x,c,b,aHere
)(Twith)/n(T)n(T
c
xmxm
x
Prof. Amr Goneid, AUC 20
Other examples forOther examples for a = c a = cxx
Recursive Power FunctionRecursive Power Function
The following function computes xn where x is of type double:
Pow (x , n)
{
if (n == 0) return 1.0; else if (n == 1) return x;
else if (n mod 2) return Pow (x*x , n / 2) * x;
else return Pow (x*x , n / 2);}
Show that the number of double arithmetic operations performed by this algorithm in the best case is T(n) = log n
Prof. Amr Goneid, AUC 21
SolutionSolution
In the best case n = 2m , so that n is always even and we
obtain the recurrence relation:
T(n) = T(n/2) + 1 , for n > 1 with T(1) = 0
Here, a = 1, b = 1, c = 2, x = 0
with the solution :
T(n) = bnx log n = log n
Prof. Amr Goneid, AUC 22
Other examples forOther examples for a = c a = cxx
Recursive log FunctionRecursive log Function
A recursive function to compute the largest integer less than or equal
to log2 n. (Hint: for n > = 2 , the value of this function for n is one
greater than for n/2 ).
int Log2(int n)
{ if (n == 1) return 0;
else return (1 + Log2(n/2)); }
The number of integer divisions has the recurrence relation:
T(n) = T(n/2) + 1 , for n > 1 with T(1) = b = 0
with the solution :
T(n) = log n = O(log n)
Prof. Amr Goneid, AUC 23
Find the Number of Binary Digits in the Binary Representation of a Positive Decimal Integer using a D&Q recursive algorithm.
int BinDig (int n)
{ if (n == 1) return 1;
else return (1 + BinDig(n/2)); }
T(n) = number of arithmetic operations.
T(n) = T(n/2) + 2 for n > 1 with T(1) = 0
a = 1, b = 2, c = 2, x = 0
T(n) = 2 log n = O(log n)
Other examples forOther examples for a = c a = cxx
Prof. Amr Goneid, AUC 24
Other examples forOther examples for a = c a = cxx
Other Recurrences with a = cx
T(n) = 4 T(n/2) + n2 for n > 1 with T(2) = 1
Hence, T(n) = n2(log n – ¾) = O(n2 log n)
T(n) = 9T(n/3) + n2 for n > 1 with T(1) = 0
Hence, T(n) = n2 log3 n
T(n) = 16 T(n/4) + n2 for n > 1 with T(1) = 0
Hence, T(n) = n2 log4 n
Prof. Amr Goneid, AUC 25
Case(b): Case(b): f(n) = b nf(n) = b nxx, a , a c cxx
Example: a Fractal AlgorithmThe following function draws recursive squares (called a fractal star).
The drawing primitive is Box (x , y , n) which draws a square of side (n)
pixels centered at (x,y) :
void STAR( int x, int y, int n)
{ if (n > 1) { STAR(x-n , y+n , n/2); STAR(x+n , y+n , n/2);
STAR(x-n , y-n , n/2); STAR(x+n , y-n , n/2);
Box(x , y , n); }
}
Prof. Amr Goneid, AUC 26
Fractal AlgorithmFractal Algorithm
The recurrence relation is:
T(n) = 4 T(n/2) + 4 n for n > 1 with T(1) = 0
Here, a = 4, b = 4, c = 2, x =1 so that a cx
The general solution is:
)n(O)n(n)(nn)n(T
)c
a(bn)(Ta)n(T
mm
i
i
im
ix
xm
21
0
1
0
1412424 Hence,
given. T(1)for 1
Prof. Amr Goneid, AUC 27
Example: MaxMin Example: MaxMin Straightforward AlgorithmStraightforward Algorithm
// Return max and min values in locations s to e in array a[ ]MaxMin (a, s , e , mx, mn){ mx = mn = as; for i = s+1 to e
{if (ai > mx) mx = ai ;if (ai < mn) mn = ai ;
}}
Invoke as MaxMin(a , 1 , n , mx , mn) ;Let T(n) be the number of comparisons of array elements.
Hence,T(n) = 2 ( n – 1) independent of the data in a[ ].
Prof. Amr Goneid, AUC 28
MaxMin D&Q AlgorithmMaxMin D&Q Algorithm
MaxMin (a, s , e , mx , mn){
if (s e-1) // one or two elementsif (as > ae) {mx = as; mn = ae;} else {mx = ae; mn = as;}
else {
m = (s+e)/2;MaxMin (a,s,m,mx,mn);Maxmin (a,m+1,e,mx1,mn1);mx = max(mx,mx1); mn = min(mn,mn1);
}}// The functions max/min(a,b) find the larger/smaller of (a,b). Each will
cost one comparison.
Prof. Amr Goneid, AUC 29
MaxMin AnalysisMaxMin Analysis
The recurrence relation is:
T(n) = 2 T(n/2) + 2 for n > 2 with T(2) = 1
Here, a = 2, b = 2, c = 2, x =0 so that a cx
The general solution is:
This is only 75% of the cost of the straightforward algorithm.
25112250 Hence,
given. T(2)for 222
1
2
0
1
n.)(n.)n(T
)(Ta)n(T
m
m
i
im
Prof. Amr Goneid, AUC 30
Other Examples withOther Examples with a a c cxx
Copy Binary Tree Algorithm
The following function receives a pointer (t) to a binary tree and
returns a pointer to an exact copy of the tree :
treeNode *CopyTree ( treeNode *t )
{ treeNode *p;
if (t)
{ p = new treeNode ;
p->left = CopyTree(t->left) ;
p->right = CopyTree(t->right);
p->info = t->info ; return p ; };
else return NULL ;
}
Prof. Amr Goneid, AUC 31
Other Examples withOther Examples with a a c cxx
Copy Binary Tree Algorithm
In the worst case, the tree is full and the number of visits is:
T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1Hence
T(n) = 2n – 1 = O(n)
Prof. Amr Goneid, AUC 32
Other Examples withOther Examples with a a c cxx
Pre-Order Traversal of a Binary Tree
void preorder (tree t)
{ if (t)
{ visit (t); preorder (t -> left); preorder (t -> right); }
}
In the worst case, the tree is full and the number of visits is:
T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1
Hence
T(n) = 2n – 1 = O(n)
Prof. Amr Goneid, AUC 33
Other Examples withOther Examples with a a c cxx
Other Algorithms T(n) = T(n/2) + n2 for n > 1 with T(1) = 0
Hence T(n) = (4/3) (n2 – 1) T(n) = 2 T(n/2) + n2 for n > 1 with T(1) = 0
Hence T(n) = 2n(n – 1) T(n) = 8 T(n/2) + n2 for n > 1 with T(1) = 0
Hence T(n) = n2 (n – 1)
Prof. Amr Goneid, AUC 34
5. MergeSort5. MergeSort (a) Merging(a) Merging Definition:
Combine two or more sorted sequences of data into a single sorted sequence.
Formal Definition: The input is two sorted sequences,
A={a1, ..., an} and B={b1, ..., bm} The output is a single sequence, merge(A,B), which is a sorted permutation of
{a1, ..., an, b1, ..., bm}.
Prof. Amr Goneid, AUC 35
MergingMerging
merge(A, B) is
A, if B is empty,
B, if A is empty,
{a1}.merge({a2, ..., an}, B) if a1 <= b1, and
{b1}.merge(A, {b2, ..., bm}) otherwise.
The symbol "." stands for concatenation, for example,
{a1, ..., an}.{b1, ..., bm} = {a1, ..., an, b1, ..., bm}.
Prof. Amr Goneid, AUC 36
Merge AlgorithmMerge Algorithm
Merge(A,p,q,r)
{
copy Ap..r to Bp..r and set Ap..r to empty
while (neither L1 nor L2 empty)
{ compare first items of L1 & L2
remove smaller of the two from its list
add to end of A
}
concatenate remaining list to end of A
return A
}
p q r
array B
q+1L1 L2
Prof. Amr Goneid, AUC 37
ExampleExample
2 9 11 20 1 15 17 30
1
i j
p q rB
A
z
Prof. Amr Goneid, AUC 38
ExampleExample
2 9 11 20 1 15 17 30
1 2
i j
p q rB
A
z
Prof. Amr Goneid, AUC 39
ExampleExample
2 9 11 20 1 15 17 30
1 2 9
i j
p q rB
A
z
Prof. Amr Goneid, AUC 40
ExampleExample
2 9 11 20 1 15 17 30
1 2 9 11
i j
p q rB
A
z
Prof. Amr Goneid, AUC 41
ExampleExample
2 9 11 20 1 15 17 30
1 2 9 11 15
i j
p q rB
A
z
Prof. Amr Goneid, AUC 42
ExampleExample
2 9 11 20 1 15 17 30
1 2 9 11 15 17
i j
p q rB
A
z
Prof. Amr Goneid, AUC 43
ExampleExample
2 9 11 20 1 15 17 30
1 2 9 11 15 17 20
i j
p q rB
A
z
Prof. Amr Goneid, AUC 44
ExampleExample
2 9 11 20 1 15 17 30
1 2 9 11 15 17 20 30
i j
p q rB
A
z
Prof. Amr Goneid, AUC 45
Worst Case AnalysisWorst Case Analysis |L1| = size of L1, |L2| = size of L2
In the worst case |L1| = |L2| = n/2
Both lists empty at about same time, so everything has to be compared.
Each comparison adds one item to A so the worst case is
T(n) = |A|-1 = |L1|+|L2|-1 = n-1 = O(n)
comparisons.
Prof. Amr Goneid, AUC 46
(b) MergeSort Methodology(b) MergeSort Methodology
Invented by Von Neumann in 1945 Recursive Divide-And-Conquer
Prof. Amr Goneid, AUC 47
MergeSort MethodologyMergeSort Methodology
Divides the sequence into two subsequences of equal size, sorts the subsequences and then merges them into one sorted sequence
Fast, but uses an extra space
Prof. Amr Goneid, AUC 48
Methodology (continued)Methodology (continued)
Divide:
Divide n element sequence into two subsequences each of n/2 elements
Conquer:
Sort the two sub-arrays recursively Combine:
Merge the two sorted subsequences to produce a sorted sequence of n elements
Prof. Amr Goneid, AUC 49
AlgorithmAlgorithm
MergeSort (A, p, r) // Mergesort array A[ ] locations p..r {
if (p < r) // if there are 2 or more elements { q = (p+r)/2; // Divide in the middle // Conquer both MergeSort (A , p , q); MergeSort(A , q+1 , r); Merge(A , p , q , r); // Combine solutions}
}
Invoke as MergeSort(a , 1 , n)
Prof. Amr Goneid, AUC 50
Merge Sort ExampleMerge Sort Example8 3 2 9 7 1 5 4
8 3 2 9 7 1 5 4
8 3 2 9 7 1 5 4
8 3 2 9 7 1 5 4
3 8 2 9 1 7 4 5
2 3 8 9 1 4 5 7
1 2 3 4 5 7 8 9
Prof. Amr Goneid, AUC 51
http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/MSort.html
http://coderaptors.com/?MergeSort
Merge Sort DemosMerge Sort Demos
Prof. Amr Goneid, AUC 52
Worst case Analysis of MergeSortWorst case Analysis of MergeSort
Merge will use n-1 comparisons operations so that:
)log(1log)(
1)12(
2)2()1(2)(
2 n for giveson substituti Successive
)1()2()4/(2
)1(}1)2/()4/(2{2)(
0)1(1)2/(2)(
2
1
0
1
0
m
2
nnOnnnnTHence
nnmnm
nmnTnT
nnnT
nnnTnT
TwithnnTnT
m
m
i
m
i
iim
Prof. Amr Goneid, AUC 53
Performance of MergeSortPerformance of MergeSort
MergeSort divides the array to two halves at each level. So, the number of levels is O(log n)
At each level, merge will cost O(n) Therefore, the complexity of MergeSort is
O(n log n) In-Place Sort No (uses extra array) Stable Algorithm Yes
This technique is satisfactory for large data sets
Prof. Amr Goneid, AUC 54
6. QuickSort6. QuickSort(a) Methodology(a) Methodology
Invented by Sir Tony Hoare in 1962 Recursive Divide-And-Conquer algorithm Partitions array around a Pivot , then sorts parts
independently Fast, in-place sorting
Prof. Amr Goneid, AUC 55
(a) Methodology(a) Methodology
pivot
Left Right
qp r
Divide:Array a[p..r] is rearranged into a sub-array a[p..q-1], a pivot a[q] and a second sub-array a[q+1..r] such that each element of the first is <= pivot and each element of the second is >= pivot ( the pivot index (q) is computed as part of this process)
Prof. Amr Goneid, AUC 56
Methodology (continued)Methodology (continued)
Conquer:Sort the two sub-arrays recursively
Combine: The sub-arrays are already sorted in place, i.e., a[p..r] is sorted.
Prof. Amr Goneid, AUC 57
(b) Algorithm(b) AlgorithmQuickSort (a, p , r)// Sorts the elements a[p],..., a[r] which reside in// the global array a[1:n] into ascending order; // a[n+1] is considered to be defined and must be >= all the// elements in a[1:n].{
if (p < r ) { // If there are more than one elementq = partition (a , p , r+1); // partition around Pivot// q is the position of the pivotQuickSort (a , p ,q-1); // Sort left sub-arrayQuickSort (a , q+1 , r); // Sort Right sub-array
}}
Prof. Amr Goneid, AUC 58
PartitioningPartitioning
int partition(a, p, r){
pivot = ap; // pivot is initially the first element i = p; j = r;do{
do {i++; } while (ai < pivot);do {j--; } while (aj > pivot);if(i < j) swap (ai , aj);
} while (i < j);ap = aj; aj = pivot;return (j);// j is the final location of the pivot
}
Prof. Amr Goneid, AUC 59
ExampleExample
5 3 1 9 8 2 4 7
2 3 1 4 5 8 9 7
1 2 3 4 5 7 8 9
1 2 3 4 5 7 8 9
1 2 3 4 5 7 8 9
1 2 3 4 5 7 8 9
Prof. Amr Goneid, AUC 60
http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/QSort.html
http://coderaptors.com/?QuickSort
Quick Sort DemoQuick Sort Demo
Prof. Amr Goneid, AUC 61
(c)Analysis of QuickSort(c)Analysis of QuickSort
Partitioning will cost (n) comparisons. Sorting costs T(i) for left
sub-array + pivot, T(n-i) for right sub-array. Hence:
T(n) = n + T(i) + T(n-i) for n > 1 with T(1) = 0
i elements n-i elements
partition cost = n
Prof. Amr Goneid, AUC 62
Best CaseBest Case
Best Case when pivot has the middle value. In this case, the array is partitioned into two almost equal halves.
T(n) = 2T(n/2) + n for n > 1 with T(1) = 0
hence, T(n) = O(n log n)
n/2 elements n/2 elements
partition cost = n
Prof. Amr Goneid, AUC 63
Worst CaseWorst Case
Worst Case when pivot is the smallest element so that i = 1
T(n) = T(n-1) + n for n > 1
with T(1) = 0
Hence, T(n) = n(n+1)/2 -1= O(n2)
In this case, Quicksort is not really quick!
1 n-1 elements
partition cost = n
Prof. Amr Goneid, AUC 64
Summary ofSummary ofPerformance of QuickSortPerformance of QuickSort
Partitioning will cost O(n) at each level Best Case: Pivot has the middle value
Quicksort divides the array to two equal halves at each level. So, the number of levels is O(log n)Therefore, the complexity of QuickSort is O(n log n)
Worst Case: Pivot is the minimum (maximum) valueThe number of levels is O(n)Therefore, the complexity of QuickSort is O(n2)
In-Place Sort Yes Stable Algorithm No This technique is satisfactory for large data sets
Prof. Amr Goneid, AUC 65
Median-of-Three PartitioningMedian-of-Three Partitioning
If we take the elements in the first, last and middle locations in the array and then take the element middle in value between these three, this will be the median of three. In this case, there is always at least one element below and one element above. The worst case is therefore unlikely
Prof. Amr Goneid, AUC 66
A Better Choice: Random PivotA Better Choice: Random Pivot
A better solution that avoids the worst case of O(n2) is to use a randomizer to select the pivot element.
Pick a random element from the sub-array (p..r) as the partition element.
Do this only if (r-p) > 5 to avoid cost of the randomizer
Prof. Amr Goneid, AUC 67
A Better Choice: Random PivotA Better Choice: Random Pivot
void RQuickSort (a , p , r){
if (p < r ) { if ((r-p) > 5) { int m = rand()%(r-p+1) + p;
swap (ap , am); }q = partition(a , p , r+1);RQuickSort(a , p ,q-1);RQuickSort(a , q+1 , r);
}}
Prof. Amr Goneid, AUC 68
Average Case AnalysisAverage Case Analysis
If the pivot is randomly chosen using a random number generator, we have (n-1) equally probable configurations:
{1,n-1},{2,n-2},..., {i, n-i},..{n-2,2},{n-1,1}
The probability of each configuration is 1/(n-1)
We obtain an average case cost given by:
1
1
})()({1
1)(
n
i
ninTiTn
nT
Prof. Amr Goneid, AUC 69
Average Case AnalysisAverage Case Analysis
nn
nT
n
nT
nniTnTnAlso
nniTnTnAnd
niTn
nTHence
iTinT
n
i
n
i
n
i
n
i
2
1
)1()(get 1st we from 2ndSubtract
)2()1()(2)1()2(
)1()(2)()1(
)(1
2)(
)()(
: thatrecall weequation, thissolve To
2
1
1
1
1
1
1-n
1i
1
1
Prof. Amr Goneid, AUC 70
Average Case AnalysisAverage Case Analysis
)log()( still is case average theSo,
log386.1log/log2ln2)(
ln1
equations, usefull from Also
0T(1) F(1) andT(n)/n F(n) that Recall
12)1()(
:solution with the
)/2()1()(
:as written becan This
2
2
nnOnT
nnennnnnTHence
ni
iFnF
nnFnF
n
i
n
i
Prof. Amr Goneid, AUC 71
See Internet Animation Sites:See Internet Animation Sites:
For Example: http://www.cs.ubc.ca/spider/harrison/Java/sorting-
demo.html
http://coderaptors.com/?All_sorting_algorithms
Prof. Amr Goneid, AUC 72
7. An Application of Quicksort:7. An Application of Quicksort:Convex HullConvex HullThe Problem:Given a set S of n points in 2-dimensional space, find
the smallest convex polygon containing all the points of
S.
Prof. Amr Goneid, AUC 73
Convex HullConvex Hull The convex hull of a set of points is an elementary
problem in computational geometry. The hull gives a rough idea of the shape or extent
of a data set. It is an important step for various geometric
algorithms. For example, finding the diameter of a set of points.
A famous algorithm is the Graham Scan
Prof. Amr Goneid, AUC 74
Graham ScanGraham Scan Select lowest-y point (p) Compute angle i between
lines p-pi and the x-axis
Sort points according to Scan through the sorted set
starting at p: select 3 successive points if they form a left turn, move one point ahead if not, delete the 2nd point and move back repeat until we reach point p again
p
x
y
1
2
3
Prof. Amr Goneid, AUC 75
Graham ScanGraham Scan Triplet = (p,1,2) Left, move to (1)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 76
Graham ScanGraham Scan Triple = (1,2,3)Right, Delete (2), back to (p)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 77
Graham ScanGraham Scan Triplet = (p,1,3) left, Move to (1)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 78
Graham ScanGraham Scan Triplet = (1,3,4) left, Move to (3)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 79
Graham ScanGraham Scan Triplet = (3,4,5) left, move to (4)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 80
Graham ScanGraham Scan Triplet = (4,5,6) Right, Delete (5), back to (3)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 81
Graham ScanGraham Scan Triplet = (3,4,6) left, move to (4)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 82
Graham ScanGraham Scan Triplet = (4,6,7) left, Move to (6)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 83
Graham ScanGraham Scan Triplet = (6,7,p) left, Move to (7), 3rd point is (p)
p
x
y
1
2
3
Prof. Amr Goneid, AUC 84
Graham ScanGraham Scan Stop, Final Hull
p
x
y
1
2
3
Prof. Amr Goneid, AUC 85
Graham ScanGraham Scan
After the scan, all points remaining will lie on the hull To distinguish a left turn from a right turn, we compute the
signed area of the triangle formed by the point triplet:signed area = (1/2) det (A)
det (A) = the determinant of the matrix:
If the area is (+)ve, we have a left turn, if (-)ve , we have a right turn. If zero, the 3 points are co-linear.
111321
321
yyy
xxx
A
Prof. Amr Goneid, AUC 86
Graham Scan AlgorithmGraham Scan Algorithm
void GScan (pointList S) {
// p is the start point in S//(p1 , p2 , p3) is a point triplet// A is the signed area of the triangle// formed by the point tripletset p1 = p;
do { p2 = succ(p1);
if (p2 is not the last point) then p3 = succ(p2); else p3 = p;
Prof. Amr Goneid, AUC 87
Graham Scan AlgorithmGraham Scan Algorithm
A = Area(p1 , p2 , p3); // signed area// if p1,p2,p3 form a left turn
// move one point ahead; // if not, delete p2 and move back.
if (A >= 0.0) p1 = succ(p1); // left turn else { // right turn delete p2; p1 = pred(p1); } } while (!((p3 == p) && (A >= 0.0))); }
Prof. Amr Goneid, AUC 88
Graham Scan AlgorithmGraham Scan Algorithm
An implementation of the Graham scan algorithm is given in the Book, p. 187
It uses a doubly linked list to represent the sorted point list.
The scan algorithm runs in O(n) time. Since sorting takes O(n log n) time, the total
time of Graham scan algorithm is
O(n log n)
Prof. Amr Goneid, AUC 89
Graham Scan DemoGraham Scan Demo
http://www.cosc.canterbury.ac.nz/people/mukundan/cgeo/Gscan.html
http://www.cs.princeton.edu/courses/archive/fall08/cos226/demo/ah/GrahamScan.html
Prof. Amr Goneid, AUC 90
8. Selection by Partitioning8. Selection by Partitioning
The Selection Problem
Let A be an array of n distinct elements. Find
the kth smallest element.
Some Applications
Finding nearest neighbor
Finding the Median
Partition around median for closest pair problem
Traveling salesman problem
Prof. Amr Goneid, AUC 91
Selection by PartitioningSelection by Partitioning
Algorithms: Several algorithms exist. Sort array (e.g. by Mergesort), kth smallest element
will be located at kth location. Complexity is
O(n logn) Insert all elements in a minimum Heap, then remove
k elements from Heap. Last removed is the kth
smallest. Complexity is O(n logn) A more efficient algorithm uses partitioning around a
pivot like Quicksort. It is a Divide and Conquer algorithm with an average cost of O(n).
Prof. Amr Goneid, AUC 92
•Selection by PartitioningSelection by PartitioningLet A [1..n] be an array of n distinct elements. Find the kth smallest element
from location p through location r. Invoke as select (A, 1, n, k). A[n+1] is
considered to be defined and must be >= all the elements in A[1:n].
Type select (Type A[ ], p, r, k){
if (p == r) return A[p]; else{ //Choose a pivot (v) from A and partition A around (v) // Let A1, A2 be the elements of A which are respectively <,> v
q = partition (A, p, r+1); // new pivot location// A1 = elements from p to q-1, A2 = elements from q+1 to rL = q – p + 1;if (k = = L) return A[q];
else if (k < L) return select (A, p, q-1, k);else return select (A, q+1, r, k − L);
}}
Prof. Amr Goneid, AUC 93
Selection by PartitioningSelection by Partitioning
A1 v A21 j j+2 nq
cost =T(j)whenk <= j
i.e.j = k .. n-1
cost =T(n-j-1)when
k > j+1i.e.
j = 0..k-2
partitioncost = n
Prof. Amr Goneid, AUC 94
Selection by PartitioningSelection by Partitioning
Analysis: The cost of partitioning is (n) Let |A1| = j and |A2| = n-j-1 The pivot position is q = j+1 with j = 0..n-1 We call select on A1 when k <= j, i.e. in the cases of j = k...n-1 We call select on A2 when k > j+1, i.e. in the cases of j = 0 .. k-2 Each case happens with equal probability (1/n) Hence the average cost will be:
0)1(
2,)}1()({1
)(1 2
0
T
nnjnTjTn
nTn
kj
k
j
Prof. Amr Goneid, AUC 95
Average Case AnalysisAverage Case Analysis
)()1(2)(solution thehas recurrence This
0)1( with 2)1()(get 1st we from 2ndSubtract
)1()(2)1()1(
)(2)(
:nby sidesboth Multiply .2/ when is maximum The
k with decreases )(
k with increases )()1(
: thatrecall weequation, thissolve To
22
2/
1
2/
2
1
2-k
0j
1
1
nOnnT
TnTnT
njTnTnAnd
njTnnTHence
nk
jT
jTjnT
n
nj
n
nj
n
kj
n
knj
Prof. Amr Goneid, AUC 96
Selection using a BSTSelection using a BST
Type select (Type A[ ], int n, int k){
Type Amin;binaryTree<Type, int> BST; for (int i = 0; i<n; i++) BST.insert(A[i],0);for (i = 1; i < k; i++) { Amin = BST.min(); BST.remove(Amin);} return BST.min();}
invoke as select (A, n, k)AnalysisInsert n elements O(n log n)Find and delete (k-1) minima O((k-1) log n)Find kth min O(log n)Total O((k+n) log n)
Prof. Amr Goneid, AUC 97
Other Selection AlgorithmsOther Selection Algorithms
Explore other selection algorithms at:http://en.wikipedia.org/wiki/Selection_algorithm
Prof. Amr Goneid, AUC 98
9. Closest Pair Problem9. Closest Pair Problem
The Closest Pair Problem in 1-DGiven a set S = {p1, p2, ..., pN} of N points on a line,
find the two points of S whose distance is smallest.
Prof. Amr Goneid, AUC 99
Closest Pair ProblemClosest Pair Problem
Simple solution
Dmin=
for each pi in S
for each pj in S and (i != j)
if (Dij < Dmin) { Dmin= Dij ; K = i; L = j;}
return K , L, Dmin
Time Complexity = O(N2) ,
Space Complexity = O(N)
Can we do better?
Prof. Amr Goneid, AUC 100
Closest Pair ProblemClosest Pair Problem
Solution with Sorting Sort the points according to their distance from the
origin. After the points are sorted, the closest pair will lie
next to each other somewhere in sorted order. Thus a linear-time scan through them completes the job.
Sorting will cost O(N log N) and the scan will cost O(N) so that the total cost is O(N log N) + O(N) = O(N log N).
Obviously, the use of sorting is much more efficient than the first method.
Prof. Amr Goneid, AUC 101
Closest Pair ProblemClosest Pair Problem
D&Q Algorithm in 1-D
Partition S, a set of points on a line, into two sets S1 and S2 at some point m such that for every point p S1 and q S2, p < q. Point m could be the median
Solve Closest Pair recursively on S1 and S2 Let {p1, p2} be the closest pair in S1 with 1 =
|p2 - p1| and {q1, q2} the closest pair in S2 with 2 =|q2 - q1|
Prof. Amr Goneid, AUC 102
Closest Pair ProblemClosest Pair Problem
Let be the smallest distance found so far:
= min (1, 2) The closest pair in S is either {p1,
p2} or {q1, q2} or some {p3, q3} with p3 S1 and q3 S2.Which one?
Prof. Amr Goneid, AUC 103
Closest Pair ProblemClosest Pair Problem
Let be the smallest separation found so
far:
= min(|p2 - p1|; |q2 - q1|)
Prof. Amr Goneid, AUC 104
Closest Pair ProblemClosest Pair Problem
How many points of S1 or S2 can lie within of m?
Since is the distance between the closest pair in either S1 or S2, there can only be at most 1 point in each side.
Therefore, the number of distance computations required to check for a closest pair {p3, q3} with p3 S1 and q3 S2 is O(1).
Prof. Amr Goneid, AUC 105
Closest Pair Problem:Closest Pair Problem:D&Q Algorithm for 1-DD&Q Algorithm for 1-DClosest-Pair (S) If |S| = 1, = else If |S| = 2, = |p2 - p1|; else do the following steps:
1. Let m = median(S).2. Divide S into S1; S2 at m.
3. 1 = Closest-Pair(S1).
4. 2 = Closest-Pair(S2).
5. 12 is minimum distance across the cut = |p3 – q3|
6. Return = min(1 , 2 ,,12).
Prof. Amr Goneid, AUC 106
Closest Pair Problem:Closest Pair Problem:D&Q Algorithm for 1-DD&Q Algorithm for 1-DAnalysis: Finding the median point (selection problem) is O(N) Partitioning around the median is O(N) Computing in each recursive call is O(1) Hence, T(N) = 2T(N/2)+O(N) so that
T(N) = O(N log N)Source Material for 1-D and 2-D cases:http://www.eecs.tufts.edu/~tklein/closestpair.htmhttp://www.cs.mcgill.ca/~cs251/ClosestPair/http://www.cs.ucsb.edu/~suri/cs235/http://www.cs.ucf.edu/courses/cot5520/lectures.htmlAnimationhttp://www.cs.mcgill.ca/~cs251/ClosestPair/ClosestPairApplet/
ClosestPairApplet.html