Transform and Conquer
description
Transcript of Transform and Conquer
Design and Analysis of Algorithms - Chapter 6 1
Transform and ConquerTransform and Conquer
Solve problem by transforming into:Solve problem by transforming into: a more convenient instance of the same problem a more convenient instance of the same problem
((instance simplificationinstance simplification))• Presorting, Gaussian elimination, matrix inversion, Presorting, Gaussian elimination, matrix inversion,
determinant computationdeterminant computation a different representation of the same instance a different representation of the same instance
((representation changerepresentation change))• balanced search trees, heaps and heapsort, polynomial balanced search trees, heaps and heapsort, polynomial
evaluation by Horner’s rule, Fast Fourier Transformevaluation by Horner’s rule, Fast Fourier Transform a different problem altogether (a different problem altogether (problem reductionproblem reduction))
• reductions to graph problems, linear programming reductions to graph problems, linear programming
Design and Analysis of Algorithms - Chapter 6 2
Instance simplification - Instance simplification - PresortingPresorting
Solve instance of problem by transforming into Solve instance of problem by transforming into another simpler/easier instance of the same problemanother simpler/easier instance of the same problem
Presorting:Presorting:Many problems involving lists are easier when list is Many problems involving lists are easier when list is
sorted.sorted. element uniquenesselement uniqueness computing the modecomputing the mode finding repeated elementsfinding repeated elements searching searching computing the median (selection problem)computing the median (selection problem)
Design and Analysis of Algorithms - Chapter 6 3
Selection ProblemSelection Problem
Find the Find the kk-th-th smallest element in A[1],…A[smallest element in A[1],…A[nn]. ]. • minimum: k minimum: k = 1= 1• maximum: k maximum: k = = nn• median: k median: k = = nn/2/2
Presorting-based algorithm Presorting-based algorithm • sort listsort list• return A[return A[kk]]
Partition-based algorithm (decrease & conquer):Partition-based algorithm (decrease & conquer):• pivot/split at A[pivot/split at A[ss] using partitioning algorithm] using partitioning algorithm• if if ss==k k return A[ return A[ss] ] • else if else if s<k s<k repeat with sublist A[repeat with sublist A[ss+1],…A[+1],…A[nn]. ]. • else if s>k else if s>k repeat with sublist A[1],…A[repeat with sublist A[1],…A[ss-1]. -1].
Design and Analysis of Algorithms - Chapter 6 4
Notes on Selection ProblemNotes on Selection Problem
Presorting-based algorithm: Presorting-based algorithm: ΩΩ((nnlglgnn) + ) + ΘΘ(1) = (1) = ΩΩ((nnlglgnn) ) Partition-based algorithm (decrease & conquer):Partition-based algorithm (decrease & conquer):
• worst case: T(worst case: T(nn) =T() =T(nn-1) + (-1) + (nn+1) -> +1) -> ΘΘ((nn22) )
• best case: best case: ΘΘ((nn) )
• average case: T(average case: T(nn) =T() =T(nn/2) + (/2) + (nn+1) -> +1) -> ΘΘ((nn) )
• Bonus:Bonus: also identifies the also identifies the kk smallest elements smallest elements
Special cases max, min: better, simpler linear Special cases max, min: better, simpler linear algorithm (brute force)algorithm (brute force)
Conclusion:Conclusion: Presorting does Presorting does notnot help in this case. help in this case.
Design and Analysis of Algorithms - Chapter 6 5
Finding repeated elementsFinding repeated elements
Presorting-based algorithm: Presorting-based algorithm: • use mergesort (optimal): use mergesort (optimal): ΘΘ((nnlglgnn) ) • scan array to find repeated scan array to find repeated adjacentadjacent elements: elements:
ΘΘ((nn) ) • in total it makes: in total it makes: ΘΘ((nnlglgnn) )
Brute force algorithm: Brute force algorithm: ΘΘ((nn22) ) Conclusion:Conclusion: Presorting yields Presorting yields significantsignificant
improvementimprovement
Design and Analysis of Algorithms - Chapter 6 6
Checking element uniquenessChecking element uniqueness
Brute force algorithm: Brute force algorithm: ΘΘ((nn22) ) AlgorithmAlgorithm PresortedElementUniqueness PresortedElementUniqueness
Sort the array ASort the array A
for i for i 0 to n-2 do 0 to n-2 do
if A[i]=A[i+1] return falseif A[i]=A[i+1] return false
else return trueelse return true Conclusion:Conclusion: Presorting again improves Presorting again improves Similar improvement for modeSimilar improvement for mode
Design and Analysis of Algorithms - Chapter 6 7
Checking modeChecking mode
ModeMode is the most often met element is the most often met element Brute force: scan the list and compute the frequencies. Brute force: scan the list and compute the frequencies.
Then find the largest frequencyThen find the largest frequency AlgorithmAlgorithm PresortedMode PresortedMode
Sort the array ASort the array Ai i 1; modefrequency 1; modefrequency 0; 0;while i while i n-1 do n-1 do
runlengthrunlength1; runvalue1; runvalueA[i];A[i];while i+runlength≤n-1 and A[i+runlength]=runvalue while i+runlength≤n-1 and A[i+runlength]=runvalue
runlength runlength runlength+1 runlength+1if runlength> modefrequencyif runlength> modefrequency
modefrequencymodefrequencyrunlength, modevaluerunlength, modevaluerunvaluerunvalueiii+runlengthi+runlength
return modevaluereturn modevalue Conclusion:Conclusion: Presorting again improves Presorting again improves
Design and Analysis of Algorithms - Chapter 6 8
Gaussian EliminationGaussian Elimination
Given a system of two linear equations with two Given a system of two linear equations with two unknownsunknowns
aa1111x + ax + a1212y = by = b11
aa2121x + ax + a2222y = by = b22
It has a unique solution unless the coefficients are It has a unique solution unless the coefficients are proportional proportional
Express one variable as function of the other and Express one variable as function of the other and substitute to solve one equation.substitute to solve one equation.
What if the system has What if the system has nn equations and equations and nn unknowns ? unknowns ?
Design and Analysis of Algorithms - Chapter 6 9
Gaussian Elimination (2)Gaussian Elimination (2)
Transform Ax=b to A’x=b’, where A’ upper triangularTransform Ax=b to A’x=b’, where A’ upper triangular Then, solution is possible with backward substitutionThen, solution is possible with backward substitution Elementary operationsElementary operations
• Exchange equationsExchange equations
• Replace an equation with a nonzero multipleReplace an equation with a nonzero multiple
• Replace an equation with a sum or difference of this equation and Replace an equation with a sum or difference of this equation and some multiple of another equationsome multiple of another equation
ExampleExample
2x2x11 - x - x22 + x + x33 = 1 = 1
4x4x11 + x + x22 - x - x33 = 5 = 5
xx11 + x + x22 + x + x33 = 0 = 0
Design and Analysis of Algorithms - Chapter 6 10
Gaussian Elimination (3)Gaussian Elimination (3)
Algorithm GaussEliminationAlgorithm GaussElimination
for i for i 1 to n do A[i,n+1] 1 to n do A[i,n+1]b[i]b[i]
for ifor i 1 to n-1 do 1 to n-1 do
for jfor ji+1 to n doi+1 to n do
for kfor ki to n+1 doi to n+1 do
A[j,k]A[j,k]A[j,k]-A[i,k]*A[j,i]/A[i,i]A[j,k]-A[i,k]*A[j,i]/A[i,i]
Potential problems if A[i,i] is zero or very smallPotential problems if A[i,i] is zero or very small
Design and Analysis of Algorithms - Chapter 6 11
Gaussian Elimination (partial Gaussian Elimination (partial pivoting)pivoting)
Algorithm GaussElimination2Algorithm GaussElimination2for i for i 1 to n do A[i,n+1] 1 to n do A[i,n+1]b[i]b[i]for ifor i 1 to n-1 do 1 to n-1 do
pivotrowpivotrowiifor jfor ji+1 to n doi+1 to n do
if |A[j,i]|>|A[pivot,i]| pivotrowif |A[j,i]|>|A[pivot,i]| pivotrowjjfor kfor ki to n+1 doi to n+1 do
swap(A[i,k],A[pivotrow,k])swap(A[i,k],A[pivotrow,k])for jfor ji+1 to n doi+1 to n do
temptempA[j,i]/A[i,i]A[j,i]/A[i,i]for kfor kI to n+1 doI to n+1 do
A[j,k]A[j,k]A[j,k]-A[I,k]*tempA[j,k]-A[I,k]*temp EfficiencyEfficiency
Design and Analysis of Algorithms - Chapter 6 12
LU decompositionLU decomposition
Byproduct of Gaussian EliminationByproduct of Gaussian Elimination Example A=LUExample A=LU
LUx=b. Denote y=Ux LUx=b. Denote y=Ux Ly=b Ly=b Solve Ly=b, then solve Ux=ySolve Ly=b, then solve Ux=y Solve with as many times with different b’s.Solve with as many times with different b’s. No extra spaceNo extra space
11 00 00
22 11 00
1/21/2 1/21/2 11
22 -1-1 11
00 33 -3-3
00 00 22
Design and Analysis of Algorithms - Chapter 6 13
Computing a matrix inverseComputing a matrix inverse
AAAA-1-1=I=I A A singularsingular matrix does not have an inverse matrix does not have an inverse A matrix is singular if and only if one of the A matrix is singular if and only if one of the
rows is a linear combination of the other rows.rows is a linear combination of the other rows. Apply Gaussian elimination. If it yields an Apply Gaussian elimination. If it yields an
upper-triangular with no zeros on the upper-triangular with no zeros on the diagonal, then the matrix is not singulardiagonal, then the matrix is not singular
AxAxjj=e=ejj
Design and Analysis of Algorithms - Chapter 6 14
Computing the determinantComputing the determinant
A well-known recursive formulaA well-known recursive formula What if n is large ? Efficiency ?What if n is large ? Efficiency ? Apply Gaussian elimination.Apply Gaussian elimination. The determinant of an upper-triangular The determinant of an upper-triangular
matrix is the product of elements on its matrix is the product of elements on its diagonal.diagonal.
Efficiency ?Efficiency ?
Cramer’s ruleCramer’s rule
Design and Analysis of Algorithms - Chapter 6 15
Taxonomy of Searching Taxonomy of Searching AlgorithmsAlgorithms
Elementary searching algorithmsElementary searching algorithms• sequential searchsequential search• binary searchbinary search• binary tree searchbinary tree search
Balanced tree searchingBalanced tree searching• AVL treesAVL trees• red-black treesred-black trees• multiway balanced trees (2-3 trees, 2-3-4 trees, B trees)multiway balanced trees (2-3 trees, 2-3-4 trees, B trees)
HashingHashing• separate chainingseparate chaining• open addressingopen addressing
Design and Analysis of Algorithms - Chapter 6 16
Balanced trees: AVL treesBalanced trees: AVL trees
For every node, difference in height between left For every node, difference in height between left and right subtree is at most 1and right subtree is at most 1
AVL property is maintained through AVL property is maintained through rotationsrotations, , each time the tree becomes unbalancedeach time the tree becomes unbalanced
lg lg nn ≤≤ hh ≤ 1.4404 lg (≤ 1.4404 lg (n n + 2) - 1.3277 + 2) - 1.3277 average:average: 1.01 lg 1.01 lg n + n + 0.1 for large 0.1 for large nn
Disadvantage: needs extra storage for maintaining Disadvantage: needs extra storage for maintaining node balancenode balance
A similar idea: red-black trees (height of subtrees A similar idea: red-black trees (height of subtrees is allowed to differ by up to a factor of 2)is allowed to differ by up to a factor of 2)
Design and Analysis of Algorithms - Chapter 6 17
AVL tree rotationsAVL tree rotations
Small examples:Small examples:• 1, 2, 31, 2, 3• 3, 2, 1 3, 2, 1 • 1, 3, 21, 3, 2• 3, 1, 23, 1, 2
Larger example: 4, 5, 7, 2, 1, 3, 6Larger example: 4, 5, 7, 2, 1, 3, 6 See figures 6.4, 6.5 for general cases of rotations; See figures 6.4, 6.5 for general cases of rotations;
Design and Analysis of Algorithms - Chapter 6 18
General case: single R-rotationGeneral case: single R-rotation
Design and Analysis of Algorithms - Chapter 6 19
Double LR-rotationDouble LR-rotation
Design and Analysis of Algorithms - Chapter 6 20
Balance factorBalance factor Algorithm maintains Algorithm maintains balance factorbalance factor for each node. For for each node. For
example:example:
Design and Analysis of Algorithms - Chapter 6 21
HeapsortHeapsort
Definition:Definition:A A heapheap is a binary tree with the following conditions: is a binary tree with the following conditions: it is essentially complete:it is essentially complete:
The key at each node is The key at each node is ≥ keys at its children≥ keys at its children
Design and Analysis of Algorithms - Chapter 6 22
Definition implies:Definition implies:
Given Given n,n, there exists a unique binary tree with there exists a unique binary tree with nn nodes that is essentially complete, with nodes that is essentially complete, with hh= lg = lg nn
The root has the largest keyThe root has the largest key
The subtree rooted at any node of a heap is also a The subtree rooted at any node of a heap is also a heapheap
Design and Analysis of Algorithms - Chapter 6 23
Heapsort Algorithm:Heapsort Algorithm:
1.1. Build heapBuild heap
2.2. Remove root –exchange with last (rightmost) leafRemove root –exchange with last (rightmost) leaf
3.3. Fix up heap (excluding last leaf)Fix up heap (excluding last leaf)
Repeat 2, 3 until heap contains just one node.Repeat 2, 3 until heap contains just one node.
Design and Analysis of Algorithms - Chapter 6 24
Heap constructionHeap construction
Insert elements in the order given breadth-first in Insert elements in the order given breadth-first in a binary treea binary tree
Starting with the last (rightmost) parental node, Starting with the last (rightmost) parental node, fix the heap rooted at it, if it does not satisfy the fix the heap rooted at it, if it does not satisfy the heap condition:heap condition:1.1. exchange it with its largest childexchange it with its largest child
2.2. fix the subtree rooted at it (now in the child’s position)fix the subtree rooted at it (now in the child’s position)
Example: 2 3 6 7 5 9Example: 2 3 6 7 5 9
Design and Analysis of Algorithms - Chapter 6 25
Root deletionRoot deletion
The root of a heap can be deleted and the heap fixed The root of a heap can be deleted and the heap fixed up as follows:up as follows:
exchange the root with the last leaf exchange the root with the last leaf compare the new root (formerly the leaf) with each compare the new root (formerly the leaf) with each
of its children and, if one of them is larger than the of its children and, if one of them is larger than the root, exchange it with the larger of the two. root, exchange it with the larger of the two.
continue the comparison/exchange with the continue the comparison/exchange with the children of the new root until it reaches a level of children of the new root until it reaches a level of the tree where it is larger than both its children the tree where it is larger than both its children
Design and Analysis of Algorithms - Chapter 6 26
RepresentationRepresentation
Use an array to store breadth-first traversal of heap tree:Use an array to store breadth-first traversal of heap tree: Example:Example:
Left child of node Left child of node jj is at 2 is at 2jj Right child of node Right child of node jj is at 2 is at 2jj+1+1 Parent of node Parent of node jj is at is at j j /2/2 Parental nodes are represented in the first Parental nodes are represented in the first nn /2 locations /2 locations
9
1
5 3
4 2
1 2 3 4 5 6
9 5 3 1 4 2
Design and Analysis of Algorithms - Chapter 6 27
Bottom-up heap construction Bottom-up heap construction algorithmalgorithm
Design and Analysis of Algorithms - Chapter 6 28
Analysis of HeapsortAnalysis of Heapsort
See algorithm HeapBottomUp in section 6.4See algorithm HeapBottomUp in section 6.4 Fix heap with “problem” at height Fix heap with “problem” at height j: j: 2 2jj
comparisonscomparisons For subtree rooted at level For subtree rooted at level ii it does 2( it does 2(hh--ii) )
comparisonscomparisons Total for heap construction phase:Total for heap construction phase:
Σ 2(h-i) 2i = 2 ( n – lg (n + 1)) = Θ(n)i=0
h-1
# nodes at level i
Design and Analysis of Algorithms - Chapter 6 29
Analysis of Heapsort Analysis of Heapsort (continued)(continued)
Recall algorithm:Recall algorithm:1.1. Build heapBuild heap
2.2. Remove root –exchange with last (rightmost) leafRemove root –exchange with last (rightmost) leaf
3.3. Fix up heap (excluding last leaf)Fix up heap (excluding last leaf)
Repeat 2, 3 until heap contains just one node.Repeat 2, 3 until heap contains just one node.
Θ(n)
Θ(log n)
n – 1 times
Total:Total: Θ(n) + Θ( n log n) = Θ(n log n)
• Note:Note: this is the worst case. Average case also Θ(n log n).
Design and Analysis of Algorithms - Chapter 6 30
Priority queuesPriority queues
A A priority queue priority queue is the ADT of an ordered set with is the ADT of an ordered set with the operations:the operations:• find element with highest priority find element with highest priority
• delete element with highest priority delete element with highest priority
• insert element with assigned priority insert element with assigned priority
Heaps are very good for implementing priority Heaps are very good for implementing priority queuesqueues
Design and Analysis of Algorithms - Chapter 6 31
Insertion of a new elementInsertion of a new element
Insert element at last position in heap. Insert element at last position in heap. Compare with its parent and if it violates heap Compare with its parent and if it violates heap
condition exchange themcondition exchange them Continue comparing the new element with nodes Continue comparing the new element with nodes
up the tree until the heap condition is satisfiedup the tree until the heap condition is satisfied
Example:Example:
Efficiency:Efficiency:
Design and Analysis of Algorithms - Chapter 6 32
Bottom-up vs. Top-down heap Bottom-up vs. Top-down heap constructionconstruction
Top down:Top down: Heaps can be constructed by Heaps can be constructed by successively inserting elements into an (initially) successively inserting elements into an (initially) empty heapempty heap
Bottom-up:Bottom-up: Put everything in and then fix it Put everything in and then fix it
Which one is better?Which one is better?
Design and Analysis of Algorithms - Chapter 6 33
Horner’s ruleHorner’s rule
Horner published in early 19Horner published in early 19thth century century According to Knuth, the method was used by NewtonAccording to Knuth, the method was used by Newton Evaluate a polynomial at a point xEvaluate a polynomial at a point x
p(x) = ap(x) = annxxnn + a + an-1n-1xxn-1n-1 + … + a + … + a11x + ax + a00
p(x) = ( … (ap(x) = ( … (annx + ax + an-1n-1) x + … )x + a) x + … )x + a00
Example: evaluate p(x)=2xExample: evaluate p(x)=2x44-x-x33+3x+3x22+x-5 at x=3+x-5 at x=3
p(x) = x (x (x (2x-1) + 3) + 1) - 5p(x) = x (x (x (2x-1) + 3) + 1) - 5 Visualization by a tableVisualization by a table
Design and Analysis of Algorithms - Chapter 6 34
Horner’s rule [2]Horner’s rule [2]
Algorithm Horner(P[0..n],x)Algorithm Horner(P[0..n],x)
// Evaluate polynomial at a given point// Evaluate polynomial at a given point
// Input: an array P[0..n] of coefficients and a number x// Input: an array P[0..n] of coefficients and a number x
//Output: the value of polynomial at point x//Output: the value of polynomial at point x
p p P[n] P[n]
for i for i n-1 down to 0 do n-1 down to 0 do
p p x*p + P[i] x*p + P[i]
return preturn p Efficiency ?Efficiency ? Byproduct: coefficients of the quotient of the division of p(x) Byproduct: coefficients of the quotient of the division of p(x)
by (x-xby (x-x00))
Design and Analysis of Algorithms - Chapter 6 35
Binary exponentiationBinary exponentiation
Horner is not efficient to compute p(x)=xHorner is not efficient to compute p(x)=xnn at x=a at x=a Degenerate to brute forceDegenerate to brute force Let the binary representation n=bLet the binary representation n=bllbbl-1l-1… b… bii … b … b11bb00
p(x) = bp(x) = bllxxll + b + bl-1l-1xxl-1l-1 + … + b + … + b11x + bx + b00 and x=2 and x=2 Algorithm LeftRightBinaryExponentiationAlgorithm LeftRightBinaryExponentiation
product product a afor i for i l-1 down to 0 do l-1 down to 0 do
product product product * product product * product
if bif bii1 then product 1 then product product*a product*a
return productreturn product Example: compute aExample: compute a1313, n=13=1101, n=13=1101 EfficiencyEfficiency
Design and Analysis of Algorithms - Chapter 6 36
Binary exponentiation (2)Binary exponentiation (2)
Compute aCompute ann Consider n = bConsider n = bll22ll + b + bl-1l-122l-1l-1 + … + b + … + b112 + b2 + b00
and multiply independent powers termsand multiply independent powers terms Algorithm RightLeftBinaryExponentiationAlgorithm RightLeftBinaryExponentiation
term term a aif bif b00=1 then product =1 then product a aelse product else product 1 1for ifor i 1 to l do 1 to l do
term term term * term term * termif bif bi i = 1 then product = 1 then product product * term product * term
return productreturn product Example: compute aExample: compute a1313, n=13=1101, n=13=1101 EfficiencyEfficiency
Design and Analysis of Algorithms - Chapter 6 37
Least common multipleLeast common multiple
lcm(24,60)=120, lcm(11,5)=55lcm(24,60)=120, lcm(11,5)=55 Example: 24 = 2 x 2 x 2 x 3Example: 24 = 2 x 2 x 2 x 3
60 = 2 x 2 x 3 x 560 = 2 x 2 x 3 x 5
lcm(24,60) = (2x2x3) x 2 x 5lcm(24,60) = (2x2x3) x 2 x 5 Efficiency (a list of primes is required)Efficiency (a list of primes is required)
lcm(m,n) = mn / gcd(m,n)lcm(m,n) = mn / gcd(m,n)
Design and Analysis of Algorithms - Chapter 6 38
Counting paths in a graphCounting paths in a graph
The number of different paths of length k>0 from The number of different paths of length k>0 from node i to node j equals the (i,j) element of the Anode i to node j equals the (i,j) element of the Akk, , where A the adjacency matrix where A the adjacency matrix
Example Example EfficiencyEfficiency
Design and Analysis of Algorithms - Chapter 6 39
Reduction to graph problemsReduction to graph problems
Applies for a variety of games and puzzlesApplies for a variety of games and puzzles Build the state-space graphBuild the state-space graph
Example: peasant, wolf, goat, cabbageExample: peasant, wolf, goat, cabbage
Traverse the graph by applying what?Traverse the graph by applying what?