12 A: Algorithm Design Techniques II - NUS Computingcs1102s/slides/slides_12_A...Optimal Binary...
Transcript of 12 A: Algorithm Design Techniques II - NUS Computingcs1102s/slides/slides_12_A...Optimal Binary...
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
12 A: Algorithm Design Techniques II
CS1102S: Data Structures and Algorithms
Martin Henz
April 7, 2010
Generated on Tuesday 6th April, 2010, 14:42
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 1
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
1 Greedy Algorithms (brief review)
2 Divide and Conquer (Example)
3 Dynamic Programming
4 Backtracking Algorithms
5 Another Puzzler
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 2
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
1 Greedy Algorithms (brief review)SchedulingHuffman Codes
2 Divide and Conquer (Example)
3 Dynamic Programming
4 Backtracking Algorithms
5 Another Puzzler
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 3
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Nonpreemptive Scheduling
Input
A set of jobs with a running time for each
Desired output
A sequence for the jobs to execute on on single machine,minimizing the average completion time
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 4
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Example
Some schedule:
The optimal schedule:
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 5
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
The Multiprocessor Case
N processors
Now we can run the jobs on N identical machines. What is aschedule that minimizes the average completion time?
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 6
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 7
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
A “Slight” Variant
Miniming final completion time
If we want to minimize the final completion time (completiontime of the last task), the problem becomes NP-complete!
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 8
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Optimal Prefix Code in Table Form
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 9
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Optimal Prefix Code in Tree Form
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 10
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Huffman’s Algorithm: An Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 11
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Huffman’s Algorithm: An Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 12
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Huffman’s Algorithm: An Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 13
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Huffman’s Algorithm: An Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 14
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Huffman’s Algorithm: An Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 15
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Huffman’s Algorithm: An Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 16
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Huffman’s Algorithm: An Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 17
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Correctness of Huffman’s Algorithm
Observation 1
An optimal tree must be full; no node has only one child.
Observation 2
The two least frequent characters α and β must be the twodeepest nodes.
Observation 3
Characters at the same level can be swapped without affectingoptimality.
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 18
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
SchedulingHuffman Codes
Correctness of Huffman’s Algorithm
Observation 3
Characters at the same level can be swapped without affectingoptimality.
Initial Step of Huffman’s Algorithm
An optimal tree can be found that contains the two leastfrequent symbols as siblings; the first step in Huffman’salgorithm is not a mistake.
Observation 4
Every step of Huffman’s algorithm produces a simplifiedproblem, resulting from treating two characters asindistinguishable.
Each Step of Huffman’s AlgorithmCS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 19
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
1 Greedy Algorithms (brief review)
2 Divide and Conquer (Example)
3 Dynamic Programming
4 Backtracking Algorithms
5 Another Puzzler
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 20
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Closest-Points Problem
Input
Set of points in a plane
Euclidean distance between p1 and p2
[(x1 − x2)2 + (y1 − y2)
2]1/2
Required output
Find the closest pair of points
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 21
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 22
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Naive Algorithm
Exhaustive search
Compute the distance between each two points and keep thesmallest
Run time
There are N2 pairs to check, thus O(N2)
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 23
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Idea
Preparation
Sort points by x coordinate; O(N log N)
Divide and Conquer
Split point set into two halves, PL and PR.Recursively find the smallest distance in each half.Find the smallest distance of pairs that cross the separationline.
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 24
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Partitioning with Shortest Distances Shown
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 25
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Two-lane Strip
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 26
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Brute Force Calculation of min(δ, dC)
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 27
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Better Idea
Sort points by y coordinate
This allows a scan of the strip.
Sort points by y coordinate
This allows a scan of the strip.
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 28
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Only p4 and p5 Need To Be Considered
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 29
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Refined Calculation of min(δ, dC)
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 30
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
At Most Eight Points Fit in Rectangle
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 31
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
1 Greedy Algorithms (brief review)
2 Divide and Conquer (Example)
3 Dynamic ProgrammingFibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
4 Backtracking Algorithms
5 Another Puzzler
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 32
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Inefficient Algorithm
public s t a t i c i n t f i b ( i n t n ) {i f ( n <= 1)
return 1;else
return f i b ( n − 1) + f i b ( n − 2 ) ;}
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 33
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Trace of Recursion
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 34
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Memoization
i n t [ ] f i b s = new i n t [ 1 0 0 ] ;public s t a t i c i n t f i b ( i n t n ) {
i f ( f i b s [ n ] ! = 0 ) return f i b s [ n ] ;i f ( n <= 1) return 1;i n t new f ib = f i b ( n − 1) + f i b ( n − 2 ) ;f i b s [ n ] = new f ib ;return new f ib ;
}
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 35
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
A Simple Loop for Fibonacci Numbers
public s t a t i c i n t f i b ( i n t n ) {i f ( n <= 1) return 1;i n t l a s t = 1 , nextToLast = 1 ; answer = 1;for ( i n t i = 2 ; i <= n ; i ++) {
answer = l a s t + nextToLast ;nextToLast = l a s t ;l a s t = answer ;
}return answer ;
}
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 36
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Sample Input
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 37
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Three Possible Binary Search Trees
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 38
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Comparison of the Three Trees
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 39
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Structure of Optimal Binary Search Tree
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 40
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Idea
Proceed in order of growing tree size
For each range of words, compute optimal tree
Memoization
For each range, store optimal tree for later retrieval
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 41
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Computation of Optimal Binary Search Tree
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 42
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Run Time
For each cell of table
Consider all possible roots
Overall runtime
O(N3)
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 43
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Fibonacci NumbersOptimal Binary Search TreeAll-pairs Shortest Path
Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 44
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
1 Greedy Algorithms (brief review)
2 Divide and Conquer (Example)
3 Dynamic Programming
4 Backtracking Algorithms
5 Another Puzzler
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 45
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Example: The Turnpike Reconstruction Problem
From points to distances
It is easy to calculate for a set of points on a line all distancesbetween points.
From distances to points
Consider the reverse: given the set of all distances between thepoints, compute the positions of the points!
Placement of first point
By default, the first point is placed at position 0.
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 46
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Example
Input set for 6 points
D = {1, 2, 2, 2, 3, 3, 3, 4, 5, 5, 5, 6, 7, 8, 10}
First and last point
First point goes to 0, which means that the last point goes to 10.
Placement of third point
The third point can be placed either at position 2 or position 8.This choice is arbitrary; we can flip a solution to get from one tothe other!
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 47
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Decision Tree for the Example
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 48
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Summary of Backtracking
Decision tree
Backtracking explores decision tree; typically exponential in size
Pruning
At each node, reasoning is applied to avoid exploring thesubtree
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 49
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
1 Greedy Algorithms (brief review)
2 Divide and Conquer (Example)
3 Dynamic Programming
4 Backtracking Algorithms
5 Another Puzzler
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 50
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Remember Lecture 2 A: Parameter Passing
Java uses pass-by-value parameter passing.
public s t a t i c void t ryChanging ( i n t a ) {a = 1;return ;
}. . .i n t b = 2;tryChanging ( b ) ;System . out . p r i n t l n ( b ) ;
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 51
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Remember Lecture 2 A: Parameter Passing withObjects
public s t a t i c void t ryChanging ( SomeObject ob j ) {obj . someField = 1;ob j = new SomeObject ( ) ;ob j . someField = 2;return ;
}. . .SomeObject someObj = new SomeObject ( ) ;t ryChanging ( someObj ) ;System . out . p r i n t l n ( someObj . someField ) ;
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 52
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Remember Lecture 7 A: Sorting
Input
Unsorted array of elements
Behavior
Rearrange elements of array such that the smallest appearsfirst, followed by the second smallest etc, finally followed by thelargest element
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 53
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Will This Work?
public s t a t i c <AnyType extends Comparable<? super Anvoid mergeSort ( AnyType [ ] a ) {
AnyType [ ] r e t = . . . . ; / / dec lare he lper ar ray. . . . / / here goes a program t h a t places. . . . / / the element o f ” a ” i n t o ” r e t ” so. . . . / / t h a t ” r e t ” i s sor teda = r e t ;return ;
}. . .I n t ege r [ ] myArray = . . . ;I t e ra t i veMergeSor t . mergeSort ( myArray ) ;
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 54
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Will This Work?
public s t a t i c <AnyType extends Comparable<? super Anvoid mergeSort ( AnyType [ ] a ) {
AnyType [ ] r e t = . . . . ; / / dec lare he lper ar ray. . . . / / here goes a program t h a t places. . . . / / the element o f ” a ” i n t o ” r e t ” so. . . . / / t h a t ” r e t ” i s sor teda = r e t ;return ;
}. . .I n t ege r [ ] myArray = . . . ;I t e ra t i veMergeSor t . mergeSort ( myArray ) ;
Answer: No! The assignment a = ret ; has no effect onmyArray!
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 55
Greedy Algorithms (brief review)Divide and Conquer (Example)
Dynamic ProgrammingBacktracking Algorithms
Another Puzzler
Next Lecture
Friday:
Search trees with external storage (Section 4.7)
Preview: Extendible hashing (Section 5.7)
Preview: External sorting (Section 7.10)
CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 56