Post on 14-Dec-2015
§3 Dynamic Programming
Use a table instead of recursion
1. Fibonacci Numbers: F(N) = F(N – 1) + F(N – 2)
int Fib( int N ) { if ( N <= 1 ) return 1; else return Fib( N - 1 ) + Fib( N - 2 ); }
T(N) T(N – 1) + T(N – 2)
T(N) F(N)
1/17
§3 Dynamic ProgrammingF6
F2
F1 F0
F3
F1
F2
F1 F0
F2
F1 F0
F3
F1 F2
F1 F0
F3
F1
F2
F1 F0
F4
F4F5
Trouble-maker: The growth of redundant calculations is explosive.Solution: Record the two most recently computed values to avoid
recursive calls.
int Fibonacci ( int N ) { int i, Last, NextToLast, Answer; if ( N <= 1 ) return 1; Last = NextToLast = 1; /* F(0) = F(1) = 1 */ for ( i = 2; i <= N; i++ ) { Answer = Last + NextToLast; /* F(i) = F(i-1) + F(i-2) */ NextToLast = Last; Last = Answer; /* update F(i-1) and F(i-2) */ } /* end-for */ return Answer; } T(N) = O(N)
2/17
§3 Dynamic Programming
2. Ordering Matrix Multiplications
〖 Example〗 Suppose we are to multiply 4 matrices M1 [ 1020 ] M2 [ 2050 ] M3 [ 501 ] M4 [ 1100 ] .
If we multiply in the order
M1 [ 1020 ] ( M2 [ 2050 ] ( M3 [ 501 ] M4 [ 1100 ] ) )Then the computing time is
50 1 100 + 20 50 100 + 10 20 100 = 125,000
If we multiply in the order
( M1 [ 1020 ] ( M2 [ 2050 ] M3 [ 501 ] ) ) M4 [ 1100 ] Then the computing time is
20 50 1 + 10 20 1 + 10 1 100 = 2,200
Problem: In which order can we compute the product of n matrices with minimal computing time?
3/17
§3 Dynamic Programming
Let bn = number of different ways to compute M1 M2 Mn. Th
en we have b2 = 1, b3 = 2, b4 = 5,
Let Mij = Mi Mj . Then M1n = M1 Mn = M1i Mi+1 n
bn bi bni.1 and 1 where 1
1
1
bnbbb
n
iinin
)( 4nnn
n
Ob /* Catalan number */
Suppose we are to multiply n matrices M1 Mn where Mi is an ri1 ri matrix. Let mij be the cost of the optimal way
to compute Mi Mj . Then we have the recurrence eq
uations:
ijrrrmm
jim
jlijliljli
ij if}{min
if0
11
There are only O( N2 ) v
alues of Mij .
If j – i = k , then the only values Mxy required to
compute Mij satisfy y – x < k .
4/17
§3 Dynamic Programming
/* r contains number of columns for each of the N matrices */ /* r[ 0 ] is the number of rows in matrix 1 */ /* Minimum number of multiplications is left in M[ 1 ][ N ] */ void OptMatrix( const long r[ ], int N, TwoDimArray M ) { int i, k, Left, Right; long ThisM; for( Left = 1; Left <= N; Left++ ) M[ Left ][ Left ] = 0; for( k = 1; k < N; k++ ) /* k = Right - Left */ for( Left = 1; Left <= N - k; Left++ ) { /* For each position */
Right = Left + k; M[ Left ][ Right ] = Infinity; for( L = Left; L < Right; L++ ) { ThisM = M[ Left ][ L ] + M[ L + 1 ][ Right ]
+ r[ Left - 1 ] * r[ L ] * r[ Right ]; if ( ThisM < M[ Left ][ Right ] ) /* Update min */
M[ Left ][ Right ] = ThisM; } /* end for-L */
} /* end for-Left */}
T(N) = O(N3 )
To record the ordering please refer to Figure 10.46 on p.388
5/17
§3 Dynamic Programming3. Optimal Binary Search Tree —— The best for static searching (without insertion and deletion)
N
iii dpNT
1
)1()(
Given N words w1 < w2 < …… < wN, and the probability of searching for each wi is pi . Arrange these words in a binary search tree in a way that minimize the expected
total access time.
〖 Example〗 Given the following table of probabilities:word
probability
double else if intfloat for while0.22 0.18 0.25 0.020.20 0.05 0.08
Greedy Method
if
float
else for
while
int
double
AVL Method
Time = 2.43
for
else
double float
int
if while
Time = 2.70
Optimal Tree
float
double
else
if
for while
int
Time = 2.15
6/17
§3 Dynamic ProgrammingTi j ::= OBST for wi , ……, wj ( i < j )
ci j ::= cost of Ti j ( ci i = 0 )
ri j ::= root of Ti j ( ri i = 0 )
wi j ::= weight of Ti j = ( wi i = pi )
j
ikkp
T1N with root r1N,weight w1N, and
cost c1N .
wk
Lwi…wk1
Rwk+1…wj
Ti j
ci j = ?pk + cost( L ) + cost( R ) + weight( L ) + weight( R )
= pk + ci, k1 + ck+1, j + wi, k1 + wk +1, j = wi j + ci, k1 + ck +1, j
Ti j is optimal ri j = k is such that }{min ,11, jllijijli
ji ccwc
7/17
§3 Dynamic Programming
word
probability
double else if intfloat for while0.22 0.18 0.25 0.020.20 0.05 0.08
}{min ,11, jllijijli
ji ccwc
double..double while..whileint..intif..iffloat..float for..forelse..else
0.22 double 0.18 else 0.20 float 0.05 for 0.25 if 0.02 int 0.08 while
double..else
0.58 double
else..float
0.56 float
float..for
0.30 float
for..if
0.35 if
if..int
0.29 if
int..while
0.12 while
double..float
1.02 else
else..for
0.66 float
float..if
0.80 if
for..int
0.39 if
if..while
0.47 if
double..for
1.17 else
else..if
1.21 float
float..int
0.84 if
for..while
0.57 if
double..if
1.83 float
else..int
1.27 float
float..while
1.02 if
double..int
1.89 float
else..while
1.53 float
double..while
2.15 float
float
double
else for while
if
intT(N) = O(N3 )
Please read 10.33 on p.419 for an O( N2 ) algorithm.
8/17
§3 Dynamic Programming4. All-Pairs Shortest Path
For all pairs of vi and vj ( i j ), find the shortest path between.
Method 2 Define Dk[ i ] [ j ] = min{ length of path i { l k } j }
and D1[ i ] [ j ] = Cost [ i ] [ j ]. Then the length of the shortest path from i to j is DN1[ i ] [ j ].
AlgorithmAlgorithmStart from D1 and successively generate D0, D1, ..., DN1. If Dk1 is done, then either
k the shortest path i { l k } j Dk = Dk1 ; or
k the shortest path i { l k } j = { the S.P. from i to k } {the S.P. from k to j } Dk [ i ] [ j ] = Dk1[ i ] [ k ] + Dk1[ k ] [ j ]
0]},][[]][[],][[min{]][[ 111 kjkDkiDjiDjiD kkkk
Method 1 Use single-source algorithm for |V| times.
T = O( |V|3 ) – works fast on sparse graph.
9/17
§3 Dynamic Programming
/* A[ ] contains the adjacency matrix with A[ i ][ i ] = 0 */ /* D[ ] contains the values of the shortest path */ /* N is the number of vertices */ /* A negative cycle exists iff D[ i ][ i ] < 0 */ void AllPairs( TwoDimArray A, TwoDimArray D, int N ) { int i, j, k; for ( i = 0; i < N; i++ ) /* Initialize D */ for( j = 0; j < N; j++ )
D[ i ][ j ] = A[ i ][ j ]; for( k = 0; k < N; k++ ) /* add one vertex k into the path */ for( i = 0; i < N; i++ )
for( j = 0; j < N; j++ ) if( D[ i ][ k ] + D[ k ][ j ] < D[ i ][ j ] )
/* Update shortest path */ D[ i ][ j ] = D[ i ][ k ] + D[ k ][ j ];
}
T(N) = O(N3 ), but faster in a dense graph.
To record the paths please refer to Figure 10.53 on p.393
Works if there are negative edge costs, but no negative-cost cycles.
10/17
Bonus Problem 2
Diagon Alley
§3 Dynamic Programming
Due: Monday, January 15th, 2007 at 10:00pm
Detailed requirements can be downloaded from http://10.71.45.99/list.asp?boardid=47
Courseware Download
11/17
§5 Backtracking Algorithms
A sure-fire way to find the answer to a problem is to make a list of all candidate answers, examine each, and following the examination of all or some of the candidates, declare the identified answer.
Suuuure — if the list is finite and it is possible to identify the answer following
the examinations. AND, if there arenot too many candidates.
Backtracking enables us to eliminate the explicit examination of a large subset of the candidates while still guaranteeing that the answer will be found if the algorithm is run to termination.
The basic idea is that suppose we have a partial solution ( x1, ... , xi ) where each xk Sk for 1 k i < n. First we add xi+1 Si+1 and check if ( x1, ... , xi, xi+1 ) satisfies the constrains. If the answer is “yes” we continue to add the next x, else we delete xi and backtrack to the previous partial solution ( x1, ... , xi1 ).
pruning
§4 Randomized Algorithms – self-study
12/17
§5 Backtracking Algorithms
Find a placement of 8 queens on an 8 8 chessboard such that no two queens attack.
Two queens are said to attack iff they are in the same row, column, diagonal, or antidiagonal of the chessboard.
1 2 3 4 5 6 7 8
87
65
43
21 Q
Q
Q
Q
Qi ::= queen in the i-th row
xi ::= the column index in which Qi is
Solution = ( x1, x2, ... , x8 )= ( 4, 6, 8, 2, 7, 1, 3, 5 )
Constrains: Si = { 1,2,3,4,5,6,7,8 } for 1 i 8
This implies 88 candidates in the solution space.
xi xj if i j ( xi xj ) / (i j ) 1
This implies that the solution must
be a permutation of 1, 2, ... , 8. Thus the number of candidates
in the solution spaceis reduced to 8!.
For the problem with For the problem with nn queens, queens, there are there are nn! candidates ! candidates in the solution space.in the solution space.
1. Eight Queens
13/17
§5 Backtracking Algorithms
Method: Take the problem of 4 queens as an example
Step 1: Construct a game tree1
5
4
3
7
6
10
9
8
12
11
15
14
13
17
16
2
21
20
19
23
22
26
25
24
28
27
31
30
29
33
32
18
37
36
35
39
38
42
41
40
44
43
47
46
45
49
48
34
53
52
51
55
54
58
57
56
60
59
63
62
61
65
64
50
x1 = 1 2 3 4
x2 = 2 3 4 1 3 4 1 2 4 1 32
x3 = 3 4 2 4 2 3 3 4 1 4 1 3 2 4 1 4 1 2 2 3 1 3 1 2
x4 = 4 3 4 2 3 2 4 3 4 1 3 1 4 2 4 1 2 1 3 2 3 1 2 1
4 ! leaves
Each path from the root to a leaf defines an element of the solution space.
14/17
§5 Backtracking Algorithms1
5
4
3
7
6
10
9
8
12
11
15
14
13
17
16
2
21
20
19
23
22
26
25
24
28
27
31
30
29
33
32
18
37
36
35
39
38
42
41
40
44
43
47
46
45
49
48
34
53
52
51
55
54
58
57
56
60
59
63
62
61
65
64
50
x1 = 1 2 3 4
x2 = 2 3 4 1 3 4 1 2 4 1 32
x3 = 3 4 2 4 2 3 3 4 1 4 1 3 2 4 1 4 1 2 2 3 1 3 1 2
x4 = 4 3 4 2 3 2 4 3 4 1 3 1 4 2 4 1 2 1 3 2 3 1 2 1
Step 2: Perform a depth-first search ( post-order traversal ) to examine the paths
QQ QQ Q
QQ Q QQ
Q
( 2, 4, 1, 3 )
Note: No tree is actually constructed. The game tree is just an abstract concept.
Note: No tree is actually constructed. The game tree is just an abstract concept.
15/17
§5 Backtracking Algorithms
2. The Turnpike Reconstruction Problem
Given N points on the x-axis with coordinates x1 < x2 < …< xN . Assume that x1 = 0. There are N ( N – 1 ) / 2 distances between every pair of points.Given N ( N – 1 ) / 2 distances. Reconstruct a point set from the distances.
〖 Example〗 Given D = { 1, 2, 2, 2, 3, 3, 3, 4, 5, 5, 5, 6, 7, 8, 10 }
Step 1: N ( N – 1 ) / 2 = 15 implies N = ?6
Step 2: x1 = 0 and x6 = 10
x3 = 6 x2 = 4 x3 = 4 x4 = 6
x3 = 5
x4 = 7 x2 = 3
x5 = 8 x2 = 2
Step 3: find the next largest distance and check
( 0, 3, 5, 6, 8, 10 )
Please read Figure 10.64 and 10.65 for details.
16/17
§5 Backtracking Algorithms
17/17
Bonus Problem 3 Review of Programming
Contest Rules
Due: Monday, January 15th, 2007 at 10:00pm
Detailed requirements can be downloaded from http://10.71.45.99/list.asp?boardid=47
Courseware Download