Dynamic Programming

Lets begin by looking at the Fibonacci sequence.

The Fibonacci sequence is defined by the recursive formula Fib(x) = Fib(x-1) + Fib(x-2)

However, actually using recursion to calculate Fibonacci numbers is extremely inefficient.

Does anyone remember why? Consider what happens if we attempt to calculate

Fib(100) using recursion. We get a binary tree with leaf nodes at a minimum

depth of 100/2 and a maximum depth of 100. More than 250 calculations will be needed.

Fib(100)

Fib(98)Fib(99)

Fib(96) Fib(97) Fib(97) Fib(98)

97 more levels48 more levels

Fib(100)

Fib(0) Fib(1)Fib(0) Fib(1) Fib(0) Fib(1)

Clearly, the recursive approach gives us an Ω(2n/2) algorithm.

It is much faster to use the recursive formula to build from the bottom up.

Fib(0) = 1, Fib(1) = 1, Fib(2) = 2, Fib(3) = 3,

Fib(4) = 5, Fib(5) = 8 etc. up to Fib(100) This approach gives us an Θ(n) algorithm. If building a table of Fibonacci numbers instead of

just one, the algorithm is Θ(c), since each new number only requires two additions.

The Fibonacci sequence is NOT a dynamic programming problem, since it does not involve optimization.

However, it does illustrate an important point: Sometimes it is much faster to use a recursive formula to work from the bottom up, building a table of values as you go. Each new value depends on the values that were calculated previously.

This is similar to the dynamic programming approach.

Colonel Motors Problemfrom Cormen Leiserson Rivest Stein Second Edition Chapter 15 pages 324 to 330

Problem: Two assembly lines both produce the same cars, but take different times at each station.

Partially completed cars can move to the next station on the same line, or cross over to the next station on other line. Crossing over takes extra time.

Problem is to find the fastest path through the entire line.

If there are n stations and 2 lines, how many different ways can we select a path?

Brute force method is an exponential problem since there are 2n possible ways to select stations.

We will use dynamic programming.

First step is to define the structure of an optimal solution.

Suppose we have taken the fastest path through station S

1, j

There are only 2 ways we could have arrived there, from S

1, j-1 or by crossing over from S

2, j-1

If the fastest path was from S1, j-1

then the car must have taken the fastest path from the start to S

1, j-1

If there was a faster path to S1, j-1

we could substitute this faster path to get a faster path to S

1, j as well. But

this is a contradiction since we assumed we already had the fastest path to S

1, j.

We can use similar reasoning for the path crossing over from S

2, j-1

This property is called optimal substructure, and it is necessary for a problem to have this property for a dynamic programming solution to be applicable.

The next step is to define the recursive formula. Denote fastest path as f*.

Denote fi, j

as the fastest time to get from the start through station S

i, j

Recursive formula (on board) Now fill the total cost array using the recursive

formulas.

F1, 1

= (2+7) = 9 from base case

F2, 1

= (4+8) = 12 from base case

j 1 2 3 4 5 6 exit

f1, j

9

f2, j

12

F1, 2

= min(9 + 9, 12 + 2 + 9) = 18

F2, 2

= min(12 + 5, 9 + 2 + 5) = 16

j 1 2 3 4 5 6 exit

f1, j

9 18

f2, j

12 16

Continue in this fashion until the entire table is filled. The last column just adds the exit times.

Are we done? No. We still do not know the optimum selection of

assembly stations. With some problems (e.g. edit distance) we only

need the optimum number, not the sequence of steps need to achieve it. Here, we need the sequence.

j 1 2 3 4 5 6 exit

f1, j

9 18 20 24 32 35 38

f2, j

12 16 22 25 30 37 39

Can anyone suggest a method of retrieving the station sequence from the total cost array?

There are two ways. One – Maintain a system of pointers that tells us

where each optimum was obtained from. For example use an up arrow ↑ to indicate the value was obtained by moving from the previous station on the same line, and a right arrow → to indicate is was obtained by crossing over from the other line.

j 1 2 3 4 5 6 exit

f1, j

9 18 ↑ 20 → 24 ↑ 32 ↑ 35 → 38

f2, j

12 16 → 22 ↑ 25 → 30 ↑ 37 ↑ 39

Trace back using arrows to give shaded path.

j 1 2 3 4 5 6 exit

f1, j

9 18 ↑ 20 → 24 ↑ 32 ↑ 35 → 38

f2, j

12 16 → 22 ↑ 25 → 30 ↑ 37 ↑ 39

The other alternative is to trace back through the total cost array using the recursive formula.

j 1 2 3 4 5 6 exit

f1, j

9 18 20 24 32 35 38

f2, j

12 16 22 25 30 37 39

For example, f1, 6

= min(32 + 4, 30 + 1 + 4)

35 = min(36, 35)

So the vaue 35 was obtained by crossing from the other line. Repeat this process to the start.

Dynamic programming steps:

1. Verify the problem has optimal substructure.

2. Define the recursive formula.

3.Use the recusive formula to build a table of partial solutions from the bottom up.

4. If required, trace back to obtain the sequence to steps comprising the optimal solution.

Matrix Chain Multiplication

Review of matrix multiplication Matrix multiplication is not commutative. AB ≠ BA Matrix multiplication is associative. So that

ABC = (AB)C = A(BC) For AB to be defined the number of colums in A

must equal the number of rows in B. If A is pxq and B is qxr AB will be pxr.

If A is pxq and B is qxr then C = AB is given by the following formula:

C has pr elements and each element takes q multiplications to compute so matrix multiplicaiton is θ(pqr).

Consider the multiplication ABC where A is 10x100, B is 100x5 and C is 5x50. There are two possible parenthesizations.

1. ((AB)C) first = 10*100*5 = 5000 multiplications. Result is a 10x5 matrix. Second = 10*5*50 = 2500 multiplications. Total = 7500.

2.(A(BC)) first = 100*5*50 = 25,000 multiplications. Result is a 100x50 matrix. Second = 10*100*50 = 50,000 multiplications. Total = 75,000

Brute force is a poor strategy since it is

Ω(4n / n3/2 ).

Step 1. Demonstrate that the problem has optimal substructure.

Suppose (A1....A

k) (A

k+1....A

n) is an optimum

parenthesization of A1....A

n for some k. Then

(A1....A

k) and (A

k+1....A

n) must also be optimal. (Use

the normal proof by contradiction here.) Step 2. Define a recursive solution. Let m[i][j] be the minimum number of

multiplications needed to compute Ai....A

j where i<j.

The cost of the best overall solution is A1....A

n

If i=j then m[i][j] = 0 since there is only 1 matrix.

Suppose we are looking for the optimum way to

parenthesize Ai....A

j We will place the parenthesis

between Ak and A

k+1 where i ≤ k < j. The trick is to

find the best value for k to minimize m[i][j]. m[i][j] = 0 for i = j

= Min( m[i][k] + m[k+1][j] + Pi-1

* Pk * P

j) for i ≤ k < j

where Pi-1

* Pk * P

j is the cost of multiplying the resulting matrices

of size Pi-1

x Pk and P

k x P

j

Here Pi refers to the number of columns in A

i (or the

number of rows in Ai+1

)

It is important to note that when we use the recursive formula we must already have calculated

m[i][k] and m[k+1][j]. Thus we should fill the cost array in increasing order of the length of the chain. Thus we fill in this order:

1. m[1][2], m[2][3], m[3][4]... all chains of length 2

2. m[1][3], m[2][4], m[3][5]....all chains of length 3

3. m[1][4], m[2][5], m[3][6]....all chains of length 4

4. …

5. m[1][n] the only chain of length n

Fill this diagonal first j = 1 to n final result m[1][n]

i = 1 to n

0

0

0

0

0

0

0

Example with 4 matrices A1 = 5x4, A

2= 4x6, A

3= 6x2,

A4 = 2x7. This makes P

0 = 5, P

1 = 4, P

2 = 6, P

3 = 2, P

4 = 7.

0

0

0

0

i =1

2

3

4

J =1 2 3 4

M[1][2] = Min( m[1][k] + m[k+1][2] + P0* P

k* P

2) for 1 ≤ k < 2

= Min( m[1][1] + m[2][2] + P0* P

1* P

2) for k = 1

= 0 + 0 + 5*4*6 = 120

0 120

0

0

0

i =1

2

3

4

J = 1 2 3 4

M[2][3] = Min( m[2][k] + m[k+1][3] + P1* P

k* P

3) for 2 ≤ k < 3

= Min( m[2][2] + m[3][3] + P1* P

2* P

3) for k = 2

= 0 + 0 + 4*6*2 = 48

0 120

0 48

0

0

i =1

2

3

4

J = 1 2 3 4

M[3][4] = Min( m[3][k] + m[k+1][4] + P2* P

k* P

4) for 3 ≤ k < 4

= Min( m[3][3] + m[4][4] + P2* P

3* P

4) for k = 3

= 0 + 0 + 6*2*7 = 84

0 120

0 48

0 84

0

i =1

2

3

4

J = 1 2 3 4

M[1][3] = Min( m[1][k] + m[k+1][3] + P0* P

k* P

3) for 1 ≤ k < 3

= Min ( m[1][1] + m[2][3] + P0* P

1* P

3) where k = 1

( m[1][2] + m[3][3] + P0* P

2* P

3) where k = 2

= Min ( 0 + 48 + 5*4*2), ( 120 + 0 + 5*6*2) = 88

0 120 88

0 48

0 84

0

i =1

2

3

4

J = 1 2 3 4

M[2][4] = Min( m[2][k] + m[k+1][4] + P1* P

k* P

4) for 2 ≤ k < 4

= Min ( m[2][2] + m[3][4] + P1* P

2* P

4) where k = 2

( m[2][3] + m[4][4] + P1* P

3* P

4) where k = 3

= Min ( 0 + 84 + 4*6*7), ( 48 + 0 + 4*2*7) = 104

0 120 88

0 48 104

0 84

0

i =1

2

3

4

J = 1 2 3 4

M[1][4] = Min( m[1][k] + m[k+1][4] + P0* P

k* P

4) for 1 ≤ k < 4

= Min ( m[1][1] + m[2][4] + P0* P

1* P

4) where k = 1

( m[1][2] + m[3][4] + P0* P

2* P

4) where k = 2

( m[1][3] + m[4][4] + P0* P

3* P

4) where k = 3

= Min ( 0 + 104 + 5*4*7), ( 120 + 84 + 5*6*7), ( 88 + 0 + 5*2*7) = 158

0 120 88 158

0 48 104

0 84

0

i =1

2

3

4

J = 1 2 3 4

Are we done? No. Just like the Colonel Motors problem, we have

the lowest cost, but we don't know the sequence needed to achieve it.

The best way is to maintain a separate array s[n][n] that stores the k value that was used to achieve the optimal splitting in the total cost array.

The correct placement of parenthesis can then be recovered with the following recursive algorithm.

PrintBrackets( i, j)

if i = j

then print “Ai”

else print “(“

PrintBrackets( i, s[i][j])

PrintBrackets( s[i][j] + 1, j)

print “)”