Dynamic Dynamic ProgrammingProgramming

Dynamic ProgrammingDynamic Programming

Like divide-and-conquer algorithms:Like divide-and-conquer algorithms: Combines solutions to subproblemsCombines solutions to subproblems

Dynamic programming is useful when Dynamic programming is useful when subproblems share subproblemssubproblems share subproblems Not necessary to recomputeNot necessary to recompute Save the answer in a tableSave the answer in a table

Dynamic programming applies to Dynamic programming applies to optimization problems (finding min or optimization problems (finding min or max)max)

Developement of DPDevelopement of DP

1.1. Characterize the structure of an Characterize the structure of an optimal solutionoptimal solution

2.2. Recursively define the value of an Recursively define the value of an optimal solutionoptimal solution

3.3. Compute the value of an optimal Compute the value of an optimal solution (bottom-up)solution (bottom-up)

4.4. Construct an optimal solution from Construct an optimal solution from computed information (may be computed information (may be omitted)omitted)

The Assembly LineThe Assembly Line

Imagine two assembly lines, with Imagine two assembly lines, with nn stationsstations

The The iithth station performs the same thing, station performs the same thing, but in different times (denoted but in different times (denoted aaii,,jj ))

There is a time to transfer from one line There is a time to transfer from one line to another costs to another costs ttii,,jj

The problem: determine the fastest way The problem: determine the fastest way through the factorythrough the factory

How many possibilities are there?How many possibilities are there?

Example: Assembly LineExample: Assembly Line

a1,1 a1,2 a1,3 a1,n-1 a1,n

a2,1 a2,2 a3,3 a2,n-1 a2,n

e1

e2

t1,1

t2,1

t1,2

t2,2

t1,n-1

t2,n-1

x1

x2

Station S1,1 Station S1,2 Station S1,3 Station S1,n-1Station S1,n

chassisenters

Station S2,1 Station S2,2 Station S2,3 Station S2,n-1Station S2,n

completedauto exits

Step 1: the structureStep 1: the structure

Because there are Because there are (2 (2nn)) possibilities, possibilities, this won’t work.this won’t work.

Consider the fastest possible way Consider the fastest possible way through station Sthrough station S1, 1, jj

If If jj = 1, then there is only one way = 1, then there is only one way If If jj > 1, then there are two choices: > 1, then there are two choices:

came from station Scame from station S1, 1, j j -1-1

came from station Scame from station S2, 2, j j -1-1, incurring cost t, incurring cost t2, 2, j j -1-1

Key ObservationKey Observation

The chassis must have taken the fastest The chassis must have taken the fastest way from the start to station Sway from the start to station S1, 1, j j -1-1

Why? If there were a faster way, we Why? If there were a faster way, we could substitute it to yield a faster way: could substitute it to yield a faster way: contradiction!contradiction!

In other words, for it to be the fastest In other words, for it to be the fastest way, the subproblem had to be optimal.way, the subproblem had to be optimal.

Optimal substructure: a hallmark of Optimal substructure: a hallmark of dynamic programmingdynamic programming

Thus...Thus...

The fastest way is either The fastest way is either the fastest way through station Sthe fastest way through station S1,j-11,j-1 and and

directly to Sdirectly to S1, j1, j

the fastest way through station Sthe fastest way through station S2,j-12,j-1 with a transfer, and then through Swith a transfer, and then through S1, j1, j

To find the fastest way through To find the fastest way through station j, we solve the subproblemsstation j, we solve the subproblems

Step 2: A recursive Step 2: A recursive solutionsolution

Let Let ffii[[jj]] denote the fastest time from denote the fastest time from the start to station the start to station SSi,ji,j

Fastest way through the factory Fastest way through the factory f*f* is is therefore therefore min(min(ff11[[nn]+]+xx11, , ff22[[nn]+]+xx22))

ff11[[jj] = min(] = min(ff11[[jj-1]+-1]+aa1,1,jj, , ff22[[jj-1]+-1]+tt2,2,jj-1-1++aa1,1,jj))

ff22[[jj] = min(] = min(ff22[[jj-1]+-1]+aa2,2,jj, , ff11[[jj-1]+-1]+tt1,1,jj-1-1++aa2,2,jj))

Step 3: Computing the Step 3: Computing the fastest timefastest time

Top down yields a Top down yields a 22nn solution. solution. As a note As a note ff11[1][1] is calculated is calculated 22nn-1-1 times! times!

By computing the By computing the ffii[[jj]] values in values in increasingincreasing order order jj, time is reduced to , time is reduced to ((nn))

7 9 3 8 4

8 5 6 5 7

2

4

2

2

3

1

4

1

3

2

Station S1,1 Station S1,2 Station S1,3 Station S1,5 Station S1,6

chassisenters

Station S2,1 Station S2,2 Station S2,3 Station S2,5 Station S2,6

completedauto exits

4

1

2

4

3

2

Station S1,4

Station S2,4

9 18 20 24 32 35

12 16 22 25 30 37

f1[ j ]

f2[ j ]f * = 38

Matrix-Chain Matrix-Chain MultiplicationMultiplication

Given a sequence of matrices, find the best Given a sequence of matrices, find the best way to parenthesize them.way to parenthesize them.

For example:For example: (A(A11(A(A22(A(A33AA44))))))

(A(A11((A((A22AA33)A)A44))))

((A((A11AA22)(A)(A33AA44))))

(A(A11(A(A22AA33)A)A44))

(((A(((A11AA22)A)A33)A)A44))

There are actually There are actually (2(2nn) different ) different parenthesizations!parenthesizations!

Why This is ImportantWhy This is Important The way that we parenthesize them The way that we parenthesize them

dramatically affects the amount of work to dramatically affects the amount of work to do:do:

For Example:For Example: AA11=10=10100, A100, A22=100=1005, A5, A33=5=55050 ((A((A11AA22)A)A33) )

10101001005 = 5000 (for A5 = 5000 (for A11AA22) leaving a 10) leaving a 105 plus5 plus 10105550 = 2500 = 7500 total ops50 = 2500 = 7500 total ops

(A(A11(A(A22AA33)))) 1001005550 = 25,000 (for A50 = 25,000 (for A22AA33) leaving a 100) leaving a 10050 plus50 plus 101010010050 = 50,000 = 75,000 total ops50 = 50,000 = 75,000 total ops

NotationNotation

Let ALet Ai..ji..j denote the matrix results for denote the matrix results for AAiiAAii+1+1...A...Ajj

Must pick a split point Must pick a split point ii<<kk<<jj for some for some kk, we first compute A, we first compute Aii....kk and A and Akk+1...+1...jj, and , and

then multiply them togetherthen multiply them together How do you pick How do you pick kk??

Note the optimal substructure:Note the optimal substructure: AAii....kk must be optimal must be optimal If there was a less-costly way, we could cut-n-If there was a less-costly way, we could cut-n-

paste!paste!

Building from the Building from the Ground UpGround Up

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

1

1

2

2

3

34

5

6

4

5

6

Building from the Building from the Ground UpGround Up

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

15,750 2,625 750 1,000 5,0001

1

2

2

3

34

5

6

4

5

6

Building from the Building from the Ground UpGround Up

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

15,750 30x15

2,625 35x5

75015x10

1,000 5x20

5,000 10x25

1

1

2

2

3

34

5

6

4

5

6

?

Note: Can be either (A1(A2A3)) or ((A1A2)A3) 15,750 + (30155) = 18,000 vs2,625 + (30355) = 7,875

Building from the Building from the Ground UpGround Up

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

15,750 30x15

2,625 35x5

75015x10

1,000 5x20

5,000 10x25

1

1

2

2

3

34

5

6

4

5

6

7,875 30x5

4,375 35x10

2,500 15x20

3,500 5x25

One Last ExampleOne Last Example

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

15,750 30x15

2,625 35x5

75015x10

1,000 5x20

5,000 10x25

1

1

2

2

3

34

5

6

4

5

6

7,875 30x5

4,375 35x10

2,500 15x20

3,500 5x25

?

One Last ExampleOne Last Example

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

15,750 30x15

2,625 35x5

75015x10

1,000 5x20

5,000 10x25

1

1

2

2

3

34

5

6

4

5

6

7,875 30x5

4,375 35x10

2,500 15x20

3,500 5x25

?

0 + 2,500 + (351520) vs

One Last ExampleOne Last Example

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

15,750 30x15

2,625 35x5

75015x10

1,000 5x20

5,000 10x25

1

1

2

2

3

34

5

6

4

5

6

7,875 30x5

4,375 35x10

2,500 15x20

3,500 5x25

?

0 + 2,500 + (351520) vs2,625 + 1000 + (35520) vs

One Last ExampleOne Last Example

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

15,750 30x15

2,625 35x5

75015x10

1,000 5x20

5,000 10x25

1

1

2

2

3

34

5

6

4

5

6

7,875 30x5

4,375 35x10

2,500 15x20

3,500 5x25

?

0 + 2,500 + (351520) vs2,625 + 1000 + (35520) vs4,375 + 0 + (351020)

One Last ExampleOne Last Example

c

matrix DimensionA1 3035A2 3515A3 155A4 510A5 1020A6 2025

A1 A2 A3 A4 A5 A6

0 0 0 0 0 0

15,750 30x15

2,625 35x5

75015x10

1,000 5x20

5,000 10x25

1

1

2

2

3

34

5

6

4

5

6

7,875 30x5

4,375 35x10

2,500 15x20

3,500 5x25

7,125 35x20

0 + 2,500 + (351520) vs2,625 + 1000 + (35520) vs4,375 + 0 + (351020)

=7,125

Brain TeaserBrain Teaser

Using this tabular method, can you:Using this tabular method, can you: create a dynamic merge-sort?create a dynamic merge-sort? create a dynamic twelve days of create a dynamic twelve days of

Christmas?Christmas? *find the total number of ways to make *find the total number of ways to make

change for change for nn cents? cents?

In SummaryIn Summary

Dynamic Programming exhibits Dynamic Programming exhibits optimal substructureoptimal substructure

Overlapping SubproblemsOverlapping Subproblems Tables for holding intermediate Tables for holding intermediate

resultsresults