Dynamic Programming CISC4080, Computer Algorithms CIS ...

Dynamic ProgrammingCISC4080, Computer Algorithms

CIS, Fordham Univ.

Instructor: X. Zhang

Outline

• Introduction via example: Fibonacci, rod cutting • Characteristics of problems that can be

solved using dynamic programming • More examples:

• Maximal subarray problem • Longest increasing subsequence

problem • Two dimensional problem spaces

• Longest common subsequence • Matrix chain multiplication

• Summary 2

Dynamic Programming

3

Rod Cutting Problem

• A company buys long steel rods (of length n), and cuts them into shorter one to sell • integral length only • cutting is free • rods of diff lengths sold for

diff. price • Goal: cut rod into shorter ones

to sell for maximal profit

4

• Input: • given rod’s length n • p[i]: selling price of rod of length i, for example:

• Output: maximal achievable total settling price, Rn, over all possible ways to cut out n to shorter pieces and sell

• For example, if n=4, we could cut it into: • {4}: do not cut ==> total selling price is $9 • {3,1} ==> selling price $8+$1=$9 • {2,2} ==> $10 • {2,1,1} ==> $7 • {1,1,1,1}==> $4

Rod Cutting Problem

5

multiset:allow duplicate elementsorder does not matter

Solution Space

• Evaluate all possible ways, and pick one with highest total selling price? • how many ways to write n as sum of positive integers?

• n=4, there are 5 ways: 4=4, 4=1+3, 4=2+2, 4=1+1+1+1, 4=2+1+1

• n=5: … • # of ways to cut n: # of ways to express n as positive

integer linear combination of 1,2, …10,

• this leads to exponential running timeeπ 2n/3 /4n 3

6

• //return maximum profit achievable with a rod of length n

• CutRod (n, p[1…k])

• What’s the smallest problem(s) that we can solve trivially? • if n=0, n=1

• How to reduce the problem to smaller problems?

Thinking Recursively

7

• CutRod (n, p[1…k]) //return rn • What’s the smallest problem(s) that we can solve trivially?

• if n=0, n=1 • How to reduce the problem to smaller problems?

• consider the first rod to cut out and sell, c1 • c1 can be 1, 2, 3, … , min (n, k)

• we sell it for p[c1], and have a rod of length n-c1 remaining ==> a smaller problems

• Just try all different c1 and solve subproblems recursively:

rn = maxc1=1,2...min{n,k}

{p[c1] + rn−c1}

Recursion!

8

• Optimal substructure: Optimal solution to a problem of size n incorporates optimal solutions to problems of smaller size (n-1, n-2,… ).

rn = maxc1=1...min{n,k}

{p[c1] + rn−c1}

Optimal substructure

9

• //return max. profit one can make with a rod of length n • CutRod (n, p[1…k]) //What’s the smallest problem(s) that we can solve trivially?

if n=0 return 0 if n=1 return p[1] //general case: consider the first rod to cut out and sell, curMax=0 for c1=1, 2, 3, … , min (n, k)

curProfit = p[c1]+MaxProf (n-c1) curMax = max (curProfit, curMax)

// done evaluating all possibilities return curMax

Optimal Substructure => recursive solution

10

• //return max. profit one can make with a rod of length n • CutRod (n, p[1…k]) //What’s the smallest problem(s) that we can solve trivially?

if n=0 return 0 if n=1 return p[1] //general case: consider the first rod to cut out and sell, curMax=0 for c1=1, 2, 3, … , min (n, k)

curProfit = p[c1]+CutRod (n-c1) curMax = max (curProfit, curMax)

// done evaluating all possibilities return curMax

Running time?

11

T(n) = T(n − 1) + T(n − 2) + . . . + T(n − k) + C// for n > k

Recursive Trees

12

How many times CutRod (2) is called?How about CutRod(1)?

Overlapping of Subproblems

13

• Recursive calling tree shows overlapping of subproblems • i.e., n=4 and n=3 share

overlapping subproblems (2,1,0)

• Idea: avoid recomputing subproblems again and again • store subproblem

solutions in memory/table (hence “programming”)

Dynamic Programming: two approach

• memoization (recursive, top-down) • improve recursive solution by storing subproblem

solution in a table • when need solution of a subproblem, check if it has

been solved before, • if not, calculate it and store result in table • if yes, access result stored in table

• a top-down approach: • start with the problem at hand (a larger problem),

and solve subproblems as needed (recursively)

//return max. profit one can make with a rod of length n CutRod (n, p[1…k]) 1. create an array r[1…n], filled with -1 (indicate “not calculated yet”) 2. CutRodHelper (n, p, r)

CutRodHelper (n, p[1…k], r[]) 1. if r[n] >=0 return r[n] //if it has been calculated already 2. // no need to recalculate, return the stored result 3. 4. if n==0 return 0 //base case 5. //general case 6. curMax=0 7. for c1=1, 2, 3, … , min (n, k) 8. curProfit = p[c1] + CutRodHelper (n-c1) 9. curMax = max(curProfit, curMax) 10. 11. r[n]= curMax //save result in r[] for future reference 12. return curMax

Memoization illustrated in code

15

Dynamic Programming:Tabulation

• Memoization (recursive, top-down) • Tabulation

• Iteratively solve smaller problems first, move the way up to larger problems

• bottom-up method: as we solve smaller problems first, and then larger and larger one • => when solving a problem, all subproblems

solutions that are needed have already been calculated

• //return max. profit one can make with a rod of length n • CutRodBottomUp (n, p[1…k]) 1. create an array r[1…n] //store subproblem solutions 2. r[0] = 0 3. for i=1 to n // solve smaller problems first … 4. // calculate ri , max revenue for rod length i 5. curMax=0 6. for c1=1, 2, 3, … , min (i, k) 7. curProfit = p[c1]+ r[i-c1] 8. curMax = max(curProfit, curMax) 9. r[i]= curMax //save result in r[] for future reference 10. 11. return r[n]

Bottom-up

17

Recap

• We analyze rod cutting problem • Two characteristics of problems that can benefit

from dynamic programming: • optimal substructure: a recursive formular

• overlapping subproblems

18

rn = maxc1=1,2...min{n,k}

{p[c1] + rn−c1}

Recap (2)

• How dynamic programming works: • Memoization: recursion with table • Tabulation: iteratively solve all possible

subproblems, and work our way from small problems to large problems

19

Exercises

• Copy given C++ program for rod cutting • copyRodCut

• cd CutRod; Compile and run • Exercise #1:

• add some cout statement to find out how many times MaxProf (3) is called when MaxProf(6) is called from main.

• Exercise #2: • So far, we focus on finding , max. revenue…

• but what cutting achieves ? • How to extend the functions to output/return the cutting?

• e.g., n=4, cutting 2, 2, r_4=…

rnrn

20

//return max. profit one can make with a rod of length n CutRod (n, p[1…k]) 1. create an array r[1…n], filled with -1 (indicate “not calculated yet”) 2. CutRodHelper (n, p, r)

CutRodHelper (n, p[1…k], r[]) 1. if r[n] >=0 return r[n] //if it has been calculated already 2. // no need to recalculate, return the stored result 3. 4. if n==0 return 0 //base case 5. //general case 6. curMax=0 7. for c1=1, 2, 3, … , min (n, k) 8. curProfit = p[c1] + CutRodHelper (n-c1) 9. curMax = max(curProfit, curMax) 10. 11. r[n]= curMax //save result in r[] for future reference 12. return curMax

Tracing: CutRod(n=3,p)

21

Tracing with n=3

table r :

c1curProfitcurMax

• //return max. profit one can make with a rod of length n • CutRodBottomUp (n, p[1…k]) 1. create an array r[1…n] //store subproblem solutions 2. r[0] = 0 3. for i=1 to n // solve smaller problems first … 4. // calculate ri , max revenue for rod length i 5. curMax=0 6. for c1=1, 2, 3, … , min (i, k) 7. curProfit = p[c1]+ r[i-c1] 8. curMax = max(curProfit, curMax) 9. r[i]= curMax //save result in r[] for future reference 10. 11. return r[n]

Tabulation: Tracing n=5

22

• Input • given weight capacity of a knapsack, W • n different objects, with weights and values given by

array w[ ], v[ ] • i-th type of objects weighs w[i] and has a value of v[i] • there are infinite quantities of each object type

• Output: objects to put into knapsack so that total value is maximized and total weights <= W • For now, focus on max. total value

• Discussion: • how is this similar to Rod Cutting?

Discussion: Knapsack

23

• Input: weight capacity of a knapsack, W; n different objects, with weights and values given by array w[ ], v[ ] • i-th type of objects weighs w[i] and has a value of v[i] • there are infinite quantities of each object type

• Output: objects to put into knapsack so that total value is maximized and total weights <= W

• For example: W=13, w[4]={5,3,8,4}, v[4]={10,20,25,8} in the figure • Let Vk be max total value possible when weight capacity is k • V13 = ? Consider first object to put in knapsack:

• if object #1 => total max. value would be 10 + max value possible with 13-5

• if object #2 => total max. value would be ….

• if object #i => total max. value would be

What would be V13?

Discussion: Knapsack

24

• Input: weight capacity of a knapsack, W; n different objects (of infinite quantities) with weights and values given by array w[ ], v[ ]

• Output: objects so that total value is maximized and total weights <= W • For example: W=13, w[4]={5,3,8,4}, v[4]={10,20,25,8} in the figure • Let Vk be max total value possible when weight capacity is k • Recursive formula for Vk

• A illustration of calculation of W13

• Run the following command to copy the sample code: copyRodCut

Vk = maxi such that w[i]<k

{v[i] + Vk−w[i]}

Optimal substructure in Knapsack

25

• Input: weight capacity of a knapsack, W; n different objects (of infinite quantities) with weights and values given by array w[ ], v[ ]

• Output: objects so that total value is maximized and total weights <= W • Let Vk be max total value possible when weight capacity is k

• What’s the set of objects (multi-set) that achieve Vk? • The i that achieves the max value above • and then the object i1 chosen for k-w[i] • and then the object i2 chosen for k-w[i]-w[i1] • … until the weight capacity ==0 or < min(w)

• a table obj[]: obj[k] store the first object to chose when capacity is k (the one that maximize …)

• Output the multi-set achieving Vk, using obj[ ] array • Demo…

Vk = maxi such that w[i]<k

{v[i] + Vk−w[i]}

Knapsack: extension

26

Discussion(2)

• Cutting cloth to make products. How to maximize profit?

27

Outline

• Introduction via example: rod cutting • Characteristics of problems that can be





• Summary 28

maximum (contiguous) subarray

• Problem: find contiguous subarray within an array (containing at least one number) which has largest sum • If given the array [-2,1,-3,4,-1,2,1,-5,4], • contiguous subarray [4,-1,2,1] has largest sum = 6

• Brute-force? • brute-force: n2 or n3

• Divide-and-conquer: T(n)=2 T(n/2)+O(n), T(n)=nlogn • Dynamic programming?

29

Analyze optimal solution

• Problem: find contiguous subarray with largest sum • Sample Input: [-2,1,-3,4,-1,2,1,-5,4] (array of size n=9) • How does solution to this problem relates to smaller

subproblem? • If we divide up array into two halves • [-2,1,-3,4,-1,2,1,-5,4] //find MaxSub in this array

• [-2,1,-3,4,-1] [2,1,-5,4] • still need to consider subarray that spans both halves • This does not lead to a dynamic programming sol.

• Need a different way to define smaller subproblems!

30

• Problem: find contiguous subarray with largest sum

A Index

• MSE(k), max. subarray ending at pos k, among all subarray ending at k (A[i…k] where i<=k), the one with largest sum • MSE(1), max. subarray ending at pos 1, is A[1..1], sum is -2 • MSE(2), max. subarray ending at pos 2, is A[2..2], sum is 1 • MSE(3) is A[2..3], sum is -2 • MSE(4)?


31

• A

• Index • MSE(k) and optimal substructure

• MSE(3): A[2..3], sum is -2 (red box) • MSE(4): two options to choose

• (1) either grow MSE(3) to include pos 4 • subarray is then A[2..4], sum is

MSE(3)+A[4]=-2+A[4]=2 • (2) or start afresh from pos 4

• subarray is then A[4…4], sum is A[4]=4 (better) • Choose the one with larger sum, i.e.,

• MSE(4) = max (A[4], MSE(3)+A[4])


32

How a problem’s optimal solution can be derived from optimal solution to smaller problem

• A

• Index • • MSE(k) and optimal substructure

• Max. subarray ending at k is the larger between A[k…k] and Max. subarray ending at k-1 extended to include A[k]

MSE(k) = max (A[k], MSE(k-1)+A[k]) • MSE(5)= , subarray is • MSE(6) • MSE(7) • MSE(8) • MSE(9)


33

MSE(4)=4, array is A[4…4]

• A • Index

• Once we calculate MSE(1) … MSE(9) • MSE(1)=-2, the subarray is A[1..1] • MSE(2)=1, the subarray is A[2..2] • MSE(3)=-2, the subarray is A[2..3] • MSE(4)=4, the subarray is A[4…4] • … MSE(7)=6, the subarray is A[4…7] • MSE(9)=4, the subarray is A[9…9]

• What’s the maximum subarray of A? • well, it either ends at 1, or ends at 2, …, or ends at 9 • Whichever yields the largest sum!


34

Practice: 1) fill in ?? 2) How to find out starting and ending index of the max. subarray?

• Idea 1) Calculate MSE(1) … MSE(n)

MSE(1)= A[1] MSE(i) = max (A[i], A[i]+MSE(i-1));

2) Return max of MSE(i), for i=1, 2, …n

35

Idea to Pseudocode

int MaxSubArray (int A[1…n], int & start, int & end) { // Use array MSE to store the MSE(i) MSE[1]=A[1]; maxSubArray = MSE[1]; //use tabulation/bottom up to calculate MSE for (int i=2;i<=n;i++) { MSE[i] = ??

maxSubArray = max(maxSubArray, MSE[i]); } return maxSubArray; }

36

Running time Analysisint MaxSubArray (int A[1…n], int & start, int & end) { // Use array MSE to store the MSE(i) MSE[1]=A[1]; max_MSE = MSE[1]; for (int i=2;i<=n;i++) { MSE[i] = ??

if (MSE[i] > max_MSE) { max_MSE = MSE[i]; } } return max_MSE; }

• It’s easy to see that running time is O(n) • a loop that iterates

for n-1 times • Recall other solutions:

• brute-force: n2 or n3

• Divide-and-conquer: nlogn

• Dynamic programming wins!

What is DP? When to use?

• We have seen several optimization problems • brute force solution • divide and conquer • dynamic programming

• To what kinds of problem is DP applicable? • Optimal substructure: Optimal solution to a

problem of size n incorporates optimal solution to problem of smaller size (1, 2, 3, … n-1).

• Overlapping subproblems: small subproblem space and common subproblems

37

Optimal substructure• Optimal substructure: Optimal solution to a

problem of size n incorporates optimal solution to problem of smaller size (1, 2, 3, … n-1).

• Rod cutting: find rn (max. revenue for rod of len n)

rn = max (p[1]+rn-1, p[2]+rn-2, p[3]+rn-3,…, p[n-1]+r1, p[n]) • A recurrence relation (recursive formula)

• => Dynamic Programming: Build an optimal solution to the problem from solutions to subproblems • We solve a range of sub-problems as needed

38

Sol to probleminstance of size n

Sol to probleminstance of size n-1, n-2, … 1

Optimal substructure in Max. Subarray Problem

• Optimal substructure: Optimal solution to a problem of size n incorporates optimal solution to problem of smaller size (1, 2, 3, … n-1). • Max. Subarray Problem:

• MSE(i) = max (A[i], MSE(i-1)+A[i])

• Max Subarray = max (MSE(1), MSE(2), …MSE(n))

39

Max. Subarray Ending at position i is the either the max. subarray ending at pos i-1 extended to pos i; or just A[i]

Overlapping Subproblems

• space of subproblems must be “small” • total number of distinct subproblems is a polynomial

in input size (n) • a recursive algorithm revisits same problem

repeatedly, i.e., optimization problem has overlapping subproblems.

• DP algorithms take advantage of this property • solve each subproblem once, store solutions in a

table • Look up table for sol. to repeated subproblem using

constant time per lookup. • In contrast: divide-and-conquer solves new

subproblems at each step of recursion.40

Outline






• Summary 41

Longest Increasing Subsequence

• Input: a sequence of numbers given by an array a • Output: a longest subsequence (a subset of the

numbers taken in order) that is increasing (ascending order)

• Example, given a sequence • 5, 2, 8, 6, 3, 6, 9, 7 • There are many increasing subsequence: 5, 8, 9;

or 2, 9; or 8 • The longest increasing subsequence is:

2, 3, 6, 9 (length is 4)

42

LIS as a DAG• Find longest increasing subsequence of a

sequence of numbers given by an array a 5, 2, 8, 6, 3, 6, 9, 7

Observation: • If we add directed edge from smaller number to larger one, we get

a DAG. • A path (such as 2,6,7) connects nodes in increasing order • LIS corresponds to longest path in the graph. 43

Graph Traversal for LIS• Find longest increasing subsequence of a

sequence of numbers given by an array a 5, 2, 8, 6, 3, 6, 9, 7

Observation: • LIS corresponds to longest path in DAG • Finding Longest path in a general graph is a hard problem

44

http://www.apple.com

• Find Longest Increasing Subsequence of a sequence of numbers given by an array a

Let L(n) be the length of LIS ending at n-th number L(1) = 1, LIS ending at pos 1 is 5 L(2) = 1, LIS ending at pos 2 is 2 L(7)= // how to relate to L(1), …L(6)? • Consider LIS ending at a[7] (i.e., 9). What’s the number before 9? .… ? ,9

Dynamic Programming Sol: LIS

45

1 2 3 4 5 6 7 8

• Given a sequence of numbers given by an array a

Let L(n) be length of LIS ending at n-th number Consider all increasing subsequence ending at a[7] (i.e., 9). • What’s the number before 9?

• It can be either NULL, or 6, or 3, or 6, 8, 2, 5 (all those numbers pointing to 9)

• If the number before 9 is 3 (a[5]), what’s max. length of this seq? L(5)+1 where the seq is …. 3, 9


46

1 2 3 4 5 6 7 8

LIS ending at pos 5


Let L(n) be length of LIS ending at n-th number Consider all increasing subsequence ending at a[7] (i.e., 9). • It can be either NULL, or 6, or 3, or 6, 8, 2, 5 (all those numbers pointing to 9)

• L(7)=max(1, L(6)+1, L(5)+1, L(4)+1, L(3)+1, L(2)+1, L(1)+1)

• L(8)=?


47

Pos: 1 2 3 4 5 6 7 8


Let L(n) be length of LIS ending at n-th number. Recurrence relation:

Note that the i’s in RHS is always smaller than the j • How to implement? Running time? • LIS of sequence = Max (L(i), 1<=i<=n) // the longest among all


48

Pos: 1 2 3 4 5 6 7 8

Outline


solved using dynamic programming • More examples (skipped)

• Maximal subarray problem • Longest increasing subsequence problem

• Two dimensional problem spaces • Add up to N problem • Knapsack with one of each • Longest common subsequence • Matrix chain multiplication

• Summary 49

• Problem: Given an array of int values and integer K, find out whether there is a subset of these numbers that add up to K (and output one such subset) • cannot use a number multiple times

• bool AddUpToK (int numbers[], int n, int K) • K=100, n=9, numbers:

• In lab2: 1) use binary number to enumerate all 2n subsets, 2) use recursive function to check all possible subsets (include numbers[i] or not)

• Now: use dynamic programming to improve recursive solution

K-Sum Problem (add up to K)

50

22 34 18 30 76 1 3 19 80

1 2 3 4 5 6 7 8 9

• Problem: find out whether there is a subset of these numbers that add up to K (and output one such subset)

• bool AddUpToK (int numbers[], int num_len, int K) • K=100, num_len=9 numbers:

• Think recursively! • base case:

• general case: how to reduce it to smaller problem(s)? • hint: recall how we did it in Knapsack and RodCut?


51

22 34 18 30 76 1 3 19 80 1 2 3 4 5 6 7 8 9



• base cases • if K==0 return true • if K>0 and num_len==0 return false

• general case: //how reduce to smaller problem? • consider making one decision: include last number, 20, or not? • if included it, then see if we can add up to 100-20 using the rest of

the numbers • if not, see if we can add up to 100 using the rest of the numbers. • two subproblems (underlined above)


52

22 34 18 80 76 1 3 19 20

1 2 3 4 5 6 7 8 9

Problem: find out whether there is a subset of these numbers that add up to K (and output one such subset) bool AddUpToK (int numbers[], int num_len, int K) { //base cases

if K==0 return true if K>0 and num_len==0 return false

// general case: consider include last number, 20 (in this case), or not? if (AddUpToK (numbers, num_len-1, K-numbers[num_len-1])) return true; //we can make K by including last num… else if (AddUpToK (numbers, num_len-1, K)) return true; //we can make K by not including last num else return false; //not possible }


53

22 34 18 80 76 1 3 19 20

1 2 3 4 5 6 7 8 9



• Overlapping subproblems? • can you give an example of overlapping subproblems? • sol: • if we include 20, but not 19,3,1,76 => AddUpToK(…4,

80) //use numbers[1…4] to make 80 • if we include 19,1 and not 20,3,76 => AddUpToK(…, 4,

80): same subproblems


54

22 34 18 80 76 1 3 19 20

1 2 3 4 5 6 7 8 9



• Overlapping subproblems? • can you give an example of overlapping subproblems? • sol: • if we include 20, but not 19,3,1,76 => AddUpToK(…4,

80) //use numbers[1…4] to make 80 • if we include 19,1 and not 20,3,76 => AddUpToK(…, 4,

80): same subproblems • Memoization or Tabulation: store subproblem solutions


55

22 34 18 80 76 1 3 19 20

1 2 3 4 5 6 7 8 9

/*whether there is a subset of these numbers that add up to K, and output one such subset */ bool AddUpToK_Tabulation (int numbers[], int num_len, int K) { bool C[K+1][num_len+1]; // C[k][n]: can we add up to k using numbers[1…n]

for n=0 to num_len C[0][n] = 0 // if K==0 return true for k=1 to K C[k][0] = false //fill in array row by row, left to right for k=1 to K for n=1 to num_len

if C[k][n-1]==true C[k][n]=true //we can make k without last number

//otherwise, can we include numbers[n] to make k? else if k==numbers[n] C[k][n] = true;

else if k>numbers[n] and C[k-numbers[n]][n-1]==true C[k][n] = true; else //k<numbers[n] or cannot make k-numbers[n] using numbers[1…n-1]

C[k][n] = false return C[K][num_len]; }

K-Sum Problem: tabulation

56

22 34 18 80 76 1 3 19 20

1 2 3 4 5 6 7 8 9

• Input: given weight capacity of a knapsack, W; and n different objects, with weights and values given by array w[ ], v[ ] • there is one of each type object: one gold bar, one diamond… • finding a subset of objects …

• Output: objects to select so that total value is maximized and total weights <= W

• We could enumerate all possible subsets (as in lab2) • => Running time

• Plan: • recursive solution • use memoization or tabulation to improve

Ω(2n)

Knapsack without repetition

57

/* Output max. value, and objects to select @param W: given weight capacity, >=0 @ param n: # of objects, n>=0 @param w, v: weights and values @return max value achievable from obj [1…n] */

Knapsack_Norepeat (W, w, v, n) { if W==0 or n == 0 //base case return 0

//general case: reduce to smaller problems? //Q: include n-th obj or not, which one yields larger value? if (w[n] > W) //it’s possible to include n-th obj V1 = Knapsack_Norepeat (W-w[n], w, v, n-1) + V[n] //if we include n-th obj V2 = Knapsack_Norepeat (W, w, v, n-1) //if we don’t include n-th obj return max(V1, V2) else //impossible to include n-th obj // only one option: do not include n-th obj return Knapsack_Norepeat (W, w, v, n-1) }

58

Just Recursion!

• Problem size has two dimensions: W and n

• Recursive calls reduce problem size in one or both dimension

• eventually reach base case • Running time

• in worst case T(n)=2T(n-1) , n is the number of objects

• ==> T(n) = 2n

Knapsack_Norepeat (W, w, v, n) { initialize V[W+1][n+1] to store solutions to subproblems // V[w][n] is the max value achievable with weight capacity w and obj 1…n fill V with -1 Knapsack_Norepeat_helper (W, w, v, n, V); }

Knapsack_Norepeat_helper (W, w, v, n, V) { if (V[w][n]>=0) // If the problem has been solved before…. return V[w][n] if (w==0 or n==0)

V[w][n]= 0 return V[w][n]

//general case: reduce to smaller problems? //Q: include n-th obj or not, which one yields larger value? if (w[n] > W) //it’s possible to include n-th obj V1 = Knapsack_Norepeat_helper (W-w[n], w, v, n-1, V) + V[n] V2 = Knapsack_Norepeat_helper (W, w, v, n-1, V) V[w][n] = max (V1, V2) return V[w][n] else //impossible to include n-th obj // only one option: do not include n-th obj V[w][n] = Knapsack_Norepeat_helper (W, w, v, n-1, V) return V[w][n]; }

59

Recursion and Memoization

Running time analysis? T(w,n) = O (nm)

Outline• Introduction via example: rod cutting • Characteristics of problems that can be solved

using dynamic programming • More one dimensional examples

• Knapsack with repetition, Maximal subarray problem, Longest increasing subsequence

• use one dimensional array to store subproblem solutions

• Two dimensional problem spaces • K-Sum • Knapsack without repetition • Longest common subsequence • Matrix chain multiplication (skipped)

• Summary 60

Longest Common Subseq.

• Given a sequence, X = 〈x1, x2, …, xm〉, where xi is a letter from a certain alphabet, a subsequence of X is a subset of elements in sequence X taken in order but not necessarily consecutive • e.g., X = 〈A, B, C, B, D, A, B〉, here are some subsequences of X: 〈A, B, D〉,〈B, C, D, B〉,〈A〉,

〈〉, 〈A, B, C, B, D,A, B〉 • How many possible subsequences are there for X?

• e.g., DNA alphabet is {A, C, G, T} • Denote length of a sequence X by |X|, which is the

number of letters in the sequence • e.g., X = 〈A, B, C, B, D, A, B〉, |X|=7

Longest Common Subseq.

• Given two sequences, X = 〈x1, x2, …, xm〉, Y = 〈y1, y2, …, yn〉 where xi, yi are letters, find a longest common subsequence (LCS) of X and Y • a sequence that is subsequence of X, and is a

subsequence of Y

• and is of the same or longer length than all common subsequences of X and Y

LCS: sequence alignment

X = 〈A, B, C, B, D, A, B〉 X = 〈A, B, C, B, D, A, B〉

Y = 〈B, D, C, A, B, A〉 Y = 〈B, D, C, A, B, A〉

• 〈B, C, B, A〉 and 〈B, D, A, B〉 are both longest common subsequences of X and Y (length = 4)

• BCBA = LCS(X,Y): functional notation, but is it not a function

• 〈B, C, A〉 is a common subsequence of X, Y, but it is not a LCS

63

Brute-Force Solution1. /* Check every subsequence of X[1 . . m] to see if it is also a subsequence of Y[1 .. n].

*/ 2. LCS (X, Y) 3. { 4. for each of 2m subsequence, s, of X 5. //check if s is a subsequence of Y, O(n) time 6. k = |s| //k is length of s 7. j = 1 8. for i=1 to k 9. while ( Y[j]!=s[i] and j<=n) 10. j++ // scan Y for a letter matching s[i] 11. if j>n // cannot match s[i] 12. break and s not a subsequence Y 13. 14. s is subsequence of Y, update longest 15. 16.} Worst-case running time: O(n2m) 64

Thinking about subproblems• Given a sequence X = 〈x1, x2, …, xm〉, we define i-th prefix of X (for i

= 0, 1, 2, …, m) as Xi = 〈x1, x2, …, xi〉

• for example, X = 〈A, B, C, B, D, A, B〉, we have the following prefixes:

• X2 = 〈A, B〉, X4 = 〈A, B, C, B〉, • X0 = 〈〉, a special prefix that is an empty sequence • X7 = 〈A, B, C, B, D, A, B〉, a special prefix that is original

sequence • What’s X5 ?

• We focus on length of LCS first, • Define: c[i, j] = | LCS (Xi, Yj) = |LCS(X[1..i], Y[1..j])|, i.e., the length

of a LCS of i-th prefix of X, Xi = 〈x1, x2, …, xi〉, and j-th prefix of Y, Yj = 〈y1, y2, …, yj〉.

65

Recursive Solution. Case 1/* Return the longest subsequence of X, Y @param X: is a string @param Y: is a string @return the longest common subsequence of X and Y*/ LCS (X, Y) m = |X|, n = |Y| if X[m] ==Y[n]

e.g.: X = 〈A, B, D, E〉

Y = 〈Z, B, E〉• find a LCS of Xm-1 and Yn-1, (here, X3 = 〈A, B, D〉 and Y2 = 〈Z, B〉, <B>)

• append X[m] to the LCS of Xm-1 and Yn-1 => <B, E>66

Recursive Solution. Case 2if X[m] ≠ Y[n]

e.g.: X = 〈A, B, D, G〉

Y = 〈Z, B, D〉

• Must solve two subproblems

LCS(X, Y) = Longer { LCS(Xm - 1, Y), LCS (X, Yn-1)}

67

Either the G or the D is not in the LCS

(they cannot be both in LCS)

If we ignore last element in X

If we ignore last element in Y

/* Return the longest subsequence of X, Y @param X: is a string @param Y: is a string @return the longest common subsequence of X and Y*/ LCS (X, Y) { m=|X|, n=|Y| if (|X| == 0 or |Y|==0) return “ “; //empty string //general case // can we match last letters of X and Y? if X[m] == Y[n] return LCS(Xm-1,Yn-1)+X[m] //concatenated with last letter else // Xm and Yn cannot be both in LCS s1 = LCS(Xm-1,Yn) s2 = LCS(Xm, Yn-1) return longer of s1 and s2 }

68

X = 〈A, B, D, E〉Y = 〈Z, B, E〉

X = 〈A, B, D, G〉Y = 〈Z, B, D〉

Recursion

Three Questions?

Here we need to calculate prefix (Xm-1, Yn-1), and pass them to recursive calls

/* Return the longest subsequence of X, Y @param X : is a string @param Y: is a string @return the longest common subsequence of Xm and Yn*/ LCS (X, Y, m, n ) { if (m== 0 or n==0) return “ “; //empty string //general case // can we match last letters of X and Y? if X[m] == Y[n] return LCS(X,Y, m-1, n-1)+X[m] //concatenated with last letter else // Xm and Yn cannot be both in LCS s1 = LCS(X, Y, m-1, n ) s2 = LCS(X, Y, m, n-1) return longer of s1 and s2 }

// keep X, Y as they are, use parameters to specify prefix length // subproblem’s size is given by m and n, at least one dimension is decreased 69

X = 〈A, B, D, E〉Y = 〈Z, B, E〉

X = 〈A, B, D, G〉Y = 〈Z, B, D〉

Recursion

Three Questions?

Optimal substructure & Overlapping Subproblems

• A recursive solution contains a “small” number of distinct subproblems repeated many times. • e.g., LCS (5,5) depends on LCS(4,4), LCS(4,5), LCS(5,4) • Exercise: Draw subproblem dependence graph

• each node is a subproblem • directed edge represents “calling”, “uses solution of” relation

• Small number of distinct subproblems: • total number of distinct LCS subproblems for two

strings of lengths m and n is mn.

70

Tabulation to avoid recalculation• Given two sequences X = 〈x1, x2, …, xm〉, Y = 〈y1, y2, …, yn〉

• To ease tracing, we focus on finding length of LCS

c[i, j] = | LCS (Xi, Yj) | // length of LCS

71

c[i-1, j-1] + 1 if X[i]= Y[j] c[i, j] = max(c[i, j-1], c[i-1, j]) otherwise (i.e., if X[i] ≠ Y[j])

Tabulation

C[2,3] C[2,4]

C[3,3]C[3,4]

Y A B C B D A B X

B

D

C

A

B

A

LCS (X, Y) X=<B, D, C, A, B, A> Y=<A, B, C, B, D, A, B> |X|=6, |Y|=7

c is 7x8 array

Goal: calculate C[6,7] (bottom right corner)

72

0 1 2 3 4 5 6 7

0

1

2

3

4

5

6

C[3,4]= length of LCS (X3, Y4) = Length of LCS (BDC, ABCB)

3rd row, 4th column element

Memoization algorithm Memoization: After computing a solution to a subproblem, store it in a table.

Subsequent calls check the table to avoid redoing work. LCS (X, Y) create and initialize table c[m][n] to -1 LCS_helper(X, Y, |X|, |Y|, c)

LCS_helper(X, Y, i, j, c) if c[i, j] = NIL // LCS(i,j) has not been solved yet then if i==0 or j==0 c[i,j]=0 return 0 if x[i] = y[j] c[i, j] ←LCS_helper(x, y, i–1, j–1, c) + 1 else c[i, j] ←max{ LCS_helper(x, y, i–1, j, c), LCS_helper(x, y, i, j–1,c) } else return c[i,j]

73

Tracing Memoization

C[2,3] C[2,4]

C[3,3]C[3,4]

Y A B C B D A B X

B

D

C

A

B

A

LCS (X, Y) X=<B, D, C, A, B, A> Y=<A, B, C, B, D, A, B> |X|=6, |Y|=7

c is 7x8 array

Start from C[6,7] C[5,7] and C[6,6]

74

0 1 2 3 4 5 6 7

0

1

2

3

4

5

6

C[3,4]= length of LCS (X3, Y4) = Length of LCS (BDC, ABCB)

3rd row, 4th column element

Bottom-Up

C[2,3] C[2,4]

C[3,3]C[3,4]

Y A B C B D A B X

B

D

C

A

B

A

Initialization: base case c[i,j] = 0 if i=0, or j=0

//Fill table row by row // from left to right for (int i=1; i<=m;i++) for (int j=1;j<=n;j++) update c[i,j]

return c[m, n]

Running time = Θ(mn)

75

0 1 2 3 4 5 6 7

0

1

2

3

4

5

6C[3,4]= length of LCS (X3, Y4) = Length of LCS (BDC, ABCB)

3rd row, 4-th column element

Dynamic-Programming AlgorithmA B C B D A B

B

D

C A

B

A

0 0 0 0 0 0 0 0

0 0 1 1 1 1 1 1

1 1 1 2 2 20 0

1 2 2 2 2 20 0

1 2 2 2 3 30 1

2 2 3 3 3 40 1

2 2 3 3 4 40 1

Reconstruct LCS tracing backward: Where do we get value of C[i,j] from?

• C[i-1,j-1]+1 ( ) • C[i-1,j] ( ) • C[i, j-1] ( )

Use a table b[i][j] to store above arrows (Up, Left, LU)

76Output

AOutput

BOutput

COutput

B

Remark

• Longest common subsequence algorithm is similar to • minimum edit distance problem (used by spell

checker to suggest a correction)

• Needleman-Wansh Alg. (used in bioinformatics to align protein or nucleotide sequences) 77

MatrixMatrix: a 2D (rectangular) array of numbers, symbols, or expressions, arranged in rows and columns.

e.g., a 2 × 3 matrix (there are two rows and three columns)

Each element of a matrix is denoted by a variable with two subscripts, a2,1 element at second row and first column of a matrix A.

an m × n matrix A:

78

Matrix Multiplication:

Matrix Multiplication

79

Dimension of A, B, and A x B?

Total (scalar) multiplication: 4x2x3=24

Total (scalar) multiplication: n2xn1xn3

Multiplying a chain of Matrix Multiplication

Given a sequence/chain of matrices, e.g., A1, A2, A3, there are different ways to calculate A1A2A3

1. (A1A2)A3)

2. (A1(A2A3))

Dimension of A1: 10 x 100

A2: 100 x 5

A3: 5 x 50

all yield the same result

But not same efficiency

80

Matrix Chain MultiplicationGiven a chain <A1, A2, … An> of matrices, where matrix Ai has dimension pi-1x pi, find optimal fully parenthesize product A1A2…An that minimizes number of scalar multiplications.

Chain of matrices <A1, A2, A3, A4>: five distinct ways

A1: p1 x p2 A2: p2 x p3 A3: p3 x p4 A4: p4 x p5

81

# of multiplication: p3p4p5+ p2p3p5+ p1p2p5

Find the one with minimal multiplications?

Matrix Chain Multiplication• Given a chain <A1, A2, … An> of matrices, where matrix Ai has

dimension pi-1x pi, find optimal fully parenthesize product A1A2…An that minimizes number of scalar multiplications.

• Let m[i, j] be the minimal # of scalar multiplications needed to calculate AiAi+1…Aj (m[1…n]) is what we want to calculate)

• Recurrence relation: how does m[i…j] relate to smaller problem • First decision: pick k (can be i, i+1, …j-1) where to divide AiAi+1…Aj

into two groups: (Ai…Ak)(Ak+1…Aj)

• (Ai…Ak) dimension is pi-1 x pk, (Ak+1…Aj) dimension is pk x pj

82

Summary

• Keys to DP • Recursive algorithm => optimal Substructure • overlapping subproblems

• Write recurrence relation for subproblem: i.e., how to calculate solution to a problem using sol. to smaller subproblems

• Implementation: • memoization (table+recursion) • bottom-up table based (smaller problems first)

• Insights and understanding comes from practice!

83

Dynamic Programming CISC4080, Computer Algorithms CIS ...

Documents

Transcript of Dynamic Programming CISC4080, Computer Algorithms CIS ...