lecture 24
description
Transcript of lecture 24
![Page 1: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/1.jpg)
04/10/23 1
CS 332 - Algorithms
Dynamic programming
Longest Common Subsequence
![Page 2: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/2.jpg)
04/10/23 2
Dynamic programming
It is used, when the solution can be
recursively described in terms of solutions
to subproblems (optimal substructure)
Algorithm finds solutions to subproblems
and stores them in memory for later use
More efficient than “brute-force methods”,
which solve the same subproblems over
and over again
![Page 3: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/3.jpg)
04/10/23 3
Longest Common Subsequence (LCS)
Application: comparison of two DNA stringsEx: X= {A B C B D A B }, Y= {B D C A B A}
Longest Common Subsequence:
X = A B C B D A B
Y = B D C A B A
Brute force algorithm would compare each subsequence of X with the symbols in Y
![Page 4: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/4.jpg)
04/10/23 4
LCS Algorithm if |X| = m, |Y| = n, then there are 2m
subsequences of x; we must compare each with Y (n comparisons)
So the running time of the brute-force algorithm is O(n 2m)
Notice that the LCS problem has optimal substructure: solutions of subproblems are parts of the final solution.
Subproblems: “find LCS of pairs of prefixes of X and Y”
![Page 5: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/5.jpg)
04/10/23 5
LCS Algorithm First we’ll find the length of LCS. Later we’ll
modify the algorithm to find LCS itself. Define Xi, Yj to be the prefixes of X and Y of
length i and j respectively Define c[i,j] to be the length of LCS of Xi and
Yj
Then the length of LCS of X and Y will be c[m,n]
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjic
jyixjicjic
![Page 6: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/6.jpg)
04/10/23 6
LCS recursive solution
We start with i = j = 0 (empty substrings of x
and y)
Since X0 and Y0 are empty strings, their LCS
is always empty (i.e. c[0,0] = 0)
LCS of empty string and any other string is
empty, so for every i and j: c[0, j] = c[i,0] = 0
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjic
jyixjicjic
![Page 7: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/7.jpg)
04/10/23 7
LCS recursive solution
When we calculate c[i,j], we consider two
cases:
First case: x[i]=y[j]: one more symbol in
strings X and Y matches, so the length of LCS
Xi and Yj equals to the length of LCS of
smaller strings Xi-1 and Yi-1 , plus 1
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjic
jyixjicjic
![Page 8: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/8.jpg)
04/10/23 8
LCS recursive solution
Second case: x[i] != y[j]
As symbols don’t match, our solution is not
improved, and the length of LCS(Xi , Yj) is the
same as before (i.e. maximum of LCS(Xi, Yj-1)
and LCS(Xi-1,Yj)
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjic
jyixjicjic
Why not just take the length of LCS(Xi-1, Yj-1) ?
![Page 9: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/9.jpg)
04/10/23 9
LCS Length AlgorithmLCS-Length(X, Y)1. m = length(X) // get the # of symbols in X2. n = length(Y) // get the # of symbols in Y
3. for i = 1 to m c[i,0] = 0 // special case: Y0
4. for j = 1 to n c[0,j] = 0 // special case: X0
5. for i = 1 to m // for all Xi
6. for j = 1 to n // for all Yj
7. if ( Xi == Yj )8. c[i,j] = c[i-1,j-1] + 19. else c[i,j] = max( c[i-1,j], c[i,j-1] )10. return c
![Page 10: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/10.jpg)
04/10/23 10
LCS ExampleWe’ll see how LCS algorithm works on the
following example: X = ABCB Y = BDCAB
LCS(X, Y) = BCBX = A B C BY = B D C A B
What is the Longest Common Subsequence of X and Y?
![Page 11: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/11.jpg)
04/10/23 11
LCS Example (0)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
X = ABCB; m = |X| = 4Y = BDCAB; n = |Y| = 5Allocate array c[5,4]
ABCBBDCAB
![Page 12: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/12.jpg)
04/10/23 12
LCS Example (1)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
for i = 1 to m c[i,0] = 0 for j = 1 to n c[0,j] = 0
ABCBBDCAB
![Page 13: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/13.jpg)
04/10/23 13
LCS Example (2)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
0
ABCBBDCAB
![Page 14: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/14.jpg)
04/10/23 14
LCS Example (3)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
0 0 0
ABCBBDCAB
![Page 15: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/15.jpg)
04/10/23 15
LCS Example (4)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
0 0 0 1
ABCBBDCAB
![Page 16: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/16.jpg)
04/10/23 16
LCS Example (5)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
000 1 1
ABCBBDCAB
![Page 17: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/17.jpg)
04/10/23 17
LCS Example (6)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
0 0 10 1
1
ABCBBDCAB
![Page 18: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/18.jpg)
04/10/23 18
LCS Example (7)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 1 11
ABCBBDCAB
![Page 19: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/19.jpg)
04/10/23 19
LCS Example (8)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 1 1 1 2
ABCBBDCAB
![Page 20: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/20.jpg)
04/10/23 20
LCS Example (10)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj )c[i,j] = c[i-1,j-1] + 1
else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
21 1 11
1 1
ABCBBDCAB
![Page 21: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/21.jpg)
04/10/23 21
LCS Example (11)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 11
1 1 2
ABCBBDCAB
![Page 22: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/22.jpg)
04/10/23 22
LCS Example (12)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj )c[i,j] = c[i-1,j-1] + 1
else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 1
1 1 2
1
22
ABCBBDCAB
![Page 23: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/23.jpg)
04/10/23 23
LCS Example (13)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 1
1 1 2
1
22
1
ABCBBDCAB
![Page 24: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/24.jpg)
04/10/23 24
LCS Example (14)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj )c[i,j] = c[i-1,j-1] + 1
else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 1
1 1 2
1
22
1 1 2 2
ABCBBDCAB
![Page 25: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/25.jpg)
04/10/23 25
LCS Example (15)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3
ABCBBDCAB
![Page 26: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/26.jpg)
04/10/23 26
LCS Algorithm Running Time
LCS algorithm calculates the values of each entry of the array c[m,n]
So what is the running time?
O(m*n)
since each c[i,j] is calculated in constant time, and there are m*n elements in the array
![Page 27: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/27.jpg)
04/10/23 27
How to find actual LCS So far, we have just found the length of LCS,
but not LCS itself. We want to modify this algorithm to make it
output Longest Common Subsequence of X and Y
Each c[i,j] depends on c[i-1,j] and c[i,j-1]
or c[i-1, j-1]
For each c[i,j] we can say how it was acquired:
2
2 3
2 For example, here c[i,j] = c[i-1,j-1] +1 = 2+1=3
![Page 28: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/28.jpg)
04/10/23 28
How to find actual LCS - continued Remember that
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjic
jyixjicjic
So we can start from c[m,n] and go backwards Whenever c[i,j] = c[i-1, j-1]+1, remember
x[i] (because x[i] is a part of LCS) When i=0 or j=0 (i.e. we reached the
beginning), output remembered letters in reverse order
![Page 29: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/29.jpg)
04/10/23 29
Finding LCSj 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
Yj BB ACD
0
0
00000
0
0
0
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3B
![Page 30: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/30.jpg)
04/10/23 30
Finding LCS (2)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
Yj BB ACD
0
0
00000
0
0
0
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3B
B C BLCS (reversed order):
LCS (straight order): B C B (this string turned out to be a palindrome)
![Page 31: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/31.jpg)
04/10/23 31
Knapsack problem
Given some items, pack the knapsack to get the maximum total value. Each item has some weight and some value. Total weight that we can carry is no more than some fixed number W.So we must consider weights of items as well as their value.
Item # Weight Value 1 1 8 2 3 6 3 5 5
![Page 32: lecture 24](https://reader036.fdocuments.us/reader036/viewer/2022062617/54c6dac34a79597d0e8b4574/html5/thumbnails/32.jpg)
04/10/23 32
Knapsack problem
There are two versions of the problem:(1) “0-1 knapsack problem” and(2) “Fractional knapsack problem”
(1) Items are indivisible; you either take an itemor not. Solved with dynamic programming(2) Items are divisible: you can take any fraction of an item. Solved with a greedy algorithm.