CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces...
Transcript of CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces...
![Page 1: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/1.jpg)
CS 312: Algorithm Design & Analysis
Lecture #35: Branch and Bound Design Options: State Spaces
Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, and Sean Warnick
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.
![Page 2: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/2.jpg)
Announcements
Homework #25 due now Questions about bounding functions?
Screencast and Quiz Due: end of day Thursday
Project #7: TSP We continue discussing design options today Help sessions: Thursday (3/26) Whiteboard experience: Monday (3/30) Early: Wednesday 4/8 Due: Friday 4/10
![Page 3: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/3.jpg)
Objectives
Consider an alternate way to structure the state space. i.e., an alternate method for state expansion
Provide more design options to complete the project.
![Page 4: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/4.jpg)
Review the Ingredients of B&B
![Page 5: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/5.jpg)
Review: A Bound on TSP Tour
1
2
3 4
58
67
4
4
3
2
19
1012
999 9 999 8 999999 999 4 999 2999 3 999 4 999999 6 7 999 12
1 999 999 10 999
![Page 6: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/6.jpg)
1. Reduce Rows
999 1 999 0 999999 999 2 999 0999 0 999 1 999999 0 1 999 6
0 999 999 9 999
1
2
3 4
50
01
2
1
0
0
01
96
999 9 999 8 999999 999 4 999 2999 3 999 4 999999 6 7 999 12
1 999 999 10 999
![Page 7: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/7.jpg)
2. Reduce Columns
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 0 0 999 6
0 999 999 9 999
1
2
3 4
50
00
2
1
0
0
01
96
999 1 999 0 999999 999 2 999 0999 0 999 1 999999 0 1 999 6
0 999 999 9 999
![Page 8: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/8.jpg)
Review: Feasibility
Part of our state expansion function The feasibility test may include any
condition you might think of: Vertex doubly entered Vertex doubly exited Cycle among sub-set of cities No options remaining – dead end.
![Page 9: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/9.jpg)
Review: Pruning
Under what conditions can a search state be pruned?
Just one condition: the (lower) bound for state BSSF (current upper bound)
When? Right after the state is created
Don’t even add it to the agenda
Either / or: Eager pruning: When the BSSF is updated Lazy pruning: When the state comes off the agenda
![Page 10: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/10.jpg)
B&B Pseudo-Code: Eager Pruning
function BandB()s init_state()BSSF quick-solution(s) // BSSF.cost holds costAgenda.clear()Agenda.add(s, s.bound)while !Agenda.empty() and time remains and BSSF.cost != Agenda.first().bound do
u Agenda.first()Agenda.remove_first()children = successors(u)
for each w in children doif ! time remains then breakif (w.bound is better than BSSF.cost) then
if criterion(w) thenBSSF wAgenda.prune(BSSF.cost)
elseAgenda.add(w, w.bound)
return BSSF
Proj. #7 option:Use this pseudo-code for eager B&B.
![Page 11: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/11.jpg)
B&B Pseudo-Code: Lazy Pruning
function BandB()s init_state()BSSF quick-solution(s) // BSSF.cost holds costAgenda.clear()Agenda.add(s, s.bound)while !Agenda.empty() and time remains and BSSF.cost != Agenda.first().bound do
u Agenda.first()Agenda.remove_first()if (u.bound is better than BSSF.cost) then
children = successors(u)for each w in children do
if ! time remains then breakif (w.bound is better than BSSF.cost) then
if criterion(w) thenBSSF w
elseAgenda.add(w, w.bound)
return BSSF
Proj. #7 option:Use this pseudo-code for lazy B&B.
![Page 12: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/12.jpg)
Structure of State Space
Recall the structure of the state space from last time.
![Page 13: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/13.jpg)
Review: State Space #1 Generate all children one step away
each child state is generated by including an edge from city j, to a previously unvisited city (where j is the destination of the last edge in the parent state)
Pro: Does not include premature cycles. Why not? Pro: Simple algorithm: pick all neighbors that remain from
vertex j, (leave the starting vertex for the end) Con: On many instances, while searching with B&B, will
result in excessively deep (possibly exponential) agendas. Why?
Is B&B in this state space optimal, given enough time and space?
Includeedge (j,1) (j,2) (j,3) … (j,N)
(i,j)
![Page 14: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/14.jpg)
Structure of State Space
Could we structure the space another way?
How else could we generate children states?
![Page 15: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/15.jpg)
State Space #2
Include edge (i,j) Exclude edge (i,j)
Use same representation for states & same bounding function.
New method for generating children:
Why is branch and bound in this state space optimal, given enough time?
![Page 16: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/16.jpg)
How: Excluding an Edge
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 0 0 999 6
0 999 999 9 999
1
2
3 4
50
00
2
1
0
0
01
96
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 999 0 999 6
0 999 999 9 999
What happens to a state’s cost matrix when an edge is excluded?
Exclude 4 2
![Page 17: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/17.jpg)
Size of the State Space
Start with Complete Graph
![Page 18: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/18.jpg)
State Space: Strategy #2
![Page 19: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/19.jpg)
![Page 20: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/20.jpg)
Which edge to select?
What strategies for edge selection can you think of?
Idea: try to reduce the size of the part of the state-space you actually need to explore!
Avoid going into the deep (exclude) side, if possible.
![Page 21: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/21.jpg)
0
Try #1: Consider edge (3,2)
include (3,2) exclude (3,2)
999 999 999 0 999999 999 1 999 0999 999 999 999 999999 999 0 999 6
0 999 999 9 999
999 1 999 0 999999 999 1 999 0999 999 999 1 999999 0 0 999 6
0 999 999 9 9991
2
3 4
50
01
0
0
96
1
2
3 4
50
00
1
1
0
0
01
96
bound = 22bound = 21
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 0 0 999 6
0 999 999 9 999
Parent:
bound = 21
Both children similarlycompetetive.
![Page 22: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/22.jpg)
Try #2: Consider edge (2,3)
include (2,3) exclude (2,3)
bound = 21bound = 28
bound = 21
1
2
3 4
50
01
1
0
01
96
1
2
3 4
50
00
1
1
0
0
01
96
999 1 999 0 999999 999 999 999 999999 0 999 1 999999 0 999 999 6
0 999 999 9 999
999 1 999 0 999999 999 999 999 0999 0 999 1 999999 0 0 999 6
0 999 999 9 999
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 0 0 999 6
0 999 999 9 999
Parent:
Left child more likely tobe pruned.
![Page 23: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/23.jpg)
Try #3: Consider edge (5,1)
include (5,1) exclude (5,1)
bound = 30bound = 21
bound = 21
1
2
3 4
50
00
1
1
0
0
01
96
1
2
3 4
50
00
1
1
0
0
01
6
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 0 0 999 6999 999 999 999 999
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 0 0 999 6999 999 999 9 999
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 0 0 999 6
0 999 999 9 999
Parent:
Right child more likelyto be pruned. Let’s use thisapproach for our example.
![Page 24: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/24.jpg)
State Space so far
include (5,1) exclude (5,1)
bound = 30bound = 21
bound = 21
![Page 25: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/25.jpg)
Consider edge (2,5)
include (2,5) exclude (2,5)
bound = 28bound = 21
bound = 21
1
2
3 4
50
00
1
0
0
1
999 1 999 0 999999 999 999 999 999999 0 999 1 999999 0 0 999 999999 999 999 999 999
999 1 999 0 999999 999 1 999 999999 0 999 1 999999 0 0 999 6999 999 999 0 999 1
2
3 4
50
00
1
1
0
0
1
6
999 1 999 0 999999 999 1 999 0999 0 999 1 999999 0 0 999 6999 999 999 999 999
Parent:
Again, right child morelikely to be pruned.
![Page 26: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/26.jpg)
Search Space so far
include (5,1) exclude (5,1)
3021
21
include (2,5) exclude (2,5)
2821
![Page 27: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/27.jpg)
Consider edge (3,2)
include (3,2) exclude (3,2)
bound = 22bound = 21
bound = 21
999 999 999 0 999999 999 999 999 999999 999 999 999 999999 999 0 999 999999 999 999 999 9991
2
3 4
50
00
1
2
3 4
50
00
0
0
1
999 1 999 0 999999 999 999 999 999999 999 999 1 999999 0 0 999 999999 999 999 0 999
999 1 999 0 999999 999 999 999 999999 0 999 1 999999 0 0 999 999999 999 999 999 999
Parent:
![Page 28: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/28.jpg)
Search Space so far
include (5,1) exclude (5,1)
3021
21
include (2,5) exclude (2,5)
2821
include (3,2)
21
exclude (3,2)
22
![Page 29: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/29.jpg)
Search Space so far
include (5,1) exclude (5,1)
3021
21
include (2,5) exclude (2,5)
2821
include (3,2)
21
exclude (3,2)
22
1
2
3 4
50
0
...
21
include (4,3),(1,4)
![Page 30: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/30.jpg)
Summarize: State Space #2 Pick an edge e = (i,j) and generate two children:
1. Include edge e2. Exclude edge e
One might pick an edge by: maximizing the difference between the included edge child and
the excluded edge child maximizing the bound on the excluded edge child minimizing the bound on the included edge child
One must be careful not to create a cycle prematurely How to prevent?
Includeedge (i,j)
Excludeedge (i,j)
![Page 31: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/31.jpg)
Conclusions
You have options!
Having some control without sacrificing optimality is important.
You may be able to prune more aggressively without sacrificing optimality.
![Page 32: CS 312: Algorithm Design & Analysis Lecture #35: Branch and Bound Design Options: State Spaces Slides by: Eric Ringger, with contributions from Mike Jones,](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f2c5503460f94c478eb/html5/thumbnails/32.jpg)
Assignment
HW #26 Due Friday
Read the project instructions and lecture notes
Have your “whiteboard experience” for Project #7 Consider your options Make your choice Design your algorithm Work and talk through an example