1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.
-
Upload
claud-walton -
Category
Documents
-
view
214 -
download
0
Transcript of 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.
![Page 1: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/1.jpg)
1
Sequence Grammars for Very Large Scale
Neighborhood Search
Agustin Bompadre
James Orlin
ISMP 2006
![Page 2: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/2.jpg)
2
On Combinatorial Optimization
Recent incredible advances in our ability to solve IPs
But… combinatorial optimization problems are still difficult to solve, and always will be
We will always rely on other approaches
– Neighborhood search
– Other heuristics
![Page 3: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/3.jpg)
3
Very Large Scale Neighborhood (VLSN) Search
Neighborhood search:
– Iteratively replace a current solution with the
best solution in a neighborhood VLSN Search:
– The neighborhoods are too large to search explicitly
• exponentially large
– Neighborhood can be searched efficiently
• Polynomial time
![Page 4: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/4.jpg)
4
Background on VLSN Search:Collaborators with Ravi Ahuja and Jim OrlinRicha Agarwal, Georgia Institute of
Technology, Atlanta
Agustin Bompadre, MIT
Ozlem Ergun, Georgia Institute of Technology, Atlanta
Jon Goodstein, United Airlines, Chicago
Wei Huang, University of Florida, Gainesville
Krishna C. Jha, University of Florida, Gainesville
Arvind Kumar, University of Florida, Gainesville
Gilbert Laporte, University of Montreal, Montreal
Jian Liu, University of Florida, Gainesville
D. Romero Morales, Maastricht University, Netherlands
Amit Mukherjee, United Airlines, Chicago
Stefano Pallottino, University of Pisa, Pisa, Italy
Abraham P. Punnen, University of New Brunswick, Fredericton
H. Edwin Romeijn, University of Florida, Gainesville
Maria P. Scaparra, University of Pisa, Pisa, Italy
Maria G. Scutellà, University of Pisa, Pisa, Italy
Dushyant Sharma, University of Michigan, Ann Arbor
Zuo J. Shen, University of Florida, Gainesville
Larry Shughart, Charles River Associates, Boston
![Page 5: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/5.jpg)
5
VLSN Search has been applied to …
Classic Combinatorial Problems
– TSP, QAP, GAP, and more Vehicle routing Problems Partitioning Problems Airline Fleet Scheduling Locomotive Scheduling Block Scheduling for Rail and more Weapon Target Assignment Timetabling Problems Telecomunications etc
![Page 6: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/6.jpg)
6
Some References
http://www.ise.ufl.edu/ahuja/vlsn/index.htm
http://web.mit.edu/jorlin/www
R.K. Ahuja, O. Ergun, J.B. Orlin, and A.P. Punnen. 2002.
R.K. Ahuja, O. Ergun, J.B. Orlin, and A.P. Punnen. 2005
Deineko and Woeginger [2000]
![Page 7: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/7.jpg)
7
Overview of this talk
Unifying structure for a variety of VLSN search techniques for sequencing problems
– Based on context free grammars and regular grammars
– Encompasses a number of previous results for the TSP
– Extends to other sequencing problems
![Page 8: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/8.jpg)
8
Two Sequencing Problems
minimize (f() : Sn)
– Sn is the set of permutations on n objects
Example 1: TSP1
( ), (1) ( ), ( 1)1( )
n
n i iif c c
![Page 9: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/9.jpg)
9
,, : ( ) ( )( ) i ji j i j
f c
Linear Ordering Problem (LOP)
e.g., in ranking college football teams
A little known result
M Groetschel, M Juenger, G
Reinelt , 1984
![Page 10: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/10.jpg)
10
Assume initial sequence is 1, …, n
Example: the swap neighborhood
1 2 3 4 5 6 7 8
swap [4,7] 1 5 62 3 7 84
5 3 8 1 2 7 6 4
swap [4,7] 5 3 8 6 2 7 1 4
swap [4,7] swap elements in positions 4 and 7.
4 7
1 6
A neighborhood of the permutation 1, 2, …, n induces a neighborhood for any other permutation.
![Page 11: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/11.jpg)
11
A very large Neighborhood: Pyramidal tours Klyaus [1976]
N = { : 1 = (1) < (2) < … < (k) for some k and (k) > (k+1) > … > (n) }
1 4 6 8 7 5 3 2
Size of neighborhood: 2n-1
Time to search the neighborhood: O(n2)
![Page 12: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/12.jpg)
12
Another Very Large Scale Neighborhood
Example: Balas-Simonides nbhd [2001]
N = { : (1) = 1 and if (i) < (j) then i < j+K for all i, j}
1 3 4 2 5 8 6 7
an example when K = 2
Exponential size: even if K = 2.
Time to search the neighborhood: O(n K2 2K)
![Page 13: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/13.jpg)
13
The research
Uses grammars to describe exponentially large sequence grammars.
Polynomial time DP algorithm for finding the min
cost neighbor for
– the TSP
– the LOP
– many other problems
![Page 14: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/14.jpg)
14
What is a grammar?
A grammar consists of a set of symbols, and production rules that explain how to replace one or more symbols by a different set of symbols.
Context free grammars were developed by Chomsky [1956]
![Page 15: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/15.jpg)
15
What is a context Free Grammar?
S aB
S ab
B Sb
S
ab aB
aSb
aabb aaBb
aaSbb
aaaBbbaaabbbThis generates the language consisting of sequences of the form anbn.
In a Context free grammar each production rule has one non-terminal on the left.
Example
![Page 16: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/16.jpg)
16
Features of a Context Free Grammar
A “start symbol” {S}
A set of “terminals” {a, b}
A set of “non-terminals” {S, B}
Production Rules: S aB
S ab
B Sb
Language: {anbn: n = 1, 2, …}
S aB
S ab
B Sb
![Page 17: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/17.jpg)
17
Sequence grammars (Bompadre and Orlin, 2005)
Terminals of the grammar: {1, 2, …, n}.
– one terminal for each “city.”
Non-terminals of the grammar: a set of symbols.
– May be thought of as subsets of cities
– Applying production rules to non-terminal A will eventually generate a string of cities
– e.g., A 5 - 3 - 9 - 2
– A 9 - 2 - 5 - 3
For each non-terminal A, there is a set of cities called Cities(A). e.g., Cities(A) = {2, 3, 5, 9}.
– A always generates strings on Cities(A)
![Page 18: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/18.jpg)
18
The pyramidal tour neighborhoodKlyaus [1976]
Let Cities(Tj) = { j, j+1, j+2, …, n}
The start non-terminal is T1
Rules
T1 1, T2
Tj j, Tj+1 for j = 2 to n-1
Tj Tj+1, j for j = 2 to n-1
Tn n
24 6 7 5 31 8
T21
1 2T3
21 3T4
231 4 T5
24 31 5T6
24 5 31 6 T7
24 6 5 31 7T8O(n) rules
24 6 7 5 31 8
![Page 19: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/19.jpg)
19
The semi-twisted neighborhood
Let cities(Tij) = { i, i+1, i+2, …, j}
The start non-terminal is T1n.
Rules
T1n 1, T2,n
Tij i, Ti+1,j for i < j
Tij Ti+1,j, i for i < j
Tij j, Ti,j-1 for i < j
Tij Ti,j-1, j for i < j
Tjj j
23 5 6 4 81 7
T2,81
1 2T3,8
21 8T3,7
281 3 T4,7
23 81 4T5,7
23 4 81 5 T6,7
23 5 4 81 6T7,7
23 5 6 4 81 7
If we consider items added from last to first, they are consecutive.O(n2) rules.
![Page 20: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/20.jpg)
20
The twisted neighborhood: Aurenheimer [1988]
Let cities(Tij) = { i, i+1, i+2, …, j}
The start non-terminal is S.
Rules
S T1n
Tik Tij, Tj+1,k for i j k
Tik Tj+1,k , Tij for i j k
Tjj j for all j
O(n3) rules.
T1,3T4,8
T4,8 T1,2 3
T1,2 3T4,5 T6,8
3T1,2T6,85 4
35 4 T6,8 2 1
T1,8
35 4 2 1T6,78
35 4 2 178 6
![Page 21: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/21.jpg)
21
Can every polynomially sized neighborhood be generated in polynomial time?
• A 1, 2, 3, 4, 5
• A 2, 1, 3, 4, 5
• A 3, 2, 1, 4, 5
• A 4, 2, 3, 1, 5
• A 5, 2, 3, 4, 1
• A 1, 3, 2, 4, 5
. . .
Yes!
Example: The two swap neighborhood.
Let cities(A) = {1, 2, 3, 4, 5}
There is a non-terminal called the start non-terminal. In this case it is city A
![Page 22: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/22.jpg)
22
What is the advantage of representing neighborhoods as sequence grammars?
We have a “meta dynamic program” for grammar-induced neighborhood for the TSP
If the grammar has K-rules, the generated DP runs in O(Kn3) time for finding the best neighbor
Often the bound can be improved.
DP Generator
Rules of the grammar
Input
DP for searching the neighborhood
Output
![Page 23: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/23.jpg)
23
More on the dynamic program
DP running time.
– O(Kn3) time for finding the best neighbor
– Time analysis can be improved in some cases
O(n2) time for pyramidal tour nbhd.
O(n3) time for semi-twisted nbhd.
O(n6) time for twisted nbhd.
– previous best was O(n7).
![Page 24: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/24.jpg)
24
Dynamic Programming Illustration
Some rules for pyramidal tour neighborhood
pj: Tj j, Tj+1
qj: Tj Tj+1, j for j = 2 to n-1
Let f(i, Tj, k) = min length, starts at i, ends at k, visits Tj
T4
4 T5
4T5
orf(4, T4, j) = mink {c4k+ f(k, T5, j)}
f(j, T4, 4) = mink {f(j, T5, k) + ck4}
![Page 25: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/25.jpg)
25
On Pyramidal Tours
Rules
O(n) rules
O(n2) states
O(n2) time to run the recursion.
What’s new?
The DP takes the rules as input and forms the DP automatically.
It separates the solver from the formulation
![Page 26: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/26.jpg)
26
Running times for the TSP
Neighborhood Running time
2-opt O(n2)
Pyramidal Tour O(n2)
Tree-based grammars with K rules O(Kn3)
Twisted Sequence O(n6) **
Semi-twisted Sequence O(n3)
Dynasearch Nbhds O(n2)
Balas-Simonetti O(k2kn)
Improves upon O(n7) algorithm by Deineko and Woeginger [2000]
![Page 27: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/27.jpg)
27
The Linear Ordering Problem
g(k) is the min cost ordering of items k to n
T4
4 T5
4T5
or
g(4) = g(5) + c45 + c46 + c47 + c48
Consider the pyramidal tour neighborhood.
g(4) = g(5) + c54 + c64 + c74 + c84
O(n2) time. O(n) time if data is preprocessed.
![Page 28: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/28.jpg)
28
Running times for the LOP
Neighborhood Running time
2-opt O(n2)
Pyramidal Tour O(n2)
Tree-based grammars with K rules O(Kn2)
Twisted Sequence O(n3)
Semi-twisted Sequence O(n2)
Dynasearch Nbhds O(n2)
Balas-Simonetti O(k2kn)
![Page 29: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/29.jpg)
29
Summary so far
Use context free grammars to create exponentially large neighborhoods
time to search the neighborhoods optimally is polynomial time in n and K for TSP and LOP
next: some natural questions arising from this research
![Page 30: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/30.jpg)
30
Do we get new neighborhoods?
Yes! e.g., we can create new grammars by taking unions or subsets
– The union of two sequence grammars induces a new neighborhood
– Any subset of a sequence grammar induces a new grammar
Open question: can one create interesting new neighborhoods?
![Page 31: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/31.jpg)
31
1 3 5 7 9
2 6 84
Can all polynomially searchable neighborhoods be efficiently represented as sequence grammars?
keep odd cities fixed; optimally insert even cities.
1 2 3 4 5 6 7 8 9
This nbhd requires an exponential number of rules if represented as a sequence grammar.
No! e.g., Sarvanov and Doroshko [1981] assignment neighborhood for the TSP..
![Page 32: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/32.jpg)
32
Given a context free grammar, can one efficiently recognize if it is a sequence grammar?
Suppose that G is a context free grammar.
e.g., T1 1, T2
Tj j, Tj+1 for j = 2 to n-1Tj Tj+1, j for j = 2 to n-1Tn n
1. To recognize G, use recursion to determine Cities(A) for all non-terminals A.
2. Then verify that Cities(A) is unique for each A, and that if A B, C, then cities(A) = cities(B) cities(C)…
Tn n implies that Cities(Tn) = {n}
Tn-1 Tn, n-1 implies that Cities(Tn-1} = {n-1, n}…
Yes!
![Page 33: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/33.jpg)
33
A natural extension is “context sensitive grammars”. Are these useful in VLSN Search?
CSGs permit rules such as : ABC ADdC
– This rule involving B is sensitive to its context
– WLOG, we can also permit rules of the form AB BA
The following grammar generates all tours
S A1, A2, …, An
Aj j for j = 1 to n
AiAj AjAi for all i, j
It has O(n2) rules, and the nbhd cannot be searched in polynomial time.
No!
![Page 34: 1 Sequence Grammars for Very Large Scale Neighborhood Search Agustin Bompadre James Orlin ISMP 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062805/5697c0131a28abf838cccb80/html5/thumbnails/34.jpg)
34
Summary and next directions
Sequence grammars for representing many exponential sized neighborhoods for sequence problems
Dynamic programming algorithms for searching these neighborhoods for the TSP, linear ordering problem, and several other sequence problems.
Next directions: grammars that are useful for VLSN Search for other combinatorial problems