Slide 1

Vittorio Maniezzo - University of Bologna - Transportation Logistics 1/65 TSP: an introduction Vittorio Maniezzo University of Bologna

Slide 2

Vittorio Maniezzo - University of Bologna - Transportation Logistics 2/65 Travelling Salesman Problem (TSP) n locations given (home of salesperson and n-1 customers). [d ij ] distance matrix given, contains distances (Km, time, ) between each pair of locations. The TSP asks for the shortest tour starting home, visiting each customer once and returning home Symmetric / asymmetric versions.

Slide 3

Vittorio Maniezzo - University of Bologna - Transportation Logistics 3/65 Introduction Which is the shortest tour? Difficult to say

Slide 4

Vittorio Maniezzo - University of Bologna - Transportation Logistics 4/65 TSP history The general form of the TSP appears to have been first studied by mathematicians starting in the 1930s by Karl Menger in Vienna. Mathematical problems related to the traveling salesman problem were treated in the 1800s by the Irish mathematician Sir William Rowan Hamilton. The picture shows Hamilton's Icosian Game that requires players to complete tours through the 20 points using only the specified connections. (http://www.math.princeton.e du/tsp/index.html)

Slide 5

Vittorio Maniezzo - University of Bologna - Transportation Logistics 5/65 Past practice

Slide 6

Vittorio Maniezzo - University of Bologna - Transportation Logistics 6/65 Current practice Actual tours of persons or vehicles (deliveries to customers) Given: Graph of streets, where some nodes represent customers. Find: shortest tour among customers. Precondition: Solve shortest path problems Distance matrix

Slide 7

Vittorio Maniezzo - University of Bologna - Transportation Logistics 7/65 Application: PCB Minimization of time for drilling holes in PCBs (printed circuit boards)

Slide 8

Vittorio Maniezzo - University of Bologna - Transportation Logistics 8/65 Application: setup costs Minimization of total setup costs (or setup times) when setup costs are sequence dependent All product types must be produced and the setup must return to initial configuration i \ j12345 1018172021 2180211618 3169801920 4171620017 518 19160

Slide 9

Vittorio Maniezzo - University of Bologna - Transportation Logistics 9/65 Application: paint shop Minimization of total setup costs (or setup times) when setup costs are sequence dependent All colors must be processed i \ jWhitePinkRedBlue White0577 Pink10025 Red15705 Blue151050

Slide 10

Vittorio Maniezzo - University of Bologna - Transportation Logistics 10/65 Solution Methods n nodes ( n-1 )! possible tours Some possible solution method: Complete enumeration IP: Branch and Bound (and Cut, and Price, ) Dynamic programming Approximation algorithms Heuristics Metaheuristics n 5 10 15 20 n!1203,628.8001,31*10^122,43*10^18

Slide 11

Vittorio Maniezzo - University of Bologna - Transportation Logistics 11/65 Standard Example Example with 5 customers (1 home, 4 customers) Zimmermann, W., O.R., Oldenbourg, 1990: e.g. cost of 1-2-3-4-5: 18+21+19+17+18 = 93 First: simplify by reducing costs 161918 5 17 2016174 2019 98163 181621 182 21201718 1 54321i \ j

Slide 12

Vittorio Maniezzo - University of Bologna - Transportation Logistics 12/65 Lower bound Subtract minimum from each row & column Reduction constant = 17+16+16+16+16 + 1 = 82 all tours have costs which are 82 greater, e.g. cost of 1-2-3-4-5: 1+5+3+0+2 = 11 [= 93 - 82] Optimal tour does not change (see [Toth, Vigo, 02]). 03225 0 4014 33 8203 105 22 3301 1 54321i \ j

Slide 13

Vittorio Maniezzo - University of Bologna - Transportation Logistics 13/65 Complete Enumeration 5 customers 4! = 2 3 4 = 24 tours

Slide 14

Vittorio Maniezzo - University of Bologna - Transportation Logistics 14/65 A wider scope A combinatorial optimization problem is defined over a set C = {c 1, , c n } of basic components. A solution of the problem is a subset S C ; F 2 C is the subset of feasible solutions, (a solution S is feasible iff S F). z: 2 C is the cost function, the objective is to find a minimum cost feasible solution S , i.e., to find S F such that z(S) z(S), S F. Failing this, the algorithm anyway returns the best feasible solution found, S * F.

Slide 15

Vittorio Maniezzo - University of Bologna - Transportation Logistics 15/65 TSP as a CO example TSP is defined over a weighted undirected complete graph G=(V,E,D), where V is the set of vertices, E is the set of edges and D is the set of edge weights. The component set corresponds to E (C=E), F corresponds to the set of Hamiltonian cycles in G z(S) is the sum of the weights associated to the edges belonging to the solution S.

Slide 16

Vittorio Maniezzo - University of Bologna - Transportation Logistics 16/65 LP formulation is a binary LP (IP) which is almost identical to the assignment problem: plus conditions to avoid short cycles LP-Formulation

Slide 17

Vittorio Maniezzo - University of Bologna - Transportation Logistics 17/65 Optimal solution of LP without conditions to avoid short cycles = solution of assignment problem short cycles 1-3-1 and 2-5-4-2 conditions to avoid short cycles are needed Example for short cycles 12345

Slide 18

Vittorio Maniezzo - University of Bologna - Transportation Logistics 18/65 Conditions to avoid short cycles Several formulations in the literature e.g. Dantzig-Fulkerson-Johnson exponentially many!

Slide 19

Vittorio Maniezzo - University of Bologna - Transportation Logistics 19/65 Example: avoid short cycles Above example with n = 5 cities: must consider subsets Q: {1, 2}; {1, 3}; {1, 4}; {1, 5}; {2, 3}; {2, 4}; {2, 4}; {3, 4}; {3, 5}; {4, 5} constraint for subset Q = {1, 2} and V-Q = {3, 4, 5} x 13 + x 14 + x 15 + x 23 + x 24 + x 25 1 n = 6 subsets (constraints) exponentially many!

Slide 20

Vittorio Maniezzo - University of Bologna - Transportation Logistics 20/65 Heuristics for the TSP Vittorio Maniezzo University of Bologna

Slide 21

Vittorio Maniezzo - University of Bologna - Transportation Logistics 21/65 Computational issues The size of the instances met in real world applications rules out the possibility of solving them to optimality in an acceptable time. Nevertheless, these instances must be solved. Thus the need to look for suboptimal solutions, provided that they are of acceptable quality and that they can be found in acceptable time.

Slide 22

Vittorio Maniezzo - University of Bologna - Transportation Logistics 22/65 Heuristic algorithms How to cope with NP-completeness: small instances; polynomial special cases; approximation algorithms guarantee to find a solution within a known gap from optimality; probabilistic algorithms guarantee that for instances big enough, the probability of getting a bad solution is very small; heuristic algorithms: no guarantee, but historically, on the average, these algorithms have the best quality/time trade off for the problem of interest.

Slide 23

Vittorio Maniezzo - University of Bologna - Transportation Logistics 23/65 heuristics: focus on solution structure Simplest heuristics exploit structural properties of feasible solutions in order to quickly come to a good one. They belong to one of two main classes: constructive heuristics or local search heuristics.

Slide 24

Vittorio Maniezzo - University of Bologna - Transportation Logistics 24/65 Constructive heuristics 1. Sort the components in C by increasing costs. 2. Set S*= and i=1. 3. Repeat If (S* c i is a partial feasible solution) then S* = S* c i. i=i+1. Until S* F.

Slide 25

Vittorio Maniezzo - University of Bologna - Transportation Logistics 25/65 Constructive heuristics A constructive approach can yield optimal solutions for certain problems, eg. the MST. In other cases it could be unable to construct a feasible solution. TSP: order all edges by increasing edge cost, take the least cost one and add to it increasing cost edges, provided they do not close subtours, until a Hamiltonian circuit is completed. More involved constructive strategies give rise to well- known TSP heuristics, like the Farthest Insertion, the Nearest Neighbor, the Best Insertion or the Sweep algorithms.

Slide 26

Vittorio Maniezzo - University of Bologna - Transportation Logistics 26/65 Nearest Neighbor Heuristic 0. Start from any city; e.g. the first one 1.Find nearest neighbor (to the last city) not already visited 2. repeat 1 until all cities are visited. Then connect last city with starting point Ties can be broken arbitrarily. Starting from different starting points gives different solutions! Some will be bad some will be good. GREEDY the last connections tend to be long and improvement heuristics should be used afterwards

Slide 27

Vittorio Maniezzo - University of Bologna - Transportation Logistics 27/65 Best Insertion Heuristic 0.Select 2 cities A, B and start with short cycle A-B-A 1.Insert next city (not yet inserted) in the best position in the short cycle 2.Repeat 1. until all cities are visited Using different starting cycles and different choices of the next city different solutions are obtained Often when symmetric: starting cycle = 2 cities with maximum distance next city = maximizes min distance to inserted cities

Slide 28

Vittorio Maniezzo - University of Bologna - Transportation Logistics 28/65 An approximation algorithm In some cases, it is possible to construct solutions which are guaranteed to be not too bad. Approximation algorithms provide a guarantee on worst possible value for the ratio (z h z*)/z*. TSP hypothesis: it is always cheaper to go straight from a node u to another node v instead of traversing an intermediate node w (triangular inequality).

Slide 29

Vittorio Maniezzo - University of Bologna - Transportation Logistics 29/65 Approx-TSP An approx. Algorithm for the TSP with triangular inequality is: Approx-TSP-Tour(G,w) 1select a root vertex r V 2construct a MST T for G rooted in r 3let L be the list of vertices visited in a preorder tree walk of T 4return the hamiltonian cycle H which visits all vertices in the order L

Slide 30

Vittorio Maniezzo - University of Bologna - Transportation Logistics 30/65 Approx-TSP: example a b c h d e fg

Slide 31

Vittorio Maniezzo - University of Bologna - Transportation Logistics 31/65 Approx-TSP: example a b c h d e fg

Slide 32

Vittorio Maniezzo - University of Bologna - Transportation Logistics 32/65 Approx-TSP: example a b c h d e fg

Slide 33

Vittorio Maniezzo - University of Bologna - Transportation Logistics 33/65 Approx-TSP: example a b c h d e fg

Slide 34

Vittorio Maniezzo - University of Bologna - Transportation Logistics 34/65 Approx-TSP: performance Theorem Approx-TSP-Tour is an approximation algorithm with a ratio bound of 2 for the traveling salesman problem with triangle inequality. Proof Because the optimal tour solution must have one more edge than the MST of T, so c(MST) c(T*). c(Tour) = 2c(MST) Because of the assumption of triangle inequality, c(H) c(Tour). Therefore, c(H) 2c(T*). Since the input is a complete graph, so the implementation of Prims algorithm runs in O(V 2 ). Therefore the total time complexity of Approx-TSP-Tour also is O(V 2 ).

Slide 35

Vittorio Maniezzo - University of Bologna - Transportation Logistics 35/65 Local Search: neighborhoods The neighborhood of a solution S, N(S), is a subset of 2 C defined by a neighborhood function N : 2 C 2 2 c. Often only feasible solutions considered, thus neighborhood functions N : F 2 F. The specific function used has a deep impact on the algorithm performance and its choice is left to the algorithm designer.

Slide 36

Vittorio Maniezzo - University of Bologna - Transportation Logistics 36/65 Local search 1. Generate an initial feasible solution S. 2.Find S' N(S), such that z(S')=min z(S^), S^ N(S). 3.If z(S') < z(S) then S=S' go to step 2. 4. S* = S. The update of a solution in step 3 is called a move from S to S'. It could be made to the first improving solution found.

Slide 37

Vittorio Maniezzo - University of Bologna - Transportation Logistics 37/65 Local search There are problems for which a local search approach guarantees to find an optimal solution (ex. the simplex algorithm). For TSP, two LS are 2-opt and 3-opt, which take a solution (a list of n vertices) and exhaustively swap the elements of all pairs or triplets of vertices in it. More sophisticated neighborhoods give rise to more effective heuristics, among which Lin and Kernighan [LK73].

Slide 38

Vittorio Maniezzo - University of Bologna - Transportation Logistics 38/65 Local search heuristics 2 opt (arcs): try all inversions of some subsequence if improvement start again from beginning 3 opt (arcs): try all shifts of some subsequence to different positions if improvement start again from beginning 2-opt (nodes): swap every pair of nodes in the permutation representing the solution. if improvement start again from beginning 3-opt (nodes): swap every triplet of nodes in the permutation representing the solution. if improvement start again from beginning

Slide 39

Vittorio Maniezzo - University of Bologna - Transportation Logistics 39/65 Metaheuristcs for the TSP (some of them) Vittorio Maniezzo University of Bologna, Italy

Slide 40

Vittorio Maniezzo - University of Bologna - Transportation Logistics 40/65 Metaheuristics: focus on heuristic guidance Simple heuristics can perform very well, but can get trapped in poor quality solutions (local optima). New approaches have been presented starting from the mid '70ies. They are algorithms which manage the trade-off between search diversification, when search is going on in bad regions of the search space, and intensification, aimed at finding the best solutions within the region being analyzed. These algorithms have been named metaheuristics.

Slide 41

Vittorio Maniezzo - University of Bologna - Transportation Logistics 41/65 Metaheuristics Metaheuristics include: Simulated Annealing Tabu Search GRASP Genetic Algorithms Variable Neighborhood Search ACO Particle Swarm Optimization (PSO)

Slide 42

Vittorio Maniezzo - University of Bologna - Transportation Logistics 42/65 Simulated Annealing Simulated Annealing (SA) [AK89] modifies local search in order to accept, in probability, worsening moves. 1.Generate an initial feasible solution S, initialize S* = S and temperature parameter T. 2.Generate S N(S). 3.If z(S') z(S)) S* = S else accept to set S=S' with probability p = e -(z(S')-z(S))/kT. 4.If ( annealing condition ) decrease T. 5.If not( end condition ) go to step 2.

Slide 43

Vittorio Maniezzo - University of Bologna - Transportation Logistics 43/65 SA example: ulysses 16 Simulated annealing trace

Slide 44

Vittorio Maniezzo - University of Bologna - Transportation Logistics 44/65 Tabu search Tabu Search (TS) [GL97] escapes from local minima by moving onto the best solution of the neighborhood at each iteration, even though it is worse than the current one. A memory structure called tabu list, TL, forbids to return to already explored solutions. 1.Generate an initial feasible solution S, set S* = S and initialize TL = . 2.Find S' N(S), such that z(S')=min {z(S^), S^ N(S), S^ TL}. 3. S=S', TL=TL {S}, if (z(S*) > z(S)) set S* = S. 4.If not( end condition ) go to step 2.

Slide 45

Vittorio Maniezzo - University of Bologna - Transportation Logistics 45/65 TS example: ulysses 16 Tabu Search trace

Slide 46

Vittorio Maniezzo - University of Bologna - Transportation Logistics 46/65 GRASP GRASP (Greedy Randomized Adaptive Search Procedure) [FR95] restarts search from another promising region of the search space as soon as a local optimum is reached. GRASP consists in a multistart approach with a suggestion on how to construct the initial solutions. 1.Build a solution S (=S* in the first iteration) by a constructive greedy randomized procedure based on candidate lists. 2.Apply a local search procedure starting from S and producing S'. 3.If z(S')