Tabu Search - Exampleszebpe83/heuristic/lectures/TS_applications.pdf · Heuristic Algorithms for...
Transcript of Tabu Search - Exampleszebpe83/heuristic/lectures/TS_applications.pdf · Heuristic Algorithms for...
Heuristic Algorithms for Combinatorial Optimization Problems 1
Petru Eles, 2010
Tabu Search - Examples
Petru Eles
Department of Computer and Information Science (IDA)Linköpings universitet
http://www.ida.liu.se/~petel/
2
Petru Eles, 2010
u Search Examples
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Tab
Hardware/Software Partitioning
Travelling Salesman
3
Petru Eles, 2010
ftware Partitioning
em:
f there exists a direct
esponding processes
edge:
ability for hardware
rocess.
ommunication between pro-
ardware and software
Inp
■
Ou
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TS Examples: Hardware/Sout:
The process graph: an abstract model of a syst
❚ Each node corresponds to a process.
❚ An edge connects two nodes if and only i
communication channel between the corr
❚ Weights are associated to each node and
- Node weights reflect the degree of suit
implementation of the corresponding p
- Edge weights measure the amount of c
cesses
tput:
Two subgraphs containing nodes assigned to h
respectively.
4
Petru Eles, 2010
ftware Partitioning
P11
P8
P9
P12
P10
Software
P14
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TS Examples: Hardware/So
P6
P4P1
P7
P5
P3
P2
Hardware
P13
5
Petru Eles, 2010
ftware Partitioning
measure of the computation
of operations in process i;
m inside process i;
re implementation;
We
W 2
KiC
KiU
KiP
KiSO
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TS Examples: Hardware/Soight assigned to nodes:
iN = M CL Ki
CL M U KiU M P Ki
P M SO KiSO×–×+×+×
L is equal to the RCL of process i, and thus is aload of that process;
= Nr_o pi
Nr_kind_o pi------------------------------- ; Ki
U is a measure of the uniformity
= Nr_o pi
L_pathi------------------ ; Ki
P is a measure of potential parallelis
=wop j
op j SPi∈∑Nr_o pi
----------------------------- ; KiSO captures suitability for softwa
6
Petru Eles, 2010
ing: Cost Function
2iN
----------
ts
Th
Re
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partition
e cost function:
Q1 W 1ijE
ij( ) cut∈∑× Q2
W 2ijE
ij( )∃∑
W 1iN
----------------------i( ) Hw∈∑
N H---------------------------------------× Q3
W 2iN
i( ) Hw∈∑
N H------------------------------
Wi( ) Sw∈∑
N S-------------------–
×–+
strictions:
H_costii H∈∑ Max
H≤
S_costii H∈∑ Max
S≤
W iN Lim1≥ i Hw∈⇒
W iN Lim1≤ i Sw∈⇒
amount of Hw-Sw comm.
Ratio com/cmp of Hw part.
Difference of average weigh
7
Petru Eles, 2010
ves&Neighborhood
lution xnow is the set of
g a node from its current
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partitioning: Mo
Moves:
❚ The neighborhood N(xnow) of a certain so
solutions which can be obtained by movin
partition to the other one.
Hw/Sw Partitioning: TS Algorithm
• Construct initial configuration xnow= (Hw0, Sw0)start:for each solution xk ∈ N(xnow) do
• Compute change of cost function ∆Ck = C(xk) - C(xnow) end forfor each ∆Ck < 0, in increasing order of ∆Ck do
if not tabu(xk) or tabu_aspirated(xk) thenxnow = xkgoto accept
end ifend forfor each solution xk ∈ N(xnow) do
• Compute ∆C′k = ∆Ck + penalty(xk)end forfor each ∆C′k in increasing order of ∆C′k do
if not tabu(xk) thenxnow = xkgoto accept
end ifend for• Generate xnow by performing the least tabu moveaccept:if iterations since previous best solution < Nr_w_b then
goto startend ifif restarts < Nr_r then
• Generate initial configuration xnow considering frequenciesgoto start
end ifreturn solution corresponding to the minimum cost function
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
8
Petru Eles, 2010
9
Petru Eles, 2010
ning: TS Algorithm
ssible) is tried.
d the best possible non-tabu
u state is executed.
■
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partitio
First attempt:
❚ An improving non-tabu move (the best po
Second attempt:
❚ Frequency based penalties are applied an
move is tried;
Third attempt:
❚ The move which is closest to leave the tab
10
Petru Eles, 2010
ning: The Tabu-List
u-list. Their reverse is tabu.
lem and of the
with large tabu tenures.
tabu restriction: stronger
variable:
olution
iven interval.
■
■
■
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partitio
The last τ moves performed are stored in the tab
τ = tabu tenure (length of the tabu list)
The tabu tenure depends on the size of the prob
neighborhood: large problem sizes are coupled
The tabu tenure depends on the strength of the
restrictions are coupled with smaller sizes.
Tabu tenures are tuned experimentally or can be
❚ too small tenures ⇒ cycling
❚ too large tenures ⇒ deterioration of the s
❚ Recommended values: 7 ÷ 25
Tabu tenures can be selected randomly from a g
11
Petru Eles, 2010
ng: Tabu Aspiration
ion produced is better than
■Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partitioni
The tabu status of a move is ignored if the solut
the best obtained so far.
12
Petru Eles, 2010
Long Term Memory
ns each node has spent in
for diversification:
on, which favors the transfer eir current partition.
y of occurrences of the node rtain threshold.
e started from an initial e encountered previously.
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partitioning:
Long term memory stores the number of iteratio
the hardware partition. This information is used
1. Application of a penalty to the cost functiof nodes that have spent a long time in th
2. A move is forbidden (tabu) if the frequencin its current partition is smaller than a ce
3. If the system is frozen a new search can bconfiguration which is different from thos
13
Petru Eles, 2010
artitioning: Penalty
k spent in the Hw partition.
∈ Hw
∈ Sw
Coefficients experimentally set to:
❚ CH=0.4
❚ CS=0.15.
Th
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Pe penalized cost function:
∆C'k ∆Ck
∆Cii
∑Nr_of_nodes------------------------------- pen k( )×+=
where
pen k( )
CH
Node_in_Hwk
N iter----------------------------------×–
CS 1Node_in_Hwk
N iter----------------------------------–
×–
=
❚ Node_in_Hwk : number of iterations node
❚ Niter : total number of iterations;
❚ Nr_of_nodes : total number of nodes;
if nodek
if nodek
14
Petru Eles, 2010
for Node Movement
ccurrences of the node in its
:
H
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
w/Sw Partitioning: Thresholds
A move is forbidden (tabu) if the frequency of o
current partition is smaller than the threshold:
Node_in_Hwk
N iter---------------------------------- T H>
1Node_in_Hwk
N iter----------------------------------–
T S>
The thresholds have been experimentally set to
❚ TH=0.2
❚ TS=0.4.
if nodek ∈ Hw
if nodek ∈ Sw
15
Petru Eles, 2010
perimental Results
arch partitioning
CPU time (s)(time with SA)
0.008(0.23)
0.04(1.27)
0.19(2.33)
30.5(769)
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partitioning: Some Ex
Parameters and CPU times for Tabu Se(SPARCstation 10)
numbers of nodes
τ Nr_w_b Nr_r
20 7 30 0
40 7 50 0
100 7 50 0
400 18 850 2
Variation of cost function for TS partitioning with 400 nodes
1.95e+06
2e+06
2.05e+06
2.1e+06
2.15e+06
2.2e+06
2.25e+06
2.3e+06
2.35e+06
2.4e+062.45e+06
0 500 1000 1500 2000 2500 3000
1.99e+06
1.992e+06
1.994e+06
1.996e+06
1.998e+06
2e+06
2.002e+06
2.004e+06
0 500 1000 1500 2500 3000
1.99e+06
1.992e+06
1.994e+06
1.996e+06
1.998e+06
2e+06
1920 1925 1930 1935 1940 1945 1950
2000Number of iterations
Cos
t fu
ncti
on v
alue
Number of iterations
Number of iterations
Cos
t fu
ncti
on v
alue
Cos
t fu
ncti
on v
alue
optimum at iteration 1941
optimum at iteration 1941
optimum at iteration 1941
Embedded Systems for Real-Time Applications: Analysis and Synthesis
Tabu Search
16
September 2010
17
Petru Eles, 2010
perimental Results
h 100 nodes
100 120 140
timum at iteration 76
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partitioning: Some Ex
Variation of cost function for TS partitioning wit
38000
40000
42000
44000
46000
48000
50000
52000
54000
56000
0 20 40 60 80
Number of iterations
Cos
t fu
ncti
on v
alue
op
18
Petru Eles, 2010
perimental Results
1000400
SA
TS
KL
ithmic)
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Hw/Sw Partitioning: Some Ex
Partitioning times with SA, TS, and KL
0.001
0.01
0.1
1
10
100
1000
10 10020 40
Number of graph nodes (logar
Exe
cuti
on t
ime
(s)
(log
arit
hmic
)
19
Petru Eles, 2010
Salesman Problem
then return to the initial city; find the tour with minimum
graph where the nodes nt the distance between ered in traversing all cities.
A seacdis
In g
Finrepciti
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TS Examples: Travelling
alesman has to travel to a number of cities and h city has to be visited once. The objective is totance.
raph theoretical formulation:
d the shortest Hamiltonian circuit in a completeresent cities. The weights on the edges represees. The cost of the tour is the total distance cov
20
Petru Eles, 2010
TSP: Cost Function
ny tour can be represented
j.
are adjacent cities in the
that minimizes:
■
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
If the problem consists of n cities ci, i = 1, .., n, a
as a permutation of numbers 1 to n.
d(ci,cj) = d(cj,ci) is the distance between ci and c
Given a permutation π of the n cities, vi and vi+1
permutation. The permutation π has to be found
d vi vi 1+( , ) d vn v1( , )+
i 1=
n 1–
∑
The size of the solution space is (n-1)!/2
21
Petru Eles, 2010
ves&Neighborhood
se tours obtained by
nt set of k links, in a way that
the tour after two links have
imated in order to select the
y a sequence of such
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Mo
k-neighborhood of a given tour is defined by tho
removing k links and replacing them by a differe
maintains feasibility.
For k = 2, there is only one way of reconnecting
been removed.
❚ Size of the neighborhood: n(n - 1) / 2
❚ As opposed to SA, all alternatives are est
appropriate move.
❚ Any tour can be obtained from any other b
moves.
22
Petru Eles, 2010
ves&Neighborhood
02
4
67
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Mo
5
3
1
Permutation:[0 2 4 6 7 5 3 1]
23
Petru Eles, 2010
ves&Neighborhood
02
4
67
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Mo
5
3
1
links (v3,v1), (v4,v6)are removed
24
Petru Eles, 2010
ves&Neighborhood
02
4
67
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Mo
5
3
1
Permutation:[0 2 4 3 5 7 6 1]
25
Petru Eles, 2010
ves&Neighborhood
in the permutation):
by the pair i, j so that i < j.
ed incrementally:
■
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Mo
vi is the city in position i of the tour (ith position
remove (vi, vi+1) and (vj, vj+1)
connect vi to vj and vi+1 to vj+1
All 2-neighbors of a certain solution are defined
The change of the cost function can be comput
∆C = d(vi,vj) + d(vi+1,vj+1) - d(vi,vi+1) - d(vj,vj+1)
26
Petru Eles, 2010
Tabu-Classification
everal pairs of cities have
e tour before the swap.
utes)?
ecoming tabu?
■
Qu
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Move Attributes for
We have performed a move as result of which s
swapped their position in the tour:
❚ Cities x and y are such a pair;
position(x) and position(y): positions in th
position(x) < position(y).
estions:
1. What information do we store (move attrib
2. Using this information, which moves are b
27
Petru Eles, 2010
Tabu-Classification
ur with city x and city y
pectively.
ur with city x occupying
.
n(x).
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Move Attributes for
1. Vector(x, y, position(x), position(y))
❚ To prevent any swap from resulting in a to
occupying position(x) and position(y) res
2. Vector(x, y, position(x), position(y))
❚ To prevent any swap from resulting in a to
position(x) or city y occupying position(y)
3. Vector(x, position(x))
❚ To prevent city x from returning to positio
28
Petru Eles, 2010
Tabu-Classification
n(y).
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Move Attributes for
4. City x
❚ To prevent city x from moving LEFT.
5. City x
❚ To prevent city x from moving.
6. Vector(y, position(y))
❚ To prevent city y from returning to positio
29
Petru Eles, 2010
Tabu-Classification
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Move Attributes for
7. City y
❚ To prevent city y from moving RIGHT.
8. City y
❚ To prevent city y from moving.
9. City x and y
❚ To prevent both cities from moving.
30
Petru Eles, 2010
Tabu-Classification
ts the smallest amount of
ts a large amount of moves).
iveness.
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Move Attributes for
❚ Condition 1 is the least restrictive (preven
moves).
❚ Condition 9 is the most restrictive (preven
❚ Conditions 3, 4, 5 have increasing restrict
31
Petru Eles, 2010
TSP: Tabu List Size
n be relatively small.
to be larger.
possibly good vicinity.
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
τ has to be experimentally tuned.
❚ For highly restrictive tabu conditions τ ca
❚ For less restrictive tabu conditions τ has
❚ τ too small ⇒ cycling
❚ τ too large ⇒ exploration driven away from
32
Petru Eles, 2010
TSP: Tabu List Size
ies/3
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
Tabu list size:
❚ for conditions 4 and 7: nr_cities/4 ÷ nr_cit
❚ for conditions 5, 8, 9: nr_cities/5
❚ for conditions 1, 2, 3, 6: ≈nr_cities
Best results for conditions 4 and 7
33
Petru Eles, 2010
Long Term Memory
es an edge is visited.
ng tour is generated
frequently.
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP:
Long term memory maintains the number of tim
After a certain number of iterations a new starti
consisting of edges that have been visited less
34
Petru Eles, 2010
perimental Results
■
■
Heuristic Algorithms for Combinatorial Optimization Problems
Tabu Search
TSP: Some Ex
100 city problem; optimal solution: C = 21247.
❚ Best solution C = 21352 (21255 for SA)
❚ Time = 210 s (Sun4/75) - (1340 s for SA)
❚ Standard deviation over 10 trials: 30.3;
(randomly generated starting tour!)
❚ Average cost: 21372
57 city problem; optimal solution: C = 12955
❚ Optimal solution in 109 s (673 s for SA).
(Sequent Balance 8000)