GRASP: Greedy Randomized Adaptive Search Procedures

92
slide 1 GRASP GRASP: Greedy Randomized Adaptive Search Procedures A metaheuristic for combinatorial optimization Maurício G. C. Resende Algorithms & Optimization Research Dept. AT&T Labs Research Florham Park, New Jersey [email protected] http://www.research.att.com/~mgcr May 2000 ECCO’2000 – Capri, Italy

Transcript of GRASP: Greedy Randomized Adaptive Search Procedures

Page 1: GRASP: Greedy Randomized Adaptive Search Procedures

slide 1 GRASP

GRASP: Greedy Randomized Adaptive Search Procedures

A metaheuristic for combinatorial optimization

Maurício G. C. ResendeAlgorithms & Optimization Research Dept.

AT&T Labs ResearchFlorham Park, New Jersey

[email protected]://www.research.att.com/~mgcr

May 2000

ECCO’2000 – Capri, Italy

Page 2: GRASP: Greedy Randomized Adaptive Search Procedures

slide 2 GRASP

Outline

• Introductionl combinatorial optimization & local searchl random multi-start local searchl greedy and semi-greedy algorithms

• A basic (standard) GRASP• Enhancements to the basic GRASP

l enhancements to local searchl asymptotic behaviorl automatic choice of RCL parameter αl use of long-term memoryl GRASP in hybrid metaheuristicsl parallel GRASP

• Survey of applications in literaturel operations research & computer sciencel industrial

Page 3: GRASP: Greedy Randomized Adaptive Search Procedures

slide 3 GRASP

Combinatorial Optimization

• Given:l discrete set of solutions Xl objective function

• Objective:l find

f x x X R( ): ∈ →

x X f x f y y X∈ ≤ ∀ ∈: ( ) ( ),

Page 4: GRASP: Greedy Randomized Adaptive Search Procedures

slide 4 GRASP

Origins

• Probabilistic algorithm (GRASP) for difficult set covering problems [Feo& R., 1989]

• GRASP was related to previous work, e.g.:l random multistart local search [e.g.

Lin & Kernighan, 1973]l semi-greedy heuristics [e.g. Hart &

Shogan, 1987]

Page 5: GRASP: Greedy Randomized Adaptive Search Procedures

slide 5 GRASP

Local Search

• To define local search, one needs to specify a local neighborhood structure.

• Given a solution x , the elements of the neighborhood N (x ) of x are those solutions y that can be obtained by applying an elementary modification (often called a move) to x.

Page 6: GRASP: Greedy Randomized Adaptive Search Procedures

slide 6 GRASP

Local Search Neighborhoods

• Consider x = (0,1,0) and the 1-flip neighborhood of a 0/1 array.

x = (0,1,0)

(1,1,0)

(0,0,0)(0,1,1)

N (x )

Page 7: GRASP: Greedy Randomized Adaptive Search Procedures

slide 7 GRASP

Local Search Neighborhoods

• Consider x = (2,1,3) and the 2-swap neighborhood of a permutation array.

x = (2,1,3)

(3,1,2)

(2,3,1)(1,2,3)

N (x )

Page 8: GRASP: Greedy Randomized Adaptive Search Procedures

slide 8 GRASP

Local Search

• Local search: Given an initial solution x0, a neighborhood N (x ), and function f (x ) to be minimized:

x = x0 ;while ( ){

x = y ;}

At the end, x is a local minimum of f (x ) .Time complexity of local search can be exponential.

∃ ∈ <y N x f y f x( )| ( ) ( )

check for bettersolution in neighborhoodof x

move to bettersolution y

Page 9: GRASP: Greedy Randomized Adaptive Search Procedures

slide 9 GRASP

Local Search(ideal situation)

f (0,0,0) = 3

f (1,1,1) = 3

f (0,1,0) = 4

f (0,0,1) = 0

f (1,0,0) = 5

f (0,1,1) = 1

f (1,1,0) = 6

f (1,0,1) = 2

With any starting solution Local Search finds the global optimum.

globalminimum

Page 10: GRASP: Greedy Randomized Adaptive Search Procedures

slide 10 GRASP

Local Search(more realistic situation)

f (1,1,1) = 2

f (0,1,0) = 2

f (0,0,1) = 0

f (1,0,0) = 1

f (0,1,1) = 3

f (1,1,0) = 6

f (1,0,1) = 3

f (0,0,0) = 3globalminimum

localminima

localminimum

But some starting solutions lead Local Search to a localminimum.

Page 11: GRASP: Greedy Randomized Adaptive Search Procedures

slide 11 GRASP

Local Search

• Effectiveness of local search depends on several factors:l neighborhood structurel function to be minimizedl starting solution

usually pre-determined

usually easier tocontrol

Page 12: GRASP: Greedy Randomized Adaptive Search Procedures

slide 12 GRASP

Local search with randomstarting solutions

Generate solution at random

In basin of attractionof global optimum?No

Yes

Local search leads to global optimum.

LOOP

By repeating LOOPover and over, w.p. 1outcome is Yes

Page 13: GRASP: Greedy Randomized Adaptive Search Procedures

slide 13 GRASP

Random Multistart Local Search

best_obj = HUGE; /* minimization */repeat many times{

x = random_construction( );x = local_search(x);if ( obj_function(x) < best_obj ){

x* = x;best_obj = obj_function(x);

}}

Page 14: GRASP: Greedy Randomized Adaptive Search Procedures

slide 14 GRASP

The greedy algorithm

• To define a semi-greedy heuristic, we must first consider the greedy algorithm.

• Greedy algorithm: constructs a solution, one element at a time:l Defines candidate elements.l Applies a greedy function to each

candidate element.l Ranks elements according to greedy

function value.l Add best ranked element to solution.re

peat

unt

il d

one

Page 15: GRASP: Greedy Randomized Adaptive Search Procedures

slide 15 GRASP

The greedy algorithmAn example

• Minimum spanning tree: Given graph G = (V, E), with weighted edges, find least weighted spanning tree.l greedy algorithm builds solution, one

element (edge) at a timel greedy function: edge weight of

edges that do not form cycle when added to current solution

Page 16: GRASP: Greedy Randomized Adaptive Search Procedures

slide 16 GRASP

The greedy algorithmAn example

22

13

22

4 2 5 4

3

4

5

22

13

22

4 2 5 4

3

4

5

22

13

22

4 2 5 4

3

4

5

Edges of weight 1 & 2 Edges of weight 3 & 4

Global minimum

Page 17: GRASP: Greedy Randomized Adaptive Search Procedures

slide 17 GRASP

The greedy algorithmAnother example

• Maximum clique: Given graph G = (V, E), find largest subgraph of G such that all vertices are mutually adjacent.l greedy algorithm builds solution, one

element (vertex) at a timel greedy function: degree of

unselected vertex that is adjacent to all selected vertices with respect to all unselected vertices adjacent to all selected vertices.

Page 18: GRASP: Greedy Randomized Adaptive Search Procedures

slide 18 GRASP

The greedy algorithmAnother example

432

0

2

3 3 3

2

1 2 10

0

a)

b) c)

global maximum

Page 19: GRASP: Greedy Randomized Adaptive Search Procedures

slide 19 GRASP

The greedy algorithmAnother example

2

4

4

3

33a)

0 0

0

0

b) c)sub-optimalclique

Page 20: GRASP: Greedy Randomized Adaptive Search Procedures

slide 20 GRASP

Semi-greedy heuristic

• A semi-greedy heuristic tries to get around convergence to non-global local minima.

• repeat until solution is constructedl For each candidate element

l apply a greedy function to element

l Rank all elements according to their greedy function values

l Place well-ranked elements in a restricted candidate list (RCL)

l Select an element from the RCL at random & add it to the solution

Page 21: GRASP: Greedy Randomized Adaptive Search Procedures

slide 21 GRASP

Semi-greedy heuristic

min max

RCL

greedy functionvalue

Candidate elements are ranked according togreedy function value.

RCL is a set of well-ranked candidate elements.

Page 22: GRASP: Greedy Randomized Adaptive Search Procedures

slide 22 GRASP

Semi-greedy heuristic

• Hart & Shogan (1987) propose two mechanisms for building the RCL:l Cardinality based: place k best candidates

in RCLl Value based: place all candidates having

greedy values better than α*best_value in RCL, where α ε [0,1].

• Feo & R. (1989) proposed semi-greedy construction, independently, as a basic component of GRASP.

Page 23: GRASP: Greedy Randomized Adaptive Search Procedures

slide 23 GRASP

Hart-Shogan Algorithm(maximization)

best_obj = 0; repeat many times{

x = semi-greedy_construction( );if ( obj_function(x) > best_obj ){

x* = x;best_obj = obj_function(x);

}}

Page 24: GRASP: Greedy Randomized Adaptive Search Procedures

slide 24 GRASP

A Basic GRASP

• GRASP tries to capture good features of greedy & random constructions.

• iterativelyl samples solution space using a greedy

probabilistic bias to construct a feasible solution (semi-greedy construction)

l applies local search to attempt to improve upon the constructed solution

• keeps track of the best solution found

Page 25: GRASP: Greedy Randomized Adaptive Search Procedures

slide 25 GRASP

GRASP (for maximization)

best_obj = 0;repeat many times{

x = semi-greedy_construction( );x = local_search(x);if ( obj_function(x) > best_obj ){

x* = x;best_obj = obj_function(x);

}}

bias towards greedinessgood diverse solutions

Page 26: GRASP: Greedy Randomized Adaptive Search Procedures

slide 26 GRASP

minmax α - percentage based RCL

min max

min + α (max - min)

0 ≤ ≤ 1α

RCL

greedy functionvalue

α = 0: random assignmentα = 1: greedy assignment

Page 27: GRASP: Greedy Randomized Adaptive Search Procedures

slide 27 GRASP

Random vs greedy construction

• random constructionl high variancel low qualityl almost always

sub-optimal

• greedy constructionl low (no)

variancel good qualityl usually sub-

optimal

α = 0 α = 1

Page 28: GRASP: Greedy Randomized Adaptive Search Procedures

slide 28 GRASP

Random vs greedy construction

0

200

400

600

800

1000

405000 410000 415000 420000 425000 430000 435000

0

200

400

600

800

1000

405000 410000 415000 420000 425000 430000 435000

random

greedy

Page 29: GRASP: Greedy Randomized Adaptive Search Procedures

slide 29 GRASP

0

2 00

4 00

6 00

8 00

10 00

40 000 0 4 0500 0 41 000 0 4 1500 0 42 000 0 4 2500 0 43 000 0 4 3500 0 44 000 0

a lp h a = 0.0

0

200

400

600

800

1 000

4 000 00 4050 00 4 100 00 4150 00 4 200 00 4250 00 4 300 00 4350 00 4 400 00

a lp h a = 0.2

0

200

400

600

800

1 000

4 000 00 4050 00 4 100 00 4150 00 4 200 00 4250 00 4 300 00 4350 00 4 400 00

a lp h a = 0.4

0

200

400

600

800

1 000

4 000 00 4050 00 4 100 00 4150 00 4 200 00 4250 00 4 300 00 4350 00 4 400 00

a lp h a = 0.6

0

200

400

600

800

1 000

4 000 00 4050 00 4 100 00 4150 00 4 200 00 4250 00 4 300 00 4350 00 4 400 00

a lp h a = 0.8

0

200

400

600

800

1 000

4 000 00 4050 00 4 100 00 4150 00 4 200 00 4250 00 4 300 00 4350 00 4 400 00

a lp h a = 1.0

α = 0.0

α = 0.2

α = 0.4

α = 0.6

α = 0.8

α = 1.0

random

greedy

Construction phase solutions

Page 30: GRASP: Greedy Randomized Adaptive Search Procedures

slide 30 GRASP

How do methods compare?

• Local search from random starting solution:l high variancel avg solution

worse than avggreedy

l best solution usually better than best greedy

l slow convergence

• Local search from greedy starting solution:l low (no)

variancel usually sub-

optimall fast convergence

GRASP tries to capture good features ofgreedy & random

constructions.

Page 31: GRASP: Greedy Randomized Adaptive Search Procedures

slide 31 GRASP

Greedy vs Random: As starting solution for local search

0

200

400

600

800

1000

437000 438000 439000 440000 441000 442000 443000 444000 445000

0

200

400

600

800

1000

437000 438000 439000 440000 441000 442000 443000 444000 445000

random

greedy

local optima

local optima

Page 32: GRASP: Greedy Randomized Adaptive Search Procedures

slide 32 GRASP

0

200

400

600

800

1000

437000 438000 439000 440000 44 1000 442000 44300 0 444000 4450 00

a lp h a = 0.0

0

200

400

600

800

1000

437000 438000 439000 440000 44 1000 442000 44300 0 444000 4450 00

a lp h a = 0.2

0

200

400

600

800

1000

437000 438000 439000 440000 44 1000 442000 44300 0 444000 4450 00

a lp h a = 0.4

0

200

400

600

800

1000

437000 438000 439000 440000 44 1000 442000 44300 0 444000 4450 00

a lp h a = 0.6

0

200

400

600

800

1000

437000 438000 439000 440000 44 1000 442000 44300 0 444000 4450 00

a lp h a = 0.8

0

200

400

600

800

1000

437000 438000 439000 440000 44 1000 442000 44300 0 444000 4450 00

a lp h a = 1.0

α = 0.0

α = 0.2

α = 0.4

α = 0.6

α = 0.8

α =1.0

random

greedy

Local search phase solutions

bestsolution

semi-greedy

semi-greedy

semi-greedy

semi-greedy

Page 33: GRASP: Greedy Randomized Adaptive Search Procedures

slide 33 GRASP

0

200

400

600

800

1000

400000 405000 410000 415000 420000 425000 430000 435000 440000 445000

0

200

400

600

800

1000

400000 405000 410000 415000 420000 425000 430000 435000 440000 445000

0

200

400

600

800

1000

400000 405000 410000 415000 420000 425000 430000 435000 440000 445000

0

200

400

600

800

1000

400000 405000 410000 415000 420000 425000 430000 435000 440000 445000

0

200

400

600

800

1000

400000 405000 410000 415000 420000 425000 430000 435000 440000 445000

0

200

400

600

800

1000

400000 405000 410000 415000 420000 425000 430000 435000 440000 445000

randomα = 0

semi-greedyα = 0.4

greedyα = 1

construction

construction

construction

local search

local search

local search

Page 34: GRASP: Greedy Randomized Adaptive Search Procedures

slide 34 GRASP

Random & local search

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

10.00

0 500 1000 1500 2000 2500 3000 3500 4000

phase 2 solnphase 1 soln

we

igh

t

iteration

random construction

local search

Page 35: GRASP: Greedy Randomized Adaptive Search Procedures

slide 35 GRASP

GRASP ( α = 0.85)

9.50

9.55

9.60

9.65

9.70

9.75

9.80

9.85

9.90

9.95

10.00

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

phase 2 solnphase 1 soln

iteration

wei

ght

GRASP construction

local search

Page 36: GRASP: Greedy Randomized Adaptive Search Procedures

slide 36 GRASP

Local search solutions

9.80

9.82

9.84

9.86

9.88

9.90

9.92

9.94

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

GRASPrandomgreedy

iteration

wei

ght

GRASP

random

greedy

Page 37: GRASP: Greedy Randomized Adaptive Search Procedures

slide 37 GRASP

Local search solutions

9.88

9.89

9.90

9.91

9.92

9.93

9.94

10 100 1000 10000 100000

GRASPrandomgreedy

wei

ght

CPU time (seconds)

GRASP

random

greedy

Page 38: GRASP: Greedy Randomized Adaptive Search Procedures

slide 38 GRASP

Local search

• Local search is done from constructed solution:l to improve constructed solution that is not

locally optimall to improve constructed solution that is

locally optimal

• Types of local search used:l exchange [e.g. Feo, R., & Smith, 1994;

Laguna, Feo, Elrod, 1994]l tabu search [Laguna & Velarde, 1991; Díaz

& Fernández, 1998]l simulated annealing [Feo & Smith, 1994]l path relinking [Laguna & Martí, 1999]l POP [Fleurent & Glover, 1999]

Page 39: GRASP: Greedy Randomized Adaptive Search Procedures

slide 39 GRASP

Probability distribution:Time to sub-optimal

• Aiex, R., & Ribeiro (2000) studied the probability distribution of “time to sub-optimal” of several GRASPsl showed that “time to sub-optimal”

fits a two-parameter (or shifted) exponential distribution

l this has important implications regarding parallel implementations of GRASP

Page 40: GRASP: Greedy Randomized Adaptive Search Procedures

slide 40 GRASP

Probability distribution:Time to sub-optimal

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10 12

probability

t ime to sub-opt imal

33333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333

33333333333333333333

333333333333333333333

3333 3 33 3 3 3

time to sub-optimal (seconds)

prob

abilit

y

200 independent runsof GRASP

Empirical distribution

Page 41: GRASP: Greedy Randomized Adaptive Search Procedures

slide 41 GRASP

Q-Q plot with variabilityinformation

-2

0

2

4

6

8

10

12

14

16

0 1 2 3 4 5 6

measured t imes

exponent ial quant iles

333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333

333333333333333333333333333333333333333333333333

3333333333333333333333333333

33333333333

3333333333

33333 3 3

3 33

exponential quantiles

mea

sure

d tim

es

200 independent runsof GRASP

plus/minus one standarddeviation

Page 42: GRASP: Greedy Randomized Adaptive Search Procedures

slide 42 GRASP

Superimposed empirical & theoretical distributions

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10 12

probability

t ime to sub-opt imal

3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333

3333333333333333333333333 3 33 3 3 3

time (seconds)

prob

abilit

y

200 independent runsof GRASP

Page 43: GRASP: Greedy Randomized Adaptive Search Procedures

slide 43 GRASP

Enhancements

• Local searchl Path relinkingl Proximate Optimality Principle (POP)

• Asymptotically convergent GRASPl Bias function

• Automatic choice of RCL parameter αl Reactive GRASP

• Use of long-term memory• GRASP and Genetic Algorithms• Parallel GRASP

Page 44: GRASP: Greedy Randomized Adaptive Search Procedures

slide 44 GRASP

Path relinking

• Laguna & Martí (1999) adapted the concept of path relinking for use within a GRASP.

Diverse elite solutions:keep best 3

Current GRASP solution (after semi-greedy construction followed by local search).

Guiding solution (chosen at random)

guide

d so

lution

s:

chec

k eac

h fo

r qua

lity

Page 45: GRASP: Greedy Randomized Adaptive Search Procedures

slide 45 GRASP

Path relinking

X is current GRASP iterateY is guiding solution from elite set∆ = symmetric difference (X,Y) Example: X = (1,1,0,1,0) Y = (1,0,0,0,1) ∆ = (*,1g0,*, 1g0, 0g1)

while ( ∆ is not empty ){evaluate each move in ∆ from Xlet δ be the best moveX = move ( X, δ )if ( X is better than X* ) { X* = X }∆ = ∆ \ { δ }

}

Best solution istested for membershipIn Elite set after P.R.

Page 46: GRASP: Greedy Randomized Adaptive Search Procedures

slide 46 GRASP

Path relinking

• Aiex, Pardalos, R., & Toraldo (2000) and Festa, R., & Pardalos (2000) added the following to the approach of Laguna & Martí:l Large elite sets (10 to 50 elements)l Back and forth path relinkingl Path relinking between solution and all elite

solutions l Test for inclusion into elite set only best

solution in pathl Intermediate and post-optimization path

relinking between all elite set solutions

Page 47: GRASP: Greedy Randomized Adaptive Search Procedures

slide 47 GRASP

Proximate Optimality Principle (POP)

• “Good solutions at one level are likely to be found ‘close to’ good solutions at an adjacent level.” [Glover & Laguna, 1997]

• GRASP interpretation of POP: imperfections introduced during steps of GRASP construction can be “ironed-out” by applying local search during (and not only at the end of) GRASP construction[Fleurent & Glover, 1999].

Page 48: GRASP: Greedy Randomized Adaptive Search Procedures

slide 48 GRASP

Proximate Optimality Principle (POP)

• POP has been applied in GRASPsfor:l QAP by Fleurent & Glover (1999)l Job shop scheduling by Binato, Hery,

Loewenstern, and R. (1999)l transmission expansion planning by

Binato & Oliveira (1999)

• In all instances, POP improved the performance of GRASP.

Page 49: GRASP: Greedy Randomized Adaptive Search Procedures

slide 49 GRASP

Convergent GRASP

• Mockus, Eddy, Mockus, Mockus, &Reklaitis (1997) pointed out that GRASP with fixed nonzero RCL parameter α may not converge (asymptotically) to a global optimum.

• Remedies:l Randomly select α uniformly from

the interval [0,1] [R., Pitsoulis, &Pardalos, 1998]

l Use bias function selection mechanism of Bresina [1996]

l Reactive GRASP [Prais & Ribeiro, 1998]

Page 50: GRASP: Greedy Randomized Adaptive Search Procedures

slide 50 GRASP

Bias function

• Bresina (1996) introduced the concept of a bias function to select a candidate element to be included in the solution.l rank all candidate elements by greedy

function valuesl assign bias(r ) to r -th ranked element

l logarithmic: bias(r ) = 1/log(r +1)l linear: bias(r ) = 1/rl polynomial(n ): bias(r ) = 1/r n

l exponential: bias(r ) = 1/er

l random: bias(r ) = 1

Page 51: GRASP: Greedy Randomized Adaptive Search Procedures

slide 51 GRASP

Bias function

l define total_bias = Σ bias(r )l assign probability of selection of the

element ranked r to be:prob(r ) = bias(r ) / total_bias

l pick r -th ranked element with probability prob(r )

• Binato, Hery, Loewenstern, & R. (2000) use bias function to select an element from the RCL.

Page 52: GRASP: Greedy Randomized Adaptive Search Procedures

slide 52 GRASP

Automatic choice of RCL parameter α

• Choice of RCL parameter α is complicated:l may be problem dependentl may be instance dependent

• Remedies:l Randomly selected RCL parameter

[R., Pitsoulis, & Pardalos, 1998].l Reactive GRASP [Prais & Ribeiro,

1998]: self-adjusting α according to previously found solutions.

Page 53: GRASP: Greedy Randomized Adaptive Search Procedures

slide 53 GRASP

Reactive GRASP

• Introduced by Prais & Ribeiro (1998)• At each GRASP iteration, a value of the

RCL parameter α is chosen from a discrete set of values {α1 , α2, ..., αm }

• The probability that αk is selected is p(αk ).

• Reactive GRASP adaptively changes the probabilities {p(α1),p(α 2), ..., p(αm )} to favor values that produce good solutions.

Page 54: GRASP: Greedy Randomized Adaptive Search Procedures

slide 54 GRASP

Reactive GRASP

• We describe Reactive GRASP for a minimization problem.

• Initially p(αi ) = 1/m, i = 1,2, …, m, i.e. values are selected uniformly.

• Define l F (S *) be the value of the incumbent

(i.e. best so far) solution.l Ai be the average value of the

solutions obtained with αi in the construction phase.

Page 55: GRASP: Greedy Randomized Adaptive Search Procedures

slide 55 GRASP

Reactive GRASP

• Compute every Nα iterations:

l qi = ( F (S *) / A i )δ, i = 1,2, …,m

l p(αi ) = qi / Σ qj , i = 1,2, …,m

• Observe that the more suitable a value αi is, the larger the value ofqi is and, consequently, the higher the value of p(αi ), making αi more likely to be selected.

• The parameter δ can be used as an attenuation parameter.

Page 56: GRASP: Greedy Randomized Adaptive Search Procedures

slide 56 GRASP

Reactive GRASP

• Has been applied to:l traffic scheduling in satellite switched time

division multi-access (SS/TDMA) systems [Prais & Ribeiro, 1998]

l single source capacitated plant location [Díaz & Fernández, 1998]

l transmission expansion planning [Binato & Oliveira, 1999]

l mobile phone frequency assignment [Oliveira, Gomes, & R., 2000]

• Extensive computational experiments described in [Prais & Ribeiro, 1999]

Page 57: GRASP: Greedy Randomized Adaptive Search Procedures

slide 57 GRASP

Long term memory

• Since GRASP iterations are independent, current iteration makes no use of information gathered in previous iterations.

• Remedies:l Path relinking [Laguna & Martí, 1999]

l Reactive GRASP [Prais & Ribeiro, 1998]

l Use set of previously generated elite solutions to guide construction phase of GRASP [Fleurent & Glover, 1999] as an intensification mechanism.

Page 58: GRASP: Greedy Randomized Adaptive Search Procedures

slide 58 GRASP

Fleurent & Glover intensification scheme

• Introduced as a way to use long term memory in multi-start heuristics such as GRASP [Fleurent& Glover, 1999]

• An elite set of solutions S is maintained. To be in S solution must be:l better than best member of S , orl better than worst and sufficiently

different from other elite solutionsl e.g. count identical vector components

and set a threshold for rejection

• Use elite set in construction phase.

Page 59: GRASP: Greedy Randomized Adaptive Search Procedures

slide 59 GRASP

Fleurent & Glover intensification scheme

• Strongly determined variables are those that cannot be changed without eroding the objective or changing significantly other variables.

• A consistent variable is one that receives a particular value in a large portion of the elite solution set.

• Let I (e ) be a measure of the strongly determined and consistent features of choice e , i.e. I (e ) becomes larger as eresembles solutions in elite set S

Page 60: GRASP: Greedy Randomized Adaptive Search Procedures

slide 60 GRASP

Fleurent & Glover intensification scheme

• Intensity function is used in the construction phasel Recall g (e ) is greedy functionl Let E (e ) = F (g (e ), I (e ))

l e.g. E (e ) = λ g (e ) + I (e )

l Bias selection from RCL to those elements with a high E (e ) value.l prob (selecting e ) = E (e ) / Σs ε RCL E (s)

• E (e ) can vary with time (e.g. changing the value of λ )l keep λ large initially, then reducel to add diversification, increase λ

Page 61: GRASP: Greedy Randomized Adaptive Search Procedures

slide 61 GRASP

Fleurent & Glover intensification scheme

• Has been applied tol QAP [Fleurent & Glover, 1999]l Job shop scheduling [Binato, Hery,

Loewenstern, & R., 1999]

Page 62: GRASP: Greedy Randomized Adaptive Search Procedures

slide 62 GRASP

GRASP in hybrid metaheuristics

• tabu search as local search phase [Laguna & González-Velarde, 1991; Colomé & Serra, 1998; Delmaire, Díaz, Fernández, & Ortega, 1999]

• simulated annealing as local search phase [Feo & Smith, 1994; Liu, Pardalos, Rajasekaran, & R., 2000]

• path relinking as additional local search phase [Laguna & Martí, 1999; Festa, Pardalos, & R., 2000; Aiex, Pardalos, R., & Toraldo, 2000]

Page 63: GRASP: Greedy Randomized Adaptive Search Procedures

slide 63 GRASP

GRASP in hybrid metaheuristics

• GRASP as initial population generator for genetic algorithms (GA) [Ahuja, Orlin, & Tiwari, 2000]

• GRASP has also been used in a GA to implement a crossover operatorthat generates perfect offspring [Ramalhinho, Paixão, & Portugal, 1998]l Given two parents, perfect offspring

are the best possible offspring and their determinations requires the solution of an optimization problem.

Page 64: GRASP: Greedy Randomized Adaptive Search Procedures

slide 64 GRASP

Parallel GRASP

• GRASP is easy to implement in parallel:l parallelization by problem

decompositionl Feo, R., & Smith (1994)

l iteration parallelizationl Pardalos, Pitsoulis, & R. (1995)l Pardalos, Pitsoulis, & R. (1996)l Alvim (1998)l Martins & Ribeiro (1998)l Murphey, Pardalos, & Pitsoulis (1998)l R. (1998)l Martins, R., & Ribeiro (1999)l Aiex, Pardalos, R., & Toraldo (2000)

Page 65: GRASP: Greedy Randomized Adaptive Search Procedures

slide 65 GRASP

Parallel GRASP

• Let Pr(t ) be the probability of not having found a given (target) solution in t time units with r independent processes.l If P1(t ) = λ e- (t - µ )/λ , where λ and µ

are real numbers (λ > 0),l Then Pr(t ) = r λ e- (t - µ )/(µλ)

• Probability of finding a target solution in time rt with one processor equals probability of finding a solution at least as good in time t with r processors.

• Experiments indicate that this is the case for memoryless implementations of GRASP [Aiex, R. & Ribeiro, 2000].

Page 66: GRASP: Greedy Randomized Adaptive Search Procedures

slide 66 GRASP

Simple parallelization

• Most straightforward scheme for parallel GRASP is distribution of iterations to different processors.

• Care is required so that two iterations never start off with same random number generator seed.l run generator and record all Ng seeds

in seed( ) arrayl start iteration i with seed seed(i )

Page 67: GRASP: Greedy Randomized Adaptive Search Procedures

slide 67 GRASP

PVM & MPI implementations

• PVM: Pardalos, Pitsoulis, & R. (1996)• MPI: Alvim (1998); Alvim & Ribeiro

(1998); Martins, R., & Ribeiro (1999);Aiex, Pardalos, R., & Toraldo (2000)

master

slave slave slave

master passesblock of seeds toslave

master waits for slaveto finish block

slave passes backbest solution of block to master

Page 68: GRASP: Greedy Randomized Adaptive Search Procedures

slide 68 GRASP

Survey of O.R. & C.S. applications in literature

• scheduling• routing• logic• partitioning• location• graph theoretic• QAP and other assignment

problems• miscellaneous problems

Page 69: GRASP: Greedy Randomized Adaptive Search Procedures

slide 69 GRASP

Scheduling

• operations sequencing [Bard & Feo, 1989]• flight scheduling [Feo & Bard, 1989]• single machine [Feo, Venkatraman, & Bard,

1991]• just-in-time scheduling [Laguna & González-

Velarde, 1991]• Constant flow allowance (CON) due date

assignment & sequencing [De, Ghosj, & Wells, 1994]

• printed wire assembly [Feo, Bard, & Holland, 1995; Bard, Feo, & Holland, 1996]

Page 70: GRASP: Greedy Randomized Adaptive Search Procedures

slide 70 GRASP

Scheduling (continued)

• single machine with sequence dependent setup costs & delay penalties [Feo, Sarathy, & McGahan, 1996]

• field technician scheduling [Xu & Chiu, 1996, 1997]

• flow shop with setup costs [Ríos-Mercado & Bard, 1997, 1998]

• school timetabling [Drexl & Salewski, 1997; Rivera, 1998]

Page 71: GRASP: Greedy Randomized Adaptive Search Procedures

slide 71 GRASP

Scheduling (continued)

• bus-driver scheduling [Ramalhinho, Paixão, & Portugal, 1998]

• vehicle scheduling [Atkinson, 1998]

• job shop scheduling [R., Binato, Hery, & Loewenstern, 2000]

Page 72: GRASP: Greedy Randomized Adaptive Search Procedures

slide 72 GRASP

Routing

• vehicle routing with time windows [Kontoravdis & Bard, 1995]

• vehicle routing [Hjorring, 1995]

• aircraft routing [Argüello, Bard, & Yu, 1997]

• inventory routing problem with satellite facilities [Bard et al., 1998]

• Vehicle routing with backhauls [Carreto & Baker, 2000]

Page 73: GRASP: Greedy Randomized Adaptive Search Procedures

slide 73 GRASP

Logic

• SAT [R. & Feo, 1996]

• MAX-SAT [Pardalos, Pitsoulis, & R., 1996, 1997, 1998]

• inferring logical clauses from examples [Deshpande & Triantaphyllou, 1998]

Page 74: GRASP: Greedy Randomized Adaptive Search Procedures

slide 74 GRASP

Partitioning

• graph two-partition [Laguna, Feo, & Elrod, 1994]

• number partitioning [Argüello, Feo, &Goldschmidt, 1996]

• circuit partitioning [Areibi & Vannelli, 1997; Areibi, 1999]

Page 75: GRASP: Greedy Randomized Adaptive Search Procedures

slide 75 GRASP

Location and Layout

• p - hub location [Klincewicz, 1992]

• pure integer capacitated plant location [Delmaire et al., 1997]

• location with economies of scale [Holmqvist, Migdalas, & Pardalos, 1997]

• traffic concentrator [R. & Ulular, 1997]

• single source capacitated plant location [Díaz & Fernández, 1998]

• maximum covering [R., 1998]

• dynamic facility layout [Urban, 1998]

• uncapacitated location problem [Gomes & Silva, 1999]

Page 76: GRASP: Greedy Randomized Adaptive Search Procedures

slide 76 GRASP

Graph theoretic

• max independent set [Feo, R., & Smith, 1994; R., Feo, & Smith, 1998]

• max clique with weighted edges [Macambira & Souza, 1997]

• graph planarization [R. & Ribeiro, 1997;Ribeiro & R., 1997]

• 2-layer straight line crossing minimization [Laguna & Martí, 1999]

• sparse graph coloring [Laguna & Martí, 1998]

Page 77: GRASP: Greedy Randomized Adaptive Search Procedures

slide 77 GRASP

Graph theoretic (continued)

• maximum weighted edge subgraph[Macambira & Meneses, 1998]

• Steiner problem [Martins, Pardalos, R., & Ribeiro, 1998; Martins & Ribeiro, 1998; Martins, R., & Ribeiro, 1999; Martins, R., Ribeiro, & Pardalos, 2000]

• feedback vertex set [Qian, Pardalos, & R., 1998; Festa, Pardalos, & R., 1999]

• maximum clique [Abello, Pardalos, & R., 1998; Pardalos, R., & Rappe, 1998]

• capacitated minimum spanning tree [Ahuja, Orlin, & Sharma, 1998]

Page 78: GRASP: Greedy Randomized Adaptive Search Procedures

slide 78 GRASP

Graph theoretic (continued)

• traveling salesman [Silveira, 1999]

• maximum cut [Festa, Pardalos, & R., 2000]

Page 79: GRASP: Greedy Randomized Adaptive Search Procedures

slide 79 GRASP

QAP & other assignment problems

• QAP [Li, Pardalos, & R., 1994]• parallel GRASP for QAP [Pardalos,

Pitsoulis, & R., 1995]• Fortran subroutines for dense QAPs [R.,

Pardalos, & Li, 1996]• initial population for GA for QAP [Ahuja,

Orlin, & Tiwari, 2000]• long term memory GRASP for QAP

[Fleurent & Glover, 1999]• biquadratic assignment problem

[Mavridou, Pardalos, Pitsoulis, & R., 1997]

Page 80: GRASP: Greedy Randomized Adaptive Search Procedures

slide 80 GRASP

QAP & other assignment problems (continued)

• Fortran subroutines for sparse QAPs[Pardalos, Pitsoulis, & R., 1997]

• multidimensional knapsack [Labat &Mynard, 1997]

• data association multidimensional assignment problem [Murphey,Pardalos, & Pitsoulis, 1998]

• multidimensional assignment problem [Robertson, 1998]

• modified local search in GRASP for QAP [Rangel, Abreu, Boaventura-Netto, &Boeres, 1998]

Page 81: GRASP: Greedy Randomized Adaptive Search Procedures

slide 81 GRASP

QAP & other assignment problems (continued)

• 3-index assignment problem [Aiex, Pardalos, R., & Toraldo, 2000]

Page 82: GRASP: Greedy Randomized Adaptive Search Procedures

slide 82 GRASP

Miscellaneous problems

• set covering [Feo & R., 1989]• concave-cost network flow problem

[Holmqvist, Migdalas, & Pardalos, 1998]

• maximum diversity [Ghosj, 1996]• protein folding [Krasnogor et al., 1998]• clustering [Areibi & Vannelli, 1997;

Areibi, 1999]• consumer choice in competitive location

models [Colomé & Serra, 1998]• time series analysis [Medeiros, R., &

Veiga, 1999; Medeiros, Veiga, & R., 1999]

Page 83: GRASP: Greedy Randomized Adaptive Search Procedures

slide 83 GRASP

Survey of industrial applications in literature

• manufacturing• transportation• telecommunications• automatic drawing• electrical power systems• VLSI design• military

Page 84: GRASP: Greedy Randomized Adaptive Search Procedures

slide 84 GRASP

Manufacturing

• discrete parts [Bard & Feo, 1989]

• cutting path & tool selection [Feo & Bard, 1989]

• equipment selection [Bard & Feo, 1991]

• component grouping [Klincewicz &Rajan, 1994]

• printed wiring board assembly [Feo, Bard, & Holland, 1995; Bard, Feo, & Holland, 1996]

Page 85: GRASP: Greedy Randomized Adaptive Search Procedures

slide 85 GRASP

Transportation

• flight scheduling & maintenance base planning [Feo & Bard, 1989]

• intermodal trailer assignment [Feo &González-Velarde, 1995]

• aircraft routing in response to groundings & delays [Argüello, Bard, & Yu, 1997]

• rail car unloading [Bard, 1997]

• airline fleet assignment [Sosnowska, 1999]

Page 86: GRASP: Greedy Randomized Adaptive Search Procedures

slide 86 GRASP

Telecommunications

• design of SDH mesh-restorable networks [Poppe, Pickavet, Arijs, &Demeester, 1997]

• Steiner tree in graphs [Martins, Pardalos, R., & Ribeiro, 1998; Martins & Ribeiro, 1998; Martins, R., & Ribeiro, 1999]

• permanent virtual circuit (PVC) routing [Resende & R., 1997; Resende & R., 1999; Festa, Resende, & R., 2000]

• traffic scheduling in satellite switched time division multi-access (SS/TDMA) systems [Prais & Ribeiro, 1998]

Page 87: GRASP: Greedy Randomized Adaptive Search Procedures

slide 87 GRASP

Telecommunications (continued)

• point of presence (PoP) location [R., 1998]

• frequency assignment [Pasiliao, 1998; Liu,Pardalos, Rajasekaran, & R., 1999; Oliveira, Gomes, & R., 2000]

Page 88: GRASP: Greedy Randomized Adaptive Search Procedures

slide 88 GRASP

Automatic drawing

• seam drawing in mosaicking of aerial photographic maps [Fernández& Martí, 1997]

• graph planarization [R. & Ribeiro, 1997; Ribeiro & R., 1997]

• 2-layer straight line crossing minimization [Laguna & Martí, 1999]

Page 89: GRASP: Greedy Randomized Adaptive Search Procedures

slide 89 GRASP

Electrical power systems

• transmission expansion planning [Binato, Oliveira, & Araújo, 1998; Binato& Oliveira, 1999]

Page 90: GRASP: Greedy Randomized Adaptive Search Procedures

slide 90 GRASP

VLSI design

• circuit partitioning [Areibi & Vannelli, 1997; Areibi, 1999]

Page 91: GRASP: Greedy Randomized Adaptive Search Procedures

slide 91 GRASP

Military

• multitarget multisensor tracking [Murphey, Pardalos, & Pitsoulis, 1998]

Page 92: GRASP: Greedy Randomized Adaptive Search Procedures

slide 92 GRASP

Conclusion

• Online at my web site:l Up-to-date survey of GRASP [R.,

1998]:http://www.research.att.com/~mgcr/doc/sgrasp.ps

l Up-to-date bibliography:http://www.research.att.com/~mgcr/doc/graspbib.ps.Z

http://www.research.att.com/~mgcr/doc/graspbib.bib

l Up-to-date annotated bibliography [Festa & R., 2000]:

http://www.research.att.com/~mgcr/doc/gabib.pdf