Business Process ModelDevelopment – Phase 2 DiscussionAugust 19, 2002

James B. OrlinJames B. OrlinMIT Sloan School MIT Sloan School of Managementof ManagementVery Large-Scale

Neighborhood Search For Timetabling

Survey is joint with Carol MeyersSurvey is joint with Carol Meyers

2

Overview

Very large-scale neighborhood (VLSN) search

The cyclic exchange neighborhood

Functional annealing

Optimized crossover for genetic algorithms

Other approaches to VLSN Search

3

Neighborhood Search Algorithms

Start with a feasible solution x

Define a neighborhood of x

Identify an improved neighbor y

Replace x by y and repeat

xx

x

x

xx

x x

x x x

x

x

x

xx

xx

x

x

x xxxx

xx

x

x

x

x

x

x

xx

x

x

x

xx

x

x

xx

x x

x x x

x

x

x

xx

xx

x

x

x xxxx

xx

x

x

x

x

x

x

xx

x

x

x

xx

x

x

xx

x x

x x x

x

x

x

xx

xx

x

x

x xxxx

xx

x

x

x

x

x

x

xx

x

x

x

xx

x

x

xx

x x

x x x

x

x

x

xx

xx

x

x

x xxxx

xx

x

x

x

x

x

x

xx

x

x

x

……………..

4

VLSN Search

Quality of the local optimal solution

Better as the size of the neighborhood increases

Running time increases with the increase in size

VLSN Search

Very large neighborhoods

Neighborhoods enumerated implicitly

Use of network/discrete optimization algorithms

5

VLSN Search dates back a long way

If Simplex is viewed as a search technique, then

column generation is VLSN Search

Sending flows around cycles to find a min cost flow

is VLSN Search

Augmenting path algorithms are VLSN search

bipartite matching

non-bipartite matching

max flow

6

Overarching Goal

Promoting VLSN Search

VLSN Search is a collection of tools that may be of

use in timetablingranges from very obvious approaches to some

subtle approachescan be combined with tabu search or other

heuristics it has only recently been used. There are many

more opportunities for use of VLSN search

Note: I am available to discuss ideas over the next

two days

7

Surveys on VLSN Search

Ahuja, Ergun, Orlin, Punnen [2002], [2006]

Meyers and Orlin [2006]

Deĭneko and Woeginger [2000]

See also http://www.ise.ufl.edu/ahuja/

http://web.mit.edu/jorlin/www/

8

The Cyclic Exchange

Neighborhood

9

1

10

7

89

1112

14 1315

17 19

201816

3

2 4

5

6

Partitioning Problems

Given a set S of

20 objects.

Partition the set S

into four subsets,

S1, S2, S3, S4.

Find the partition

such that i c(Si) is

minimum.

S1 S2

S4S3

1 10

141920

8

13

1518

3 1112

17 16

45

7

92

6

10

A Two-Exchange Neighborhood

Number of possible exchanges n2

1 10

1920

8

13

1518

1112

17 16

7

92

6

543

S4S3

S2S1 14

11

A Multi-Exchange Neighborhood-I

A Cyclic Multi-Exchange

1 10

1920

8

13

1511

17 16

92

6

543

S1 S2

S4S3

Number of possible

exchanges K

nK!

K

14

1812

7

12

A Multi-Exchange Neighborhood-II

A Path Multi-Exchange

1 10

1920

8

13

1518

11

17 16

92

6

5

43

S1 S2

S4S3

14

12

7

13

Sizes of the Two Neighborhoods

Number of

Neighbors

Number of

Neighbors

n = 100, K = 10)

Two-

Exchanges n2 10,000

Multi-

Exchanges 3.63*1016

KnK!

K

Suppose that we can evaluate 1 million neighbors per second.

Estimated

CPU Time for

Enumeration

10-2 seconds

1,150 years

14

Multi-Exchanges w.r.t a solution S

Directed Cycles in G(S)

Multi-exchanges are transformed into directed

cycles in an appropriately defined graph G(S),

called the improvement graph.

Enumerating Multi-Exchanges

15

Improvement Graph

Improvement Graph G:

A node i for every item

i in partition.

An arc (i, j) signifies a

move of item i to the

subset containing item

j and item j leaving its

subset.

1 10

141920

8

1315

18

1112

17 16

7

926

543

S1 S2

S4 S3

1

19

20

10

146

7

9

2

17

12

4

16

11

5 15

3

18

13

8

16

Improvement Graph (contd.)

Cost of the multi-exchange =

Cost of the directed cycle

Profitable multi-exchange =

a negative cost cycle

c14,7

c12,8

c8,14 c7,12

12

714

8

1 10

1920

13

1511

17 16

92

6

543

S1 S2

S4S3

14

812

7

18

17

A Theorem on Improvement Graphs

Theorem. There is a one-to-one correspondence

between cyclic exchanges w.r.t S and subset-disjoint

cycles in G(S).

18

The Generic Algorithm

Start with a feasible partition S = (S1,S2,.., Sk).

Construct the improvement graph G(S).

Repetitive Loop

Identify a negative-cost (subset-disjoint) cycle in

G(S)

Obtain the profitable multi-exchange w.r.t S

Perform the multi-exchange and update S

Update the improvement graph G(S)

19

Uses of the Cyclic Exchange Neighborhood

Facility location problems Ahuja, Orlin, Pallottino, Scaparra and Scutella [2002]

Vehicle routing problems Agarwal, Ahuja, Laporte and Shen [2003]

Weapon-target assignment problem Ahuja, Kumar, Jha and Orlin [2003]

Examination Timetabling Abdullah, Ahmadi, Burke, Dror PATAT [2004]

Course Scheduling Jha PhD dissertation [2004]

20

Examination SchedulingAbdullah, Ahmadi, Burke, Dror [2004]

No person scheduled for two exams in one period.

For each person, there is a penalty for exams that are close to each other.

21

Examination SchedulingAbdullah, Ahmadi, Burke, Dror [2004]

Day 1

Period 1

Day K

Period 2

Period 3

22

Some Empirical Results

23

Course Timetabling : Jha, Ahuja [2004]

Assign subjects to time slots and classrooms

minimize student conflicts on class preferences

maximize professors’ preferences

The partitions are time slots.

Each cyclic exchange moves classes between the time slots.

M W F 10 AM T Th F 11 AM

M Th F 3 PM

Typically within a few percent of optimum on those problems solvable by IPs.

24

Simulated Annealing

Simulated Annealing (in its usual form) is not a

good choice for VLSN Search

No. of

Neighbors

No. of Neighbors

n = 100, K = 10)

Two-

Exchanges n2 10,000

Multi-

Exchanges 3.63*1016

Choosing one of 1016 neighbors at random can lead to very slow convergence to the optimum.

25

Functional Annealing

Sharma and Sukhapesna [2005 and 2006]

Iteratively perturb the objective function

The objective function converges to the correct

objective function

Combined with statistical learning, the authors

claim impressive results for the QAP.

26

The Functioning Annealing Algorithm

Algorithm Functional annealing

begin

choose an initial solution x0 in F

set k = 0

while stopping criteria not met do

let c’ = c + w, where w is obtained using an exp dist

find an improved neighbor wrt c’

end

end

27

GAs and Optimized Crossover

Give a very brief overview of GAs since the

audience already knows this stuff.

Then explain that one can in principle look for the

optimized crossover.

application to independent set (one slide) on

technique plus a slide on computational results

application to TSP (one slide)

possible application to timetabling

Mutation

http://press2.nci.nih.gov/sciencebehind/cancer/images/cancer42.gif

29

two chromosomes

a crossover (chiasma)

the chromosomes after recombination

http://www.emc.maricopa.edu/faculty/farabee/BIOBK/BioBookmeiosis.html

Crossovers

30

The evolutionary process

maintain a population of individuals

the gene pool is renewed through mutations

Natural selection (better solutions are more likely to

be parents), and crossover (mixing of genes), leads

to improvements in successive generations

But, can we do better than nature? What if we

create optimal children?

31

Optimized Crossover for Independent Sets

Agarwal, Orlin, Tai [1997]

1

8

23

4

5

67

Parent 1

1 2 3 4 5 6 7 8

0 1 0 0 1 0 0 1

1

8

23

4

5

67

Parent 2

1 2 3 4 5 6 7 8

1 0 0 1 0 1 0 0

1

8

23

4

5

67

Opt child

1 2 3 4 5 6 7 8

0 1 0 1 0 1 0 1

32

A possible application to timetabling

Consider allocating subjects to time slots

The optimal crossover of two parents has the

following property:

if subject j is allocated to time slot s1 by parent 1 and

time slot s2 by parent 2, then in any child it is

allocated to time slot s1 or s2.

Questions to address Can one find the optimized crossover? How would one implement this as part of a GA? Would it work well in practice?

33

Other Work on VLSN Search

Diameter of a Neighborhood

see e.g., Bozejko and Wodecki. PATAT06

Dominance [Punnen and Glover]

Extended Neighborhood [Orlin and Sharma]

Using Grammars to express neighborhoods

[Bompadre and Orlin]

Exponential Nbhds searchable in polynomial time

34

A simple approach relying on IPs

If the original problem is formulated as an IP, then

one can have IP based neighborhoods.

ORIGINAL IP FOR SOME TIMETABLING PROBLEM

minimize cx

subject to x P

Suppose that x* is feasible but not optimal

LOCAL SEARCH

minimize cx

subject to || x – x* || K

x P

35

Uses of the Simple IP Approach

Great for interactive scheduling (person – machine

scheduling.)

Can be used to modify the original problem and

specify that the solution is not too far away.

e.g., in exam scheduling, a user can require that a

subject’s exam move to a different period. The user

can then reoptimize subject to the constraint that at

most 10 other subjects change time slots.

36

Conclusion Slide

Some successful applications of VLSN search in

timetabling

Lots of successes for VLSN search in other

domains

Opportunities for more applications of VLSN search

in timetabling.