Program Does not Equal Program: Constraint Programming and Its Relationship to Mathematical...

Copyright � 2001 INFORMS0092-2102/01/3106/0029/$05.001526–551X electronic ISSN

PROGRAMMING—INTEGER—ALGORITHMS—BRANCHANDBOUND COMPUTERS/COMPUTER SCIENCE—ARTIFICIALINTELLIGENCE PROGRAMMING—CONSTRAINTThis paper was refereed.

INTERFACES 31: 6 November–December 2001 (pp. 29–53)

Program Does Not Equal Program: ConstraintProgramming and Its Relationship toMathematical ProgrammingIrvin J. [email protected]

ILOG1080 Linda Vista AvenueMountain View, California 94043

Jean-Francois [email protected]

ILOG9 rue VerdunBP 85Gentilly Cedex, France

Arising from research in the computer science community,constraint programming is a fairly new technique for solvingoptimization problems. For those familiar with mathematicalprogramming, a number of language barriers make it difficultto understand the concepts of constraint programming. In thisshort tutorial on constraint programming, we explain how itrelates to familiar mathematical programming concepts andhow constraint programming and mathematical programmingtechnologies are complementary. We assume a minimal back-ground in linear and integer programming.

George Dantzig [1963] invented thesimplex method for linear program-

ming in 1947 and first described it in apaper entitled “Programming in a linearstructure” [Dantzig 1948, 1949]. Fifty yearslater, linear programming is now a strate-gic technique used by thousands of busi-nesses trying to optimize their globaloperations. In the mid-1980s, researchersdeveloped constraint programming as acomputer science technique by combiningdevelopments in the artificial intelligence

community with the development of newcomputer programming languages. Fifteenyears later, constraint programming isnow being seen as an important techniquethat complements traditional mathematicalprogramming technologies as businessescontinue to look for ways to optimize theirbusiness operations.

Developed independently as a techniquewithin the computer science literature,constraint programming is now getting at-tention from the operations research com-

LUSTIG, PUGET

INTERFACES 31:6 30

munity as a new and sometimes betterway of solving certain kinds of optimiza-tion problems. We provide an introduc-tion to constraint programming for thosefamiliar with traditional mathematicalprogramming.The Word Programming

For those familiar with mathematicalprogramming, one of the confusions withregard to understanding constraint pro-gramming is the fact that the names ofboth areas share the word programming. Infact, the two disciplines use this worddifferently.

The field of mathematical programmingarose from its roots in linear program-ming. In his seminal textbook, Dantzig[1963, p. 1–2] introduces linear program-ming by describing a few different plan-ning problems:

Nevertheless, it is possible to abstract the un-derlying essential similarities in the manage-ment of these seemingly disparate systems. Todo this entails a look at the structure and stateof the system, and at the objective to be ful-filled, in order to construct a statement of theactions to be performed, their timing, and theirquantity (called a “program” or “schedule”),which will permit the system to move from agiven status toward the defined objective.

The problems that Dantzig studied whiledeveloping the simplex algorithm were“programming problems,” because theUnited States Defense Department in thepost-World War II era was supporting re-search to devise programs of activities forfuture conflicts. T. C. Koopmans suggestedthat Dantzig use the term linear program-ming as opposed to programming in a linearstructure, and Al Tucker then suggestedthat Dantzig call the linear programmingproblem a linear program [Dantzig andThapa 1997]. Therefore, the term program,

previously used to describe a plan of ac-tivities, became associated with a specificmathematical problem in the operationsresearch literature.

Constraint programming is often calledconstraint logic programming, and it origi-nates in the artificial intelligence literaturein the computer science community. Here,the word programming refers to computerprogramming. Knuth [1968, p. 5] defines acomputer program as “an expression of acomputational method in a computer lan-guage.” A computer program can beviewed as a plan of action for the opera-tions of a computer, and hence the com-mon concept of a plan is shared with theplanning problems studied in the develop-ment of the simplex method. Constraintprogramming is a computer programmingtechnique, with a name that is in the spiritof other programming techniques, such asobject-oriented programming, functionalprogramming, and structured program-ming. Logic programming is a declarative,relational style of programming based onfirst-order logic, where simple resolutionalgorithms are used to resolve the logicalstatements of a problem. Constraint logicprogramming extends this concept by us-ing more powerful algorithms to resolvethese statements. Van Hentenryck [1999,p. 4] writes:

The essence of constraint programming is atwo-level architecture integrating a constraintand a programming component. The constraintcomponent provides the basic operations of thearchitecture and consists of a system reasoningabout fundamental properties of constraint sys-tems such as satisfiability and entailment. Theconstraint component is often called the con-straint store, by analogy to the memory store oftraditional programming languages. . . . Oper-ating around the constraint store is aprogramming-language component that speci-

CONSTRAINT PROGRAMMING

November–December 2001 31

fies how to combine the basic operations, oftenin non-deterministic ways.

Hence, a constraint program is not a state-ment of a problem as in mathematical pro-gramming, but is rather a computer pro-gram that indicates a method for solving aparticular problem. It is important to em-phasize the two-level architecture of a con-straint programming system. Because it isfirst and foremost a computer program-ming system, the system contains repre-sentations of programming variables,which are representations of memory cellsin a computer that can be manipulatedwithin the system. The first level of theconstraint programming architecture al-lows users to state constraints over theseprogramming variables. The second levelof this architecture allows users to write acomputer program that indicates how thevariables should be modified so as to findvalues of the variables that satisfy theconstraints.

The roots of constraint programmingcan be traced back to the work on con-straint satisfaction problems in the 1970s,with arc-consistency techniques[Mackworth 1977] on the one hand andthe language ALICE [Lauriere 1978] thatwas designed for stating and solving com-binatorial problems on the other hand. Inthe 1980s, work in the logic programmingcommunity showed that the Prolog lan-guage could be extended by replacing thefundamental logic programming algo-rithms with more powerful constraintsolving algorithms. For instance, in 1980,Prolog II used a constraint solver to solveequations and disequations on terms. Thisidea was further generalized in the con-straint logic programming scheme and im-

plemented in several languages[Colmerauer 1990; Jaffar and Lassez 1987;Van Hentenryck 1989; and Wallace et al.1997]. Van Hentenryck [1989] used the arc-consistency techniques developed in theconstraint satisfaction problem frameworkas the algorithm for the basic constraintsolving. This concept was termed finitedomain constraints.

In the 1990s, researchers transformedconstraint logic programming based onProlog to constraint programming by pro-viding constraint programming features ingeneral-purpose programming languages.Examples includes Pecos for Lisp [Puget1992], ILOG Solver for C�� [ILOG 1999],and Screamer for Common Lisp [Siskindand McAllester 1993]. This developmentmade possible powerful constraint solvingalgorithms in the context of mainstreamprogramming languages [Puget andLeconte 1995]. Another rich area of re-search in constraint programming hasbeen the development of special-purposeprogramming languages to allow peopleto apply the techniques of constraint pro-gramming to different classes of problems.Examples include Oz [Smolka 1993] andCLAIRE [Caseau and Laburthe 1995]. Indesigning such languages, their develop-ers have sought to provide complete lan-guages for doing computer programmingand hence the languages allow users toimplement constraint solving algorithms.Departing from this approach, VanHentenryck [1999] designed OPL (Optimi-zation Programming Language) to make iteasy to solve optimization problems bysupporting constraint programming andmathematical programming techniques.He did not deem the completeness of the

LUSTIG, PUGET

INTERFACES 31:6 32

language for computer programming orthe ability to program constraint solvingalgorithms as important. Instead, he de-signed the language to support the declar-ative representation of optimization prob-lems, providing the facilities to use anunderlying constraint programming en-gine combined with the ability to describe,via computer programming techniques, asearch strategy for finding solutions toproblems. The OPL language is not a com-plete computer programming language,

Constraint programming is acomputer programmingtechnique.

but rather a language that is designed toallow people to solve optimization prob-lems using either constraint programmingor mathematical programming techniques.An advantage of OPL is that the same lan-guage is used to unify the representationsof decision variables from traditionalmathematical programming with pro-gramming variables from traditional con-straint programming. Some of the exam-ples we present are related to those VanHentenryck [1999] presents.

Van Hentenryck was motivated to de-sign OPL by the increased interest inmathematical programming modeling lan-guages, such as AMPL [Fourer, Gay, andKernighan 1993] and GAMS [Bisschop andMeeraus 1982], and the recent use of con-straint programming to solve combinato-rial optimization problems. These NP-hardproblems include feasibility problems aswell as optimization problems. Constraintprogramming is often used when peoplewant a quick feasible solution to a prob-

lem rather than a provably optimal solu-tion. Because a constraint programmingsystem provides a rich declarative lan-guage for stating problems combined witha programming language for describing aprocedural strategy to find a solution, con-straint programming is an alternative tointeger programming for solving a varietyof combinatorial optimization problems.

Van Hentenryck [1989], Marriott andStuckey [1999], and Hooker [2000] de-scribe some of the underlying theory ofconstraint programming. In his bookabout OPL, Van Hentenryck [1999] gives anumber of examples of how constraintprogramming can be applied to real prob-lems. Unfortunately, we have yet to find agood book that gives the techniques forapplying constraint programming to solveoptimization problems and is written inthe spirit of the book by Williams [1999]for mathematical programming. Williamsdemonstrates a wide variety of modelingtechniques for solving problems usingmathematical programming.Constraint Programming Formulations

To explain the constraint programmingframework, we will first characterize theways that constraint programming can beapplied to solve combinatorial optimiza-tion problems by developing a notation todescribe these problems. Then we willshow formulations of feasibility problemsand optimization problems using the OPLlanguage.

We first formally define a constraint sat-isfaction problem, using some of the ter-minology of mathematical programming.Given a set of n decision variables x1, x2,. . . , xn, the set Dj of allowable values foreach decision variable xj, j� 1, . . . , n, is



called the domain of the variable xj. Thedomain of a decision variable can be anypossible set, operating over any possibleset of symbols. For example, the domainof a variable could be the even integers be-tween 0 and 100 or the set of real numbersin the interval [1,100] or a set of people{Tom, Judy, Jim, Ann}. There is no restric-tion on the type of each decision variable,and hence decision variable can take oninteger values, real values, set elements, oreven subsets of sets.

Formally, a constraint c(x1, x2, . . . , xn) isa mathematical relation, that is, a subset Sof the set D1 � D2 � . . . � Dn, such thatif (x1, x2, . . . , xn) � S, then the constraint issaid to be satisfied. Alternatively, we candefine a constraint as a mathematical func-tion f : D1 � D2 � . . . � Dn → {0,1} suchthat f(x1, x2, . . . , xn)�1 if and only if c(x1,x2, . . . , xn) is satisfied. Using this func-tional notation, we can then define a con-straint satisfaction problem (CSP) asfollows:

Given n domains D1, D2, . . . , Dn and mconstraints f1, f2, . . . , fm, find x1, x2, . . . , xnsuch that

f (x , x , . . . , x )�1, 1 � k � m;k 1 2 n

x � D , 1 � j � n.j j

This problem is only a feasibility problem,and no objective function is defined. Nev-ertheless, CSPs are an important class ofcombinatorial optimization problems.Here the functions fk do not necessarilyhave closed mathematical forms (for ex-ample, functional representations) and canbe defined simply by providing the set Sdescribed above. A solution to a CSP issimply a set of values of the variables suchthat the values are in the domains of the

variables and all of the constraints aresatisfied.Optimization Problems

We have defined a constraint satisfac-tion problem as a feasibility problem. Withregards to optimization, constraint pro-gramming systems also allow specificationof an objective function. Notationally, wedenote the objective function as g: D1 � D2

� . . . � Dn → �, so that at any feasiblepoint to the CSP, the function g(x1, x2, . . . ,xn) can be evaluated. For ease of exposi-tion, we will assume that we are minimiz-ing this objective function. An optimiza-tion problem can then be stated as follows:

Minimize g(x , x , . . . , x )1 2 n

Subject tof (x , x , . . . , x )�1, 1 � k � m;k 1 2 n

x � D , 1 � j � n.j j

Simple ExamplesTo present some simple examples of

constraint satisfaction and optimizationproblems using constraint programmingtechniques, we will need a language thatallows us to describe the decision vari-ables x1, x2, . . . , xn, the constraints f1, f2,. . . , fm, and the objective function g. Be-cause we want to indicate the relation-ships between constraint programmingand mathematical programming, we willuse OPL for these representations. We dis-cuss programming a search strategy later.OPL includes a default search strategy forfinding a solution to the problem, so wedo not always need to describe such astrategy. A later example demonstrateswhere a search strategy is required to beable to solve the problem. In our presenta-tions of OPL, we include line numbers onthe left to facilitate discussion.

LUSTIG, PUGET

INTERFACES 31:6 34

Map ColoringMap coloring is an NP-complete combi-

natorial optimization problem. Consider aset of countries and a set of colors with agiven set of adjacency relationships amongthe countries. We need to assign the colorsto the countries so that no two adjacentcountries have the same color. This is anOPL statement for this problem:1 enum Countries...;2 enum Colors...;3 struct Neighbor {4 Countries c1;5 Countries c2;6 };7 setof(Neighbor) neighbors � ...;8 var Colors color[Countries];9 solve {10 forall (n in neighbors) {11 color[n.c1] �� color[n.c2];12 }13 };

Line 1 specifies the set of countries, andLine 2, the set of colors, with each of thesesets given in a data file. Lines 3 through 6are an OPL construct to define a record,where a record consists of two elementsfrom the set of countries. Line 7 indicatesthat the set of adjacency relationships (aset called neighbors containing recordsof type Neighbor) is also provided asdata for the problem. This data is specifiedas a list of pairs of adjacent countries.

Line 8 contains the decision variablesfor the problem, with a decision variablecolor[j] for each country j, and thevalue of this decision variable is one of theelements of the set Colors. The domainsof the variables are elements of a set. Line9 contains the keyword solve, indicatingthat a constraint satisfaction problem is tobe solved. The forall statement in Line10 is an OPL quantifier that says that theconstraints on Line 11 will be stated foreach member of the set neighbors, that

is, for each pair of countries that are adja-cent. The constraints for this problem, inLines 10 and 11, are simple to state. The“��” operator is used to specify that foreach member n of the set neighbors,with each member consisting of a pair ofcountries n.c1 and n.c2, the color of thefirst country n.c1 is different from that ofthe second country n.c2 of the pair.

This example illustrates that variablescan be set-valued and that constraints canbe written as mathematical relations. Witha normal mathematical programming rep-resentation, it is not possible to write somemathematical relations, such as strict in-equalities on continuous variables. Inmathematical programming, one normallyexpresses these relations by introducingadditional binary variables and additionalconstraints into a mixed integer program-ming representation.The Stable Marriage Problem

The stable marriage problem is a well-studied problem in the operations researchliterature. In fact, entire books have beenwritten on the subject, for example,Gusfield and Irving [1989]. Given a set ofn women and an equal number of men,we assume that each of the women hasprovided a preference ordering of the menby assigning each man a unique valuefrom the integers 1 through n. Similarly,we assume that each of the men has pro-vided a preference ordering of the womenby assigning each woman a unique valuefrom the integers 1 through n. Each manhas a woman he most prefers (a womanhe gives a ranking of 1), while eachwoman also has a man that she prefers (aman she gives a ranking of 1). A stablemarriage (for the men) is an assignment of



the men to the women so that if a man Aprefers another man’s (B’s) wife, she pre-fers her current husband B to man A. Sim-ilarly, a stable marriage (for the women) isan assignment of the men to the womenso that if a woman A prefers anotherwoman’s (B’s) husband, he prefers his cur-rent wife B to woman A. The stable mar-riage problem is a simplified version ofthe problem of assigning prospectivemedical residents to hospitals, that the Na-tional Resident Matching Program solvesannually.

The conditions for stability are easilystated as a constraint satisfaction problemin OPL:1 enum Women ...;2 enum Men ...;3 int rankWomen[Women,Men] � ...;4 int rankMen[Men,Women] � ...;5 var Women wife[Men];6 var Men husband[Women];7 solve {8 forall(m in Men)9 husband[wife[m]] � m;10 forall(w in Women)11 wife[husband[w]] � w;12 forall(m in Men & o in Women)13 rankMen[m,o]�rankMen[m,wife[m]]��14 rankWomen[o,husband[o]]�rankWomen[o,m];15 forall(w in Women & o in Men)16 rankWomen[w,o]�rankWomen[w,husband[w]]��17 rankMen[o,wife[o]]�rankMen[o,w];18 };

Line 1 specifies the set of women, andLine 2, the men. The elements of these setscan be used to index into arrays, and hencethe two-dimensional array rankWomenspecifies the women’s preference rankingsof the men, while the two-dimensional ar-ray rankMen specifies the men’s prefer-ence rankings of the women. The decisionvariables for this problem, specified inLines 5 and 6, are given as two arrays. Foreach element m in the set Men, wife[m]is an element of the set Women that indi-cates the wife of man m. Similarly, for eachelement w in the set Women, husband[w]

is an element of the set Men that indicatesthe husband of wife w. The values of thedecision variables are specific elements ofa set and are not numerically valued.

As in the map-coloring problem, Line 7indicates that we want to find a solutionto a constraint satisfaction problem. Again,we use the forall constructor to indicatea group of constraints. On Lines 8 and 9,we state a constraint that specifies that thehusband of the wife of man m must beman m. We state a similar constraint foreach woman w on Lines 10 and 11. Notethat the decision variables wife[m] andhusband[w] are being used to index intothe arrays of decision variables husband[]and wife[], respectively. This kind ofconstraint is called an element constraint.

Lines 12 through 14, and subsequentlyLines 15 through 17, specify the stabilityrelationships, with the rule for the menstated before the rule for the women. Theoperator “��” is the logical implicationrelation, while the “�” is a less-thanrelation. The constraint in Lines 13 and14 can be read as, “If rankMen[m,o] �

rankMen[m,wife[m]], then it must bethe case that rankWomen[o, husband[o]]� rankWomen[o,m].” This constraint issatisfied as long as it is not the case thatthe left-hand side of the implication is trueand the right-hand side of the implicationis simultaneously false. An alternative rep-resentation of this constraint would be toreplace the implication relation “��”with the less-than-or-equal relation “��”,which, in this case, would be an equiva-lent representation. This is because boththe left-hand and right-hand sides arethemselves constraints (each using the“�” relation), and these constraints evalu-

LUSTIG, PUGET

INTERFACES 31:6 36

ate to 0 or 1 and hence can be numericallycompared. This type of composition ofconstraints is termed a metaconstraint, sinceit is a constraint stated over otherconstraints.A Sequencing Problem

A set of blocks in a plant needs to bepainted. Each block has a designatedcolor, and we want to sequence the blocksto minimize the cost of the number oftimes we change the paint color duringthe sequence. In addition, the blocks areeach given an initial position wi in the se-quence and a global interval g such that ifa block is painted before its initial positionor after the position wi � g, a penalty costis assessed. This problem is a simplifiedversion of a paint-shop problem solvedwith constraint programming at Daimler-Chrysler. The following provides an OPLmodel for this problem:

1 int nblocks � ...;2 enum Colors � ...;3 range blockrng 1..nblocks;4 blockrng whenblock[blockrng] � ...;5 Colors color[blockrng] � ...;6 int� OKinterval � ...;7 int swapcost � ...;8 assert forall (i in 1..nblocks�1)

whenblock[i] �� whenblock[i�1];910 setof(int) whencolor[c in Colors] �11 {i | i in blockrng : color[i]�c};12 var blockrng position[blockrng];13 var blockrng whichblock[blockrng];14 var int colorChanges in card(Colors)

�1 ..nblocks�1;15 var int pencost in 0..nblocks*nblocks;1617 minimize swapcost*colorChanges � pencost18 subject to {19 colorChanges � sum (i in 1..nblocks�1)20 (color[whichblock[i]]��color

[whichblock[i�1]]);21 pencost � sum (i in blockrng)22 (maxl(whenblock[i]�position[i],0)�23 maxl(position[i]�(whenblock[i]

�OKinterval),0));24 alldifferent(position);25 alldifferent(whichblock);26 forall (i in blockrng) {27 whichblock[position[i]] � i;28 position[whichblock[i]] � i;29 };30 forall (c in Colors) {

31 forall (k in whencolor[c] : k ��whencolor[c].last()) {

32 position[k] � position[whencolor[c].next(k)];

33 }34 }35 };

Lines 1 and 2 specify the number of blocksand the set of paint colors. Line 3 uses theOPL keyword range to create a type(blockrng) for representing the possiblepositions (1 through the number of blocks)in which we can place each block. This typecan then act as a set of integers, just as thetype Colors represents the set of colors.Line 4 declares the array whenblock,which corresponds to the values wi. We as-sume that these values are given in increas-ing order, and this is checked in Line 8. Line5 specifies the color of each block, whileLine 6 declares the global interval g. Line 7declares the cost of changing the color inmidsequence. Lines 10 and 11 determine,for each possible color, the set of blocks tobe painted this color.

The model to solve this problem beginsat Line 12. Here, the array position indi-cates the ordinal position of each block,while the array whichblock declared inLine 13 indicates which specific block is ineach position. Line 14 declares a variableto count the number of times that thecolor can be changed. A lower bound forthis is the number of colors less 1, whilean upper bound is the number of blocksless 1. For ease of exposition, the penaltycost is declared as a decision variable inLine 15. Line 17 indicates that the objectivefunction is being minimized. The numberof changes in color is determined by com-paring the color of adjacent blocks in Lines19 and 20. As in the previous section, thisis done via a metaconstraint, where the in-



equality color[whichblock[i]] ��

color[whichblock[i�1]] is evalu-ated for each pair of adjacent blocks. Thisinequality has a zero-one value, and thesevalues are summed for each pair to com-pute the number of changes in color. Thepenalty costs are computed in Lines 21through 23. The OPL function maxl (a,

b) computes the maximum of the argu-ments. The penalty is then computed bypenalizing both earliness and lateness,with no penalty assessed when the condi-tion whenblock[i] �� position[i]

�� whenblock[i]�Okinterval

holds.Lines 24 and 25 illustrate two examples

of global constraints. The constraintalldifferent(position) indicatesthat each member of the position arrayshould have a different value. In essence,this single constraint on an array ofnblocks variables is a statement ofnblocks*(nblocks�1) pairwise in-equalities. In this example, because the ar-ray position has nblocks elements,each with an integer value between 1 andnblocks, the constraint is equivalent tosaying that the array position containsan assignment of each block to a uniqueposition. An equivalent representation us-ing mathematical programming tech-niques would require nblocks*nblocksbinary variables, and 2*nblocks con-straints. The element constraints in Lines27 and 28 represent that the arrays posi-tion and whenblock are inverses of eachother.

The constraints stated in Lines 30through 34 help reduce the search space.For each color, the blocks of that color areconsidered in the order of the values in

the whenblock array. Because of thestructure of the penalties, it is easy to seethat blocks of the same color should be or-dered in the same order in which they ap-pear in the input data. The constraints arewritten for each of the blocks of each color.The expression whencolor[c].next(k)indicates the block that follows block k inthe set whencolor[c].Algorithms for Constraint Programming

Up to now, we have not discussed thealgorithm that a constraint programmingsystem uses to determine solutions to con-straint satisfaction and optimization prob-lems. In traditional constraint program-ming systems, the user is required toprogram a search strategy that indicateshow the values of the variables shouldchange so as to find values that satisfy theconstraints. In OPL, a default search strat-egy is used if the user does not provide asearch strategy. However, users often pro-gram a search strategy to effectively applyconstraint programming to solve a prob-lem. We will describe the fundamental al-gorithm underlying a constraint program-ming system and then indicate themethodologies used to program searchstrategies.Constraint Propagation and DomainReduction

We defined a constraint as a mathemati-cal function f (x1, x2, . . . , xn) of the vari-ables. Within this environment, assumethere is an underlying mechanism that al-lows the domains of the variables to bemaintained and updated. When a vari-able’s domain is modified, the effects ofthis modification are then communicatedto any constraint that interacts with thatvariable. This communication is called con-

LUSTIG, PUGET

INTERFACES 31:6 38

straint propagation. For each constraint, adomain reduction algorithm modifies the do-mains of all the variables in that con-straint, given the modification of one ofthe variables in that constraint. The do-main reduction algorithm for a particularkind of constraint discovers inconsisten-cies among the domains of the variables inthat constraint by removing values fromthe domains of the variables. If the algo-rithm discovers that a particular variable’sdomain becomes empty, then it can be de-termined that the constraint cannot be sat-isfied, and an earlier choice can be un-done. A similar methodology is found inthe bound-strengthening algorithms usedin modern mathematical programmingsolvers and discussed by Brearley, Mitra,and Williams [1975]. A crucial differencebetween the procedures used in mathe-matical programming presolve implemen-tations and domain reduction algorithmsis that in constraint programming, the do-mains can have holes, while in mathemati-cal programming, domains are intervals.

We can best demonstrate this methodol-ogy with an example (Figure 1). Consider

two variables x and y, where the domainsof each variable are given asDx�{1,2,3,4, . . . ,10} andDy�{1,2,3,4, . . . ,10}, and the single con-straint y�2x. If we first consider the vari-able y and this constraint, we know that ymust be even, and hence the domain of ycan be changed to Dy�{2,4,6,8,10}. Now,considering the variable x, we see thatsince y � 10, it follows that x � 5, andhence the domain of x can be changed toDx�{1,2,3,4,5}. Suppose that we add aconstraint of the form (x modulo 2)�1.This is equivalent to the statement that x isodd. We can then reduce the domain of xto be Dx�{1,3,5}. Now, reconsidering theoriginal constraint y�2x, we can removethe values of 4 and 8 from the domain of yand obtain Dy�{2,6,10}.

Some constraint programming systems(for example, ILOG Solver, Oz, andECLiPSe) allow the programmer to takeadvantage of existing propagators forbuilt-in constraints that cause domain re-ductions and allow the programmer tobuild his or her own propagation and do-main reduction schemes for user-defined

Figure 1: Constraint propagation and domain reduction are used to reduce the domains of thevariables x and y. The constraints y�2x, y � 10, (x modulo 2)�1, and y�2x are applied in anorder determined by constraint propagation, due to reductions in the domains of each of thevariables.



constraints. However, many constraintprogramming systems (for example, OPL,ILOG Solver, and CHIP) are now strongenough that they provide large libraries ofpredefined constraints, with associatedpropagation and domain reduction algo-rithms, and it is often not necessary to cre-ate new constraints with specialized prop-agation and domain reduction algorithms.

Given a set of variables with their do-mains and a set of constraints on thosevariables, a constraint programming sys-tem will apply the constraint propagationand domain reduction algorithm in an it-erative fashion to make the domains ofeach variable as small as possible, whilemaking the entire system arc consistent.Given a constraint fk as stated above and avariable xj, value d � Dj is consistent withfk if at least one assignment of the vari-ables exists such that xj�d and fk�1 withrespect to that assignment. A constraint isthen arc consistent if all of the values in thedomains of all the variables involved inthe constraint are consistent. A constraintsystem is arc consistent if all of the corre-sponding constraints are arc consistent.The term arc is used because the first CSPswere problems with constraints stated onpairs of variables, and hence this systemcan be viewed as a graph, with nodes cor-responding to the variables and arcs corre-sponding to the constraints. Arc consis-tency enables the domains of the variablesto be reduced while not removing poten-tial solutions to the CSP.

Researchers have developed a numberof algorithms to efficiently propagate con-straints and reduce domains so as to cre-ate systems that are arc consistent. One al-gorithm, called AC-5, was developed by

Van Hentenryck, Deville, and Teng [1992].Their article is important for constraintprogramming because it unified the re-search directions of the constraint satisfac-tion community and the logic program-ming community by introducing theconcept of developing different domainreduction algorithms for different con-straints as implementations of the basicconstraint propagation and domain reduc-tion principle.Constraint Programming Algorithms forOptimization

As compared to linear and mixed-integer programming, a weakness of aconstraint programming approach whenapplied to a problem with an objectivefunction to minimize is that a lowerbound may not exist. A lower bound maybe available if the expression representingthe objective function has a lower boundthat can be derived from constraint propa-gation and domain reduction. This is un-like integer programming, in which alower bound always exists because of thelinear programming relaxation of theproblem. Constraint programming sys-tems offer two methods for optimizingproblems, called standard and dichotomicsearch.

The standard search procedure is to firstfind a feasible solution to the CSP, whileignoring the objective function g(x1, x2, . . . ,xn). Let y1, y2, . . . , yn represent such a fea-sible point. The search space can then bepruned by adding the constraint g(y1, y2,. . . , yn) � g(x1, x2, . . . , xn) to the systemand continuing the search. The added con-straint specifies that any new feasiblepoint must have a better objective valuethan the current point. Propagation of this

LUSTIG, PUGET

INTERFACES 31:6 40

constraint may cause the domains of thedecision variables to be reduced, thus re-ducing the size of the search space. As thesearch progresses, new points will haveprogressively better objective values. Theprocedure concludes when no feasiblepoint is found. When this happens, thelast feasible point can be taken as the opti-mal solution.

Dichotomic search depends on having agood lower bound L on the objective func-tion g(x1, x2, . . . , xn). Before optimizing theobjective function, a procedure must findan initial feasible point, which determinesan upper bound U on the objective func-tion. A dichotomic search procedure is es-sentially a binary search on the objectivefunction. The procedure computes themidpoint M�(U � L)/2 of the twobounds and then solves a CSP by takingthe original constraints and adding theconstraint g(x1, x2, . . . , xn) � M. If it findsa new feasible point, then it updates theupper bound and continues the search inthe same way with a new midpoint M. Ifit finds the system to be infeasible, then itupdates the lower bound, and the searchagain continues with a new midpoint M.Dichotomic search is effective when thelower bound is strong, because the com-putation time to prove that a CSP is infea-sible can often be large. The use of dicho-tomic search in cooperation with a linearprogramming solver may be effective ifthe linear programming representation canprovide a good lower bound. The differ-ence between this procedure and a branch-and-bound procedure for mixed-integerprogramming is that the dichotomicsearch stresses the search for feasible solu-tions, whereas branch-and-bound proce-

dures usually emphasize improving thelower bound.Programming a Search Strategy

Given a CSP, one can apply the con-straint propagation and domain reductionalgorithms to reduce the domains of thevariables so as to arrive at an arc-consistent system. However, while doingthis may determine whether the CSP is in-feasible, it does not necessarily find solu-tions to a CSP. To do this, one must pro-gram a search strategy (or use a defaultsearch strategy if the constraint program-ming system provides one). Traditionally,the search facilities that constraint pro-gramming systems provide have beenbased on depth-first search. The root nodeof the search tree contains the initial val-ues of the variables. At each node, theuser programs a goal, which is a strategythat breaks the problem into two (or more)parts and decides which part should beevaluated first. A simple strategy might beto pick a variable and to try to set thatvariable to the different values in the vari-able’s domain. This strategy creates a setof leaves in the search tree and createswhat is called a choice point, with each leafcorresponding to a specific choice. Thegoal also orders the leaves amongst them-selves within the choice point. In the nextlevel of the tree, the results of the choicemade at the leaf are propagated, and thedomains are reduced locally in that part ofthe tree. This will either produce a smallerarc-consistent system or a proof that thechoice made for this leaf is not possible. Inthis case, the system automatically back-tracks to the parent and tries other leavesof that parent. The search thus proceeds ina depth-first manner, until it finds a solu-



tion at a node low in the tree or until it ex-plores the entire tree, in which case itfinds the CSP to be infeasible. The searchstrategy is enumerative, with constraintpropagation and domain reduction em-ployed at each node to help prune thesearch space.

A simple example will illustrate thisidea. Consider the following OPL exam-ple, which shows a simple CSP on threevariables, each with the same domain.

1 var int x in 1..3;2 var int y in 1..3;3 var int z in 1..3;4 solve {5 x � y � 1;6 (z � x) \/ (z � y);7 };

The constraint in Line 6 illustrates a logi-cal constraint using the logical or operator

(\/) indicating that either (or both) of theconditions (z�x) or (z�y) must hold. Adefault search strategy for this problemwould try the different values for x, y,and z in order, producing the search treeshown in Figure 2.

ILOG Solver 4.4 and ILOG OPL Studio2.1 incorporate a recent innovation in con-straint programming systems [ILOG 1999],allowing the programmer to use otherstrategies beyond depth-first search. Con-straint programming systems have tradi-tionally used depth-first search because, inviewing constraint programming as a par-ticular computer programming methodol-ogy, depth-first search dramatically sim-plifies memory management issues. ILOGSolver 4.4 and ILOG OPL Studio 2.1(which uses ILOG’s Solver technology)

Figure 2: In this diagram, each internal node represents a choice point. The nodes shaded withhorizontal lines represent nodes that are never created, because the corresponding values havebeen removed from the variable’s domain. The nodes that are shaded with vertical lines repre-sent nodes that are also never created, because there is a single choice for the decision variabley once the selection for x has been determined. The nodes that are shaded black correspond tosolutions that are found. It is worthwhile to consider the node that exists as a result of thechoice x�3. After this choice is made, the constraint propagation and domain reduction algo-rithms automatically remove the values 1 and 3 from the domain of y and there is no need for achoice point corresponding to a choice for y. Since y�2 is the only remaining value, the searchfor z can begin immediately.

LUSTIG, PUGET

INTERFACES 31:6 42

allow the programmer to use best-firstsearch [Nilsson 1971], limited-discrepancysearch [Harvey and Ginsberg 1995], depth-bounded-discrepancy search [Walsh 1997],and interleaved depth-first search[Meseguer 1997]. In ILOG Solver, the basicidea is that the user programs node evalua-tors, search selectors, and search limits. Nodeevaluators contain code that looks at eachopen node in the search tree and choosesone to explore next. Search selectors orderthe different choices within a node, andsearch limits allow the user to terminatethe search after it reaches some globallimit (for example, time or node count).With these basic constructs in place, onecan easily program any search strategythat systematically searches the entiresearch space by choosing nodes to explore(that is, programming node evaluators),by dividing the search space at nodes (thatis, programming goals and creating choicepoints), and by picking the choice to eval-uate next within a specific node (that is,

programming search selectors). Constraintprogramming systems provide a frame-work for describing enumeration strate-gies for solving search problems in combi-natorial optimization.An Example of Search: Graceful Graphs

A graph G�(V,E) with n�|V| nodesand m�|E| edges is said to be graceful ifthere are unique node labelsf :V r {0,1, 2, . . . , m} and unique edge la-bels g:E r {1, 2, . . . , m} such thatg(i, j)�|f (i)�f ( j )| for each edge e�Ewith e�(i, j ) (Figure 3). Graceful graphshave applications in radio astronomy andcryptography.

An OPL solution to the problem of find-ing a graceful labeling of a graph follows:

1 int numnodes � ...;2 range Nodes 1..numnodes;3 struct Edge {4 Nodes i;5 Nodes j;6 };7 setof(Edge) edges � ...;8 int numedges � card(edges);9 range Labels 0..numedges;

Figure 3: In this graph with a graceful labeling, the numbers in an italic font are the labels forthe nodes, while the other numbers are the labels for the edges.



10 var Labels nl[Nodes];11 var Labels el[edges];12 solve {13 alldifferent(nl);14 alldifferent(el);15 forall (e in edges) {16 el[e]�abs(nl[e.i]�nl[e.j]);17 el[e]�0;18 }19 };20 search {21 generate(nl);22 };

Line 1 declares that the number of nodesis input data, while Line 2 declares a typeto represent the set of nodes. Lines 3through 6 declare a type to hold the pairsof edges, while Line 7 declares the set ofedges as input data. A type Labels is cre-ated at Line 9 to represent the potential la-bels of the nodes and edges. Lines 10 and11 declare the arrays nl and el for the la-bels of the nodes and edges, respectively.The constraints for the problem are statedin Lines 12 through 19. Lines 13 and 14use the global constraint alldifferentto indicate that all the node labels must bedifferent as well as the edge labels. Lines15 through 18 indicate the relationship be-tween the edge labels and the node labels.Because all node labels must be distinct,the constraint in Line 17 indicates that thezero value is not possible for the edgelabels.

Lines 20 through 22 indicate a very sim-ple search procedure for solving this prob-lem. The statement generate(nl) indi-cates that the different possible values forthe array nl should be generated in a non-deterministic way by choosing the variablewith the smallest domain size and instan-tiating that variable to the smallest valuein its domain. After this selection is made,constraint propagation will cause values tobe removed from the domains of other

variables. A new variable is then chosenand a value is instantiated for it. As thesearch progresses, a new variable is cho-sen, always using the smallest domain sizeas the metric. It should be noted that with-out the declaration of this procedure, it isnot possible to solve the example in Figure3. In fact, until constraint programmingwas applied to this particular example, itwas not known if the graph in this figurehad a graceful labeling.An Example of Search: A WarehouseLocation Problem

Consider the problem of assigningstores to warehouses while simultaneouslydeciding which warehouses to open. Thedata that is given is the cost of assigningeach store to each warehouse, a fixed costthat is incurred when the warehouse isopened, and the capacity (in terms ofnumber of stores) of each potential ware-house. An OPL model for this problem us-ing constraint programming constructsand a search strategy for solving the prob-lem follows:

1 int fixed � ...;2 int nbStores � ...;3 enum Warehouses ...;4 range Stores 1..nbStores;5 int capacity[Warehouses] � ...;6 int supplyCost[Stores,Warehouses] � ...;7 int maxCost � max(s in Stores, w in

Warehouses) supplyCost[s, w];8 Warehouses wlist[w in Warehouses] � w;910 var int open[Warehouses] in 0..1;11 var Warehouses supplier[Stores];12 var int cost[Stores] in 0..maxCost;1314 minimize15 sum(s in Stores) cost[s] � sum(w in

Warehouses) fixed * open[w]16 subject to {17 forall(s in Stores)18 cost[s]�supplyCost[s,supplier[s]];19 forall (s in Stores)20 open[supplier[s]] � 1;21 atmost(capacity, wlist, supplier);22 };2324 search {

LUSTIG, PUGET

INTERFACES 31:6 44

25 forall(s in Stores ordered by decreasingregretdmin(cost[s]))

26 tryall(w in Warehouses27 ordered by increasing supplyCost[s,w])28 supplier[s] � w;29 };

Lines 1 through 6 declare the data for theproblem and use the OPL keyword range

to create a type (Stores) to correspond tothe range of integers 1 through the num-ber of stores. In Line 7, the maximum costof assigning any store to any warehouse iscomputed. Line 8 computes the list ofwarehouses as an array for use in theatmost constraint in Line 21.

Lines 10 through 12 contain the decisionvariables for the problem. Line 10 declaresan array of binary variables to indicatewhether a particular warehouse is open.Line 11 declares an array that will containwhich warehouse is assigned to a particu-lar store. For any store s, supplier[s]is an element of the set Warehouses. Wealso will create an array of decision vari-ables cost[] that will contain the actualcost of assigning a particular store to itswarehouse. This will be used in the searchstrategy described in Lines 24 through 29.

Lines 14 and 15 indicate that we wish tominimize a cost function. Lines 17 and 18compute the cost of assigning a particularstore by looking up the cost in the data ar-ray supplyCost[]. Lines 19 and 20 de-scribe the constraints that require that, if astore is assigned to a warehouse, the ware-house must be open. Finally, the con-straint expressed in Line 21 is a particularkind of global constraint, described earlier.The atmost constraint in this particularcase says that for each value wlist[w](which, in this case, is each element w ofthe set Warehouses), the number of timesthat this value may appear in the array

supplier is at most capacity[w]. Ef-fectively, this single constraint is countingthe number of times each warehouse isused and placing a bound on that count.

Lines 24 through 29 describe a searchstrategy for solving this problem. The ba-sic idea behind the search strategy is to or-der the stores via some merit function andthen, for each store, to try different ware-houses to which it can be assigned, afterordering the warehouses by a differentmerit function. The statement in Line 25orders the stores by what is known as theminimal-regret heuristic. For a particularstore s, we can easily see that the domainof possible values of the variable cost[s]is initialized to be the values in the datamatrix supplyCost[s,w] for each possi-ble warehouse w. Hence, if there were fivewarehouses, the initial domain ofcost[s] would have five values. As thesearch progresses toward a solution, val-ues are eliminated from the domain of thisvariable. At any particular time, we canconsider the two lowest costs in the do-main of cost[s]. The valueregretdmin(cost[s]) is defined as thedifference between these two values. Theabbreviation dmin corresponds to domainminimum and the abbreviation regret

indicates the regret, or the cost of switch-ing from the minimum-cost warehouse forthat store to the next-higher-cost ware-house. The statement in Line 25 dynami-cally orders the stores by this value, thatis, it orders the stores by ranking first thestore that would incur the highest cost ofswitching from the lowest-cost warehouseto the next-lowest-cost warehouse. Lines26 and 27 then order the warehouses forthat store by ranking first the warehouse



with the lowest cost. In combination withLine 28, the tryall instruction in Line 26says that the system should try to assigneach warehouse in turn to this store. Afterone store is assigned, the next store that isranked according to the minimal-regretheuristic is assigned to its warehouse. Ifthis assignment is found to be infeasible,the system will backtrack and try a differ-ent warehouse for the last store that wasassigned. Since no search strategy is speci-fied for the variables cost and open, theOPL system will use a default search strat-egy for those variables after the suppliervariables have been assigned. In this par-ticular case, the values of cost and open

are determined by the constraints in Lines18 and 20, respectively, so no specificsearch strategy for those variables isrequired.

This search strategy uses knowledgeabout the problem to guide the search.First, it chooses to assign stores to ware-houses and then to open the warehouses.By ranking the stores according to the costof switching and by ranking the ware-houses by the cost of assigning the ware-houses to a fixed store, the search strategyprunes the search space by trying thelower-cost warehouses first in the searchfor a solution. In one particular instance,without this search strategy, ILOG OPLStudio 3.0 needed 1,894 choice points tofind a solution. With this search strategy,ILOG OPL Studio 3.0 needed only 147choice points.Connections to Integer Programming

We have emphasized how constraintprogramming can be applied to combina-torial optimization problems. The searchstrategies used in constraint programming

are related to those used when solvingmixed-integer-programming problems viabranch-and-bound procedures. Further-more, the problems that are solved oftenhave integer programmingrepresentations.

For those familiar with integer program-ming, the concept of search strategies wehave described should seem familiar. In fact,branch and bound, which is an enumera-tive search strategy, has been used to solveinteger programs since the mid-1960s.Lawler and Wood [1966] give an early sur-vey, while Garfinkel and Nemhauser [1972],in their classic text, describe branch andbound in the context of an enumerativeprocedure. Nemhauser and Wolsey [1988]provide a more recent discussion. In sys-tems developed for integer programming,users are often given the option of choos-ing a variable selection strategy and anode selection strategy. These are clearlyequivalent to the search selectors andnode evaluators we described.

A constraint programming frameworkextends the basic branch-and-bound pro-cedures implemented in typical mixed-integer programming solvers in two funda-mental ways. First, in most implementationsof branch-and-bound procedures formixed-integer programming, the imple-mentation creates two branches at eachnode after a variable x with a fractionalvalue � has been chosen to branch on. Theimplementation then divides the searchspace into two parts by creating a choicepoint based on the two choices of (x � �)and (x � �). In the constraint program-ming framework, the choices that are cre-ated can be any set of constraints that di-vides the search space. For example, given

LUSTIG, PUGET

INTERFACES 31:6 46

two integer variables x1 and x2, one couldcreate a choice point consisting of thethree choices (x1 � x2), (x1 � x2), (x1 � x2).

The second way that a constraint pro-gramming framework extends the basicbranch-and-bound procedures is with re-spect to the variable selection strategy. Inmost branch-and-bound implementations,the variable selection strategy uses noknowledge about the model of the prob-lem to make the choice of variable tobranch on. The integer program is treatedin its matrix form, and different heuristicsare used to choose the variable to branchon based on the solution of the linear pro-gramming relaxation that is solved at eachnode. In a constraint programming ap-proach, the user specifies the branchingstrategy in terms of the formulation of theproblem. Because a constraint program isa computer program, the decision vari-ables of the problem can be treated ascomputer programming variables, and oneprograms a strategy using the language ofthe problem formulation. Hence, to effec-tively apply constraint programming tech-niques, one uses problem-specific knowl-edge to help guide the search strategy soas to efficiently find a solution. In this way,a constraint programming system, whencombined with a linear programming opti-mizer, can be viewed as a framework thatallows users to program problem-specificbranch-and-bound search strategies forsolving mixed-integer programming prob-lems by using the same programming ob-jects for declaring the decision variablesand for programming the search strate-gies. Combinations of linear programmingand constraint programming have ap-peared in Prolog III [Colmerauer 1990],

CLP(R) [Jaffar and Lassez 1987], CHIP[Dincbas et al. 1988], ILOG Solver andILOG Planner [ILOG 1999] and ECLiPSe[Wallace, Novello, and Schimpf 1997].

A number of branch-and-bound imple-mentations for mixed-integer program-ming allow users to program customsearch strategies, including MINTO[Nemhauser, Savelsbergh, and Sigismondi1994], IBM’s OSL [1990], Dash AssociatesXPRESS-MP [2000], and ILOG CPLEX[2000]. In particular, MINTO allows usersto divide the search space into more thanone part at each node. However, the cru-cial difference between constraint pro-gramming systems and these mixed-integer programming branch-and-boundsolvers is that the mixed-integer program-ming systems require the users to specifytheir search strategy in terms of a matrixformulation of the problem, whereas aconstraint programming system uses a sin-gle programming language for both mod-eling the problem to be solved and forspecifying a search strategy to solve theproblem. The key point is that the decisionvariables of an optimization problem canbe treated as programming language vari-ables within a computer programmingenvironment.Contrasting Formulations

Earlier we gave an example of solving awarehouse location problem using con-straint programming techniques. It isworth contrasting this formulation with apure integer programming formulation:1 int fixed � ...;2 int nbStores � ...;3 enum Warehouses ...;4 range Stores1..nbStores;5 int capacity[Warehouses] � ...;6 int supplyCost[Stores,Warehouses] � ...;78 var int open[Warehouses] in 0..1;



9 var int supply[Stores,Warehouses] in 0..1;1011 minimize12 sum(w in Warehouses) fixed*open [w] �13 sum(w in Warehouses, s in Stores)

supply Cost [s,w] * supply [s,w]14 subject to {15 forall(s in Stores)16 sum(w in Warehouses) supply[s,w]�1;17 forall(w in Warehouses, s in Stores)18 supply [s,w] �� open[w];19 forall(w in Warehouses)20 sum(s in Stores) supply[s,w]

�� capacity[w];21 };

Lines 1 through 6 are identical to the con-straint programming formulation earlierand specify the data for the problem. Themeaning of the array open[] is also thesame. For each potential pair of a store sand a warehouse w, the Boolean variablesupply[s,w] indicates whether store s isassigned to warehouse w. The objectivefunction is stated in Lines 12 through 13.The cost of assigning store s, which wasrepresented by a decision variable in theconstraint programming formulation, isimplicitly computed in the expression inLine 13. Lines 15 and 16 specify the con-straint that each store must be assigned toexactly one warehouse. Lines 17 and 18specify that a store can be assigned only toan open warehouse, while Lines 19 and 20specify the constraint that limits the num-ber of stores that can be assigned to anyparticular warehouse.

The constraint programming formula-tion uses a linear number of variables withlarger domains to describe which storesare assigned to which warehouses, whilethe integer programming formulation usesa quadratic number of binary variables.The constraint that a store can be assignedto exactly one warehouse is implicit inthe constraint programming formulationbecause there is a decision variable

supplier that indicates the specific ware-house that is assigned. In the integer pro-gramming formulation, constraints explic-itly enforce this restriction. In the integerprogramming formulation, the assignmentof stores to open warehouses is enforcedby an inequality, while the constraint pro-gramming formulation enforces this re-striction via an element constraint. Finally,the constraint programming formulationuses a global constraint to enforce the re-striction on the number of stores per ware-house, while this restriction is enforcedwith a set of linear constraints in the inte-ger programming formulation.

Is one formulation better than the otheris? Making a direct comparison is difficultbecause mixed-integer programming solv-ers like CPLEX have advanced techniques,such as cut generation and presolve reduc-tions, that improve the performance of theMIP optimizer. In addition, the perfor-mance of the constraint programming al-gorithms sometimes depends on the un-derlying data for the problem as well asthe effectiveness of the search strategy. Anadditional consideration is whether one isinterested in obtaining a proof of optimal-ity or just interested in a good feasible so-lution within a certain time limit. We needmore research to better understand how tomake this kind of comparison.Hybrid Strategies

One of the exciting avenues of researchis to explore how constraint programmingand traditional mathematical program-ming approaches can be used together tosolve difficult problems. We indicatedhow a constraint programming system canserve as a framework for programming abranch-and-bound procedure for integer

LUSTIG, PUGET

INTERFACES 31:6 48

programming that takes advantage of theproblem structure to influence the variableand node selection process. There areother avenues in which the two ap-proaches can cooperate. In the first, weconsider stating formulations that containa mixture of linear and “not-linear” con-straints. In the second, we consider de-composing a problem to use constraintprogramming to solve one part of theproblem and mathematical programmingto solve a second part.A Hybrid Formulation of the WarehouseLocation Problem

Another way of solving the warehouselocation problem is to combine the twoformulations. In the following OPL model,we omit the data section and the searchstrategy (which are the same as in the pre-vious formulations) and just show the de-cision variables and the constraints.

1 var int open[Warehouses] in 0..1;2 var int supply[Stores,Warehouses] in 0..1;3 var Warehouses supplier[Stores];4 var int cost[Stores] in 0..maxCost;56 minimize with linear relaxation7 sum (w in Warehouses) fixed*open[w] �8 sum(w in Warehouses, s in Stores)

supplyCost[s,w]*supply[s,w]9 subject to {10 forall(s in Stores)11 sum(w in Warehouses) supply[s,w] � 1;12 forall(w in Warehouses, s in Stores)13 supply[s,w] �� open[w];14 forall (w in Warehouses)15 sum(s in Stores) supply[s,w]��capacity[w];1617 forall(s in Stores)18 cost[s] � supplyCost[s,supplier[s]];19 forall(s in Stores)20 open[supplier[s]] � 1;21 atmost(capacity, wlist, supplier);2223 forall(s in Stores)24 supply[s,supplier[s]] � 1;25 forall(s in Stores)26 cost[s] � sum(w in Warehouses)

supplyCost[s,w]*supply[s,w]27 };

Lines 1 through 4 combine the variables ofthe two formulations and have the same

meaning. In Line 6, the phrase “withlinear relaxation” indicates thatOPL should solve the problem by extract-ing the linear constraints and using theobjective function on the linear relaxationof those constraints to determine a lowerbound that can be used to prove optimal-ity of the procedure. (We do not use di-chotomic search here.) Lines 7 through 8are the same objective function. Lines 10through 15 are the constraints from thelinear formulation, while Lines 17through 21 are the constraints from theconstraint programming formulation.Lines 23 through 26 are the constraintsthat link the two formulations. Lines 23and 24 relate the supply and supplier

variables, while Lines 25 and 26 deter-mine the value of the cost variable interms of the supply variables. The addi-tional constraints help the combined for-mulation to prune the search space andreduce the overall solution time. For ex-ample, we can solve a simple five-warehouse, five-store problem using thepure constraint programming formulationwith 147 choice points, while we cansolve the same problem using the hybridformulation with 53 choice points. Forthis particular data set, we can solve allthree formulations in a fraction of a sec-ond, making their comparison difficult.

With the advances in available technol-ogy, exploring these kinds of mixed for-mulations is now possible. Such tech-niques might prove to be effective solutionstrategies for hard combinatorial optimiza-tion problems that benefit from multiplerepresentations. Jain and Grossman [2000]have investigated these concepts for somemachine-scheduling applications.



Column Generation and CrewScheduling

Constraint programming can be usedwhen implementing column-generationapproaches for solving different kinds ofcombinatorial optimization problems. Theclassic example is the cutting stock prob-lem, in which one solves knapsack prob-lems to generate the possible patterns touse and then solves linear programs to de-termine how many cuts of each pattern touse. The dual solutions to each linear pro-gram change the cost structure of theknapsack problem, which one solves togenerate a new column of the linearprogram.

Another example is crew scheduling.For a number of years, people have solvedcrew scheduling problems by writingcomputer programs to generate the poten-tial pairings of crews to flights. The pro-grams have to generate pairings, corre-sponding to a sequence of flights for asingle crew, that are low in cost and sat-isfy the complex duty rules included involumes of carrier regulations. The typicalapproach used in the past, which is stillused today, is a generate-and-test approach:One generates a complete pairing andthen tests the feasibility of this pairingagainst all of the rules programmed intothe system. After generating a suitablenumber of pairings, one solves a set-partitioning problem by letting each pair-ing correspond to one column of the set-partitioning problem.

Constraint programming can be used togenerate the pairings. Given a set of flightsand a flight schedule, one can declare vari-ables that correspond to the sequence offlights covered by one pairing. One can

then state constraints that dictate the rulesabout sequences of flights, how manyhours can be in a total sequence of flights,the required time between flights, and soforth. One can then carry out a search pro-cedure to generate potential flightsequences. As the search proceeds, the un-derlying constraint propagation and do-main reduction algorithms prune thesearch space. This becomes more of a test-and-generate method, where the constraintsthat define possible pairings are consis-tently used to help guide the search forpossible feasible solutions. After generat-ing a suitable set of pairings, one can usethem as columns for a set-partitioning prob-lem which is solved by an integer pro-gramming solver. An example of this ap-proach can be found on the ILOG Web sitein the OPL model library at http://www.ilog.com/products/oplstudio/.

Many operations research applicationsincorporate various kinds of enumerationstrategies that are usually embedded incomplex computer programs. Constraintprogramming provides an attractive alter-native for implementing these kinds ofenumeration schemes. We prefer to callthis constrained enumeration.Constraint-Based Scheduling

Another interesting application of con-straint programming is in schedulingproblems. We define a scheduling problemas a problem of determining a sequence ofactivities with given precedence relation-ships, subject to constraints on resourcesthat the activities compete for. For exam-ple, the classic job shop scheduling prob-lem, in which a set of jobs consisting ofsequential tasks must be scheduled on ma-chines, fits into this framework. To sup-

LUSTIG, PUGET

INTERFACES 31:6 50

port this class of problems, many con-straint programming systems extend theirframeworks to directly represent theseproblems. Following is the job shop prob-lem stated in OPL:

1 int nbMachines � ...;2 range Machines 1..nbMachines;3 int nbJobs � ...;4 range Jobs 1..nbJobs;5 int nbTasks � ...;6 range Tasks 1..nbTasks;78 Machines resource[Jobs,Tasks] � ...;9 int� duration[Jobs,Tasks] � ...;

10 int totalDuration � sum(j in Jobs, t in Tasks)duration[j,t];

1112 scheduleHorizon � totalDuration;13 Activity task[j in Jobs, t in Tasks]

(duration[j,t]);14 Activity makespan(0);1516 UnaryResource tool[Machines];1718 minimize19 makespan.end20 subject to {21 forall(j in Jobs)22 task[j, nbTasks] precedes makespan;2324 forall(j in Jobs)25 forall(t in 1..nbTasks-1)26 task[j,t] precedes task[j,t � 1];2728 forall(j in Jobs)29 forall(t in Tasks)30 task[j,t] requires

tool[resource[j,t]];31 };

The first nine lines define the input dataof the problem, consisting of the numberof machines, the number of jobs, and thenumber of tasks. We use the OPL key-word range to create a type to corre-spond to an integer range. The array re-source, declared in Line 8, is input dataconsisting of the identity of the machinethat is needed to do a particular taskwithin a job. Line 9 declares the array du-ration that is the time required to exe-cute each task within each job. Line 10computes the maximum duration of theentire schedule, which is used in Line 12to set the schedule horizon for OPL. Line

13 declares a decision variable task[j,t]for each job j and task t that is an activ-ity. By default, the keyword Activity

implicitly indicates a decision variable. Anactivity consists of three parts—a starttime, an end time, and a duration. In thedeclaration given here, the durations ofeach activity are given as data. When anactivity is declared, the constraint

task[j,t].start � task[j,t].duration

� task[j,t].end

is automatically included in the system.Line 14 declares an activity makespan ofduration 0 that will be the final activity ofthe schedule. Line 16 declares the ma-chines to be unary resources. A unary re-source is also a decision variable, and weneed to decide what activity will be as-signed to that resource at any particulartime.

The problem is then stated in Lines 18through 31. Lines 18 and 19 indicate thatour objective is to finish the last activity assoon as possible. Lines 21 and 22 indicatethat the last task of each job should pre-cede this final activity. Lines 24 through 26indicate the precedence order of the activi-ties within each job. The word precedes

is a keyword of the language. In the caseof Line 26, the constraint is internallytranslated to the constraint

task[j,t].end ��

task[j,t�1].start

Finally, Lines 28 through 30 indicate therequirement relationship between activi-ties and the machines by using therequires keyword. The declaration ofthe set of requirements causes the creationof a set of disjunctive constraints. For a



particular machine m described by a unaryresource tool[m], let task[j1,t1] andtask[j2,t2] be two activities that re-quire that machine m. In other words, sup-pose that the data is given such thatresource[j1,t1]�resource[j2,t2]�m.Then the following disjunctive constraint,created automatically by the system, de-scribes the fact that the two activities can-not be scheduled at the same time:

task[j1,t1].end��task[j2,t2].start\/

task[j2,t2].end��task[j1,t1].start

Here, the symbol “\/” means “or,” andthe constraint states that eithertask[j1,t1] precedes task[j2,t2] orvice versa.

In practice, the kinds of schedulingproblems that are solved using constraintprogramming technologies have more var-ied characteristics than the simple job shopscheduling problem. Activities can bebreakable, allowing the representation ofactivities that must be inactive on week-ends. Resources can be single machines,discrete resources such as a budget, or res-ervoirs that are both consumed and pro-duced by different activities. Constraintprogramming is an effective technologyfor solving these kinds of problems for anumber of reasons. First, the nature andstrength of the constraint propagation anddomain reduction algorithms developedspecifically for scheduling help to immedi-ately prune the search space and deter-mine bounds on the start and end times ofactivities. Second, in practice one does notnecessarily need to find a provably opti-mal solution to such problems but toquickly find a good feasible solution. Thearchitecture of a constraint programming

system is suited to finding such solutions,and problems with millions of activitieshave been successfully solved in this way.Conclusions

Our experience indicates that constraintprogramming is better than integer pro-gramming in applications that concern se-quencing and scheduling, and for prob-lems in which an integer programmingformulation contains much symmetry. Inaddition, strict feasibility problems thatare in essence constraint satisfaction prob-lems are good candidates for applyingconstraint programming techniques. Inte-ger programming seems to be superior forproblems in which the linear program-ming relaxations provide strong boundsfor the objective function. Hybrid tech-niques are relatively new, although Jainand Grossman [2000] describe their effec-tive use in some machine-schedulingapplications.

Sessions at recent INFORMS meetingsgive evidence of growing interest in howconstraint programming technologies andmathematical programming approachescan complement each other. Because of theintroduction of such languages as OPLand such systems as ECLiPSe, we can eas-ily explore alternative and hybrid ap-proaches to solving difficult problems. Wethink that these explorations will continueand that additional difficult industrialproblems will be solved using combina-tions of the two techniques. We hope thatoperations research professionals will be-come as familiar with constraint program-ming as they are today with linearprogramming.Acknowledgments

We would like to acknowledge the com-

LUSTIG, PUGET

INTERFACES 31:6 52

ments of the two anonymous referees aswell as the copyeditor. Their commentsgreatly improved the presentation of thispaper.ReferencesBisschop, Johannes and Meeraus, Alexander

1982, “On the development of a general alge-braic modeling system in a strategic planningenvironment,” Mathematical ProgrammingStudy 20, pp. 1–29.

Brearley, A. L.; Mitra, Gautam; and Williams,H. P. 1975, “Analysis of mathematical pro-gramming problems prior to applying thesimplex algorithm,” Mathematical Program-ming, Vol. 8, No. 1, pp. 54–83.

Caseau, Yves and Laburthe, F. 1995, “TheClaire documentation,” LIENS Report 96-15,Ecole Normale Superieure, Paris.

Colmerauer, Alain 1990, “An introduction toPROLOG III,” Communications of the ACM,Vol. 33, No. 7, pp. 70–90.

Dantzig, George B. 1948, “Programming in alinear structure,” Comptroller, USAF, Wash-ington, DC, February.

Dantzig, George B. 1949, “Programming ofinterdependent activities, II, mathematicalmodel,” Econometrica, Vol. 17, Nos. 3 and 4(July–October), pp. 200–211.

Dantzig, George B. 1963, Linear Programmingand Extensions, Princeton University Press,Princeton, New Jersey.

Dantzig, George B. and Thapa, Mukund N.1997, Linear Programming 1: Introduction,Springer, New York.

Dash Associates 2000, XPRESS-MP OptimiserSubroutine Library XOSL Reference Manual(Release 12), Dash Associates, Blisworth,United Kingdom.

Dincbas, Mehmet; Van Hentenryck, Pascal;Simonis, Helmut; Aggoun, Abderrahmane;Graf, T.; and Berthier, F. 1988, “The con-straint logic programming language CHIP,”Proceedings of the International Conference onFifth Generation Computer Systems, Tokyo,Japan, December.

Fourer, Robert; Gay, David M.; and Kernighan,Brian W. 1993, AMPL: A Modeling Languagefor Mathematical Programming, Scientific Press,San Francisco, California.

Garfinkel, Robert S. and Nemhauser, George L.1972, Integer Programming, John Wiley and

Sons, New York.Gusfield, Dan and Irving, Robert W. 1989, TheStable Marriage Problem: Structure and Algo-rithms, MIT Press, Cambridge,Massachusetts.

Harvey, William D. and Ginsberg, Matthew L.1995, “Limited discrepancy search,” Proceed-ings of the International Joint Conference on Ar-tificial Intelligence (IJCAI), Vol. 1, pp. 607–613.

Hooker, John 2000, Logic-Based Methods for Opti-mization: Combining Optimization and Con-straint Satisfaction, John Wiley and Sons, NewYork.

IBM 1990, Optimization Subroutine Library Guideand Reference (Release 1), IBM, Kingston, NewYork.

ILOG 1999, ILOG Solver 4.4 Users Manual,ILOG, Gentilly, France.

ILOG 2000, ILOG CPLEX 7.0 Reference Manual,ILOG, Gentilly, France.

Jaffar, Joxan and Lassez, Jean-Louis 1987, “Con-straint logic programming,” Conference Recordof the Fourteenth Annual ACM Symposium onPrinciples of Programming Languages, Munich,Germany, pp. 111–119.

Jain, Vipul and Grossman, Ignacio 2000, “Algo-rithms for hybrid MILP/CP models for aclass of optimization problems,” working pa-per, Department of Chemical Engineering,Carnegie Mellon University.

Knuth, Donald E. 1968, Fundamental Algorithms,The Art of Computer Programming, Vol. 1, sec-ond edition, Addison-Wesley, Reading,Massachusetts.

Lauriere, Jean-Louis 1978, “A language and aprogram for stating and solving combinato-rial problems,” Artificial Intelligence, Vol. 10,No. 1, pp. 29–127.

Lawler, Eugene L. and Wood, D. E. 1966,“Branch-and-bound methods: A survey,”Operations Research, Vol. 14, No. 4, pp. 699–719.

Mackworth, Alan K. 1977, “Consistency in net-works of relations,” Artificial Intelligence, Vol.8, No. 1, pp. 99–118.

Marriott, Kim and Stuckey, Peter J. 1999, Pro-gramming with Constraints: An Introduction,MIT Press, Cambridge, Massachusetts.

Meseguer, Pedro 1997, “Interleaved depth-firstsearch,” Proceedings of the International JointConference on Artificial Intelligence (IJCAI), Vol.2, pp. 1382–1387.



Nemhauser, George W. and Wolsey, LaurenceA. 1988, Integer Programming, John Wiley andSons, New York.

Nemhauser, George W.; Savelsbergh, MartinW. P.; and Sigismondi, Gabriele C. 1994,“MINTO, a Mixed INTeger Optimizer,”Operations Research Letters, Vol. 15, No. 1, pp.47–58.

Nilsson, Nils J. 1971, Problem Solving Methods inArtificial Intelligence, McGraw-Hill, NewYork.

Puget, Jean-Francois 1992, “Pecos: A high levelconstraint programming language,” Proceed-ings of the Singapore International Conference onIntelligent Systems (SPICIS), Singapore, pp.137–142.

Puget, Jean-Francois and Leconte, Michel 1995,“Beyond the glass box: Constraints as ob-jects,” Logic Programming: Proceedings of the1995 International Symposium, ed. John Lloyd,MIT Press, Cambridge, Massachusetts, pp.513–527.

Siskind, Jeffrey M. and McAllester, David A.1993, “Nondeterministic Lisp as a substraitfor constraint logic programming,” Proceed-ings of the Twelfth National Conference on Arti-ficial Intelligence (AAAI-93), pp. 133–138.

Smolka, G. 1993, “Survey of Oz: A higher-orderconcurrent constraint language,” Proceedingsof ICLP ’93: Post-Conference Workshop of Con-current Constraint Programming, Budapest,Hungary, June 24–25, 1993.

Van Hentenryck, Pascal 1989, Constraint Satis-faction in Logic Programming, MIT Press, Cam-bridge, Massachusetts.

Van Hentenryck, Pascal 1999, The OPL Optimi-zation Programming Language, MIT Press,Cambridge, Massachusetts.

Van Hentenryck, Pascal; Deville, Yves; andTeng, C. M. 1992, “A generic arc-consistencyalgorithm and its specializations,” ArtificialIntelligence, Vol. 57, No. 2, pp. 291–321.

Wallace, Mark; Novello, Stefano; and Schimpf,Joachim 1997, “ECLiPSe: A platform for con-straint logic programming,” ICL Systems Jour-nal, Vol. 12, No. 1, pp. 159–200.

Walsh, Toby 1997, “Depth-bounded discrep-ancy search,” Proceedings of the InternationalJoint Conference on Artificial Intelligence(IJCAI), Vol. 2, pp. 1388–1393.

Williams, H. P. 1999, Model Building in Mathe-matical Programming, fourth edition, John Wi-ley and Sons, New York.

Program Does not Equal Program: Constraint Programming and Its Relationship to Mathematical...

Education

Transcript of Program Does not Equal Program: Constraint Programming and Its Relationship to Mathematical...