Constraint Programming: modelling

Toby WalshNICTA and UNSW

Golomb rulers

• Mark ticks on a ruler Distance between any two ticks (not just

neighbouring ticks) is distinct

• Applications in radio-astronomy, cystallography, … http://www.csplib.org/prob/prob006

Golomb rulers

• Simple solution Exponentially long ruler Ticks at 0,1,3,7,15,31,63,…

• Goal is to find minimal length rulers turn optimization problem into sequence of satisfaction

problemsIs there a ruler of length m?Is there a ruler of length m-1?….

Optimal Golomb rulers

• Known for up to 23 ticks• Distributed internet project to find large rulers

0,10,1,3

0,1,4,60,1,4,9,11

0,1,4,10,12,170,1,4,10,18,23,25

Solutions grow as approximately O(n^2)

Modelling the Golomb ruler

• Variable, Xi for each tick• Value is position on ruler

• Naïve model with quaternary constraints For all i>j,k>l>j |Xi-Xj| \= |Xk-Xl|

Problems with naïve model

• Large number of quaternary constraints O(n^4) constraints

• Looseness of quaternary constraints Many values satisfy |Xi-Xj| \= |Xk-Xl| Limited pruning

A better non-binary model

• Introduce auxiliary variables for inter-tick distances Dij = |Xi-Xj| O(n^2) ternary constraints

• Post single large non-binary constraint alldifferent([D11,D12,…]). Tighter constraints and denser constraint graph

Other modeling issues

• Symmetry A ruler can always be reversed! Break this symmetry by adding constraint:

D12 < Dn-1,n Also break symmetry on Xi

X1 < X2 < … Xn Such tricks important in many problems

Other modelling issues

• Additional (implied) constraints Don’t change set of solutions But may reduce search significantly

E.g. D12 < D13, D23 < D24, …E.g. D1k at least sum of first k integers

• Pure declarative specifications are not enough!

Solving issues

• Labeling strategies often very important Smallest domain often good idea Focuses on “hardest” part of problem

• Best strategy for Golomb ruler is instantiate variables in strict order Heuristics like fail-first (smallest domain) not

effective on this problem!

Experimental results

Runtime/sec Naïve model Alldifferent model

8-Find 2.0 0.1

8-Prove 12.0 10.2

9-Find 31.7 1.6

9-Prove 168 9.7

10-Find 657 24.3

10-Prove > 10^5 68.3

Something to try at home?

• Circular (or modular) Golomb rulers Inter-tick distance

variables more central, removing rotational symmetry?

• 2-d Golomb rulers

All examples of “graceful” graphs

Summary

• Modelling decisions: Auxiliary variables Implied constraints Symmetry breaking constraints

• More to constraints than just declarative problem specifications!

Case study 2: all interval series

All interval series

• Prob007 at www.csplib.org• Comes from musical composition

Traced back to Alban Berg Extensively used by Ernst Krenek

Op.170 “Quaestio temporis”

All interval series

• Take the 12 standard pitch classes c, c#, d, .. Represent them by numbers 0, .., 11

• Find a sequence so each occurs once Each difference occurs once

All interval series

• Can generalize to any n (not just 12)

Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are all distinct

• Finding one solution is easy

All interval series

• Can generalize to any n (not just 12)

Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are all distinct

• Finding one solution is easy[n,1,n-1,2,n-2,.., floor(n/2)+2,floor(n/2)-1,floor(n/2)+1,floor(n/2)]Giving the differences [n-1,n-2,..,2,1]

Challenge is to find all solutions!

Basic methodology

• Devise basic CSP model What are the variables? What are the

constraints?• Introduce auxiliary variables if needed• Consider dual or combined models• Break symmetry• Introduce implied constraints

Basic CSP model

• What are the variables?

Basic CSP model

• What are the variables?Si = j if the ith note is j

• What are the constraints?

Basic CSP model

• What are the variables?Si = j if the ith note is j

• What are the constraints? Si in [0,n)

All-different([S1,S2,… Sn]) Forall i<i’ |Si+1 - Si| =/ |Si’+1 - Si’|

Will this model be any good? If so, why?If not, why not?

Basic methodology

• Devise basic CSP model What are the variables? What are the

constraints?• Introduce auxiliary variables if needed• Consider dual or combined models• Break symmetry• Introduce implied constraints

Improving basic CSP model

• Is it worth introducing any auxiliary variables? Are there any loose or messy constraints we

could better (more compactly?) express via some auxiliary variables?

Improving basic CSP model

• Is it worth introducing any auxiliary variables? Yes, variables for the pairwise differences

Di = |Si+1 - Si|

• Now post single large all-different constraintDi in [1,n-1]All-different([D1,D2,…Dn-1])

Basic methodology

• Devise basic CSP model What are the variables? What are the

constraints?• Introduce auxiliary variables if needed• Consider dual or combined models• Break symmetry• Introduce implied constraints

Break symmetry

• Does the problem have any symmetry?

Break symmetry

• Does the problem have any symmetry? Yes, we can reverse any sequence

S1, S2, … Sn is an all-inverse seriesSn, …, S2, S1 is also

• How do we eliminate this symmetry?

Break symmetry

• Does the problem have any symmetry? Yes, we can reverse any sequence

S1, S2, …, Sn is an all-inverse seriesSn, …, S2, S1 is also

• How do we eliminate this symmetry?• As with Golomb ruler!

D1 < Dn-1

Break symmetry

• Does the problem have any other symmetry?

Break symmetry

• Does the problem have any other symmetry? Yes, we can invert the numbers in any sequence

0, n-1, 1, n-2, … map x onto n-1-xn-1, 0, n-2, 1, …

• How do we eliminate this symmetry?

Break symmetry

• Does the problem have any other symmetry? Yes, we can invert the numbers in any sequence

0, n-1, 1, n-2, … map x onto n-1-xn-1, 0, n-2, 1, …

• How do we eliminate this symmetry?S1 < S2

Basic methodology

• Devise basic CSP model What are the variables? What are the

constraints?• Introduce auxiliary variables if needed• Consider dual or combined models• Break symmetry• Introduce implied constraints

Implied constraints

• Are there useful implied constraints to add?

Implied constraints

• Are there useful implied constraints to add? Hmm, unlike Golomb ruler, we only have

neighbouring differences So, no need to consider transitive closure

Implied constraints

• Are there useful implied constraints to add? Hmm, unlike Golomb ruler, we are not

optimizing So, no need to improve propagation for

optimization variable

Performance

• Basic model is poor• Refined model able to compute all solutions

up to n=14 or so GAC on all-different constraints very beneficial As is enforcing GAC on Di = |Si+1-Si|

This becomes too expensive for large nSo use just bounds consistency (BC) for larger n

Case study 3: orthogonal Latin squares

Modelling decisions

• Many different ways to model even simple problems

• Combining models can be effective Channel between models

• Need additional constraints Symmetry breaking Implied (but logically) redundant

Latin square

• Each colour appears once on each row

• Each colour appears once on each column

• Used in experimental design Six people Six one-week drug

trials

Orthogonal Latin squares

• Find a pair of Latin squares Every cell has a

different pair of elements

• Generalized form: Find a set of m Latin

squares Each possible pair is

orthogonal

Orthogonal Latin squares

1 2 3 4 1 2 3 42 1 4 3 3 4 1 23 4 1 2 4 3 2 14 3 2 1 2 1 4 3 11 22 33 44 23 14 41 32 34 43 12 21 42 31 24 13

• Two 4 by 4 Latin squares

• No pair is repeated

History of (orthogonal) Latin squares

• Introduced by Euler in 1783 Also called Graeco-Latin or Euler squares

• No orthogonal Latin square of order 2 There are only 2 (non)-isomorphic Latin

squares of order 2 and they are not orthogonal

History of (orthogonal) Latin squares

• Euler conjectured in 1783 that there are no orthogonal Latin squares of order 4n+2 Constructions exist for 4n and for 2n+1 Took till 1900 to show conjecture for n=1 Took till 1960 to show false for all n>1

• 6 by 6 problem also known as the 36 officer problem“… Can a delegation of six regiments, each of which sends a colonel,

a lieutenant-colonel, a major, a captain, a lieutenant, and a sub-lieutenant be arranged in a regular 6 by 6 array such that no row or column duplicates a rank or a regiment?”

More background

• Lam’s problem Existence of finite projective plane of order 10 Equivalent to set of 9 mutually orthogonal Latin

squares of order 10 In 1989, this was shown not to be possible after 2000

hours on a Cray (and some major maths)• Orthogonal Latin squares also used in

experimental design

A simple 0/1 model

• Suitable for integer programming Xijkl = 1 if pair (i,j) is in row k column l, 0 otherwise Avoiding advice never to use more than 3 subscripts!

• Constraints Each row contains one number in each square

Sum_jl Xijkl = 1 Sum_il Xijkl = 1 Each col contains one number in each square

Sum_jk Xijkl = 1 Sum_ik Xijkl = 1

A simple 0/1 model

• Additional constraints Every pair of numbers occurs exactly once

Sum_kl Xijkl = 1 Every cell contains exactly one pair of numbers

Sum_ij Xijkl = 1

Is there any symmetry?

Symmetry removal

• Important for solving CSPs Especially for proofs of optimality?

• Orthogonal Latin square has lots of symmetry Permute the rows Permute the cols Permute the numbers 1 to n in each square

• How can we eliminate such symmetry?

Symmetry removal

• Fix first row 11 22 33 …

• Fix first column112332..

• Eliminates all symmetry?

What about a CSP model?

• Exploit large finite domains possible in CSPs Reduce number of variables O(n^4) -> ?

• Exploit non-binary constraints Problem states that squares contain pairs that

are all different All-different is a non-binary constraint our

solvers can reason with efficiently

CSP model

• 2 sets of variables Skl = i if the 1st element in row k col l is i Tkl = j if the 2nd element in row k col l is j

• How do we specify all pairs are different? All distinct (k,l), (k’,l’) if Skl = i and Tkl = j then Sk’l’=/ i or Tk’l’ =/ j

O(n^4) loose constraints, little constraint propagation!

What can we do?

CSP model

• Introduce auxiliary variables Fewer constraints, O(n^2) Tightens constraint graph => more propagation Pkl = i*n + j if row k col l contains the pair i,j

• Constraints 2n all-different constraints on Skl, and on Tkl All-different constraint on Pkl Channelling constraint to link Pkl to Skl and

Tkl

CSP model v O/1 model

• CSP model 3n^2 variables Domains of size n, n

and n^2+n O(n^2) constraints Large and tight non-

binary constraints

• 0/1 model n^4 variables Domains of size 2 O(n^4) constraints Loose but linear

constraints• Use IP solver!

Solving choices for CSP model

• Variables to assign Skl and Tkl, or Pkl?

• Variable and value ordering• How to treat all-different constraint

GAC using Regin’s algorithm O(n^4) AC using the binary decomposition

Good choices for the CSP model

• Experience and small instances suggest: Assign the Skl and Tkl variables Choose variable to assign with Fail First

(smallest domain) heuristic• Break ties by alternating between Skl and Tkl

Use GAC on all-different constraints for Skl and Tkl

Use AC on binary decomposition of large all-different constraint on Pkl

Performance

n 0-1 modelFails t/sec

CSP model ACFails t/sec

CSP model GACFails t/sec

4 4 0.11 2 0.18 2 0.38

5 1950 4.05 295 1.39 190 1.55

6 ? ? 640235 657 442059 773

7* 20083 59.8 91687 51.1 57495 66.1

Case study 4: Langford’s problem

Langford’s problem

• Prob024 @ www.csplib.org

• Find a sequence of 8 numbers Each number [1,4]

occurs twice Two occurrences of i

are i numbers apart• Unique solution

41312432

Langford’s problem

• L(k,n) problem To find a sequence of k*n

numbers [1,n] Each of the k successive

occrrences of i are i apart We just saw L(2,4)

• Due to the mathematician Dudley Langford Watched his son build a

tower which solved L(2,3)

Langford’s problem

• L(2,3) and L(2,4) have unique solutions• L(2,4n) and L(2,4n-1) have solutions

L(2,4n-2) and L(2,4n-3) do not Computing all solutions of L(2,19) took 2.5 years!

• L(3,n) No solutions: 0<n<8, 10<n<17, 20, .. Solutions: 9,10,17,18,19, ..

A014552Sequence: 0,0,1,1,0,0,26,150,0,0,17792,108144,0,0,39809640,326721800,

0,0,256814891280,2636337861200

Basic model

• What are the variables?

Basic model

• What are the variables?Variable for each occurrence of a number

X11 is 1st occurrence of 1X21 is 1st occurrence of 2..X12 is 2nd occurrence of 1X22 is 2nd occurrence of 2..

• Value is position in the sequence

Basic model

• What are the constraints? Xij in [1,n*k] Xij+1 = i+Xij Alldifferent([X11,..Xn1,X12,..Xn2,..,X1k,..Xnk

])

Recipe

• Create a basic model Decide on the variables

• Introduce auxiliary variables For messy/loose constraints

• Consider dual, combined or 0/1 models

• Break symmetry• Add implied constraints• Customize solver

Variable, value ordering

Break symmetry

• Does the problem have any symmetry?

Break symmetry

• Does the problem have any symmetry? Of course, we can invert any sequence!

Break symmetry

• How do we break this symmetry?

Break symmetry

• How do we break this symmetry? Many possible ways For example, for L(3,9)

• Either X92 < 14 (2nd occurrence of 9 is in 1st half)• Or X92=14 and X82<14 (2nd occurrence of 8 is in

1st half)

Recipe

• Create a basic model Decide on the variables

• Introduce auxiliary variables For messy/loose constraints

• Consider dual, combined or 0/1 models

• Break symmetry• Add implied constraints• Customize solver

Variable, value ordering

What about dual model?

• Can we take a dual view?

What about dual model?

• Can we take a dual view?• Of course we can, it’s a permutation!

Dual model

• What are the variables? Variable for each position i

• What are the values?

Dual model

• What are the variables? Variable for each position i

• What are the values? If use the number at that position, we cannot

use an all-different constraint Each number occurs not once but k times

Dual model

• What are the variables? Variable for each position i

• What are the values? Solution 1: use values from [1,n*k] with the

value i*n+j standing for the ith occurrence of j Now want to find a permutation of these

numbers subject to the distance constraint

Dual model

• What are the variables? Variable for each position i

• What are the values? Solution 2: use as values the numbers [1,n] Each number occurs exactly k times Fortunately, there is a generalization of all-different

called the global cardinality constraint (gcc) for this

Global cardinality constraint

• Gcc([X1,..Xn],l,u) enforces values used by Xi to occur between l and u times All-different([X1,..Xn]) = Gcc([X1,..Xn],1,1)

• Regin’s algorithm enforces GAC on Gcc in O(n^2.d) Regin’s papers are tough to follow but this

seems to beat his algorithm for all-different!?

Dual model

• What are the constraints? Gcc([D1,…Dk*n],k,k) Distance constraints?

Dual model

• What are the constraints? Gcc([D1,…Dk*n],k,k) Distance constraints:

• Di=j then Di+j+1=j

Combined model

• Primal and dual variables• Channelling to link them

What do the channelling constraints look like?

Combined model

• Primal and dual variables• Channelling to link them

Xij=k implies Dk=i

Solving choices?

• Which variables to assign? Xij or Di

Solving choices?

• Which variables to assign? Xij or Di, doesn’t seem to matter

• Which variable ordering heuristic? Fail First or Lex?

Solving choices?

• Which variables to assign? Xij or Di, doesn’t seem to matter

• Which variable ordering heuristic? Fail First very marginally better than Lex

Recipe

• Create a basic model Decide on the variables

• Introduce auxiliary variables For messy/loose constraints

• Consider dual, combined or 0/1 models

• Break symmetry• Add implied constraints• Customize solver

Variable, value ordering