Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van...
-
Upload
gavin-baker -
Category
Documents
-
view
213 -
download
0
Transcript of Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van...
![Page 1: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/1.jpg)
Towards theoretical frameworks for comparing constraint satisfaction models and algorithms
Peter van Beek, University of WaterlooCP 2001 · Paphos, CyprusNovember 2001
![Page 2: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/2.jpg)
2
Acknowledgements
Joint work with:
Fahiem Bacchus
Xinguang Chen
Grzegorz Kondrak
Toby Walsh
![Page 3: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/3.jpg)
3
Constraint programming methodology
Model problem·specify in terms of constraints on acceptable solutions
·define/choose constraint model: variables, domains, constraints
Solve model·define/choose algorithm·define/choose heuristics
Verify and analyze solution
![Page 4: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/4.jpg)
4
Previous work
Many models, algorithms, and heuristics proposed
Choice of model, algorithm, and heuristic can greatly influence efficiency
(e.g., Nadel, 1990; Ginsberg et al., 1990;
Frost and Dechter, 1994; Tsang et al., 1995 Smith et al., 2000; Beacham et al., 2001)
![Page 5: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/5.jpg)
5
How to compare?
Empirical studies·benchmark sets·random problems
Theoretical studies·worst case·average case
(e.g. Brown & Purdom, 1981;
Nadel, 1983; Wilf, 1984)
·pair-wise comparisons-partial orders-bounded worse
Performance measures
·nodes visited·constraint checks·CPU time
![Page 6: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/6.jpg)
6
Part I
Comparing backtracking algorithms
![Page 7: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/7.jpg)
7
Some backtracking algorithms
Chronological Backtracking (BT)
Backjumping (BJ)(Gaschnig, 1978)
Conflict-Directed Backjumping (CBJ)(Prosser, 1993)
Forward Checking (FC)(Haralick & Elliott, 1980; McGregor, 1979)
Maintaining arc consistency (MAC)(Haralick & Elliott, 1980; McGregor, 1979; Sabin & Freuder, 1994)
![Page 8: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/8.jpg)
8
Example: 4-queens
4
3
2
1
x1 x2 x3 x4
xi xj and
| xi - xj | | i - j |
![Page 9: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/9.jpg)
9
Search tree for 4-queens
x1
x2x3
x4
1 2 3 4
(1,1,1,1) (4,4,4,4)(2,4,1,3) (3,1,4,2)
![Page 10: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/10.jpg)
10
4
3
2
1
{x1 1}
Backjumping
Q
Q{x1 1, x2
3}
x1
x2
x1
x2
{x1 1, x2 3 , x4
2}
Q
x3
x1 x3 x4x2
![Page 11: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/11.jpg)
11
“New” algorithms: BJ1, …, BJn
BJk :·allowed to backjump at most k times from a leaf·after that it must chronologically backtrack
Special cases:·BJ1 equivalent to BJ·BJn equivalent to CBJ
![Page 12: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/12.jpg)
12
k-consistency
Given ·any k-1 distinct variables,·any consistent assignment to those variables,·any additional variable x
there exists a consistent assignment to x
4
3
2
1
x1 x2 x3 x4
2-consistent ?
Q
Q1-consistent ?
3-consistent ?strongly 2-
consistent
yes
yes
no
![Page 13: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/13.jpg)
13
Maintaining consistency
{ x1 1, x2
4 }
4
3
2
1
x1 x2 x3 x4
Q
Qx3 x4
|x3 - x4| 1
Binary constraints
Unary constraintsx3 1, 3, 4
x4 1, 2, 4
Induced CSP
![Page 14: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/14.jpg)
14
Maintaining consistency
4
3
2
1
x1 x2 x3 x4
Q
Q
{ x1 1, x2
4 }
x3 x4
|x3 - x4| 1
Binary constraints
Unary constraintsx3 1, 3, 4
x4 1, 2, 4
FC: strong 1-consistency
![Page 15: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/15.jpg)
15
Maintaining consistency
Q
Q
{ x1 1, x2
4 }
x3 x4
|x3 - x4| 1
Binary constraints
Unary constraintsx3 1, 3, 4
x4 1, 2, 4
4
3
2
1
x1 x2 x3 x4
MAC: strong 2-consistency
Q
Q
![Page 16: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/16.jpg)
16
“New” algorithms: MC1, …, MCn
MCk :·maintains strong k-consistency on induced CSP at each node in backtrack tree
Special cases:·MC1 equivalent to FC·MC2 equivalent to MAC for binary CSPs
![Page 17: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/17.jpg)
17
Comparing backtracking algorithms
Proposal:for all CSP models for all variable orderings
algorithm a1 algorithm a2
Desirable features:·still holds if use best possible CSP model for a2
·still holds if use best possible variable ordering for a2
![Page 18: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/18.jpg)
18
Theoretical methodology
Using local consistency concepts:·formulate necessary and sufficient conditions for a node to be visited by a backtracking algorithm
Using conditions:·construct partial orders of the algorithms
![Page 19: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/19.jpg)
19
4
3
2
1
x1 x2 x3 x4
{x1 1}
{x1 1, x2
4}?
Example sufficient condition: FC
If a node is consistent and its parent is strongly 1-consistent, then FC visits the node
Q
Q
![Page 20: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/20.jpg)
20
Example necessary condition: MAC-CBJ
If MAC-CBJ visits a node, then it is consistent and its parent is strongly 2-consistent
4
3
2
1
x1 x2 x3 x4
{x1 1}
{x1 1, x2
4}?
Q
![Page 21: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/21.jpg)
21
Example conclusion: MAC-CBJ vs FC
FC visits p
MAC-CBJ visits p
p is consistent and parent(p) strongly 1-consistent
p is consistent and parent(p) strongly 2-consistent
![Page 22: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/22.jpg)
22
Partial order for nodes visited
MCk+1-CBJ
MCk-CBJ
MC1-CBJ
MCn-CBJ
MCk+1
MCk
MC1
MCn
BJk+1
BJk
BJ1
BJn
BT
...
...
...
...
...
...
less than or equal
![Page 23: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/23.jpg)
23
Partial order for nodes visited
MCk+1-CBJ
MCk-CBJ
MC1-CBJ
MCn-CBJ
MCk+1
MCk
MC1
MCn
BJk+1
BJk
BJ1
BJn
BT
...
...
...
......
...
incomparable
![Page 24: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/24.jpg)
24
Partial order for nodes visited
BT
MCk+1-CBJ
MCk-CBJ
MC1-CBJ
MCn-CBJ
MCk+1
MCk
MC1
MCn
BJk+1
BJk
BJ1
BJn
...
...
...
......
...
incomparable
![Page 25: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/25.jpg)
25
Partial order for nodes visited
MCk+1-CBJ
MCk-CBJ
MC1-CBJ
MCn-CBJ
MCk+1
MCk
MC1
MCn
BJk+1
BJk
BJ1
BJn
BT
...
...
...
...
...
...
incomparable
![Page 26: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/26.jpg)
26
Partial order for nodes visited
MCk+1-CBJ
MCk-CBJ
MC1-CBJ
MCn-CBJ
MCk+1
MCk
MC1
MCn
CBJBT
...
......
...
![Page 27: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/27.jpg)
27
Closer look: nodes visited, binary CSPs
MAC-CBJ
FC-CBJ
MAC
FC
CBJ
BJ2
BJ
BT
less than or equal
![Page 28: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/28.jpg)
28
Closer look: constraint checks, binary CSPs
MAC-CBJ
FC-CBJ
MAC
FC
CBJ
BJ2
BJ
BT
less than or equal
at most O(f) more
O(nd)
O(n2d2)
O(nd)
![Page 29: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/29.jpg)
29
Experiments: lookahead
MAC vs FC Faster? CommentsMcGregor (1979) FC 3 fasterHaralick & Elliott (1980) FC 3 fasterSabin & Freuder (1994) MAC much betterBacchus & van Run (1995) FC 3 20 fasterBessiere & Regin (1996) MAC much betterLarrosa (2000) both much better
Path vs MAC Faster? CommentsBacchus & van Run (1995) MAC much better
FC
MAC
O(nd)
Path
BT
O(nd)
O(nd)
![Page 30: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/30.jpg)
30
Example: FC vs MAC
Adapted from J. Ullman (1988); suggested by D. De Haan
Odd-even relation
x y1 21 42 12 33 23 44 14 3
variables: x1, …, xn, n odd
domains: {1, 2, 3, 4}
x1 x2 xn...
FC: 2n+1 - 4
MAC: 4
![Page 31: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/31.jpg)
31
Increasing lookahead
Problem MC2 MC3 MC4
1 > 3600 262 602 794 40 1403 7 3 34 1065 21 25 34 17 26 > 3600 125 37 58 6 48 17 3 29 163 95 210 87 4 2
Time (sec.) to solve 6 x 6 puzzlesMC4
MC3
MC2
MC1
![Page 32: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/32.jpg)
32
Experiments: lookback
FC vs FC-CBJ Faster? CommentsProsser (1993) FC-CBJ three times betterFrost & Dechter (1994) FC-CBJ somewhat betterBacchus & van Run (1995) FC-CBJ slightlySmith & Grant (1995) FC-CBJ sometimes much betterBayardo & Schrag (1996, 1997) FC-CBJ much betterLi & Anbulagan (1997) FC as good or better
FC-CBJFC
![Page 33: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/33.jpg)
33
Experiments: lookahead vs lookback
Increasing the level of consistency decreases effectiveness of CBJ
·Prosser (1993)·Bacchus & van Run (1995) ·Bessiere & Regin (1996)
MAC much better than FC-
CBJ
MAC better than MAC-CBJ
MAC-CBJMAC
FC-CBJFC
O(nd)
CBJBT
O(nd)
“CBJ becomes useless”
· Jussien, Debruyne, Boizumault (2000)MAC-DBT much better than MAC (binary)
·Chen & van Beek (2001)MGAC-CBJ much better than MAC (non-binary)
![Page 34: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/34.jpg)
34
Theory: lookahead vs lookback
MCk+1-CBJ
MCk-CBJ
MCk+1
MCk
BJk+1
BJk
BJk+1 better only if there exists a (k+1)-level backjump that is not a chronological backtrack
Number of (k+1)-level backjumps number of k-level backjumps
Therefore, as k increases, effectiveness of backjumping decreases
![Page 35: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/35.jpg)
35
Part II
Comparing CSP models
![Page 36: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/36.jpg)
36
Conversion to binary
Any non-binary CSP can be converted into one with only binary constraints
·dual graph method (Dechter & Pearl, 1989; Rossi et al. 1989)
·hidden variable method (Peirce, 1933; Rossi et al. 1989; Dechter 1990)
![Page 37: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/37.jpg)
37
Crossword puzzles
1 2 3
6
4
7
5
8
10
9
20
11
22
12
21
13
17
14
181615
23
19
aaardvarkabackabacusabaftabaloneabandon...
Mona Lisamonarchmonarchymonarda...zymurgyzyrianzythum
![Page 38: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/38.jpg)
38
Non-binary model (original)
1 2 3
6
4
7
5
8
10
9
20
11
22
12
21
13
17
14
181615
23
19
variables: one for each
unknown letter (cell)
domains: ‘a’, …, ‘z’
constraints: contiguous
letters must form words in dictionary
![Page 39: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/39.jpg)
39
Dual model (binary)
1 2 3
6
4
7
5
8
10
9
20
11
22
12
21
13
17
14
181615
23
19
variables: one for each unknown word across and down
domains: words from
dictionary
constraints intersecting words must agree on common letter
![Page 40: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/40.jpg)
40
Hidden model (binary)
1 2 3
6
4
7
5
8
10
9
20
11
22
12
21
13
17
14
181615
23
19
variables: one for each unknown letter (cell) and
one for each unknown word across and down
domains: letters, words
constraints: letter and word
variable must agree
![Page 41: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/41.jpg)
41
Comparing models
One proposal:for all backtracking algorithms for all variable orderings
model m1 model m2
Objections:·effectiveness of a model is algorithm dependent
·models may contain different variables
Desirable features:·meets objections·still holds if use best possible variable ordering for m2
Another proposal:given backtrack algorithm for all v2 for m2
there exists a v1 for m1 s. t. model m1 model m2
![Page 42: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/42.jpg)
42
Theoretical methodology
Bound difference in number of nodes visited·compare pruning power of local consistencies of alternative models
(e.g., Debruyne & Bessiere, 1997;
Stergiou & Walsh, 1999; Prosser et al.,
2000; Walsh, 2000)·establish correspondence between nodes
Bound difference in cost at each node:·local consistency·variable ordering
![Page 43: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/43.jpg)
43
Some relationships
MGAC-dual MGAC-hiddenMGAC-orig
FC-dual FC-hiddenFC-orig
BT-dual BT-hiddenBT-orig
polynomial not polynomial
![Page 44: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/44.jpg)
44
Part III
Comparing variable ordering heuristics
Future work
![Page 45: Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus.](https://reader035.fdocuments.us/reader035/viewer/2022081602/5515c9df55034693758b4a67/html5/thumbnails/45.jpg)
45
Conclusion
Technical:·comparing backtracking algorithms
partial order·comparing CSP models
bounded worse relation on algorithm-model combinations
Big picture:·theory can guide, inform experimentation