Using Symmetry to Solve Difficult Logic Puzzles

Igor MarkovUniversity of Michigan, EECS

Outline

A brief introduction to the field of Electronic Design Automation

Integrated circuits, design tools, research challengesFormal verification of digital circuits via SAT-solving (difficult logic puzzles)Exploiting symmetry as a structural propertyof SAT instances

The graph automorphism problemSymmetry-breaking predicatesImprovements in SAT-solving

portablemedia center

From The Old Personal Computer To…

pocket PC

Xbox 360

cable TV boxRoundTable

Zune

tablet PC

media center PC

Laptop PC

smart phone

portablemedia center

IPphone

desktop PCMS Surface

Source: Rico Malvar @ Microsoft Research

Source: Information Week / CMPNet

Designing New Chips QuicklyIdea 1: describe ICs in HW descr. languages (HDL)

Verilog, SystemVerilog, C, SystemC Idea 2: use software tools to compile HDL into silicon

Many intermediate representations, many passes

module foo(a, b, c, o1, o2, o3);input a, b, c;output o1, o2, o3;reg o2, o3;assign o1= a & b;always @(a, b, c) o2= a | c;…

wire sel, a, b, out;…

if (sel) out=a; else out=b;

b

a

sel

AND

AND

outOR

RTL Model

Logic SynthesisGate-level Netlist

Place & Route

Polygonal layout

Logic Synthesis & Physical OptimizationIdea 3: use software tools to optimize chips

Functional (logic) optimization A*B+A*C A*(B+C), (A+B)*(A+C) A+B*C, X1 X’

(Boolean & polynomial algebras)Physical optimization: gate locations, wire routes, etc

(computational geometry, non-convex optimization)

Source: IBM

After the Design Phase

Idea 4: use software tools to verify chipsFind bugs or prove that there are no bugs

Idea 5: use software toolsto optimize test sequencesIdea 6: use software to debug chips

Localize & diagnose bugsAutomatically suggest bug fixes

Verifying a Buggy DesignExample: Carry Select Adder

0

Design error affects results for large numbers

10

Verify Against Trusted Implementation

0

Check equivalence of output functions

+H.Katebi, I.Markov, “Large- scale Boolean Matching” DATE 2010

11

Verifying Equivalence Between Circuits

Too many input combinations to simulate16-bit adder: 232 (feasible)32-bit adder: 264 (impractical)

Key idea: formulate a theorem: x F(x)=G(x)Prove it or find a counter-exampleUse algorithms and software for theorem-proving

Further steps Agree on a simple, formal language for theoremsFormulate theorems for circuit equivalence-checkingDevelop algorithms for theorem-proving

12

Problem Transformation Miter function MFG(x) = ( F(x)G(x) ) = ( F(x) G(x) )

x F(x)G(x) x MFG(x)==1

Theorem-proving reduces to search to satisfy MFG(x)==1 – any such solution is a counter-example (bug!)Or prove that such an assignment does not exist,i.e., prove MFG(x) = 0 for all possible assignments

Circuit-SAT: satisfy one output of a circuitFind an input combination that produces 1Or prove that the output is always 0

Boolean SATisfiability (CNF-SAT)MFG(x) is represented in product-of-sums (POS) form,a.k.a conjunctive normal form (CNF)Example: (a+b+c)(a’+b’+d)(b+c)(d’)(b+e)(c+d+e)

13

CNF-SAT Example/Terminology

ClausePositive Literal

Negative Literal

( a +c ) ( b +c ) (a +b’ + c’ )

14

Ex: Circuit-SAT CNF-SAT, Solveab

c

d

e g

f

h?

= h [d=(ab)’] [e=(b+c)’] [f=d’] [g=d+e] [h=fg]= h

(a +d)(b +d)(a’ + b’ + d’)(b’ +e’)(c’ + e’)(b +c + e)(d’ +f’)(d +f)(d’ +g)(e’ +g)(d +e + g’)(f +h’)(g +h’)(f’ +g’ + h)

= h(a +d)(b +d)(a’ + b’ + d’)(b’ +e’)(c’ + e’)(b +c + e)(d’ +f’)(d +f)(d’ +g)(e’ +g)(d +e + g’)(f +h’)(g +h’)(f’ +g’ + h)

= h(a +d)(b +d)(a’ +b’ + d’)(b’ +e’)(c’ + e’)(b +c + e)(d’ +f’)(d +f)(d’ +g)(e’ +g)(d +e + g’)(f +h’)(g + h’)(f’ +g’ + h)

= h(a +d)(b +d)(a’ +b’ + d’)(b’ + e’)(c’ +e’)(b +c + e)(d’ + f’)(d +f)(d’ +g)(e’ +g)(d +e + g’)(f +h’)(g +h’)(f’ +g’ + h)

= h(a +d)(b +d)(a’ +b’ + d’)(b’ +e’)(c’ +e’)(b +c + e)(d’ +f’)(d +f)(d’ +g)(e’ +g)(d +e + g’)(f +h’)(g +h’)(f’ +g’ + h)

No branching needed in this example

Only constraint propagationLinear time

In general, branching is neededWorst-case exponential timeIn practice, often finishes quickly

15

CNF-SAT

CNF-SAT is NP-completeA polynomial-time algorithm would immediately solvenumerous important problemsAn impossibility proof would also be interesting

A general solution to SAT is worth $1M (the first Millennium problem) + famePractical SAT-solving

Dramatic progress in the last 15 yearsAlgorithms and software exist that solve many large SAT instances quicklySome small SAT instances remain difficult

SAT Can Capture Logic Puzzles (1)

The Pigeonhole Principle (PHP)Need to assign n+1 objects to n slotsNo two objects can be assigned to the same slotEvery object must be assigned to at least one slot

Encoding in terms of CNF-SATn(n+1) indicator variables matching each object with each slotn2 (n+1) mutual exclusion clauses preventing two objects from being assigned to any one slot(n+1) clauses with n literals ensure that each object

is assigned to at least one slot

Summary: PHPn ~n2 variables, ~n3 clauses, UNSAT

SAT Can Capture Logic Puzzles (2)

The Pigeonhole Principle arises in numerous applications

Trying to multiplex (n+1) signals through n outputsTrying to route (n+1) wires through n channels, etc

A competent SAT solver should handle PHPn, shouldn’t it?Relevant observations & (deep) results

All leading SAT-solvers rely on resolution (proof technique)All resolution proofs of the pigeonhole principle are exp-sizedIn practice, runtime on PHPn instances is indeed exponential

But … all pigeons are interchangeable and so are all holes !

Order holes, and impose ordering constraints on assignments

Speeding up SAT-solving in Practice

Key idea: identify structural properties of a given problem instance, then exploit them to speed up the SAT solverKey structural property of problem instances: symmetry Symmetry-breaking in search: if a, b are interchangeable, add the (a’+b) clause to reduce the branching factor

Leading-edge SAT solvers are based on branchingChallenges

Dealing with more complicated symmetries (cycles, negations)Capturing all symmetries efficientlyDetecting symmetries quicklyLimiting the size of symmetry-breaking predicates

Darga, Sakallah, Markov, “Faster Symmetry Discovery using Sparsity of Symmetries,” DAC 2008 http://vlsicad.eecs.umich.edu/BK/SAUCY/

CNF inst. GraphGraph

symmetries

CNFsymmetries

Pre-processedCNF instance

Invoke a generic SAT solver

Symmetry-breaking predicates

Shatter

Shatter

Shatter

GraphAuto: nauty or saucy

Computational Symmetry-Breaking in SAT

Aloul, Markov, Sakallah, “Shatter:

Efficient Symmetry-Breaking for

Boolean Satisfiability,” DAC 2003

Igor Markov / June 30, 2004 21

Compact Representations of Symmetries To represent the group of all symmetries of a graph

Do not list individual symmetries List generating permutations (generators)

Elementary group theory proves:If redundant generators are avoided, an N-element groupcan be represented by at most log2(N) generators

Guaranteed exponential compression (often better !)

E.g., the group Sk of all k! permutations of 1..kcan be generated by (12) and (123..k)or by (12), (23), (34),…, (k-1 k)

To use this guaranteed compression, we rely on algorithms in terms of permutation generators

Finding Symmetries of GraphsSymmetry (automorphism) of a graph

Permutation of vertices that maps edges to edges Additional constraints

Vertex colors (labels): integersEvery vertex must map into a vertex of same color

Computational Graph AutomorphismFind generators of a graph’s group of symmetriesGraphAuto NP, and is believed to P and NP-completeAlgorithms implemented in GAP(GRAPE(NAUTY))

1

2

1

2

A B

D C

Symmetries of CNF Formulae

Permutations of variables that map clauses to clauses

E.g., symmetries of (a+b+c)(d+e+f)include (ab), (abc) as well as (ad)(be)(cf)Considering single swaps only is not enough

Ditto for variable negations (aa’) and negations+permutations

E.g., symmetries of (a+b+c)(d+e’+f’)include (de’) as well as (ad)(be’)(cf’)

CNF formula colored graph Linear time and space

Find graph’s [colored] symmetriesWorst-case exponential time

Interpret graph symmetries found as symmetries of the CNF formula

Permutational symmetriesVariable-negation symmetries

Reduction to Graph Automorphism

Reduction to Graph Automorphism

Clauses: A (x’ + y + z) , B (x + y’ + z’), C (y’ + z)

Vertices of two colors: clauses and varsOne vertex per clause, two per variable

Edges of three types: (i) incidence,(ii) consistency, and (iii) 2-literal clauses

1

2

1

2

A B

C

x x’ y y’ z z’

Symmetry:(x x’)(y z’)(y’ z)

2 222

Consistency edges must map to consistency edges

we do not explicitly enforce that previous reductions do create larger graphswe reduce the input size for GraphAuto by a constant; recall that O(2n)O(2cn)

Must ensure correctness (!)a graph symmetry that maps consistency edgessomewhere else is termed spuriousspurious symmetries can and do happen

Reduction to Graph Automorphism

Igor Markov / June 30, 2004 27

Symmetry Breaking using NAUTY (=“No AUTomorphisms, Yes ?)

On all but the synthetic Urquhart instances,symmetry detection with nauty dominates run timeFurther improvements must come from improved symmetry detection

74.631.518.0123.502pipe82.613.842.4111.43XOR99.426.030.1725.86ChnlRoute93.93.450.213.24FPGAroute88.443.845.0838.76GRoute39.41.931.170.76Urq84.40.450.070.38Hole-n

% SymTotalSearchSymBenchmark

saucy 2004: Exploiting Graph Structure

nauty works very well on small graphsbut fails to scale

Runs out of memory on formulas with corresponding graphs having >50,000 vertices

saucy improvement #1: sparse representationsaucy improvement #2: can use bipartiteness

Clause vertices only connected to literalsNever connected to each other

saucy improvements #3 and #4: algorithmic(e.g., asymptotically faster partition refinement)Further dramatic improvements in saucy 2008, saucy 2010

saucy 2004: Empirical PerformanceTestcase

s#Variabl

esSAT (s) nauty

(s)% Sym saucy(s) % Sym

s4-4-3-1 10354 218.53 88.74 28.9 0.11 0.05

s4-4-3-2 9974 877.59 79.67 8.3 0.10 0.01

s4-4-3-3 9970 884.78 75.98 7.9 0.09 0.01

s4-4-3-4 10714 464.46 155.31 25.1 0.14 0.03

s4-4-3-5 11072 134.09 101.63 43.1 0.11 0.08

s4-4-3-6 9620 13.24 76.48 85.2 0.10 0.75

s4-4-3-7 10362 18.27 78.96 81.2 0.10 0.54

s4-4-3-8 6608 0.68 28.42 97.7 0.06 8.11

2pipe 3575 0.13 2.93 95.8 0.02 13.33

3pipe 10048 6.44 57.53 89.9 0.13 1.98

4pipe 21547 153.50 523.64 77.3 0.49 0.32

5pipe 38746 122.85 3144.85 96.2 1.65 1.33

http://vlsicad.eecs.umich.edu/BK/SAUCY/

saucy 2: Further Dramatic Improvements

Symmetries can be sparse too, especially the generators

In many cases, runtime now grows linearly with #generators

Early detection of hopeless branches during searchThe algorithm now races toward early termination conditionMaintain additional state to speed up some tasks:

Checking the termination conditionChecking that a permutation is a symmetryBacktracking

Several other improvements

Experiments: Symmetry Discovery Time (s)Testcases #Vertices #Generators Saucy 2004 Saucy 2.0

5pipe 38746 239 0.83 0.086pipe 65839 346 2.22 0.157pipe 100668 473 4.80 0.29LA 436535 12852 528.39 0.21IL 819138 14999 958.80 0.43CA 1679418 44439 > 30 min 0.84adaptec1 393964 15683 966.48 0.35adaptec2 471054 21788 > 30 min 0.47adaptec3 800506 36289 > 30 min 0.93adaptec4 878800 53857 > 30 min 0.99

CNF inst. GraphGraph

symmetries

CNFsymmetries

Pre-processedCNF instance

Invoke a generic SAT solver

Symmetry-breaking predicates

Shatter

Shatter

Shatter

GraphAuto: nauty or saucy

Computational Symmetry-Breaking in SAT

Aloul, Markov, Sakallah, “Shatter:

Efficient Symmetry-Breaking for

Boolean Satisfiability,” DAC 2003

How Backtrack SAT Solvers WorkLet’s solve (x’+y’)(x+y)(x’+y+z)(x+y’+z)(z’)

Try x=0Þ y=1Þ z=1 Þ violated clause, must backtrack

We just repeated similar/same steps twiceBranches x=0,y=1 and x=1,y=0 are symmetricIdeas for speed-up

Require that (x≤y), i.e., (x’+y)Consider equivalence classes under symmetryPick 1+ representative for each class, search only the reps.

This restricted search is to original

Try x=1Þ y=0Þ z=1 Þ violated clause, must

backtrack

Symmetry-Breaking Predicates

To restrict searchAdd clauses to the original CNF formula

(“symmetry-breaking” clauses)They will pick representatives of classes

Our main task is to find those clausesUse only permutations induced by generatorsPermutation group of clauses(a “symmetry-breaking” predicate)

Symmetry-Breaking Predicates (SBPs)

SBP

Classes of symmetric

truth assignments

SATisfying assignments

Generator-Based SBPsWhen adding SBPs, must ensure

No solutions are added (trivial)When solutions exist, SBPs should preserve at least one

Optimization objectiveTotal runtime of Shatter flowNot the “completeness” of symmetry-breaking – it’s OK to leave multiple symmetric solutions

TechniqueProcess each generator from GraphAuto independentlyIgnore the CNF-SAT instance, initiallyConcatenate the resulting predicates to the original CNF

Ensuring the Compatibility of SBPs

Input: permutation (usually sparse)Output: clauses that pick representativesof equivalence classes of truth assignmentsChallenge: ensuring compatibility of SBPsproduced for different generators

When conjoining SBPs, we must not lose all solutions

g1g

2

g1g

2

Lex-Leader SBPs (1)

Order the variables in the CNF formulaInduce a lexicographic ordering on truth assignments

000<001<010<011<100<101<110<111For each equivalence class, make sure that its lex-smallest element (lex-leader) is selectedTheorem: Given a CNF formula , consider its symmetry generators {gi} and build Lex-Leader SBPs {P(gi)}.Then and P(g1)P(g2)… are equi-satisfiable.Proof: No new solutions are added. If a solution exists, then the lex-smallest solution is preserved.

Lex-Leader SBPs (2)

Pg(x)= (x ≤ g(x))Where x is a truth assignment and ≤ is a lex-ordering

Prior work (in the field of AI)Defined Pg(x), but implemented it using (n2) CNF clausesDid not account for variable-negation symmetriesConjoined Pg(x) for all gAut(G)

Optimizations & improvements1. Account for negation symmetries & hybrids2. Implement Pg (x) using O(n) CNF clauses3. Implement Pg (x) using O(supp(g)) CNF clauses4. Instantiate Pg (x) only for generators of Aut(G)

Lex-Leader SBPs

Igor Markov / June 30, 2004 42

Asymptotic Results on PHPn

Proving the pigeon-hole principle w/o inductionA series of SAT instances of growing sizeConventional SAT solvers take exponential time

Our approach empirically takes polynomial time

CNF inst. GraphGraph

symmetries

CNFsymmetries

Pre-processedCNF instance

Invoke a generic SAT solver

Symmetry-breaking predicates

Shatter

Shatter

Shatter

GraphAuto: nauty or saucy

Computational Symmetry-Breaking in SAT

Aloul, Markov, Sakallah, “Shatter:

Efficient Symmetry-Breaking for

Boolean Satisfiability,” DAC 2003

Igor Markov / June 30, 2004 44

Open Challenges

Does saucy run in poly-time on bounded-degree graphs ?

It can probably be extended to do so provablyExtend Shatter to save checkable proofs of unsatisfiability using resolution + symmetriesExtend saucy to perform canonical labeling of graphs

Graph Automorphism in worst-case polynomial time ?Approximate symmetries

Igor Markov / June 30, 2004 45

Q&A

Source: Univ. Nebraska Lincoln,Chemistry Dept