Contributions à la génération de testsà base de contraintes

Arnaud Gotlieb

INRIA Rennes, France SIMULA Oslo, Norway

Soutenance d’habilitation, 12/12/2011

Several (complementary) techniques at the unit level (code level): software model-checking static-analysis based verification software testing

and Constraint-Based Testing…

Critical software development involves strong V&V requirements:

- At system testing level, safety-related properties have to be checked- At integration testing level, HW/SW integration failures must be detected- At unit testing level, programming faults must be detected and removed

At these levels, conformance to software certification standards is enforced

BCE Rafale – Dassault Electronics

TCAS - Airbus

HVAC - Thales

Java Card - Oberthur

Saturn C90 – Cisco Norway

Embedded Software Testing

Software Testing

Execution Verdict: pass / fail

implementation

Spec. / Model

Code-based TestingCorrect ?

Test case generatio

n

Test set

Model-based Testing

Constraint-Based Testing

Execution Verdict: pass / fail

Spec. / Model

Implementation

Constraint model

Constraint generatio

n

Constraint solving

Test set

Constraint-Based Testing (CBT)

Constraint-Based Testing (CBT) is the process of generating test cases against a testing objective by using constraint solving techniques

Introduced 20 years ago by Offut and DeMillo in(Constraint-based automatic test data generation IEEE TSE 1991)

Developed in the context of code-based testing and model-based testing

Lots of Research works and tools !

CBT: main tools

CEA - List (Osmose S. Bardin P.Herrmann)Univ. of Madrid (PET M. Gomez-Zamalloa, E. Albert, G. Puebla) Univ. of Stanford (EXE D. Engler, C. Cadar, P. Guo)Univ. of Nice Sophia-Antipolis (CPBPV M. Rueher, H. Collavizza, P.V. Hentenryck)INRIA - Celtique (Euclide, JAUT A. Gotlieb, F. Charreteur) …Tools with external industrial usage :

GATEL (CEA B. Marre, since 2004) Test Designer (Smartesting B. Legeard, since 2003)

PEX (Microsoft P. de Halleux, N. Tillmann, since 2009)

Tools with internal industrial usage : Inka V1 (Dassault A. Gotlieb, B. Botella, in 2001) PathCrawler (CEA N. Williams, since

2004) SAGE (Microsoft P. Godefroid, since 2010)

The automatic test data generation problem (1)

(i.e., reachability problem in infinite-state systems)

f (int x1, int x2, int x3) {

if(x1 == x2 && x2 ==x3)

if(x3==x1*x2) ... }

- Select either a path, branch, source code element, or testing criterion- Generate a test input or a test set that covers the element- Predict the expected outputs

Solving this problem would have broad industrial impact:

- increase software quality and reliability through better code coverage and more systematic test inputs generation;

- decrease testing costs through augmented automation;

- automate conformance to Software Certification Standards as they require covering testing criteria (e.g., DO-178C, ISO 2626-1,…) ;

The automatic test data generation problem (2)

Given a location k in a program under test, generate a test input that reaches k

Even when adding bounds, hard combinatorial problem

Using Random Testing, Prob{ reack k} = 2 over 232 - 232 - 232 = 2-95 = 0.00000…1.

Reachability problem in infinite-state systems is undecidable in general!

Loops (i.e., infinite-state systems) and infeasible paths Pointers, dynamic structures, higher-order computations (virtual

calls) Floating-point computations, modular computations

f (int x1, int x2, int x3) {

if(x1 == x2 && x2 ==x3)

if(x3==x1*x2) ... }

Constraint solving techniques are required!

Our thesis since fiveteen years:

Constraint Programming techniques can effectively and efficiently address these problems

Illustrated with 2 selected contributions in my today’s talk:

1) Constraint-based program exploration for automatic test data generation

2) Constraints over memory models for testing pointer programs

Outline

• Motivations

• Constraint-based program exploration for automatic test data generation(contribution 1)

• Constraints over memory models for testing pointer programs(contribution 2)

• Conclusions and further work

f( int i, … ) {a. j = 100; while( i > 1)b. { j++ ; i-- ;}

… d. if( j > 500)e. …

d

b

a

f

t

t

f

A reacheability problem

…

value of i to reach e ?

e

f( int i, … ) {a. j = 100; while( i > 1)b. { j++ ; i-- ;}

… d. if( j > 500)e. …

d

b

a

f

t

t

f

Path-oriented exploration

…

1. Path selection e.g., (a-b)14-…-d-e

2. Path condition generation (via symbolic exec.) j1=100, i1>1, j2=j1+1, i2=i1-1, i2>1,…, j15>500

3. Path condition solving unsatisfiable FAIL

Backtrack !

e

Even without loops, #pathsis exponential with #decisions

f( int i, … ) {a. j = 100; while( i > 1)b. { j++ ; i-- ;}

… d. if( j > 500)e. …

d

b

a

f

t

t

f

Constraint-based program exploration

…

1. Constraint model generation 2. Control dependencies generation; j1=100, i3 ≤ 1, j3 > 500

3. Constraint model solving j1 - j3 entailed unroll the loop 400 times i1 in 401 .. 231-1

No backtrack !

e

Constraint-based program exploration(Contribution 1)

- Based on a constraint model of the whole program (i.e., each statement is seen as a relation )

- Constraint reasoning over control structures

- Requires to build dedicated constraint solvers:

* propagation queue management with priorities

* specific propagators and meta-constraints

* structure-aware labelling heuristics

Prototype tool implementation: Euclide (Gotlieb ICST’09)

Viewing an assignment as a relation requires to normalize expressions and rename variables (through single assignment languages, e.g. SSA)

i*=++i ; i2 = (i1+1)2

Assignment as Constraint

i*=++i; /* i2 = (i1+1) 2 */

i1 = 3 ?

i2 = 16

i1 in -4..2

i2 = 9 ?

i1 in -5..3

i2 in 5..16 ? i2 = 7 ?

no

Using finite-domains bound-consistency filtering:

Statements as constraints

Type declaration: signed long x; x in -231..231-1

Assignments: i*=++i ; i2 = (i1+1)2

Memory and array accesses and updates: v=A[i] ( or p=Mem[&p] ) variations of element/3

Control structures: dedicated meta-constraints (interface, awakening conditions and filtering algorithms)

Conditionnals (SSA) if D then C1; else C2 ite

Loops (SSA) while D do C w

Conditional as meta-constraint: ite/6

ite( x > 0, j1, j2, j3, j1 = 5, j2 = 18 ) iff

if( x > 0 )

3

2

0

j2 = 18;

= …. j3 …

-( x > 0 - j1 = 5 - j3 = j1 ) - -(x > 0) - j2 = 18 - j3 = j2 -( -(x > 0) - j3 = j2 ) - x > 0 - j1 = 5 - j3 = j1 Join( x > 0 - j1 = 5 - j3 = j1 , -(x > 0) - j1 = 18 - j3 = j2 )

x > 0 - j1 = 5 - j3 = j1 -(x > 0) - j2 = 18 - j3 = j2

j1 = 5; 1

Loop as meta-constraint: w/5

v3 = -( v1 , v2 ) while( Dec )

1

2

body3

w(Dec, V1, V2, V3, body) iff DecV3V1 - bodyV3V1 - w(Dec, v2,vnew,v3, bodyV2Vnew) -DecV3V1 - v3=v1

-(DecV3V1 - bodyV3V1 ) - -DecV3V1 - v3=v1

-(-DecV3V1 - v3=v1) - DecV3V1 - bodyV3V1 - w(Dec,v2,vnew,v3,bodyV2Vnew) join(DecV3V1 - bodyV3V1 - w(Dec,v2,vnew,v3,bodyV2Vnew) , -DecV3V1 - v3=v1)

f( int i ) { j = 100; while( i > 1) { j++ ; i-- ;} … if( j > 500) …

w(i3 > 1, (i,j1), (i2,j2), (i3,j3), j2 = j3 + 1 - i2 = i3 - 1)

i = 23, j1=100 ?

i3 = 1, j3 = 122

no

i3 = 10 ?

i in 401..231-1

j1 = 100, j3 > 500 ?

w(Dec, V1, V2, V3, body) :- DecV3V1 - bodyV3V1 - w(Dec, v2,vnew,v3, bodyV2Vnew) -DecV3V1 - v3=v1

-(DecV3V1 - bodyV3V1 ) - -DecV3V1 - v3=v1

-(-DecV3V1 - v3=v1) - DecV3V1 - bodyV3V1 - w(Dec,v2,vnew,v3,bodyV2Vnew)

join(DecV3V1 - bodyV3V1 - w(Dec,v2,vnew,v3,bodyV2Vnew ,

-DecV3V1 - v3=v1)

Features of constraint-based exploration

Special meta-constraints implementation for ite and w

By construction, w is unfolded only when necessary but w may NOT terminate ! only a semi-correct test data generation procedure

Join is implemented using Abstract Interpretation operators (e.g., interval-based union, weak-join operator, widening in Euclide)

Special propagators based on linear-based relaxationsUsing Linear Programming over rationals (i.e., Q_polyhedra)

Abstraction-based relaxations

Abstraction-based relaxations

During constraint propagation, constraints can be relaxed in Abstract Domains (e.g., Q-Polyhedra, Octagons, …)

{ Z - Ya – Xc +ac ≥ 0, Xd – Z –ad + aY ≥ 0, bY – bc – Z + Xc ≥ 0, bd – bY – Xd + Z ≥ 0, a ≤ X ≤ b, c ≤ Y ≤ d}

To benefit from specialized algorithm (e.g., simplex for linear constraints) and capture global states of the constraint system

Require safe/correct over-approximation (to preserve property such as:if the Q-Polyhedra is void then the constraint system is unsatisfiable)

Q-Polyhedra in Euclide, implementing Dynamic Linear Relaxation, propagation queue with priorities

a b

c

dZ = X * Y, X in a..b, Y in c..d

Abstraction-based relaxations:weak-join operator

(Sankaranarayanan et al. VMCAI’06)

Join operations can be realized by convex hull, but usually too costly !In Euclide, we took advantage of the weak-join of Q_polyhedra

(based on simplex calculations)

AB

Abstraction-based relaxations:weak-join operator

(Sankaranarayanan et al. VMCAI’06)

AB

AB

Abstraction-based relaxations:weak-join operator

(Sankaranarayanan et al. VMCAI’06)

Foundations of the approach (Gotlieb Botella Rueher ISSTA’98, CL’2000)

Abstraction-based relaxation (Denmat Gotlieb Ducassé ISSRE’07)

Global constraint w, extended with widenning (Denmat Gotlieb Ducassé CP’07)

Euclide: A Constraint-based testing platform for C (Gotlieb ICST’09)

Application on the TCAS case study (Gotlieb KER Journal 2011)

***PhD Thesis of Tristan Denmat (Defense in June 2007)Post-doc of Pierre Rousseau (2007-2008)

*** Prototype tools implementation: Taupo (INRIA T. Denmat in 2007),

Euclide (INRIA A. Gotlieb in2009)

Constraint-based program exploration(summary of contribution 1)

Constraints over memory models for testing pointer programs

(contribution 2)

How to apply constraint-based reasoning over statement like *p := *p+1 ?

*p := *p + 1

Then fail or exception

Then a2 = a1+1

Then a2 = a1+1 or b2 = b1+1

Then p2 = p1+1, meaning thatp now refers to the next memory location

b

p a

p a

p

p

Our propositions

How to represent abstract memories and to reason on them ?

1) Constraint reasoning over Memory, as a set of graphs (Gotlieb et al., IST 2007)

p

a1 ana2....

2) Constraint reasoning over Memory, as a structured set of unbounded arrays

(Charreteur et al., JSS 2009)

Memory as a set of graphs Exploiting points-to informations for modeling use/def of dereferenced

pointers Pointer Static Single Assignment form

x1

xn

p...

)

)(

)(

,

&

...

&

,( ...

11

nn

u

xssa

xssa

x

x

pvUse of a dereferenced pointer:

)

)(

...

)(

,,

&

...

&

,(...

111

nn

d

xn

x

xssa

xssa

Expr

x

x

p

v

vDef of a dereferenced pointer:

Use of a dereferenced pointer: combinator -_u

)...,...,(

11

nn

u

V

V

P

P

PX is true iff -i such as P = Pi - X = Vi

X = *P ;

NB: When P = 0 (invalid address) then -u and -d fail or throw an exception

Def of a dereferenced pointer: combinator -_d

)...,,...,(...

111

nn

d

n V

V

Exp

P

P

P

X

Xis true iff -i such asP = Pi - Xi = Exp - {Xj = Vj}

j-i

*P = Exp ;

Global constraints to represent pointer dereferencing

Weaknesses of this Memory model

- Requires a preliminary points-to analysis that may be too imprecise when dynamic (de-)allocation is involved

- Pointers as function inputs, can point to anything on the heap

- Some conditions may constrain the shape of dynamic data structures. How to handle this in a constraint solver ?

t

next

int P(struct cell * t) { if( t == t->next ) { …

constrains t to

Memory, as a structured set of unbounded arrays

M : memoryIntegers : TABiFloats : TABfPointers : TABpStructures : [S1,S2,..]

TAB : tableau status: closed or not cont. : { @i – Vi, …}

V : integer within a finite domainType : 16,32,64 bits, signed, unsigneddom : {possible values} Min .. Max

V : float within an intervalType : float (32), double (64)dom. : Min .. Max

V:pointer possibly_null : yes, no dom : {possibles values} nondom : {non-possible values}

S : structure status : closed or not cont. : {@i}

Every statement is interpreted as constraints over Memory states

C program Constraints store

i = i + 1 -----------------------------------> load_elt(@i, I1, M1)

I2 = I1 + 1 store_elt(@i, I2, M1,M2)

*p = 3 ------------------------------------> load_elt(@p, P1, M2) DP1 = 3 store_elt(P1,DP1,M2,M3)

Store_eltM1 :Status : not closedIncludes :

i – Vij – Vjk – Vk…

M2 :Status : not closedIncludes :

i – Vi’j – Vj’ k – Vk’ …

P :Domain pointer

{i,j}

V:Domain Integer

1.. 5

store_elt(P,V,M1,M2)

Store_eltM1 :Status : not closedIncludes :

i – Vi 1.. 2j – Vj 5.. 9k – Vk 2…

M2 :Status : not closedIncludes :

i – Vi’ 3..6j – Vj’ 7..18 k – Vk’ ?…

P :Domain pointer

{i, j}

V:Domain Integer

1.. 5

Automatic deductions after the constraint propagation step :

P = i, V = Vi’ in 3..5, Vj = Vj’ in 7..9, Vk = Vk’ =2

store_elt(P,V,M1,M2)

Model for the definition of a new constraint

SVAR

success

failure

ConstraintsStore

Awake

Suspend

exit

fail

reduce

SVAR

success

failure

ConstraintsStore

Awake

Suspend

If( dom(P) = {p} and p != Null ) then add( M2[p] = V) and ..

reduce

store_elt(P,V,M1,M2)

If( M2[i] != M1[i] ) then add( P = i and M2[i]=V) and ..

If( dom(V) = - or dom(P) = -) then fail

If( dom(P) = {p} and p = Null) then fail

dom(P) {i / dom(M2[i] - dom(V) != -}dom(V) -i-dom(P) dom(M2[i]) dom(M1[i]) dom(M2[i]) - dom(M1[i]) if( i - dom(P) )dom(M2[i]) dom(M1[i]) - dom(M2[i]) if( i - dom(P) )dom(M2[i]) dom(M1[i] - dom(V)) otherwise

Constraint reasoning over Memory as graphs (Gotlieb Denmat Botella ASE’05, COMPSAC’05, IST’07)

Memory, as a structured set of unbounded arrays (Gotlieb Botella ICCSEA’06, Charreteur et al. TAIC-PART’07, JSS 2010)

Extension to Bytecode Java, inheritance and polymorphism (Charreteur Gotlieb,

ISSRE’10)

***PhD Thesis of Florence Charreteur (Defense in March 2010)

*** Prototype tools implementation: Inka V1 (Dassault A.Gotlieb, B.Botella, in 2001) InKa V2 (Dassault A. Gotlieb, B.Botella in 2006),

JAUT (INRIA F. Charreteur, in 2010)

Constraints over memory models for testing pointer programs

(summary of contribution 2)

What was left apart:

• Symmetric Testing: using symmetries to test programs (Gotlieb ISSRE’03, Gotlieb Bernard

QSIC’06)

• Constraints over floating-point variables (Botella Gotlieb Michel STVR 2006, Carlier Gotlieb ICTAI’11)

• Probabilistic constraint reasoning for statistical testing - New probablistic constraint combinators (Petit Gotlieb CP’07,

QSIC’07)- Statistical test case generation (Gotlieb Petit RT’06, JSS

2009)

• Explanation-based generalization of infeasible paths in Dynamic Symbolic Execution (Delahaye Botella Gotlieb ICST’10, TSE in revision)

• Testing constraint programs with constraints (Lazaar Gotlieb Lebbah CP’10, Constraints Journal, ICST’11)

Conclusions• Constraint Programming is worth to

Software Testing automation:

In this talk, 2 main contributions to the domain of CBT:

- Constraint-based program exploration (w)- Constraints over memory models for testing pointer programs

• Strongly involved in the development of 3 Research prototype tools :

Euclide, Inka and FPSE (more than 30KLOC Prolog, Java, C)

• Research projects: ACI V3F, ANR CAT/U3CAT, ANR CAVERN…

• Applications to the testing of embedded software

BCE Rafale – Dassault Electronics

TCAS - Airbus

Java Card - Oberthur

ConclusionsConstraint-Based Testing

• Emerging concept in code- and model-based automatic test data gener.

• Constraint Programming techniques offers:

- Global constraints modelling to handle control and data structures (while pure SAT-solving does not work well in that context)

- Versatility and flexibility of CP (while pure LP or SMT approaches are very rigid). Handles non-linear constraints over finite domains. - Generic techniques to implement new solvers, with abstraction-based relaxation, even if unsatisfiability detection has to be improved by combining techniques (e.g., SMT/CP)

• Mature tools (academic and industrial) already exist, but application on real-sized industrial cases still have to be demonstrated

Further work

- Array constraint solving.

A combined SMT/CP approach for solving constraints with arrays and arithmetics. Constraint solver CCFD and large experimental validation over random formulas. joint work with S. Bardin from CEA

- Constraint-Based Testing from feature models, in the context of the testing of highly-configurable systems

PhD A. Hervieu

VALVES project

HVAC - Thales

Saturn C90 – Cisco Norway

Thank you!

• PhD students

Tristan Denmat, Matthieu Petit, Florence Charreteur, Mickael Delahaye, Nadjib Lazaar, Aymeric Hervieu

• Post-doc

Sandrine Gouraud, Pierre Rousseau, Matthieu Carlier

• Co-authors

Olivier Lhomme, Michel Rueher, Claude Michel, Yahia Lebbah, Michel Leconte, Mireille Ducassé, Bernard Botella, Patrick Taillibert, Franck Calvet, Bruno Marre , Benjamin Blanc, Frédéric Dadeau, Nicky Williams, Catherine Dubois, Patrick Bernard, Matthieu Wattel, Benoit Baudry, Sébastien Bardin, Lionel Briand

Constraint solving over floating-point expr.for testing numerical programs

(Contribution 3)

- Numerical programs used in embedded systems contain lots of floating-point computations

- Automatic test data generation through symbolic execution is challenging for those programs as: * floating-point computations exhibit round-off errors Absorption (Xf + - == Xf), Cancellation ( Xf-Yf==K.( X-Y ) when K>>1

* floating-point variables are currently handled as reals or rationals in constraint solvers (even if floating-point numbers are used)

float foo( float x) {

float y = 1.0e12, z ;

1. if( x < 10000.0 )

2. z = x + y;

3. if( z > y)

4. …

Is the path 1-2-3-4 feasible ?

Path conditions:

x < 10000.0

x + 1.0e12 > 1.0e12

On the reals : x - (0,10000)

On the floats : no solution !

An illustrative program

float foo( float x) {

float y = 1.0e12, z ;

1. if( x > 0.0 )

2. z = x + y;

3. if( z == y)

4. …

Is the path 1-2-3-4 feasible ?

Path conditions:

x > 0.0

x + 1.0e12 = 1.0e12

On the reals : no solution

On the floats: x - (0, 32767.99…)

Conversely,

Our approach: to build a dedicated constraint solver over the floats , able to capture floating-point solutions

Prototype tool: FPSE (Botella Gotlieb Michel STVR 2006, Carlier Gotlieb 2011)

46

Our approach to solve FP constraints: Interval Propagation over FP variables

• Notations:

with [a add b] representing near(a + b)

Illustrated for the near-to-even rounding mode only

x

near(x)

a+

a-

a

mid(a,a+)

47

Floating-point projections

- Direct and indirect projections for the assignment:

proj(r, r:= a add b) (direct) [r := a add b] leads to proj(a, r:= a add b) (1st indirect) proj(b,r := a add b) (2nd indirect)

- Direct projections (over numeric fp numbers):

If Ir = [rl,rh], Ia = [al,ah] and Ib = [bl,bh] then

[r := a add b] [rl’,rh’] [al add bl, ah add bh] - [rl,rh] [r := a subs b] [rl’,rh’] [al subs bh, ah subs bl] - [rl,rh]

...

bl

bh

rl’ max(al add bl, rl)

rh’ min(ah add bh, rh)

al

ah

rh

rl

Ex: Direct projection [r := a add b]

impossible

Monotony of rounding :r1 - r2 - near(r1) - near(r2)

impossible

IEEE-754 accuracy property

49

More complex : indirect projections

If Ir = [rl,rh], Ia = [al,ah] and Ib = [bl,bh] then

1st indirect projection of [r := a add b][al’,ah’] [mid(rl,rl

-) subs bh, mid(rh,rh+) subs bl] - [al,ah]

1st indirect projection of [r := a subs b][al’,ah’] [mid(rl,rl

-) add bl, mid(rh,rh+) add bh] - [al,ah]

2nd indirect projection of [r := a subs b][bl’,bh’] [al subs mid(rh,rh

+), ah subs mid(rl,rl-)] - [bl,bh]

rl-

rh+

rl

rh

bl

bhah’ min(mid(rh,rh

+) subs bl, ah)

al’ max(mid(rl, rl

-) subs bh, al)

al

ah

Ex: 1st indirect projection [r := a add b]

impossible

not optimal, but computable with near-to-even

IEEE-754 accuracy property

FPSE: Floating-point constraint solving

- Integrated with a finite domains constraint solver (SICStus clpfd library) through a simple collaboration principle

I (long) X

FPSE clpfd

alarm(dom(J) is pruned)X (float) J

tell( I in min(X)..max(X))

- Formulas for handling basic arithmetical operators, comparisons conversions, zeros and infinities

- Labelling search heuristics based on the number of floats within an interval, rather than the real distance between two floats

Foundations of the approach (Michel Rueher Lebbah CP’01)

Symbolic Execution of FP computations (Botella Gotlieb Michel STVR 2006)

Generalization of the Marre&Michel property (Carlier Gotlieb ICTAI’11)

Extension to modular arithmetic const. solv. (Gotlieb Leconte Marre ModRef’10)

***Post-doc of Matthieu Carlier (2010-2011)

*** Prototype tools implementation: FPSE (INRIA B. Botella, A. Gotlieb in 2006), ( INRIA M. Carlier in 2011)

Constraint solving over floating-point computations

(summary of contribution 3)

How CBT relates to other bug-finding techniques ?

Static analysis aims at finding runtime errors (e.g. division-by-zero, overflows, …) at compile-time

while CBT aims at finding functional faults (e.g. P returns 3 while 2 was expected) at runtime

Software model-checking tools explores a bounded boolean structure of the program in order to prove properties or find counter-examples while CBT uses global constraints to capture the structure

Dynamic analysis approaches extract likely invariants

while CBT exploits symbolic reasoning to find counter-examples to given properties

How CBT relates to other test data generation techniques ?

Other test cases generation techniques include: - Random Testing (Uniform, Adaptive RT, Statistical structural/functional Testing…)

- Dynamic methods (program executions, Korel’s method, binary search, …)

- Evolutionary techniques (Genetic Algorithms, search-based methods, …)

By combining symbolic reasoning and numerical inference, CBT exploits program structure and data to refine the test case generation process and differs so from «blind» techniques that attempt to reach the testing objective by trials.

Dynamic symbolic evaluation

Symbolic execution of a concrete execution (also called concolic execution)

By using input values, feasible paths only are (automatically) selected

Randomized algorithm, implemented by instrumenting each statement of P

Main CBT tools: PathCrawler (CEA, Williams et al. 2005),

PEX (Microsoft, Tillman et al. 2008), SAGE (Microsoft, Godefroid et al.2008)

Comes in two ingredients…

1st ingredient: path exploration1. Draw an input at random, execute it and record path conditions

b

a

t

c

t

d

t

2. Flip a non-covered decision and solve the constraints to find a new input x

b

a

t

c

t

d

f

3. Execute with x

4. Repeat 2

b

a

t

c

t

f

de

f

t

b

a

t

c

t

f

de

f

f

…. b

a

t

c

t

f

de

fg

h

i

jk

Up to given bounds

2nd ingredient: use concrete values

Use actual values to simplify the constraint set

Flip If( x1 * x2 == 100) … (x1 = 5, x2 = 7)

(1) Exact solving -- add x1 * x2 == 100 to the constraint solver

(2) Approximate solving -- add 5 * x2 == 100 && x1 = 5

or -- add x1 * 7 == 100 && x2 = 7

(3) Useless solving -- add 35 == 100 && x1 = 5 && x2=7

Constraint solving in symbolic evaluation

• Mixed Integer Linear Programming approaches(i.e., simplex + Fourier’s elimination + branch-and-bound)

CLP(R,Q) in ATGen (Meudec 2001)lpsolve in DART/CUTE (Godefroid/Sen et al.

2005)

• SMT-solving (= SAT + Theories)

STP in EXE (Cadar et al. 2006), Z3 in PEX (Tillmann and de Halleux 2008)

• Constraint Programming techniques (constraint propagation and labelling)

Colibri in PathCrawler (Williams et al. 2005) Disolver in SAGE (Godefroid et al. 2008)

FPSE in EUCLIDE (Gotlieb 2009)