Part I Part II Part III Part IV -...
Transcript of Part I Part II Part III Part IV -...
Why Should We Care about Similarity of SatisfiabilityProblems?
Part IPart IIPart IIIPart IV
Part I - Why Does Configurable Software NeedCompositionality?
Highly-Configurable Systems
Feature Models
A Matter of Size
Why to Solve Lots of Similar SAT Problems?
Part II - Compositional Analyses with Feature-ModelInterfaces
Compositionality Principle
Feature-Model Interfaces
Reduced Feature-Model Size
Reduced Effort in Type Checking Linux?
Part III - Solving Similar SAT/BDD/SMT Instances
Profiling the SAT4J Implementation
Effect of Our SAT4J Optimization
The Influence of Large Submodels
Cumulative Solver Times with Threshold
Solving Similar SAT/BDD/SMT Instances
Conclusion
Part IV - Backup Slides
Decomposition on Demand?
Make Implicit Constraints Explicit
Decomposition with Feature-Model Interfaces
Compositionality with Feature-Model Interfaces
Reasoning Strategies
Setup for Measurements
Time to Solve SAT Queries
Time vs. Number of Selected Submodels
Threats to Validity
Why Should We Care about Similarity ofSatisfiability Problems?
Thomas Thüm, Frederik Kanning, Stephan Mennicke, Ina Schaefer, . . .FOSD Meeting 2017 in Grasellenbach, March 16, 2017
Part I
Why Does Configurable SoftwareNeed Compositionality?
Highly-Configurable Systems
An Analysis of the Variability in Forty Preprocessor-BasedSoftware Product Lines
Jörg Liebig, Sven Apel,and Christian Lengauer
University of Passau{joliebig,apel,lengauer}@fim.uni-
passau.de
Christian Kästner and Michael SchulzeUniversity of Magdeburg
{ckaestne,mschulze}@ovgu.de
ABSTRACTOver 30 years ago, the preprocessor cpp was developed toextend the programming language C by lightweight metapro-gramming capabilities. Despite its error-proneness and lowabstraction level, the preprocessor is still widely used inpresent-day software projects to implement variable software.However, not much is known about how cpp is employedto implement variability. To address this issue, we haveanalyzed forty open-source software projects written in C.Specifically, we answer the following questions: How doesprogram size influence variability? How complex are exten-sions made via cpp’s variability mechanisms? At which levelof granularity are extensions applied? Which types of ex-tension occur? These questions revive earlier discussions onprogram comprehension and refactoring in the context of thepreprocessor. To provide answers, we introduce several met-rics measuring the variability, complexity, granularity, andtypes of extension applied by preprocessor directives. Basedon the collected data, we suggest alternative implementa-tion techniques. Our data set is a rich source for rethinkinglanguage design and tool support.
Categories and Subject DescriptorsD.2.3 [Software Engineering]: Coding Tools and Tech-niques; D.2.8 [Software Engineering]: Metrics; D.3.4 [Pro-gramming Languages]: Processors—Preprocessors
General TermsEmpirical Study
KeywordsSoftware Product Lines, C Preprocessor
1. INTRODUCTIONThe C preprocessor (cpp) is a popular tool for implement-
ing variable software. It has been developed to enhance C by
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.ICSE ’10, May 2-8 2010, Cape Town, South AfricaCopyright 2010 ACM 978-1-60558-719-6/10/05 ...$10.00.
lightweight metaprogramming capabilities and is commonlyused to merge files, make arbitrary textual substitutions, anddefine conditional code fragments (a.k.a. conditional inclu-sion) [21]. As the cpp tool is line-based, it can be used withany text artifact including other programming languagessuch as Java or C#. In the past, it has been observed thatthe use of cpp causes various problems: (1) the occurrenceof syntactic and semantic errors during the generation ofsoftware products [23]; (2) code pollution due to scatteredand tangled #ifdefs (a.k.a. #ifdef hell) [32]; (3) a decrease inmaintainability and in ability to evolve [13].
The implementation of variable software is also a majorgoal of software product line engineering. A software productline (SPL) is a set of software-intensive systems sharing acommon, managed set of features that satisfy the specificneeds of a particular market segment or mission and thathave been developed from a common set of core assets in aprescribed way [8]. Software product line engineering aims atmanaging variability among the different software products(a.k.a. variants) of a product line and facilitating reuse in thatthe products of the product line share as many common coreassets as possible, such as source code artifacts [8, 11, 27].
It is widely assumed that the variability mechanism of thecpp tool is used quite frequently in the implementation ofSPLs [1, 12, 18, 33]. We believe this assumption to be trueand contribute to the discussions on the connection betweenpreprocessors and SPLs started in prior work with a substan-tial set of case studies. We answer the following questions:How does program size influence variability of SPLs? Howcomplex are the extensions applied by features via cpp’svariability mechanisms? At which level of granularity areextensions applied? Which types of extension occur? Weargue that insights into the implementation problems of fea-tures solved with cpp help to judge whether cpp usage causesproblems with regard to code pollution, error-proneness, andreduced maintainability and ability to evolve. An analysis ofcpp usage also allows developers to estimate the effort andbenefits of migrating to other, more well-founded implemen-tation techniques for SPL engineering, such as aspects [17,24] or various flavors of feature modules [5, 6].
To answer the questions above, we analyzed forty open-source software systems of different sizes (thousands to mil-lions lines of code) taken from different domains includingoperating systems, database systems, and compilers. Wepropose a set of metrics that allow us to infer and classifycpp usage patterns and to map the patterns to common SPLimplementation concepts. Our analysis reveals that cpp isused to a large extent to implement and control variability
1
Liebig et al. @ ICSE’10
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 3
Feature Models
Base
BAR
F8F7FOO
F1
F6F5F4
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 4
A Matter of Size
Feature models with thousands of features are challenging
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 5
A Matter of Size
Feature models with thousands of features are challenging
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 5
Why to Solve Lots of SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
FOO ⇒ BAR ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Why to Solve Lots of SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
FOO ⇒ BAR ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Why to Solve Lots of SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
FOO ⇒ BAR ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Why to Solve Lots of SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
FM �FOO ⇒ BAR ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Why to Solve Lots of SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
¬SAT (FM ∧ (FOO ⇒ BAR)) ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Why to Solve Lots of Similar SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
¬SAT (FM ∧ (FOO ⇒ BAR)) ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Why to Solve Lots of Similar SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
¬SAT (FM ∧ (FOO ⇒ BAR)) ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Why to Solve Lots of Similar SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
¬SAT (FM ∧ (FOO ⇒ BAR)) ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Why to Solve Lots of Similar SAT Problems?
#ifdef BAR
int x = 0;
#endif
// ...
#ifdef FOO
x++;
#endif
¬SAT (FM ∧ (FOO ⇒ BAR)) ?
Idea: reduce size of feature model
Type checking
Parsing
Dataflow analysis
Model checking
Deductive verification
Refactoring
Feature-model analysis
Configuration process
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 6
Part II
Compositional Analyseswith Feature-Model Interfaces
Compositionality Principle
(FOO ⇒ X)∧ Y ∧ Z ∧ (X ⇒ BAR) � FOO ⇒ BAR
(FOO ⇒ X)∧ (X ⇒ BAR) � FOO ⇒ BAR
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 8
Compositionality Principle
(FOO ⇒ X)∧ Y ∧ Z ∧ (X ⇒ BAR) � FOO ⇒ BAR
(FOO ⇒ X)∧ (X ⇒ BAR) � FOO ⇒ BAR
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 8
Compositionality Principle
(FOO ⇒ X)∧ Y ∧ Z ∧ (X ⇒ BAR) � FOO ⇒ BAR
(FOO ⇒ X)∧ (X ⇒ BAR) � FOO ⇒ BAR
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 8
Feature-Model Interfaces [ICSE’16]
For scalability: features eliminated from propositional formulas withexistential quantification
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 9
Feature-Model Interfaces [ICSE’16]
For scalability: features eliminated from propositional formulas withexistential quantification
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 9
Reduced Feature-Model Size [ICSE’16]
Given four monthly snapshots of an automotive feature model growingfrom 14k to 18k featuresRQ1: How small can interfaces be compared to submodels?
RQ2: How often are feature-model interfaces compatible?
Features
in V1
14,010
0%
20%
40%
60%
80%
100%
Features
in V2
17,742
Features
in V3
18,434
Features
in V4
18,616
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 10
Reduced Feature-Model Size [ICSE’16]
Given four monthly snapshots of an automotive feature model growingfrom 14k to 18k featuresRQ1: How small can interfaces be compared to submodels?RQ2: How often are feature-model interfaces compatible?
Features
in V1
14,010
0%
20%
40%
60%
80%
100%
V1↓
V2
19
Features
in V2
17,742
V2↓
V3
14
Features
in V3
18,434
V3↓
V4
13
Features
in V4
18,616
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 10
Reduced Effort in Type Checking Linux?
Family-based type checking with TypeChef
Linux kernel version 2.6.33.3 with 11,000 features
1,363 out of 7,760 files
173,845 queries
90ms per query
12s reasoning time per file
Reasoning: 60% of type checking phase
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 11
Reduced Effort in Type Checking Linux?
Family-based type checking with TypeChef
Linux kernel version 2.6.33.3 with 11,000 features
1,363 out of 7,760 files
173,845 queries
90ms per query
12s reasoning time per file
Reasoning: 60% of type checking phase
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 11
Reduced Effort in Type Checking Linux?
Family-based type checking with TypeChef
Linux kernel version 2.6.33.3 with 11,000 features
1,363 out of 7,760 files
173,845 queries
90ms per query
12s reasoning time per file
Reasoning: 60% of type checking phase
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 11
Literals and Distinct Features per Query
1 28 55 82 109
136
163
190
217
244
271
298
325
352
379
406
433
460
487
514
100
101
102
103
104
105
Literals
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 12
Literals and Distinct Features per Query
1 28 55 82 109
136
163
190
217
244
271
298
325
352
379
406
433
460
487
514
100
101
102
103
104
105
Literals
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28
Features
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 12
Apply Linux Queries to Automotive Feature Model
Problem: no decomposition for Linux feature model andno domain artifacts/queries for automotive
Idea: use queries from Linux evaluation as templates(¬A ∧ B)∨ A y (¬P ∧ Q)∨ P
Snapshot Features Constraints Clauses Submodels
1 14,010 666 237,706 442 17,742 914 342,935 453 18,434 1,300 347,557 464 18,616 1,369 350,287 46
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 13
(No) Reduced Effort for Product-Line Analysis?
1 2 3 4 5
0
1,000
2,000
3,000
4,000
Literals in query
Average
time(m
s)
fullreduced
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 14
Part III
Solving SimilarSAT/BDD/SMT Instances
Profiling the SAT4J Implementation
¬SAT (FM ∧ (FOO ⇒ BAR)) ?
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 16
Profiling the SAT4J Implementation
public void remove(T elem) {
int j = 0;
for (; this.myarray[j] != elem; j++) {
if (j == size ())
throw new NoSuchElementException ();
}
System.arraycopy(this.myarray , j + 1,
this.myarray , j, size() - j - 1);
this.myarray[--this.nbelem] = null;
}
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 17
Effect of Our SAT4J Optimization
1 2 3 4 5
0
1,000
2,000
3,000
4,000
Literals in query
Average
time(m
s)
fullreduced
full (opt.)reduced (opt.)
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 18
Cumulative Solver Time with Optimization
Snapshot1
Snapshot2
Snapshot3
Snapshot4
0
20,000
40,000
60,000
80,000
2,462
3,255
3,466
3,468
52,373
79,666 79,997 79,132
Cumulative
solver
time(m
s) fullreduced
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 19
The Influence of Large Submodels
100 101 102 103 104 105
Clauses
Snapshot1
Snapshot2
Snapshot3
Snapshot4
0
5,000
10,000
15,000
20,000
11,100
15,698
14,818
15,866
11,453
11,961
11,486
12,378
Cumulative
solver
time(m
s) fullreduced
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 20
The Influence of Large Submodels
100 101 102 103 104 105
Clauses
Snapshot1
Snapshot2
Snapshot3
Snapshot4
0
5,000
10,000
15,000
20,000
11,100
15,698
14,818
15,866
11,453
11,961
11,486
12,378
Cumulative
solver
time(m
s) fullreduced
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 20
Time vs. Added Submodel Clauses
0 0.2 0.4 0.6 0.8 1 1.2
·104
0
5
10
15
20
Added clauses
Tim
e(m
s)
fullreduced
full (approx.)reduced (approx.)
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 21
Cumulative Solver Times with Threshold
Snapshot1
Snapshot2
Snapshot3
Snapshot4
0
5,000
10,000
15,000
20,000
11,100
15,698
14,818
15,866
11,453
11,961
11,486
12,378
10,083
11,750
11,341
12,105
Cumulative
solver
time(m
s) fullreducedadaptive
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 22
Solving Similar SAT/BDD/SMT Instances
SAT: similar instances not part of satisfiability contests⇒ not optimized for this purpose
BDD: no BDD for Linux, hard to join BDDs with different variableorderings
SMT: recent developments for lightweight removal of formulas
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 23
How Should Superheroes Eat Cake?
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 24
We Should Care about Similarity of SAT Problems
Configurable software requires to solve many similar SATinstances: consumes 60% of type checking (without parsing)
Feature-model interfaces can significantly reduce the similar part
Performance gain for reasoning 10–24%
Open question: What are good decompositions of feature models?
Open question: Are solvers ready to solve similar instances?
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 25
Experience Report: How NOT to Travel to FOSD’17
Forest + Darkness= Very Late Dinner
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 26
Decomposition on Demand? [SPLC’16]
Removing x % of 18k features of an automotive feature model:
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 28
Make Implicit Constraints Explicit [FOSD’16]
What are implicit constraints?
C ⇒ D ¬C ∨ ¬D
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 29
Make Implicit Constraints Explicit [FOSD’16]
What are implicit constraints?
C ⇒ D
¬C ∨ ¬D
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 29
Make Implicit Constraints Explicit [FOSD’16]
What are implicit constraints?
C ⇒ D
¬C ∨ ¬D
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 29
Make Implicit Constraints Explicit [FOSD’16]
What are implicit constraints?
C ⇒ D ¬C ∨ ¬D
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 29
Make Implicit Constraints Explicit [FOSD’16]
RQ1: How many implicit constraints exist in subtrees?Automotive feature model with 2,513 features and 2,833 constraints
Depth 1: 6 features and 12 implicit constraintsDepth 2: 25 features and 186 implicit constraints
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 30
Make Implicit Constraints Explicit [FOSD’16]
RQ2: What is the structure of implicit constraints?
RQ3: What is the number of involved subtrees?Depth 1: max. 4/6 partial modelsDepth 2: max. 5/25 partial models
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 31
Decomposition with Feature-Model Interfaces
Base
BAR
F8F7FOO
F1
F6F5F4
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 32
Decomposition with Feature-Model Interfaces
Base
BAR
F8F7FOO
F1
F6F5F4
select subtrees
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 32
Decomposition with Feature-Model Interfaces
Base
BAR
F8F7FOO
F1
F6F5F4
select subtrees
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 32
Decomposition with Feature-Model Interfaces
Base
BAR
F8F7FOO
F1
F6F5F4
identify local features & local constraints
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 32
Decomposition with Feature-Model Interfaces
Base
BAR
F8F7FOO
F1
F6F5F4
identify local features & local constraints
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 32
Decomposition with Feature-Model Interfaces
Base
BAR
F8F7FOO
F1
F6F5F4
decompose model by removing local features
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 32
Decomposition with Feature-Model Interfaces
Base
BAR
F8F7FOO
F1
F6F5F4
decompose model by removing local features
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 32
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �
FOO ⇒ BAR ?
X
Γ ∪ ∆1 �F6 ⇒ Base ?X
Γ ∪ ∆1 ∪ ∆2 2F1 ∧ F6 ∧ FOO ?X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �
FOO ⇒ BAR ?
X
Γ ∪ ∆1 �F6 ⇒ Base ?X
Γ ∪ ∆1 ∪ ∆2 2F1 ∧ F6 ∧ FOO ?X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �FOO ⇒ BAR ?
X
Γ ∪ ∆1 �F6 ⇒ Base ?X
Γ ∪ ∆1 ∪ ∆2 2F1 ∧ F6 ∧ FOO ?X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �FOO ⇒ BAR
?
X
Γ ∪ ∆1 �F6 ⇒ Base ?X
Γ ∪ ∆1 ∪ ∆2 2F1 ∧ F6 ∧ FOO ?X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �FOO ⇒ BAR
?
X
Γ ∪ ∆1 �
F6 ⇒ Base ?
X
Γ ∪ ∆1 ∪ ∆2 2F1 ∧ F6 ∧ FOO ?X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �FOO ⇒ BAR
?
X
Γ ∪ ∆1 �F6 ⇒ Base ?
X
Γ ∪ ∆1 ∪ ∆2 2F1 ∧ F6 ∧ FOO ?X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �FOO ⇒ BAR
?
X
Γ ∪ ∆1 �F6 ⇒ Base
?
X
Γ ∪ ∆1 ∪ ∆2 2F1 ∧ F6 ∧ FOO ?X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �FOO ⇒ BAR
?
X
Γ ∪ ∆1 �F6 ⇒ Base
?
X
Γ ∪ ∆1 ∪ ∆2 �
F1 ∧ F6 ∧ FOO ?
X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �FOO ⇒ BAR
?
X
Γ ∪ ∆1 �F6 ⇒ Base
?
X
Γ ∪ ∆1 ∪ ∆2 �F1 ∧ F6 ∧ FOO ?
X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Compositionality with Feature-Model Interfaces
Base
BAR
F8F7
F1
F4
Γ
F1
F6F5F4
∆1
BAR
F8F7FOO
∆2
Γ ∪ ∆2 �FOO ⇒ BAR
?
X
Γ ∪ ∆1 �F6 ⇒ Base
?
X
Γ ∪ ∆1 ∪ ∆2 2F1 ∧ F6 ∧ FOO
?
X
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 33
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Reasoning Strategies
¬φ ∧
φ Adaptive:m(φ) < τ ?
SAT solver
¬φ ∧ ∧
Full
Reduced
CNF
CNF
CNF
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 34
Setup for Measurements
Generate sets of 1,000 and 5,000 queries
Full: pure solver time
Reduced: selection + composition + solver times
JVM: garbage collection & just-in-time compilationScalaMeter framework
Multiple measurements across different JVM runsWarm-up runsOutlier detection
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 35
Setup for Measurements
Generate sets of 1,000 and 5,000 queries
Full: pure solver time
Reduced: selection + composition + solver times
JVM: garbage collection & just-in-time compilation
ScalaMeter frameworkMultiple measurements across different JVM runsWarm-up runsOutlier detection
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 35
Setup for Measurements
Generate sets of 1,000 and 5,000 queries
Full: pure solver time
Reduced: selection + composition + solver times
JVM: garbage collection & just-in-time compilationScalaMeter framework
Multiple measurements across different JVM runsWarm-up runsOutlier detection
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 35
Time to Solve SAT Queries
0226
452
678
904
1,130
1,356
1,582
1,808
2,034
2,260
2,486
2,712
2,938
3,164
3,390
3,616
3,842
4,068
4,294
100
101
102
103
104
105
Time (ms)
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 36
Time vs. Number of Selected Submodels
0 2 4 6 8 10 12
1
2
3
4
Selected models
Average
time(m
s)full
reduced
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 37
Threats to Validity
Type checking1/6 of available filesOther analyses, product lines?
Reasoning with interfacesSemi-random queriesOnly snapshots of a single modelSingle solver implementationIsolated measurements
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 38
Threats to Validity
Type checking1/6 of available filesOther analyses, product lines?
Reasoning with interfacesSemi-random queriesOnly snapshots of a single modelSingle solver implementationIsolated measurements
Thomas Thüm Why Should We Care about Similarity of Satisfiability Problems? Slide 38