Towards Incremental Model Slicing for SPL Regression...
Transcript of Towards Incremental Model Slicing for SPL Regression...
Towards Incremental Model Slicingfor SPL Regression Testing
Sascha Lity, Hauke Baller, Ina Schaefer, May 14th, 2015
Technische Universität Braunschweig
Institut für Programmierungund Reaktive Systeme
Motivation
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 2Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Motivation
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 2Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Motivation
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 2Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Product-by-Product Testing
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 3Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Product-by-Product Testing
Reuse Artifacts/Results
Reuse Artifacts/Results
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 3Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Product-by-Product Testing
Reuse Artifacts/Results
Reuse Artifacts/Results
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 3Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Applying Regression Testing for SPLs
Regression testing allows for incremental SPL testing [TTK04, Eng10]
Regression Testing validates whether modified softwarebehaves as intended, and modifications have not adverselyimpacted the software’s quality. [Rot96]
Software version testing vs. software variant testing?
Reuse of test artifacts? ⇒ Delta Modeling [CHS15]
Incremental SPL testing? ⇒ Delta-oriented [LSKL12, Loc12, LLL+14]
Retest decision? ⇒ Slicing [AHKL93, BH93, GHS92]
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 4Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Applying Regression Testing for SPLs
Regression testing allows for incremental SPL testing [TTK04, Eng10]
Regression Testing validates whether modified softwarebehaves as intended, and modifications have not adverselyimpacted the software’s quality. [Rot96]
Software version testing vs. software variant testing?
Reuse of test artifacts? ⇒ Adaption of Delta Modeling [CHS15]
Incremental SPL testing? ⇒ Delta-oriented [LSKL12, Loc12, LLL+14]
Retest decision? ⇒ Slicing [AHKL93, BH93, GHS92]
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 4Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Program/Model Slicing
Static program projection technique by Weiser [Wei81]
Preserves syntax and semantics w.r.t. slicing criterion
Based on control/data dependencies
Model slicing [JWZC02, KLB12, ACH+13] adapts concept for state-basedmodels
Slicing Criterion
(S,V)
p p'
(S,V)(S,V)
M M‘
Slicing Criterion B2
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 5Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Variability-aware Slicing [ACLF11, KLB12, KS14]
Conditioned Model Slicing of Feature-AnnotatedState Machines
Jochen KamischkeInstitut für Programmierung
und Reaktive SystemeMühlenpfordtstr. 23
Braunschweig, [email protected]
Malte LochauInstitut für Programmierung
und Reaktive SystemeMühlenpfordtstr. 23
Braunschweig, [email protected]
Hauke BallerInstitut für Programmierung
und Reaktive SystemeMühlenpfordtstr. 23
Braunschweig, [email protected]
ABSTRACTModel-based behavioral specifications build the basis forcomprehensive quality assurance techniques for complexsoftware systems such as model checking and model-basedtesting. Various attempts exist to adopt those approachesto variant-rich applications as apparent in software productline engineering to efficiently analyze families of similar soft-ware systems. Therefore, models are usually enriched withcapabilities to explicitly specify variable parts by means ofannotations denoting selection conditions over feature pa-rameters. However, a major drawback of model-based en-gineering is still its lack of scalability. Model slicing pro-vides a promising technique to reduce models to only thoseobjects being relevant for a certain criterion under consid-eration such as a particular test goal. Here, we present anapproach for slicing feature-annotated state machine mod-els. To support feature-oriented slicing on those models,our framework combines principles of variability encodingand conditioned slicing. We also present an implementationand provide experimental results concerning the efficiencyof the slicing algorithm.
Categories and Subject DescriptorsD.2.4 [Software Engineering]: Software/Program Veri-fication; D.2.13 [Software Engineering]: Reusable Soft-ware, Reuse Models
General TermsDesign, Theory
KeywordsSoftware Product Lines, Model-Based Software Engineering.
1. INTRODUCTIONModel-based software engineering provides a rich collec-
tion of modeling languages and corresponding techniques for
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.FOSD’12, September 24–25, 2012, Dresden, Germany.Copyright 2012 ACM 978-1-4503-1309-4/12/09 ...$10.00.
the specification, documentation, maintenance, and verifi-cation/validation of high-quality software systems in a sys-tematic way. In particular, modeling approaches for specify-ing the operational behavior of a software system are oftenbased on state-transition diagrams such as UML state ma-chines [18]. Thereupon, various applications of those modelsto verification/validation techniques like model-based test-ing and model checking have been proposed [5, 1].
However, the major drawback of those approaches is stilltheir lack of scalability. This is even worse in the presenceof explicit variability at model level as apparent, e.g., insoftware product line engineering [15]. Herein, models areenriched with capabilities to specify common and variableparts occurring in a family of similar product variants. Forinstance, model elements are annotated with selection con-ditions, i.e., propositional formulas over feature parametersto guide the assembling of model variants w.r.t. a particu-lar feature configuration [5]. Hence, such feature-annotatedmodels integrate any potential behaviors of all product vari-ants of an SPL within a virtual, so-called 150% model. Thisadditional model dimension as tailored by the valid prod-uct configuration space of an underlying domain featuremodel [11] further complicates the application of model-based analysis techniques to variant-rich, real world prob-lems represented by an SPL.
Model slicing provides a promising approach to handlethe complexity problem of behavioral models by perform-ing static, i.e., syntactical model reductions that preservesome aspects of model semantics w.r.t. a slicing criterion un-der consideration [22, 21]. Therefore, model slicing extractsthose model parts affecting certain computational units onlyand, at the same time, ensuring the resulting model slice topreserve a syntactically well-formed model structure. Slicinghas gained applications in various fields of program analysisfor reverse engineering, program integration, software met-rics, component reuse, etc. [3]. Model slicing adopts theconcepts of program slicing to reduce verification/validationefforts. For instance, in model-based testing, choosing testgoals as slicing criteria allows for efficient test case genera-tion, debugging, and change impact analysis during regres-sion testing, whereas slicing along a certain model property,e.g., given as an LTL formula, decreases model checkingcomplexity. However, recent model slicing approaches areincapable to cope with models enriched with feature anno-tations. In the presence of variable model parts, the furthermodel dimension has to be taken into account when slic-ing for a particular criterion in order to yield a well-formed
9
Program Slicing in the Presence ofPreprocessor Variability
Frederik KanningTU Braunschweig
Braunschweig, GermanyEmail: [email protected]
Sandro SchulzeTU Braunschweig
Braunschweig, GermanyEmail: [email protected]
Abstract—Program slicing is a common means to supportdevelopers in examining the source code with respect to de-bugging, program comprehension, or regression testing. Whilea vast amount of techniques exist, they are mostly tailored tosingle software systems. However, with the increasing importanceof variable and highly-configurable systems, such as the Linuxkernel, the number of software variants, subject to analysis,increases dramatically. Consequently, it is infeasible to applyslicing on each variant in isolation. To overcome this problem,we propose variability-aware slicing, a technique that can dealwith source code variability, specifically conditional compilationas introduced by the C preprocessor. Particularly, we providedetails of our variability-aware dependence analysis for programslicing, point out benefits of our slicing technique, and mentioncurrent limitations and future work.
I. INTRODUCTION
Program slicing is a common and well-established methodfor detailed program analysis, which has been proposed morethan three decades ago by Weiser [1]. Basically, programslicing allows to decompose a program with respect to itscomputational dependencies. As a result, it is possible toinspect the program in more detail, for instance, to assess the(potential) influence of a set of statements (the program slice)on a particular point in the program (the slicing criterion).Meanwhile, numerous techniques have been proposed that ex-tend the original (static) slicing technique of Weiser and thus,support developers in one or more tasks, such as debugging,program comprehension, or regression testing [2], [3].
While existing techniques are very mature and applicableeven at large scale for single (monolithic) programs, they areonly of limited use for variable software systems. Such systemsusually constitute a whole family of related programs thatcan be derived from a common code base [4]. To this end,configuration options, or features allow users to specify vari-able parts in such systems. A common means to express thisvariability are preprocessor directives, in particular conditionalcompilation, as provided by the C preprocessor tool CPP [5],[6]. While this allows users to tailor a program according tospecific needs, it also gives rise to an exponential numberof configurations. For instance, the Linux kernel consists ofmore than 11, 000 features, which allow for billions of possibleconfigurations to be compiled and generated on demand.
In this paper we bridge the gap between slicing andsource code variability by proposing a technique that supportsexploration and static analysis of variable software systems.
Research problem: Obviously, statically analyzing bil-lions of configurations separately is infeasible. On the otherhand, current slicing techniques mainly work on preprocessedprograms (i.e., a particular configuration), that is, the CPPdirectives are evaluated and removed before analysis, and thus,variability is not supported. In other words, such kind ofanalysis is not variability-aware [7]. As a result, these slicingtechniques are neither applicable to variable software systemsnor can detect effects of variability, such as errors caused byfeature interactions [8].
Contribution: The main contribution of this paper is aconcept and prototypical implementation for variability-awareintraprocedural slicing that works on annotated C programs(i.e., before preprocessing CPP directives). To this end, weextend and instrument TYPECHEF, a research infrastructurefor variability-aware analysis [9], [10]. Particularly, we providedetails on our variability-aware dependence analysis and howwe use this analysis to compute precise slices that includevariability information. Moreover, we motivate our techniqueby means of an example and possible applications.
While the current implementation exhibits still some lim-itations, we argue that it provides an important foundationfor taming variability and thus, provide efficient and scalableprogram analysis for variable software systems.
II. BACKGROUND
In the following, we provide information about sourcecode variability using CPP directives. Moreover, we lay thefoundations of our slicing technique by introducing the basicconcepts of program slicing.
A. Variable Systems with the CPP
The C preprocessor CPP is a tool that is tightly integratedwith the C programming language and that allows for develop-ing variable software systems [6]. To this end, the CPP providescapabilities to annotate variable code fragments at arbitrarygranularity using conditional compilation (a.k.a. #ifdefs).As an example, we show a program excerpt in Figure 1 thatcontains two variable code fragments: One variable fragment(Line 6–8) redeclares variable c in the inner scope, while theother one (Line 13–15) assigns a constant value to b. Bothfragments are optional and their in-/exclusion is controlled byconfiguration options (a.k.a. features), by assigning (boolean)values, so-called presence conditions, to each of them. In our
2014 IEEE International Conference on Software Maintenance and Evolution
1063-6773/14 $31.00 © 2014 IEEE
DOI 10.1109/ICSME.2014.82
502
2014 IEEE International Conference on Software Maintenance and Evolution
1063-6773/14 $31.00 © 2014 IEEE
DOI 10.1109/ICSME.2014.82
501
2014 IEEE International Conference on Software Maintenance and Evolution
1063-6773/14 $31.00 © 2014 IEEE
DOI 10.1109/ICSME.2014.82
501
Slicing Feature ModelsMathieu Acher, Philippe Collet, Philippe Lahire
I3S – CNRS UMR 6070Université Nice Sophia Antipolis, France
{acher,collet,lahire}@i3s.unice.fr
Robert B. FranceComputer Science Department
Colorado State University, [email protected]
Abstract—Feature models (FMs) are a popular formalism fordescribing the commonality and variability of software productlines (SPLs) in terms of features. As SPL development increas-ingly involves numerous large FMs, scalable modular techniquesare required to manage their complexity. In this paper, we presenta novel slicing technique that produces a projection of an FM,including constraints. The slicing allows SPL practitioners to findsemantically meaningful decompositions of FMs and has beenintegrated into the FAMILIAR language.
Keywords-Feature Models; Software Product Lines; Slicing;
I. INTRODUCTION
The goal of software product line (SPL) engineering is toproduce a family of related program variants for a domain [1].SPL development starts with an analysis of the domain toidentify commonalities and differences between the membersof the family. A common way is to describe variabilities ofan SPL in terms of features which are domain abstractionsrelevant to stakeholders [2]. A Feature Model (FM) is usedto compactly represent all features in an SPL and define theirvalid combinations [3], [4].
FMs are becoming increasingly complex. There are a num-ber of factors that contribute to their growing complexity. Acontributing factor is that FMs are being used not only todescribe variability in software designs, but also variability inwider system contexts [5], [6], [7]. For example, features mayrefer to high-level requirements as well as to properties ofthe software platform. Another contributing factor is the useof multiple FMs to describe SPLs. It has been observed thatmaintaining a single large FM for the entire system may not befeasible [8], [5], [1], [9]. Following a model-based approach,several FMs are usually designed to describe software featuresat various levels of abstraction and to manage variability ofartifacts that are produced in different development phases [1],[2]. In addition, organizations are increasingly faced with thechallenge of managing variability in product parts providedby external suppliers [6], [10]. The need to support multipleSPLs (also called product populations) makes developing SPLschallenging [1]. Product populations with FMs consisting ofhundreds to thousands of features have been observed [11],[12], [13]. Finally, automated extraction of FMs from largeimplemented software systems [14], can produce FMs withthousands of features.
Managing the complexity of building FMs with a largenumber of features that are related in a variety of ways isa two-fold challenge. On the one hand FM decompositiontechniques must be provided to better manage this complexity.On the other hand, automated rigorous reasoning techniques
must be provided to relieve error-prone and tedious tasksassociated with manually analyzing FMs [13]. Naturally, withFMs being handled by several stakeholders, or even differentorganizations, techniques that provide good support for theprinciples of separation of concerns seem particularly well-suited solution to the problem of effective decompositionof FMs. Two ways to support separation of concerns is toprovide mechanisms for extracting views from large FMs [15],and to provide mechanisms for synthetizing large FMs fromsmaller composable FMs. In previous work [17], we designeda set of composition operators for FMs (insertion, merging,aggregation) and defined semantic properties that must bepreserved during composition. In this paper we present a novelslicing technique that produces a projection of an FM (a slice)with respect to a set of selected features (slicing criterion). Itallows SPL developers to automatically obtain semanticallymeaningful decompositions of FMs.
II. BACKGROUND: FEATURE MODELS
FMs were first introduced in the FODA method [8], whichalso provided a graphical representation through Feature Dia-grams. The two essential components of an FM are hierarchyand variability. A FM hierarchy (typically a tree) structuresapplication features into multiple levels of increasing detail.The variability aspect of an FM restricts the legal combinationof features and is expressed through a number of mechanisms.The features may be optional or mandatory or may form Xoror Or-groups. In addition, implies or excludes constraints thatcut across the hierarchy can be specified to express morecomplex dependencies between features. We consider thatan FM is composed of a feature diagram (see Definition 1)plus a set of constraints expressed in propositional logic (seeDefinition 2). Figure 1a shows an example of an FM.
Definition 1 (Feature Diagram): A feature diagram FD =�G, r,EMAND,FXOR,FOR, Impl, Excl� is defined as fol-lows:
• G = (F , E) is a rooted tree where F is a finite setof features and E ⊆ F × F is a finite set of edges(edges represent top-down hierarchical decomposition offeatures, i.e., parent-child relations between them) ;
• r ∈ F is the root feature ;• EMAND ⊆ E is a set of edges that defines mandatory
features with their parents ;• FXOR ⊆ P(F)×F and FOR ⊆ P(F)×F define feature
groups and are sets of pairs of child features together withits common parent feature. The child features are eitherexclusive (Xor-groups) or inclusive (Or-groups) ;
978-1-4577-1639-3/11/$26.00 c© 2011 IEEE ASE 2011, Lawrence, KS, USA
424
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 6Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
A1 A2
t1: e1/e2
B1 B2
t2: e3/e4
S
C1 C2
t4: e1/e2
t5: e2/
C
t3: e4/e5Mo
del
Incr
emen
tal
Dep
end
ency
Gra
ph
Ad
apta
tio
n
Incr
emen
tal
Slic
eC
om
pu
tati
on
DepM
S
C
A1
B1
C1
t1
t2
t4
t3
t5
A2
B2
C2
rcd gcd
rcd
gcd tcd
rcd gcd tcd
Dep
end
ency
Gra
ph
Slicing Criterion
B1 B2
t2: e3/e4
S
Slic
e
Core Variant
∆M
A2 A1
t6 : e2/e6
+δ1 : B1 B2
t2 : e3/e4-δ2 :
B1 B2
t7 : e6/e4
+δ3 : C1 t8 : e3/e7
+
δ4 :
Regression Delta
∆DG
∆Slice
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
C1 C2
t4: e1/e2
t5: e2/
t8: e3/e7
C
t3: e4/e5
DepM
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
rcd gcd tcd
sdrcd
gcd tcd
rcd gcd
gcd
tcd
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
+
++
rcd gcd tcd+
sd +rcd
gcd
+
tcd
+
rcd gcd
gcd+
tcd
B1
t2
t3
-
gcd-
tcd-
Slicing Criterion
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
A1 A2
t1: e1/e2+
t6: e2/e6
+
B1 B2
t7: e6/e4+
+ +
S
B1
B2
t2: e3/e4
-
Variant One
Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]
A1 A2
t1: e1/e2
B1 B2
t2: e3/e4
S
C1 C2
t4: e1/e2
t5: e2/
C
t3: e4/e5Mo
del
Incr
emen
tal
Dep
end
ency
Gra
ph
Ad
apta
tio
n
Incr
emen
tal
Slic
eC
om
pu
tati
on
DepM
S
C
A1
B1
C1
t1
t2
t4
t3
t5
A2
B2
C2
rcd gcd
rcd
gcd tcd
rcd gcd tcd
Dep
end
ency
Gra
ph
Slicing Criterion
B1 B2
t2: e3/e4
S
Slic
e
Core Variant
∆M
A2 A1
t6 : e2/e6
+δ1 : B1 B2
t2 : e3/e4-δ2 :
B1 B2
t7 : e6/e4
+δ3 : C1 t8 : e3/e7
+
δ4 :
Regression Delta
∆DG
∆Slice
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
C1 C2
t4: e1/e2
t5: e2/
t8: e3/e7
C
t3: e4/e5
DepM
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
rcd gcd tcd
sdrcd
gcd tcd
rcd gcd
gcd
tcd
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
+
++
rcd gcd tcd+
sd +rcd
gcd
+
tcd
+
rcd gcd
gcd+
tcd
B1
t2
t3
-
gcd-
tcd-
Slicing Criterion
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
A1 A2
t1: e1/e2+
t6: e2/e6
+
B1 B2
t7: e6/e4+
+ +
S
B1
B2
t2: e3/e4
-
Variant One
Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]
A1 A2
t1: e1/e2
B1 B2
t2: e3/e4
S
C1 C2
t4: e1/e2
t5: e2/
C
t3: e4/e5Mo
del
Incr
emen
tal
Dep
end
ency
Gra
ph
Ad
apta
tio
n
Incr
emen
tal
Slic
eC
om
pu
tati
on
DepM
S
C
A1
B1
C1
t1
t2
t4
t3
t5
A2
B2
C2
rcd gcd
rcd
gcd tcd
rcd gcd tcd
Dep
end
ency
Gra
ph
Slicing Criterion
B1 B2
t2: e3/e4
S
Slic
e
Core Variant
∆M
A2 A1
t6 : e2/e6
+δ1 : B1 B2
t2 : e3/e4-δ2 :
B1 B2
t7 : e6/e4
+δ3 : C1 t8 : e3/e7
+
δ4 :
Regression Delta
∆DG
∆Slice
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
C1 C2
t4: e1/e2
t5: e2/
t8: e3/e7
C
t3: e4/e5
DepM
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
rcd gcd tcd
sdrcd
gcd tcd
rcd gcd
gcd
tcd
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
+
++
rcd gcd tcd+
sd +rcd
gcd
+
tcd
+
rcd gcd
gcd+
tcd
B1
t2
t3
-
gcd-
tcd-
Slicing Criterion
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
A1 A2
t1: e1/e2+
t6: e2/e6
+
B1 B2
t7: e6/e4+
+ +
S
B1
B2
t2: e3/e4
-
Variant One
Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]
A1 A2
t1: e1/e2
B1 B2
t2: e3/e4
S
C1 C2
t4: e1/e2
t5: e2/
C
t3: e4/e5Mo
del
Incr
emen
tal
Dep
end
ency
Gra
ph
Ad
apta
tio
n
Incr
emen
tal
Slic
eC
om
pu
tati
on
DepM
S
C
A1
B1
C1
t1
t2
t4
t3
t5
A2
B2
C2
rcd gcd
rcd
gcd tcd
rcd gcd tcd
Dep
end
ency
Gra
ph
Slicing Criterion
B1 B2
t2: e3/e4
S
Slic
e
Core Variant
∆M
A2 A1
t6 : e2/e6
+δ1 : B1 B2
t2 : e3/e4-δ2 :
B1 B2
t7 : e6/e4
+δ3 : C1 t8 : e3/e7
+
δ4 :
Regression Delta
∆DG
∆Slice
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
C1 C2
t4: e1/e2
t5: e2/
t8: e3/e7
C
t3: e4/e5
DepM
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
rcd gcd tcd
sdrcd
gcd tcd
rcd gcd
gcd
tcd
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
+
++
rcd gcd tcd+
sd +rcd
gcd
+
tcd
+
rcd gcd
gcd+
tcd
B1
t2
t3
-
gcd-
tcd-
Slicing Criterion
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
A1 A2
t1: e1/e2+
t6: e2/e6
+
B1 B2
t7: e6/e4+
+ +
S
B1
B2
t2: e3/e4
-
Variant One
Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]
A1 A2
t1: e1/e2
B1 B2
t2: e3/e4
S
C1 C2
t4: e1/e2
t5: e2/
C
t3: e4/e5Mo
del
Incr
emen
tal
Dep
end
ency
Gra
ph
Ad
apta
tio
n
Incr
emen
tal
Slic
eC
om
pu
tati
on
DepM
S
C
A1
B1
C1
t1
t2
t4
t3
t5
A2
B2
C2
rcd gcd
rcd
gcd tcd
rcd gcd tcd
Dep
end
ency
Gra
ph
Slicing Criterion
B1 B2
t2: e3/e4
S
Slic
e
Core Variant
∆M
A2 A1
t6 : e2/e6
+δ1 : B1 B2
t2 : e3/e4-δ2 :
B1 B2
t7 : e6/e4
+δ3 : C1 t8 : e3/e7
+
δ4 :
Regression Delta
∆DG
∆Slice
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
C1 C2
t4: e1/e2
t5: e2/
t8: e3/e7
C
t3: e4/e5
DepM
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
rcd gcd tcd
sdrcd
gcd tcd
rcd gcd
gcd
tcd
S
t8
C
A1
B1
C1
t1
t7
t4
t6
t3
t5
A2
B2
C2
+
++
rcd gcd tcd+
sd +rcd
gcd
+
tcd
+
rcd gcd
gcd+
tcd
B1
t2
t3
-
gcd-
tcd-
Slicing Criterion
A1 A2
t1: e1/e2
t6: e2/e6
B1 B2
t7: e6/e4
S
A1 A2
t1: e1/e2+
t6: e2/e6
+
B1 B2
t7: e6/e4+
+ +
S
B1
B2
t2: e3/e4
-
Variant One
Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]
Discussion
Dependence on DifferencesNumber of model deltas
Impact/Distributability of changes
Worst CaseDistributed changes
No effort reduction
Average CaseLocal changes
Exploit commonality
Benefits1. Effort reduction for dependency graph generation
2. Direct derivation of slice changes
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 8Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Evaluation – BCS Case Study [LLLS12]
Body Comfort System
27 features ⇒ 11616 variants
Sampling subset: 18 Products(incl. core) [OLZG11, LSKL12]
Sampling result ⇒ Order
Model size:∅ Elements: 106∅ States: 40∅ Transition: 66∅ Hierarchy Depth: 3
Finger Protection
Body Comfort System
Power Window
Central Locking System
Automatic Locking
Remote Control
Key
Human Machine Interface
Manual Power
Window
Automatic Power
Window
Control Alarm System
Safety Function
Interior Monitoring
Alarm System
Exterior Mirror
Electric Heatable
require
require
require
Status LED
Security Door
System
LED Central Locking System
LED Power
Window
LED Exterior Mirror
LED Heatable
LED Alarm System
LED Finger Protection
Control Automatic
Power Window
require
exclude
require
Adjust Exterior Mirror
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 9Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Evaluation – Dependency Graph (First Results)
64
97
96
13
4
94
12
7
13
9
66
10
9
16
3
87
12
4
89
10
1
10
2
96
14
7
69
40
96
94
09
92
16
17
95
6
88
36
16
12
9
19
32
1
43
56
11
88
1
26
56
9
75
69
15
37
6
79
21
10
20
1
10
40
4
92
16
21
60
9
47
61
0
46
56
45
60
89
11
43
71
80
01
15
90
21
45
37
41
73
17
37
41
76
30
39
16
50
50
51
51
45
60
10
73
1
23
46
0
5000
10000
15000
20000
25000
30000
P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17
# D
ep
en
de
ncy
Ch
eck
s
Products
# Elements
# Checks Graph Norm.
# Checks Graph Incr.
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 10Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Evaluation – Slicing and Model Diff (First Results)
1,6
9
0,3
8
0,4
3
0,6
0,6
8
0,3
7
0,5
4
0,6
4
0,6
0,1
3
1,3
6
0,7
8
0,2
9
0,3
3
0,8
0,6
7
0,0
9
38
8 4
07
,05
42
8,7
8
42
5,6
39
7,5
7
39
8,2
9
41
7,5
4
40
9 4
27
,1
43
3,2
6
41
5,1
3
35
0,7
8
44
9,2
4
41
2,8
1
38
7,3
2
39
4,0
8
39
6,1
4
38
,92
34
,02
62
,96
54
,5
58
42
,27
67
,62
33
,45
97
,27
15
5,3
4
57
,5
39
,15
30
,47
32
,67
29
,68
82
,89
30
,77
0
50
100
150
200
250
300
350
400
450
500
P0 P1 (13) P2 (34) P3 (23) P4 (20) P5 (19) P6 (49) P7 (13) P8 (22) P9 (40) P10 (23) P11 (22) P12 (19) P13 (21) P14 (21) P15 (40) P16 (27) P17 (22)
Tim
e in
ms
Products (# Equal Elements)
Slice Norm. Slice Diff. Time Slice Norm. Slice Incr.
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 11Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
First Idea – Retest Coverage Criterion
Slice differences indicate retest potentials basedon the impact of model changes
What are retest requirements/goals?Elements corresponding to slice differencesElements connected to those elements
Applicable for selection and prioritization(Pairwise) coverage of retest requirements/goalsNumber of occurrences of a difference usable as weighting factorPotential retest test case generation
Retest requirements/goals refer only to retestable behavior
Model parts not influenced by changes indicate behavior not to beretested
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 12Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Conclusion & Future Work
ConclusionRegression Testing ⇒ Incremental SPL testing
Incremental model slicing ⇒ Change impact analysis
Slice differences ⇒ Retest potentials
Retest decision ⇒ Retest coverage criterion
Future WorkComprehensive evaluation
Extension of control and data dependency
Extension & optimization of implementation
Application for (evolution-aware) change impact analysis
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 13Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
Thank You for Your Attention! Any Questions?
IMoTEP Integrated Model-based Testing of
Continuously Evolving SoftwareProduct Lines
TU Braunschweig, Institute for Programming and Reactive SystemsSascha Lity
Email: [email protected]
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 14Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
References I
Kelly Androutsopoulos, David Clark, Mark Harman, Jens Krinke, and Laurence Tratt.State-based Model Slicing: A Survey.ACM Comput. Surv., 45(4):53:1–53:36, August 2013.
Mathieu Acher, Philippe Collet, Philippe Lahire, and Robert B. France.Slicing feature models.In ASE’11, pages 424–427, 2011.
Hiralal Agrawal, Joseph R. Horgan, Edward W. Krauser, and Saul London.Incremental regression testing.In Proceedings of the Conference on Software Maintenance, ICSM ’93, pages 348–357,Washington, DC, USA, 1993. IEEE Computer Society.
Samuel Bates and Susan Horwitz.Incremental program testing using program dependence graphs.In Proceedings of the 20th ACM SIGPLAN-SIGACT Symposium on Principles ofProgramming Languages, POPL ’93, pages 384–396, New York, NY, USA, 1993. ACM.
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 15Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
References II
Dave Clarke, Michiel Helvensteijn, and Ina Schaefer.Abstract Delta Modeling.Mathematical Structures in Computer Science, 25(3):482–527, 2015.
Emelie Engström.Exploring Regression testing and software product line testing - research and state ofpractice.Lic dissertation, Lund University, May 2010.
R. Gupta, M.J. Harrold, and M.L. Soffa.An approach to regression testing using slicing.In Software Maintenance, 1992. Proceerdings., Conference on, pages 299–308, Nov 1992.
Wang Ji, Dong Wei, and Qi Zhi-Chang.Slicing Hierarchical Automata for Model Checking UML Statecharts.In Formal Methods and Software Engineering, pages 435–446. Springer, 2002.
Jochen Kamischke, Malte Lochau, and Hauke Baller.Conditioned Model Slicing of Feature-Annotated State Machines.In FOSD’12, pages 9–16, 2012.
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 16Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
References III
Frederik Kanning and Sandro Schulze.Program Slicing in the Presence of Variability.In ICSME’14 - ERA Track, 2014.
Sascha Lity, Hauke Baller, and Ina Schaefer.Towards incremental model slicing for delta-oriented software product lines.In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22ndInternational Conference on, pages 530–534, March 2015.
Malte Lochau, Sascha Lity, Remo Lachmann, Ina Schaefer, and Ursula Goltz.Delta-oriented Model-based Integration Testing of Large-scale Systems.Journal of Systems and Software, 91:63–84, 2014.
Sascha Lity, Remo Lachmann, Malte Lochau, and Ina Schaefer.Delta-oriented Software Product Line Test Models - The Body Comfort System CaseStudy.Technical Report 2012-07, Technische Universität Braunschweig, 2012.
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 17Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
References IV
Malte Lochau.Model-Based Conformance Testing of Software Product Lines.PhD thesis, Technische Universität Braunschweig, 2012.
Malte Lochau, Ina Schaefer, Jochen Kamischke, and Sascha Lity.Incremental Model-Based Testing of Delta-Oriented Software Product Lines.In TAP’12, pages 67–82. Springer, 2012.
S. Oster, M. Lochau, M. Zink, and M. Grechanik.Pairwise Feature-Interaction Testing for SPLs: Potentials and Limitations.In FOSD’11, 2011.
Alessandro Orso, Saurabh Sinha, and Mary Jean Harrold.Incremental Slicing Based on Data-Dependences Types.In ICSM’01, page 158, 2001.
Gregg Rothermel.Efficient, Effective Regression Testing Using Safe Test Selection Techniques.PhD thesis, Clemson University, May 1996.
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 18Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme
References V
Antti Tevanlinna, Juha Taina, and Raine Kauppinen.Product Family Testing: A Survey.SIGSOFT Softw. Eng. Notes, 29(2):12–12, March 2004.
Heike Wehrheim.Incremental Slicing.In Formal Methods and Software Engineering, ICFEM’06, pages 514–528, 2006.
Mark Weiser.Program slicing.In ICSE’81, pages 439–449, 1981.
May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 19Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung
und Reaktive Systeme