Towards Incremental Model Slicing for SPL Regression...

28
Towards Incremental Model Slicing for SPL Regression Testing Sascha Lity , Hauke Baller, Ina Schaefer, May 14 th , 2015 Technische Universität Braunschweig Institut für Programmierung und Reaktive Systeme

Transcript of Towards Incremental Model Slicing for SPL Regression...

Towards Incremental Model Slicingfor SPL Regression Testing

Sascha Lity, Hauke Baller, Ina Schaefer, May 14th, 2015

Technische Universität Braunschweig

Institut für Programmierungund Reaktive Systeme

Motivation

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 2Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Motivation

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 2Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Motivation

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 2Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Product-by-Product Testing

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 3Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Product-by-Product Testing

Reuse Artifacts/Results

Reuse Artifacts/Results

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 3Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Product-by-Product Testing

Reuse Artifacts/Results

Reuse Artifacts/Results

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 3Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Applying Regression Testing for SPLs

Regression testing allows for incremental SPL testing [TTK04, Eng10]

Regression Testing validates whether modified softwarebehaves as intended, and modifications have not adverselyimpacted the software’s quality. [Rot96]

Software version testing vs. software variant testing?

Reuse of test artifacts? ⇒ Delta Modeling [CHS15]

Incremental SPL testing? ⇒ Delta-oriented [LSKL12, Loc12, LLL+14]

Retest decision? ⇒ Slicing [AHKL93, BH93, GHS92]

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 4Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Applying Regression Testing for SPLs

Regression testing allows for incremental SPL testing [TTK04, Eng10]

Regression Testing validates whether modified softwarebehaves as intended, and modifications have not adverselyimpacted the software’s quality. [Rot96]

Software version testing vs. software variant testing?

Reuse of test artifacts? ⇒ Adaption of Delta Modeling [CHS15]

Incremental SPL testing? ⇒ Delta-oriented [LSKL12, Loc12, LLL+14]

Retest decision? ⇒ Slicing [AHKL93, BH93, GHS92]

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 4Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Program/Model Slicing

Static program projection technique by Weiser [Wei81]

Preserves syntax and semantics w.r.t. slicing criterion

Based on control/data dependencies

Model slicing [JWZC02, KLB12, ACH+13] adapts concept for state-basedmodels

Slicing Criterion

(S,V)

p p'

(S,V)(S,V)

M M‘

Slicing Criterion B2

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 5Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Variability-aware Slicing [ACLF11, KLB12, KS14]

Conditioned Model Slicing of Feature-AnnotatedState Machines

Jochen KamischkeInstitut für Programmierung

und Reaktive SystemeMühlenpfordtstr. 23

Braunschweig, [email protected]

Malte LochauInstitut für Programmierung

und Reaktive SystemeMühlenpfordtstr. 23

Braunschweig, [email protected]

Hauke BallerInstitut für Programmierung

und Reaktive SystemeMühlenpfordtstr. 23

Braunschweig, [email protected]

ABSTRACTModel-based behavioral specifications build the basis forcomprehensive quality assurance techniques for complexsoftware systems such as model checking and model-basedtesting. Various attempts exist to adopt those approachesto variant-rich applications as apparent in software productline engineering to efficiently analyze families of similar soft-ware systems. Therefore, models are usually enriched withcapabilities to explicitly specify variable parts by means ofannotations denoting selection conditions over feature pa-rameters. However, a major drawback of model-based en-gineering is still its lack of scalability. Model slicing pro-vides a promising technique to reduce models to only thoseobjects being relevant for a certain criterion under consid-eration such as a particular test goal. Here, we present anapproach for slicing feature-annotated state machine mod-els. To support feature-oriented slicing on those models,our framework combines principles of variability encodingand conditioned slicing. We also present an implementationand provide experimental results concerning the efficiencyof the slicing algorithm.

Categories and Subject DescriptorsD.2.4 [Software Engineering]: Software/Program Veri-fication; D.2.13 [Software Engineering]: Reusable Soft-ware, Reuse Models

General TermsDesign, Theory

KeywordsSoftware Product Lines, Model-Based Software Engineering.

1. INTRODUCTIONModel-based software engineering provides a rich collec-

tion of modeling languages and corresponding techniques for

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.FOSD’12, September 24–25, 2012, Dresden, Germany.Copyright 2012 ACM 978-1-4503-1309-4/12/09 ...$10.00.

the specification, documentation, maintenance, and verifi-cation/validation of high-quality software systems in a sys-tematic way. In particular, modeling approaches for specify-ing the operational behavior of a software system are oftenbased on state-transition diagrams such as UML state ma-chines [18]. Thereupon, various applications of those modelsto verification/validation techniques like model-based test-ing and model checking have been proposed [5, 1].

However, the major drawback of those approaches is stilltheir lack of scalability. This is even worse in the presenceof explicit variability at model level as apparent, e.g., insoftware product line engineering [15]. Herein, models areenriched with capabilities to specify common and variableparts occurring in a family of similar product variants. Forinstance, model elements are annotated with selection con-ditions, i.e., propositional formulas over feature parametersto guide the assembling of model variants w.r.t. a particu-lar feature configuration [5]. Hence, such feature-annotatedmodels integrate any potential behaviors of all product vari-ants of an SPL within a virtual, so-called 150% model. Thisadditional model dimension as tailored by the valid prod-uct configuration space of an underlying domain featuremodel [11] further complicates the application of model-based analysis techniques to variant-rich, real world prob-lems represented by an SPL.

Model slicing provides a promising approach to handlethe complexity problem of behavioral models by perform-ing static, i.e., syntactical model reductions that preservesome aspects of model semantics w.r.t. a slicing criterion un-der consideration [22, 21]. Therefore, model slicing extractsthose model parts affecting certain computational units onlyand, at the same time, ensuring the resulting model slice topreserve a syntactically well-formed model structure. Slicinghas gained applications in various fields of program analysisfor reverse engineering, program integration, software met-rics, component reuse, etc. [3]. Model slicing adopts theconcepts of program slicing to reduce verification/validationefforts. For instance, in model-based testing, choosing testgoals as slicing criteria allows for efficient test case genera-tion, debugging, and change impact analysis during regres-sion testing, whereas slicing along a certain model property,e.g., given as an LTL formula, decreases model checkingcomplexity. However, recent model slicing approaches areincapable to cope with models enriched with feature anno-tations. In the presence of variable model parts, the furthermodel dimension has to be taken into account when slic-ing for a particular criterion in order to yield a well-formed

9

Program Slicing in the Presence ofPreprocessor Variability

Frederik KanningTU Braunschweig

Braunschweig, GermanyEmail: [email protected]

Sandro SchulzeTU Braunschweig

Braunschweig, GermanyEmail: [email protected]

Abstract—Program slicing is a common means to supportdevelopers in examining the source code with respect to de-bugging, program comprehension, or regression testing. Whilea vast amount of techniques exist, they are mostly tailored tosingle software systems. However, with the increasing importanceof variable and highly-configurable systems, such as the Linuxkernel, the number of software variants, subject to analysis,increases dramatically. Consequently, it is infeasible to applyslicing on each variant in isolation. To overcome this problem,we propose variability-aware slicing, a technique that can dealwith source code variability, specifically conditional compilationas introduced by the C preprocessor. Particularly, we providedetails of our variability-aware dependence analysis for programslicing, point out benefits of our slicing technique, and mentioncurrent limitations and future work.

I. INTRODUCTION

Program slicing is a common and well-established methodfor detailed program analysis, which has been proposed morethan three decades ago by Weiser [1]. Basically, programslicing allows to decompose a program with respect to itscomputational dependencies. As a result, it is possible toinspect the program in more detail, for instance, to assess the(potential) influence of a set of statements (the program slice)on a particular point in the program (the slicing criterion).Meanwhile, numerous techniques have been proposed that ex-tend the original (static) slicing technique of Weiser and thus,support developers in one or more tasks, such as debugging,program comprehension, or regression testing [2], [3].

While existing techniques are very mature and applicableeven at large scale for single (monolithic) programs, they areonly of limited use for variable software systems. Such systemsusually constitute a whole family of related programs thatcan be derived from a common code base [4]. To this end,configuration options, or features allow users to specify vari-able parts in such systems. A common means to express thisvariability are preprocessor directives, in particular conditionalcompilation, as provided by the C preprocessor tool CPP [5],[6]. While this allows users to tailor a program according tospecific needs, it also gives rise to an exponential numberof configurations. For instance, the Linux kernel consists ofmore than 11, 000 features, which allow for billions of possibleconfigurations to be compiled and generated on demand.

In this paper we bridge the gap between slicing andsource code variability by proposing a technique that supportsexploration and static analysis of variable software systems.

Research problem: Obviously, statically analyzing bil-lions of configurations separately is infeasible. On the otherhand, current slicing techniques mainly work on preprocessedprograms (i.e., a particular configuration), that is, the CPPdirectives are evaluated and removed before analysis, and thus,variability is not supported. In other words, such kind ofanalysis is not variability-aware [7]. As a result, these slicingtechniques are neither applicable to variable software systemsnor can detect effects of variability, such as errors caused byfeature interactions [8].

Contribution: The main contribution of this paper is aconcept and prototypical implementation for variability-awareintraprocedural slicing that works on annotated C programs(i.e., before preprocessing CPP directives). To this end, weextend and instrument TYPECHEF, a research infrastructurefor variability-aware analysis [9], [10]. Particularly, we providedetails on our variability-aware dependence analysis and howwe use this analysis to compute precise slices that includevariability information. Moreover, we motivate our techniqueby means of an example and possible applications.

While the current implementation exhibits still some lim-itations, we argue that it provides an important foundationfor taming variability and thus, provide efficient and scalableprogram analysis for variable software systems.

II. BACKGROUND

In the following, we provide information about sourcecode variability using CPP directives. Moreover, we lay thefoundations of our slicing technique by introducing the basicconcepts of program slicing.

A. Variable Systems with the CPP

The C preprocessor CPP is a tool that is tightly integratedwith the C programming language and that allows for develop-ing variable software systems [6]. To this end, the CPP providescapabilities to annotate variable code fragments at arbitrarygranularity using conditional compilation (a.k.a. #ifdefs).As an example, we show a program excerpt in Figure 1 thatcontains two variable code fragments: One variable fragment(Line 6–8) redeclares variable c in the inner scope, while theother one (Line 13–15) assigns a constant value to b. Bothfragments are optional and their in-/exclusion is controlled byconfiguration options (a.k.a. features), by assigning (boolean)values, so-called presence conditions, to each of them. In our

2014 IEEE International Conference on Software Maintenance and Evolution

1063-6773/14 $31.00 © 2014 IEEE

DOI 10.1109/ICSME.2014.82

502

2014 IEEE International Conference on Software Maintenance and Evolution

1063-6773/14 $31.00 © 2014 IEEE

DOI 10.1109/ICSME.2014.82

501

2014 IEEE International Conference on Software Maintenance and Evolution

1063-6773/14 $31.00 © 2014 IEEE

DOI 10.1109/ICSME.2014.82

501

Slicing Feature ModelsMathieu Acher, Philippe Collet, Philippe Lahire

I3S – CNRS UMR 6070Université Nice Sophia Antipolis, France

{acher,collet,lahire}@i3s.unice.fr

Robert B. FranceComputer Science Department

Colorado State University, [email protected]

Abstract—Feature models (FMs) are a popular formalism fordescribing the commonality and variability of software productlines (SPLs) in terms of features. As SPL development increas-ingly involves numerous large FMs, scalable modular techniquesare required to manage their complexity. In this paper, we presenta novel slicing technique that produces a projection of an FM,including constraints. The slicing allows SPL practitioners to findsemantically meaningful decompositions of FMs and has beenintegrated into the FAMILIAR language.

Keywords-Feature Models; Software Product Lines; Slicing;

I. INTRODUCTION

The goal of software product line (SPL) engineering is toproduce a family of related program variants for a domain [1].SPL development starts with an analysis of the domain toidentify commonalities and differences between the membersof the family. A common way is to describe variabilities ofan SPL in terms of features which are domain abstractionsrelevant to stakeholders [2]. A Feature Model (FM) is usedto compactly represent all features in an SPL and define theirvalid combinations [3], [4].

FMs are becoming increasingly complex. There are a num-ber of factors that contribute to their growing complexity. Acontributing factor is that FMs are being used not only todescribe variability in software designs, but also variability inwider system contexts [5], [6], [7]. For example, features mayrefer to high-level requirements as well as to properties ofthe software platform. Another contributing factor is the useof multiple FMs to describe SPLs. It has been observed thatmaintaining a single large FM for the entire system may not befeasible [8], [5], [1], [9]. Following a model-based approach,several FMs are usually designed to describe software featuresat various levels of abstraction and to manage variability ofartifacts that are produced in different development phases [1],[2]. In addition, organizations are increasingly faced with thechallenge of managing variability in product parts providedby external suppliers [6], [10]. The need to support multipleSPLs (also called product populations) makes developing SPLschallenging [1]. Product populations with FMs consisting ofhundreds to thousands of features have been observed [11],[12], [13]. Finally, automated extraction of FMs from largeimplemented software systems [14], can produce FMs withthousands of features.

Managing the complexity of building FMs with a largenumber of features that are related in a variety of ways isa two-fold challenge. On the one hand FM decompositiontechniques must be provided to better manage this complexity.On the other hand, automated rigorous reasoning techniques

must be provided to relieve error-prone and tedious tasksassociated with manually analyzing FMs [13]. Naturally, withFMs being handled by several stakeholders, or even differentorganizations, techniques that provide good support for theprinciples of separation of concerns seem particularly well-suited solution to the problem of effective decompositionof FMs. Two ways to support separation of concerns is toprovide mechanisms for extracting views from large FMs [15],and to provide mechanisms for synthetizing large FMs fromsmaller composable FMs. In previous work [17], we designeda set of composition operators for FMs (insertion, merging,aggregation) and defined semantic properties that must bepreserved during composition. In this paper we present a novelslicing technique that produces a projection of an FM (a slice)with respect to a set of selected features (slicing criterion). Itallows SPL developers to automatically obtain semanticallymeaningful decompositions of FMs.

II. BACKGROUND: FEATURE MODELS

FMs were first introduced in the FODA method [8], whichalso provided a graphical representation through Feature Dia-grams. The two essential components of an FM are hierarchyand variability. A FM hierarchy (typically a tree) structuresapplication features into multiple levels of increasing detail.The variability aspect of an FM restricts the legal combinationof features and is expressed through a number of mechanisms.The features may be optional or mandatory or may form Xoror Or-groups. In addition, implies or excludes constraints thatcut across the hierarchy can be specified to express morecomplex dependencies between features. We consider thatan FM is composed of a feature diagram (see Definition 1)plus a set of constraints expressed in propositional logic (seeDefinition 2). Figure 1a shows an example of an FM.

Definition 1 (Feature Diagram): A feature diagram FD =�G, r,EMAND,FXOR,FOR, Impl, Excl� is defined as fol-lows:

• G = (F , E) is a rooted tree where F is a finite setof features and E ⊆ F × F is a finite set of edges(edges represent top-down hierarchical decomposition offeatures, i.e., parent-child relations between them) ;

• r ∈ F is the root feature ;• EMAND ⊆ E is a set of edges that defines mandatory

features with their parents ;• FXOR ⊆ P(F)×F and FOR ⊆ P(F)×F define feature

groups and are sets of pairs of child features together withits common parent feature. The child features are eitherexclusive (Xor-groups) or inclusive (Or-groups) ;

978-1-4577-1639-3/11/$26.00 c© 2011 IEEE ASE 2011, Lawrence, KS, USA

424

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 6Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

A1 A2

t1: e1/e2

B1 B2

t2: e3/e4

S

C1 C2

t4: e1/e2

t5: e2/

C

t3: e4/e5Mo

del

Incr

emen

tal

Dep

end

ency

Gra

ph

Ad

apta

tio

n

Incr

emen

tal

Slic

eC

om

pu

tati

on

DepM

S

C

A1

B1

C1

t1

t2

t4

t3

t5

A2

B2

C2

rcd gcd

rcd

gcd tcd

rcd gcd tcd

Dep

end

ency

Gra

ph

Slicing Criterion

B1 B2

t2: e3/e4

S

Slic

e

Core Variant

∆M

A2 A1

t6 : e2/e6

+δ1 : B1 B2

t2 : e3/e4-δ2 :

B1 B2

t7 : e6/e4

+δ3 : C1 t8 : e3/e7

+

δ4 :

Regression Delta

∆DG

∆Slice

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

C1 C2

t4: e1/e2

t5: e2/

t8: e3/e7

C

t3: e4/e5

DepM

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

rcd gcd tcd

sdrcd

gcd tcd

rcd gcd

gcd

tcd

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

+

++

rcd gcd tcd+

sd +rcd

gcd

+

tcd

+

rcd gcd

gcd+

tcd

B1

t2

t3

-

gcd-

tcd-

Slicing Criterion

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

A1 A2

t1: e1/e2+

t6: e2/e6

+

B1 B2

t7: e6/e4+

+ +

S

B1

B2

t2: e3/e4

-

Variant One

Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]

A1 A2

t1: e1/e2

B1 B2

t2: e3/e4

S

C1 C2

t4: e1/e2

t5: e2/

C

t3: e4/e5Mo

del

Incr

emen

tal

Dep

end

ency

Gra

ph

Ad

apta

tio

n

Incr

emen

tal

Slic

eC

om

pu

tati

on

DepM

S

C

A1

B1

C1

t1

t2

t4

t3

t5

A2

B2

C2

rcd gcd

rcd

gcd tcd

rcd gcd tcd

Dep

end

ency

Gra

ph

Slicing Criterion

B1 B2

t2: e3/e4

S

Slic

e

Core Variant

∆M

A2 A1

t6 : e2/e6

+δ1 : B1 B2

t2 : e3/e4-δ2 :

B1 B2

t7 : e6/e4

+δ3 : C1 t8 : e3/e7

+

δ4 :

Regression Delta

∆DG

∆Slice

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

C1 C2

t4: e1/e2

t5: e2/

t8: e3/e7

C

t3: e4/e5

DepM

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

rcd gcd tcd

sdrcd

gcd tcd

rcd gcd

gcd

tcd

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

+

++

rcd gcd tcd+

sd +rcd

gcd

+

tcd

+

rcd gcd

gcd+

tcd

B1

t2

t3

-

gcd-

tcd-

Slicing Criterion

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

A1 A2

t1: e1/e2+

t6: e2/e6

+

B1 B2

t7: e6/e4+

+ +

S

B1

B2

t2: e3/e4

-

Variant One

Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]

A1 A2

t1: e1/e2

B1 B2

t2: e3/e4

S

C1 C2

t4: e1/e2

t5: e2/

C

t3: e4/e5Mo

del

Incr

emen

tal

Dep

end

ency

Gra

ph

Ad

apta

tio

n

Incr

emen

tal

Slic

eC

om

pu

tati

on

DepM

S

C

A1

B1

C1

t1

t2

t4

t3

t5

A2

B2

C2

rcd gcd

rcd

gcd tcd

rcd gcd tcd

Dep

end

ency

Gra

ph

Slicing Criterion

B1 B2

t2: e3/e4

S

Slic

e

Core Variant

∆M

A2 A1

t6 : e2/e6

+δ1 : B1 B2

t2 : e3/e4-δ2 :

B1 B2

t7 : e6/e4

+δ3 : C1 t8 : e3/e7

+

δ4 :

Regression Delta

∆DG

∆Slice

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

C1 C2

t4: e1/e2

t5: e2/

t8: e3/e7

C

t3: e4/e5

DepM

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

rcd gcd tcd

sdrcd

gcd tcd

rcd gcd

gcd

tcd

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

+

++

rcd gcd tcd+

sd +rcd

gcd

+

tcd

+

rcd gcd

gcd+

tcd

B1

t2

t3

-

gcd-

tcd-

Slicing Criterion

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

A1 A2

t1: e1/e2+

t6: e2/e6

+

B1 B2

t7: e6/e4+

+ +

S

B1

B2

t2: e3/e4

-

Variant One

Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]

A1 A2

t1: e1/e2

B1 B2

t2: e3/e4

S

C1 C2

t4: e1/e2

t5: e2/

C

t3: e4/e5Mo

del

Incr

emen

tal

Dep

end

ency

Gra

ph

Ad

apta

tio

n

Incr

emen

tal

Slic

eC

om

pu

tati

on

DepM

S

C

A1

B1

C1

t1

t2

t4

t3

t5

A2

B2

C2

rcd gcd

rcd

gcd tcd

rcd gcd tcd

Dep

end

ency

Gra

ph

Slicing Criterion

B1 B2

t2: e3/e4

S

Slic

e

Core Variant

∆M

A2 A1

t6 : e2/e6

+δ1 : B1 B2

t2 : e3/e4-δ2 :

B1 B2

t7 : e6/e4

+δ3 : C1 t8 : e3/e7

+

δ4 :

Regression Delta

∆DG

∆Slice

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

C1 C2

t4: e1/e2

t5: e2/

t8: e3/e7

C

t3: e4/e5

DepM

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

rcd gcd tcd

sdrcd

gcd tcd

rcd gcd

gcd

tcd

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

+

++

rcd gcd tcd+

sd +rcd

gcd

+

tcd

+

rcd gcd

gcd+

tcd

B1

t2

t3

-

gcd-

tcd-

Slicing Criterion

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

A1 A2

t1: e1/e2+

t6: e2/e6

+

B1 B2

t7: e6/e4+

+ +

S

B1

B2

t2: e3/e4

-

Variant One

Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]

A1 A2

t1: e1/e2

B1 B2

t2: e3/e4

S

C1 C2

t4: e1/e2

t5: e2/

C

t3: e4/e5Mo

del

Incr

emen

tal

Dep

end

ency

Gra

ph

Ad

apta

tio

n

Incr

emen

tal

Slic

eC

om

pu

tati

on

DepM

S

C

A1

B1

C1

t1

t2

t4

t3

t5

A2

B2

C2

rcd gcd

rcd

gcd tcd

rcd gcd tcd

Dep

end

ency

Gra

ph

Slicing Criterion

B1 B2

t2: e3/e4

S

Slic

e

Core Variant

∆M

A2 A1

t6 : e2/e6

+δ1 : B1 B2

t2 : e3/e4-δ2 :

B1 B2

t7 : e6/e4

+δ3 : C1 t8 : e3/e7

+

δ4 :

Regression Delta

∆DG

∆Slice

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

C1 C2

t4: e1/e2

t5: e2/

t8: e3/e7

C

t3: e4/e5

DepM

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

rcd gcd tcd

sdrcd

gcd tcd

rcd gcd

gcd

tcd

S

t8

C

A1

B1

C1

t1

t7

t4

t6

t3

t5

A2

B2

C2

+

++

rcd gcd tcd+

sd +rcd

gcd

+

tcd

+

rcd gcd

gcd+

tcd

B1

t2

t3

-

gcd-

tcd-

Slicing Criterion

A1 A2

t1: e1/e2

t6: e2/e6

B1 B2

t7: e6/e4

S

A1 A2

t1: e1/e2+

t6: e2/e6

+

B1 B2

t7: e6/e4+

+ +

S

B1

B2

t2: e3/e4

-

Variant One

Lity, Baller, Schaefer: Towards Incremental Model Slicing for Delta-oriented SPLs, SANER’15 [LBS15]

Discussion

Dependence on DifferencesNumber of model deltas

Impact/Distributability of changes

Worst CaseDistributed changes

No effort reduction

Average CaseLocal changes

Exploit commonality

Benefits1. Effort reduction for dependency graph generation

2. Direct derivation of slice changes

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 8Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Evaluation – BCS Case Study [LLLS12]

Body Comfort System

27 features ⇒ 11616 variants

Sampling subset: 18 Products(incl. core) [OLZG11, LSKL12]

Sampling result ⇒ Order

Model size:∅ Elements: 106∅ States: 40∅ Transition: 66∅ Hierarchy Depth: 3

Finger Protection

Body Comfort System

Power Window

Central Locking System

Automatic Locking

Remote Control

Key

Human Machine Interface

Manual Power

Window

Automatic Power

Window

Control Alarm System

Safety Function

Interior Monitoring

Alarm System

Exterior Mirror

Electric Heatable

require

require

require

Status LED

Security Door

System

LED Central Locking System

LED Power

Window

LED Exterior Mirror

LED Heatable

LED Alarm System

LED Finger Protection

Control Automatic

Power Window

require

exclude

require

Adjust Exterior Mirror

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 9Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Evaluation – Dependency Graph (First Results)

64

97

96

13

4

94

12

7

13

9

66

10

9

16

3

87

12

4

89

10

1

10

2

96

14

7

69

40

96

94

09

92

16

17

95

6

88

36

16

12

9

19

32

1

43

56

11

88

1

26

56

9

75

69

15

37

6

79

21

10

20

1

10

40

4

92

16

21

60

9

47

61

0

46

56

45

60

89

11

43

71

80

01

15

90

21

45

37

41

73

17

37

41

76

30

39

16

50

50

51

51

45

60

10

73

1

23

46

0

5000

10000

15000

20000

25000

30000

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17

# D

ep

en

de

ncy

Ch

eck

s

Products

# Elements

# Checks Graph Norm.

# Checks Graph Incr.

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 10Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Evaluation – Slicing and Model Diff (First Results)

1,6

9

0,3

8

0,4

3

0,6

0,6

8

0,3

7

0,5

4

0,6

4

0,6

0,1

3

1,3

6

0,7

8

0,2

9

0,3

3

0,8

0,6

7

0,0

9

38

8 4

07

,05

42

8,7

8

42

5,6

39

7,5

7

39

8,2

9

41

7,5

4

40

9 4

27

,1

43

3,2

6

41

5,1

3

35

0,7

8

44

9,2

4

41

2,8

1

38

7,3

2

39

4,0

8

39

6,1

4

38

,92

34

,02

62

,96

54

,5

58

42

,27

67

,62

33

,45

97

,27

15

5,3

4

57

,5

39

,15

30

,47

32

,67

29

,68

82

,89

30

,77

0

50

100

150

200

250

300

350

400

450

500

P0 P1 (13) P2 (34) P3 (23) P4 (20) P5 (19) P6 (49) P7 (13) P8 (22) P9 (40) P10 (23) P11 (22) P12 (19) P13 (21) P14 (21) P15 (40) P16 (27) P17 (22)

Tim

e in

ms

Products (# Equal Elements)

Slice Norm. Slice Diff. Time Slice Norm. Slice Incr.

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 11Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

First Idea – Retest Coverage Criterion

Slice differences indicate retest potentials basedon the impact of model changes

What are retest requirements/goals?Elements corresponding to slice differencesElements connected to those elements

Applicable for selection and prioritization(Pairwise) coverage of retest requirements/goalsNumber of occurrences of a difference usable as weighting factorPotential retest test case generation

Retest requirements/goals refer only to retestable behavior

Model parts not influenced by changes indicate behavior not to beretested

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 12Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Conclusion & Future Work

ConclusionRegression Testing ⇒ Incremental SPL testing

Incremental model slicing ⇒ Change impact analysis

Slice differences ⇒ Retest potentials

Retest decision ⇒ Retest coverage criterion

Future WorkComprehensive evaluation

Extension of control and data dependency

Extension & optimization of implementation

Application for (evolution-aware) change impact analysis

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 13Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

Thank You for Your Attention! Any Questions?

IMoTEP Integrated Model-based Testing of

Continuously Evolving SoftwareProduct Lines

TU Braunschweig, Institute for Programming and Reactive SystemsSascha Lity

Email: [email protected]

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 14Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

References I

Kelly Androutsopoulos, David Clark, Mark Harman, Jens Krinke, and Laurence Tratt.State-based Model Slicing: A Survey.ACM Comput. Surv., 45(4):53:1–53:36, August 2013.

Mathieu Acher, Philippe Collet, Philippe Lahire, and Robert B. France.Slicing feature models.In ASE’11, pages 424–427, 2011.

Hiralal Agrawal, Joseph R. Horgan, Edward W. Krauser, and Saul London.Incremental regression testing.In Proceedings of the Conference on Software Maintenance, ICSM ’93, pages 348–357,Washington, DC, USA, 1993. IEEE Computer Society.

Samuel Bates and Susan Horwitz.Incremental program testing using program dependence graphs.In Proceedings of the 20th ACM SIGPLAN-SIGACT Symposium on Principles ofProgramming Languages, POPL ’93, pages 384–396, New York, NY, USA, 1993. ACM.

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 15Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

References II

Dave Clarke, Michiel Helvensteijn, and Ina Schaefer.Abstract Delta Modeling.Mathematical Structures in Computer Science, 25(3):482–527, 2015.

Emelie Engström.Exploring Regression testing and software product line testing - research and state ofpractice.Lic dissertation, Lund University, May 2010.

R. Gupta, M.J. Harrold, and M.L. Soffa.An approach to regression testing using slicing.In Software Maintenance, 1992. Proceerdings., Conference on, pages 299–308, Nov 1992.

Wang Ji, Dong Wei, and Qi Zhi-Chang.Slicing Hierarchical Automata for Model Checking UML Statecharts.In Formal Methods and Software Engineering, pages 435–446. Springer, 2002.

Jochen Kamischke, Malte Lochau, and Hauke Baller.Conditioned Model Slicing of Feature-Annotated State Machines.In FOSD’12, pages 9–16, 2012.

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 16Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

References III

Frederik Kanning and Sandro Schulze.Program Slicing in the Presence of Variability.In ICSME’14 - ERA Track, 2014.

Sascha Lity, Hauke Baller, and Ina Schaefer.Towards incremental model slicing for delta-oriented software product lines.In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22ndInternational Conference on, pages 530–534, March 2015.

Malte Lochau, Sascha Lity, Remo Lachmann, Ina Schaefer, and Ursula Goltz.Delta-oriented Model-based Integration Testing of Large-scale Systems.Journal of Systems and Software, 91:63–84, 2014.

Sascha Lity, Remo Lachmann, Malte Lochau, and Ina Schaefer.Delta-oriented Software Product Line Test Models - The Body Comfort System CaseStudy.Technical Report 2012-07, Technische Universität Braunschweig, 2012.

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 17Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

References IV

Malte Lochau.Model-Based Conformance Testing of Software Product Lines.PhD thesis, Technische Universität Braunschweig, 2012.

Malte Lochau, Ina Schaefer, Jochen Kamischke, and Sascha Lity.Incremental Model-Based Testing of Delta-Oriented Software Product Lines.In TAP’12, pages 67–82. Springer, 2012.

S. Oster, M. Lochau, M. Zink, and M. Grechanik.Pairwise Feature-Interaction Testing for SPLs: Potentials and Limitations.In FOSD’11, 2011.

Alessandro Orso, Saurabh Sinha, and Mary Jean Harrold.Incremental Slicing Based on Data-Dependences Types.In ICSM’01, page 158, 2001.

Gregg Rothermel.Efficient, Effective Regression Testing Using Safe Test Selection Techniques.PhD thesis, Clemson University, May 1996.

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 18Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme

References V

Antti Tevanlinna, Juha Taina, and Raine Kauppinen.Product Family Testing: A Survey.SIGSOFT Softw. Eng. Notes, 29(2):12–12, March 2004.

Heike Wehrheim.Incremental Slicing.In Formal Methods and Software Engineering, ICFEM’06, pages 514–528, 2006.

Mark Weiser.Program slicing.In ICSE’81, pages 439–449, 1981.

May 14th , 2015 Sascha Lity, Hauke Baller, Ina Schaefer Page 19Towards Incremental Model Slicing for SPL Regression Testing Institut für Programmierung

und Reaktive Systeme