Scalable Product Line Configuration:A Straw to Break the Camel’s Back

Abdel Salam SayyadJoseph IngramTim MenziesHany Ammar

Sound bites

Real Software Product Lines…… are big, complex

… representative feature models are now available for research

… scale up or die !

2

What does it take?Multi-objective problem formulationAn optimizer that relies heavily on user preferencesRespect domain constraints, and...

… A Straw to Break the Camel’s Back

Roadmap

① State of the Art② Experiment & Results③ Conclusion

Roadmap

① State of the Art② Experiment & Results③ Conclusion

Feature Models

• Feature models = a lightweight method for defining a space of options

• De facto standard for modeling variability

5

Cross-Tree Constraints

Cross-Tree Constraints

Size ? 10 Features, 8 Rules

Feature Model Repository

6

Size ? 290 Features, 421 Rules

cardinal = 2.26x1049 *Pohl ‘11+

Feature Models of Real Software Projects

7

Size ? 6888 Features; 344,000 Rulescardinal? Forgeddaboutit!

Size isn’t all…

Property SPLOT Linux

Size Significantly smaller Significantly larger

Constraints Significantly less Significantly more

Feature Groups High ratio Low ratio

Leaves Deeper Shallower

8

T. Berger, S. She, R. Lotufo, A. Wasowski, and K. Czarnecki, "A Study of Variability Models and Languages in the Systems Software Domain," IEEE Tran Soft Eng. 2013

A Multi-Objective Formulation

Multi-objective optimization = navigating competing concerns

– Success criteria = choose features that achieve the user preferences!

9

Suppose each feature had the following metrics:1. Boolean USED_BEFORE?2. Integer DEFECTS3. Real COST

Show me the space of “best options” according to the objectives:1. That satisfies most domain constraints (0 ≤ #violations ≤ 100%)2. That offers most features3. That we have used most before4. Using features with least known defects5. Using features with least cost

No single “optimum” solution

10

Higher-level Decision Making

The Pareto Front

The Chosen Solution

State of the Art

11

Features

9

290

544

6888

SPLO

TLi

nu

x (L

VA

T)

Pohl ‘11 Lopez-Herrejon

‘11

Henard‘12

Sayyad ’13a

Olaechea‘13

Sayyad ’13bJohansen

‘11

Benavides ‘05

White ‘07, ‘08, 09a, 09b, Shi ‘10, Guo ‘11

Objectives

Multi-objectiveSingle-objective

Roadmap

① State of the Art② Experiment & Results③ Conclusion

Two Algorithms

13

1) NSGA-II [Deb et al. ‘02+Non-dominated Sorting Genetic Algorithm

2) IBEA *Zitzler and Kunzli ‘04] Indicator-Based Evolutionary Algorithm

Binary Dominance

Continuous Dominance

7 Feature Models

14

Feature Fixing

• Look for mandatory or dead features.

• Fix those feature in the evolution.

• Skip rules with only those features.

15

Experiment

16

Low Crossover & Mutation rates

Stopping Criteria

17

HV = Hypervolume of dominated region%Correct = % of solutions that are fully correct (out of 300 solutions)

Comparing %Correct

18

0%

20%

40%

60%

80%

100%

ToyBox axTLS eCosFreeBSD Fiasco

uClinuxLinux X86

NSGA-II No FF

NSGA-II with FF

IBEA No FF

IBEA with FF

i.e. % of solutions that are fully correct (out of 300 solutions)

How to crack the nut?

• What if we told IBEA what good solutions look like?

19

• Good solution? … A correct solution, with:• Minimal skeleton?• Maximum features?

• Then what?• Seed the initial population with a pre-calculated

“good” solution.

20

• How to generate good (correct) solutions?• Z3 SMT (Satisfiability Module Theory) Solver• Two-objective IBEA, min violations, max features

• Z3: super fast, low on selected features

• 2-obj IBEA: slow, but feature-rich

Correct solutions after 30 minutes for Linux Kernel feature model

21

Correct solutions after 30 minutes for Linux Kernel feature model

22

… A Straw to Break the Camel’s Back

More recent result• added the “PULL” trick (give more objective weight

to constraint violations)

23

Why is this interesting?• 20 correct solutions to choose from within 2 minutes

24

Do I have the user’s attention?

• 2 seconds…

• to stay focused

25

• 10 seconds… • to stay on task

• few minutes? • To provide

timely input at a meeting

Roadmap

① State of the Art② Experiment & Results③ Conclusion

Sound bites

ScalabilityMethod innovation is key

27

Problem FormulationGive the best (comprehensive yet concise) picture to the user.

IBEA with Feature FixingContinuous dominanceRespect domain constraints

Enter Z3?One day we will build that bridge!

Acknowledgment

This research work was funded by the Qatar

National Research Fund (QNRF) under the National Priorities Research Program

(NPRP) Grant No.: 09-1205-2-470.