1 Challenges and Results in Component Quality Certification Ralf Reussner Universität Karlsruhe...

SD Xperf=1.00Xloss=0.01

Software Design and Quality

1

Challenges and Results in Component Quality Certification

Ralf Reussner Universität Karlsruhe (TH)

Karlsruhe Institute of Technology (KIT)



Ralf Reussner, KIT, Keynote SERENE 2008, Newcastle 2

Overview

▪ Software Industrialisation & Software Quality Certification

▪ Software Engineering & Software Quality Prediction

▪ How Architectural Quality Models and Prediction Methods can be used for Softwae Quality Certification

▪ Open Issues

2

Industrialisation Engineering Certification by Prediction Open Issues




Overview




▪ Open Issues

3





Industrialisation (1)

1. Epoch in History of Technology1. Phase: steam engine as technological driver ca.

1780-18402. Phase: Electricity as driver: ca. 1840-19603. Phase: ICT as driver: since ca. 1990

2. Management-Method• Lowering of costs per unit (“Stückkosten”)





Industrialisation (2)

Goal:▪ Lowering of costs per unitThrough:▪ Standardisation▪ Specialisation▪ Automation

Standardisation eases specialisation and automation

Technologies as drivers: ▪ Steam engine as driver for automation, ▪ Enabled technologies demand standardisation (e.g., railroading) ▪ Which allowed specialisation (e.g., locomotive, wagons, rails, etc.)





Ways of Industrialisation

Standardisation

AutomationSpecialisation

adapted from: Buxmann, Diefenbach, Hess: Die Softwareindustrie, Springer, 2008

Technology Technology

Technologysupports / enables supports / enables

demandsdemands

Driver Driver

Driver





Development and Production

▪ Technical Production: well understood, planable, repeatable

▪ Problems of Software Engineering are problems in development, not production

Development

Production

TechnicalProduct

Requirements to

a technical product ......

Production Use

7





Software-Industrialisation

▪ What actually does: “lower costs per unit” mean for Software?– In particular, as software is not produced by

developers.

▪ What actually means Software-Production?– Not only distribution!

▪ But also: – Adaptation– Deployment– Configuration





Drivers ofSoftware-Industrialisation

▪ Software Components▪ Software Architectures / Patterns▪ Software Product Lines / Frameworks / Reference

Architectures▪ Model-driven Software Development ▪ Well-understood Development Processes /

Collaborative Processes▪ Certified Components and Architectures▪ Software Services and SOAs▪ Fixed and reasonable Technical Standards

which are not ignored.





Role of Components in an Industrialised Discipline

▪ All industries have components.▪ Important means for standardisation ▪ Components lower the degrees of freedom during

development and, hence, increase the predictability of quality attributes.

▪ The re-use of components blurs the boundaries between development of new software, evolution of software and integration of software (which reflects just the reality).

▪ Re-use of components / composition of systems is isomorphic to re-use / composition of prediction models

11





What is a component?

▪ “A component is a contractually specified building unit of software which can be readily composed or deployed.”– “readily composed or deployed”:

• without having to understand the interna as a human• these are the two main things to be done with

components

– not necessarily “black-box”: Information on interna can be available to tools.

▪ “Components are for composition, much beyond is unclear…” (Clemens Szyperski)

12





Different Abstraction of Components

CTType

IIDImplementation

Instance Description

DIDDeployment

Instance Description

RIDRuntime Instance

Description

1

*

1

*

Not considered within the Palladio ComponentModel

1

*

QoS (i, es, up, d)FP (es)

QoS (es, up, d)FP (es)

QoS (up)FP ()

i: implementationes: external servicesup: usage profiled: deployment

«implements»

«deploys»

Implementation

Type

Deployment

Runtime





Components and Services

▪ A software component is a contractually specified software building block which can be deployed or composed without understanding its internals.

▪ A service is a deployed component.▪ New role: service provider, new business model▪ SOAs are not designed, but are evolving

– well, nearly

▪ Lightweight service compositions– instead of scripting– one-use software

▪ SLA are needed to describe service, like an interface describes a module.





Quality of Services

▪ Eased modelling & prediction of quantitative properties:– Mostly synchronous calls– Deployment context and external components

fixed





Q-Impress Vision(EU FP 7 STREP)

Design Model

Prediction Model

Source Code

Resource Model

Quality impact analysis and simulation

Service Architecture

Model

Annotations

Changes

Service evolution

cycle

Service evolution

cycle

Me

tho

d v

alid

atio

n / D

em

on

stra

tor

Me

tho

d in

teg

ratio

n

Usage profile

UML

Service architecture extraction

Monitoring

Domain knowledge

Legacy Code

Service Wrappers

Legacy code wrapping

Monitoring / Benchmarking

Annotations




Certified Components and Architectures and Standards

▪ Needed to ensure trust of unknown foreign components.

▪ Needed to justify costs of architectural modelling.▪ Certification is usually related to a standard

(certification of “standard quality”)▪ Standards need to be reasonable (more than the

expression of the business interests if the members of standardisation bodies)

▪ Standards need to be known and obeyed. Lack of culture in our discipline.





A Simple Test on Software Standards

▪ IEEE 610.12-1990 – SE Terminology

▪ IEEE 730-2002 – Software Quality

▪ IEEE 1471-2000 – Software Architecture





What is Certification

▪ The verifiable demonstration of qualities of an entity according defined or prescribed standards.

▪ Entities are often technical products (cars, buildings, designs of such products, …)

▪ In the software world most often– processes (CMMi, QIP, ..) or– education / training courses

are certified.▪ If training courses are concerned with specific

software products, the certification of such courses is sometims wrongly called “product-oriented certification”.





Certification of Software Products

▪ In the software world software products are not certified.

▪ Software is different to physical technical products (no wear and tear) but also evolves and designs also need to be certified.

▪ Analogy: Typ appropriation of cars.▪ Different to verification:

– several quality attributes are of concern– entities for certification can be

• components• architectures• deployed component (aka services) • whole systems

– verifiably demonstratable– according to standards





Why Certification

▪ Software vendor / Software provider:– for “marketing” (a means to

create trust)– because of legal regulations

▪ Software customer:– because also software vendor

of composed products– because of legal regulations (of

non-software domains)

Software Developer

Customer





What Quality to Certify?

▪ Functional properties of components (close to automated verification)

▪ External non-functional properties of components:

▪ Demonstrate, that formal quality model fits to implementation (and vice versa security)





Standard Certification Scenario

24

CertificationAuthority

Software Developer

Customer

Trust

Examination &

Certification

Software Offer





Autonomic Certification Scenario

25

Software Developer

Customer

Software Offer

Checks certificate


Provides certifies tools for certificatie checks





Overview




▪ Open Issues

26





Elements of an Engineering Discipline

[Shaw&Garlan95]

Craft• Customer and Developer often the same person• Talent and Experience instead of Understanding

Manu-facturing• Division of Labour• Education of Specialists• Use of third party tools

Engineering

• Goal-driven optimisation of • Products• Processes

requires• Understanding of the effects of design decisions and changes Theories on products and processes

27





State of SE?

The same problems since 1968(first Software Engineering Conference)

▪ “The problem of achieving sufficient reliability in the data systems...”

▪ “The difficulties of meeting schedules and specifications on large software projects”

▪ “The highly controversial question of whether software should be priced separately from hardware”

28





Where stands“Software Engineering” as an Engineering Discipline?

▪ Progress: the same problems since decades, but for considerably larger and more complex systems

▪ “Planning crisis” instead of a “Software crisis” [Glass00]:– Budgets and schedules are

rarely done by the developer, much more by managers, sales persons and customers

0

200

400

600

800

1000

1979 1992 2004

KLOC

Approx. size of what is considered as “large” software systems

30





Software Engineering:Manufacturing

▪ Division of labour– Roles– Use of specialised tools

▪ (Specialised Education)▪ Design patterns as a vocabulary on proven

solutions to recurring problems

31





Problems

▪ Lack of Understanding and Professionalism – “New Motors in three month.”– “Sky scrapers in 5 days.”– Why do not we find books like:

• “Heart Transplantations for Dummies”• “Nuclear Weapons in 21 days”• “Flying the Airbus: Easy Access!”

– Sky scrapers as large garden houses

▪ Counter productive avoidance of up front costs

▪ Real problem of integrating and using legacy systems

32





Treatment of Quality Properties Today

4. Re-Implementing / Re-Designing / Re-Negotiating

1. Specification

3. Testing

33

2. Ignoring





Analogy and Role Model

▪ John L. Hennessy, David A. Patterson: “Computer Architecture. A Quantitative Approach”, Morgan Kaufman, 1992 (1st edition)

“At the core is a quantitative approach to computer design and analysis that uses empirical observation of programs, experimentation, and simulation as its tools.”





Missing Properties of an Engineering Discipline

Systematic Treatment of Quantitative Software Properties

Decomposition of global System-Requirements

“reaction time below 2 ms”

?

“?”

?

Prediction of global System-Properties

35

? ? ??

? 0.02 sec.

0.3 sec. 0.08 sec. 0.1 sec.1.8 sec.





Why do we want to predict quantitative Properties?

Dimensioning of Resources (“Sizing”)

vs.

Changes of usage profile – Scalability

vs.

Evaluation of Design Alternatives

▪ the quantifiable best of a list of many▪ trade-off decisions

– cost vs. benefits– QA a vs. QA b

View

ModelController

ViewView

View

ModelController

ViewView

vs.





Model-based Prediction of Quantitative Properties

Software Design Model

Annotated Software DesignModel

AnalysisModel

AnalysisResults

UML,ADL,

…

UML Performance Profile, QML,

…

Queuing modelsStochastic Petri-Nets,

Stochastic Process Algebra,…

Response timeThroughput,Utilisation,

…

EstimationMeasurement

Transformation (MDD)

Analysis / Simulation

Results

Automated by Tools

ExecutableSoftware

Transformation (MDD)

42





Scientific Approach to Create Quantitative Models

Software

Modell of Software(mit Annotationen)

Measured Quality

Predicted Quality

ComparisonAbstraction

Prediction

Measurement

Interpretation

Acceptance / rejection of abstract model

Improvement / Extension





Validation of Quantitative Models

▪ Type 1: Validation of Prediction Model

▪ Type 2: Validation of Applicability – Case Studies and Controlled

Experiemts with Students

▪ Typ 3: Validation of Benefits – in comparison to different methods– Limitations of the Approach– Required prerequisites– FZI– Industrial Partners






Ralf Reussner, KIT, Keynote SERENE 2008, Newcastle 4521.04.23 45




Ralf Reussner, KIT, Keynote SERENE 2008, Newcastle 4621.04.23 46

Dom. Exp.DSL Instance

Sys. Depl.DSL Instance

Soft. Arch.DSL Instance

Comp.Dev.DSL Instance

Tran

sfor

mat

ion

Stochastic Regular Expr.

Analysis

SPA with Scheduling

Transform

ation

Analysis +Simulation

QueueingNetwork

PerformancePrototype

Java CodeSkeletons

Transformation

Transformation

Transformation

Simulation

Execution +Measurement

Completion +Compilation

Instance

Part o

f

Part of

Part of

Part of

PalladioComponentModel





Roles Component Model Analysis Methods CoCoME ConclusionComponent Model

PCM Bench Screenshot




Roles Component Model Analysis Methods CoCoME ConclusionComponent Model

Tool Support




Overview


▪ Software Engineering &Software Quality Prediction


▪ Open Issues

49





Factors on Quantitative Component Properties





Certification Problem

▪ Demonstrating that the relation between quality model and implementation satisfies pre-scribed properties.




ComponentA

ComponentBa()

b()c()

ComponentCd()

Execution Time of a()?

?ms

2ms

3ms

5ms

Service Effect Specification(SEFF)

a(list, count):




<<InternalAction>>

public List getListWithLittleEntropy(List listToSort, int count) {

while(mode) {

// some simple internal actionfor(int x = 0; x < count; x++) {

listToSort.add(new Integer(x));}

if(count > 100) { //external call:collectionComponent.sort(listToSort);

}

//external call:mode = collectionComponent.

isEntropyLessThan(listToSort, count);

}return listToSort;

}

<<ExternalAction>>CollectionComponent.sort()

<<ResourceDemandingSEFF>>getListWithLittleEntropy

<<LoopAction>>

<<ExternalAction>>CollectionComponent.isEntropyLessThan()

Service Effect Specification (1)





ComponentDevelopers

BasicComponent

ServiceEffectSpecification

ResourceDemandingSEFF

ResourceDemandingBehaviour

AbstractAction

1

*

1

*

*

1

VariableUsage

NamespaceReference

VariableReference

<<enumeration>>Variable

CharacterisationTypeVALUETYPEBYTESIZENUMBER_OF_ELEMENTSSTRUCTURE

SignatureserviceName : String

*

1

*1

1

1

1

*

RandomVariablespecification : String

VariableCharacterisationtype : VariableCharacterisationType

AbstractNamedReference

referenceName : String

AbstractResourceDemandingAction

ExternalCallAction

Signature

VariableUsage

StartAction StopActionInternalAction

ParametricResourceDemand

AcquireAction

ReleaseAction

ProcessingResourceType

PassiveResourceType

in out

1

1

1 1

* *

1 *1 1

11

1

0..1

0..1

SetVariableAction11

1

1

*

ParameterparameterName : String

successor

predecessor

BranchAction AbstractLoopAction ForkAction

ResourceDemandingBehaviour

AbstractBranchTransition

GuardedBranchTransition

BranchCondition ProbabilisticBranchTransition

branchProbability : Double

LoopAction CollectionIteratorAction

IterationCount ParameterparameterName : String

1

1

1

*

1

1

1

1

1

*1

1RandomVariablespecification : String

1

1

Ecore

Service Effect Specification (2)





MediaStore - Architecture

WebBrowser MediaStoreWebGUI

UserManagement

SoundProcessing

Billing

Encoding

DigitalWatermarking

AudioDB

CommunityServices

PodcastInternetRadio

Equalizer

Accounting UserDB

DBAdapter

<<ResourceContainer>>Client

<<ResourceContainer>>Application Server

<<ResourceContainer>>DBServer1

<<ResourceContainer>>DBServer2

<<LinkingResource>>throughput = 1000unit = MBit/s

<<LinkingResource>>throughput = 1unit = MBit/s

<<VariableUsage>>StoredFiles.NUMBER_OF_ELEMENTSStoredFiles.INNER.BYTESIZE

<<VariableUsage>>probIncludeID.VALUEprobIncludeText.VALUE





<<ExternalCallAction>>queryDB

<<CollectionIteratorAction>>

<<ExternalCallAction>>

watermark

<<SetVariableAction>>

<<InternalAction>>search


<<ParametricResourceDemand>>specification = „1.49E-6 * StoredFiles.NUMBER_OF_ELEMENTS + 0.0096“<<ProcessingResourceType>>name = „HD“

<<InternalAction>>addID

<<InternalAction>>addText


probIncludeID

1-probIncludeID

probIncludeText

1-probIncludeText

<<VariableUsage>>fileToMark.BYTESIZE

<<Parameter>>desiredFiles

<<RDSEFF>>MediaStore.download

<<RDSEFF>>AudioDB.getFiles

<<VariableUsage>>filesIDs.NUMBER_OF_ELEMENTS

<<RDSEFF>>DigitalWatermarking.

watermark

<<VariableUsage>>filesToMark.BYTESIZE

<<ParametricResourceDemand>>specification = „fileToMark.BYTESIZE * 5.11E-9<<ProcessingResourceType>>name = „CPU“

<<VariableUsage>>desiredFiles.NUMBER_OF_ELEMENTS

<<InternalAction>>getFiles

<<ParametricResourceDemand>>specification = „4.0E-8 * desiredFile.BYTESIZE + 0.08“<<ProcessingResourceType>>name = „HD“





Results





Overview




▪ Open Issues

60





Standard Certification Scenario

61


Software Developer

Customer

Trust

Examination &

Certification

Software Offer





Autonomic Certification Scenario

62

Software Developer

Customer

Software Offer

Checks certificate


Provides certifies tools for certificatie checks





Specific Challenges of Component Certification

▪ Component developer has to provide checkable certificate for component,– but does not want to expose specific

know-how of the component implementation

▪ In the standard certification scenario this is not a problem,– as certification authority can see all

internal of the components but its certification needs not to contain component specific information (except the component ID and a hash value)

Different for the autonomic scenario.Certifification can just mean failed

falsification.


Software Developer





Research Questions

▪ How to demonstrate that a quality model (e.g. performance or reliability) fits to an implementation?

▪ Right level of abstraction? Model vs. code, model vs. code abstraction?

▪ How to automatical gain code abstractions?

▪ Right mix of testing, automated code-analysis and verification (in particular model-checking)





Certification levels

▪ depend on architectural analysis to be performed▪ An example hierarchy:

1. components as black boxes (no model on component behaviour) architectural dependency analysis for components. (If component fails, which other components are affected)

2. components with a model on dependency between provided and required services architectural dependency analysis for components on service granularity. (If component service fails, which other services are affected)

3. components with protocol information if required protocol changes, what is the new provides protocol)

4. components with QoS model analysis of system-wide QoS properties




Static Reconstruction (using existing

tools)Building Blocks

Genetic Algorithm

Monitoring

Static AnalysesAnalyses of

Dynamics / RuntimeMachine Learning

Process for dynamic reconstruction

Reconstructed Model

InternalCalculation

ExternalCall

01001001011010000111101100100010100110010001010111001

Provided Quality Model

InternalCalculation

ExternalCall

Control flow abstraction

Check conformance:• bisimulation• testing • simulation

Provided Component





Machine Learning

▪ Not limited to one approach– Genetic Algorithms (GA) – Support Vector Machines (SVM) – Hill-Climbing / simulated annealing– Regression (Splines / linear)– Stochastic approximation– Greedy optimization– …





What is to be learned?

▪ Loops– Break conditions– Loop number (depending on input parameters, (a, b))

▪ Branches– Branching conditions / probabilities

▪ Call of component-external services– Frequency / probability– Conditions– Call parameters (x, y) as a function of input parameters (a, b)

▪ Return value of provided service (long)

Based on (input dimensions)▪ Service input parameters (of described service) (a, b)▪ Return parameters (of component-external calls) (float)

long myService(int a, int b)

float requiredService (double x, double y)





Conclusions

▪ Prediction and Understanding of the Consequences of Design Decisions is THE central characteristic of an engineering discipline.

▪ Components and MDD lower the degrees of freedom in implementation

▪ Creativity is on design-model level▪ Quality-driven design requires prediction models

– Automatically generated from design models▪ Certification as a means to provide standardised

component with standardised quality▪ Certification approaches can make use of

verification and prediction techniques, etc, but answers a different question.





Missing:Franz Brosch

Dr. Jan KofronChristof Momm

Dr. Pierre ParrendDr. Barbora Zimmerova

1 Challenges and Results in Component Quality Certification Ralf Reussner Universität Karlsruhe...

Documents

Transcript of 1 Challenges and Results in Component Quality Certification Ralf Reussner Universität Karlsruhe...