Devnology Back to School: Empirical Evidence on Modeling in Software Development

Leiden University. The university to discover.

Dr. Michel R.V. Chaudron Leiden Institute of Advanced Computer Science

[email protected]

Zin en Onzin van Modeleren in Software Development

Leiden University. The university to discover. Dr. Michel R.V. Chaudron, Devnology, June 2012, Leiden

Outline of talk Introduction

- LIACS & Software Engineering

Modeling in Software Development

- How are models being used in practice?

- What does it cost/bring to do modeling?

Experiments

- Hafeez

- Dave (announcement)


Leiden Institute of Advanced Computer Science

3

Algorithms Prof. J.N. Kok, Prof. T. Bäck

• Novel Algorithms

• Data Mining

• Natural Computing

• Applications

• Drug Design

• Medicine

• Engineering

• Logistics

• Physics

Core Computer

Technologies Prof. H. Wijshoff, Dr. T. Stefanov

• Embedded Systems

• Parallel / Distributed Computing

• Compiler Technology

Software Engineering Dr. M.R.V. Chaudron,

Prof. F. Arbab,

• Software Design and Modeling

• Software Composition

• Formal Methods

• Coordination / Concurrency

Imagery and Media Dr. M. Lew, Dr. F. Verbeek

• Computer Vision and Audio/Video

• Bioimaging

• Multimedia Search

• Internet Technology

• Computer Graphics

4 years Leiden 10 years TU Eindhoven 2 year IT-company


Software Engineering Researchers

Werner Heijstek

Ramin Etemadi

Dave Stikkolorum Ana Fernandes Hafeez Osman Pieter Kwantes

Bilal Karasneh Peng Ye Javier Hernandez

Dr. Michel R.V. Chaudron [email protected]

Models in Global SE Updating UML Teaching SW Design UML in maintenance Domain Specific Modeling Languages

Optimizing Sw. Arch. Design

Reasoning about Sw. Arch. Design

Modeling Sw- Product Lines

mailto:[email protected]


A change can be an addition, modification, removal.

learn design

abstraction:

change change ’ change ’’

sou

rce

cod

e U

ML

des

ign

mo

del

update with

abstraction of change: () (’) (’’)

time

Automated Updating of Class Models


Extracting Model from Images

Input: .jpeg, .png

Bilal

Recognizing: - Rectangles - Lines - Text, - arrow-heads

Output: XMI for the class model


History of UML

Other modeling languages: BPML, Archimate, …


4+1 Views Representation of System Architecture

Logical View

Functionality (Decomposition)

Development View

Programmers

Configuration management

Process View System Architect

Deployment View

System topology

Delivery, installation, maintenance

Performance, Scalability, Throughput

System engineering

Use Case View

Concurrency, Communication,

Synchronization

End-user System Architect

How is the system structured?

How to build / configure ?

Where to install ? What hw\nw is used?

How does the system behave?

How does the system perform ?

What can/does the system do ?


UML modeling notations for 4+1 Views Structure model : components, packages, interfaces

Config. Mngnt model

Behaviour model :

MSC, state-diagrams

A B

C D

A B C D

Deployment model :

physical model + mapping

A

B C D

Stakeholders & Use cases view

BC/WC e2e-response times, freq. bandwidth, availability

TCP/IP over Ethernet

versioning policies

file ownership

…


Panel

Opinions on Value of Modelling

Value Agile

Formal Methods

“Loose UML”

completeness

100%

MDA


Introduction: Research Interest - What are the pay-offs of investing in

early design/architecture/modeling? Fewer defects? Cheaper maintenance? …

- Effectiveness of software modelling

- Quality of Modeling When is a model good (enough)

- Analysis of Non-Functional Quality Properties of architectures

- Many modeling approaches around; focus on UML in custom software development


12

Increasing Complexity of Software

Blo

w,

J. "G

am

e D

eve

lopm

ent:

Ha

rde

r T

han

Yo

u T

hin

k."

AC

M Q

ueu

e


Empirical Research … is a way of gaining knowledge by means of observation or experience.

Theory SE body of knowledge

Practice People/Skills, Processes, Methods,

Technology, Artefacts

Observation

Hyp

oth

esis

Prediction/Test

Validation

Topics: CMM, OO, Agile, MDA, … Methods: Experiment, Case study, Survey, Project Repositories …


contact

Dr. Michel R.V. Chaudron Associate Professor

Leiden Institute of Advanced Computer Science [email protected]

www.liacs.nl/~chaudron

If you have UML models you would like to assess?

If you want advice on Quality Assurance for modeling?

If you want to sponsor our research?

mailto:[email protected]


Empirical Life-cycle

Initial Idea

Survey

Method

Development

Industrial Case Studies

Experiment

Exploratory Interviews


Phase 1: Explore via Case Studies

Your project here? Mail me: [email protected]


Industrial use of UML diagrams

Class

How

UML i

s used

B Do

bing,

J Pars

ons -

Com

munic

ation

s of th

e ACM

, 2006

http://www.csa.com/partners/viewrecord.php?requester=gs&collection=TRD&recid=20061190153974CI


Styles of Modeling Sketch

Communication

Recipe for construction


Modelling style and model purpose

understanding

communicating

managing complexity

implementation blueprint

Styles of using UML – as a sketch – thinking tool/understanding

– for communicating system design

– as a blueprint – guide the implementation work

– as a implementation (MDA) - code generation

C.F.J.L

ange,

M.R.

V. C

haud

ron, J.

Muske

ns: In

Pract

ice: U

ML So

ftware

Ar

chitec

ture a

nd D

esign

Desc

riptio

n. IEE

E Soft

ware

23(2):

40-46

(2006

)

http://www.informatik.uni-trier.de/~ley/db/journals/software/software23.html


The Modeling Spectrum

Model driven architecture: Principles and practice, Brown, A.W., Software and Systems Modeling, 2004


Doing modeling is

a quality improving

activity

Uses of Software Models

+ Understanding

+ Analyzing / Predicting

+ Communicating

+ Guiding


UML in practice Sketch

Communication

Recipe for construction

More effort More expensive


Modeling & Documentation in Agile Development

- Agile principles: working software over comprehensive documentation

Tending towards: we need a bit more

Christoph J. Stettina and Werner Heijstek, Necessary and Neglected? An Empirical Study of Internal Documentation in Agile Software Development Teams 29th ACM Int. Conf. on Design of Communication , Pisa, Italy

Modeling is compatible with agile development

Survey under 75+ agile developers


Does use of UML improve software quality?

A large number of developers indicated the use of UML improves understandability and modularity

45% 41%

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

Cover Requirements

Correctness Modularity Testability Understandability

Percen

tag

e o

f th

e r

esp

on

den

ts

Reduce

Somewhat Reduce

Neutral

Somewhat Improve

Improve

Ariad

i Nug

roho,

Miche

l R. V

. Cha

udron

: A Su

rvey o

f the P

ractic

e of D

esign

- Co

de Co

rrespo

nden

ce am

ongst

Profe

ssion

al So

ftware

Engin

eers.

ESEM

2007:

467

-469


Economic Model for Cost of Quality From: H. Krasner, Cost of Quality, 1998

Cost of SQ = Achievement Cost + Non-conformance Cost


Cost of Modeling

Focus on effort “Is there a correlation between class-count and the

effort spent in modeling?”

Sources: ‐ Experiment ‐ Set of industrial case studies


Empirical Data

Experiment

106 MSc students in (TU/e) Organized in 35 teams

Task: Model Car Navigation System

Duration: 6 weeks assignment

Three treatments were applied: 1. No modeling conventions

2. With modeling conventions

3. Tool-supported modeling conventions


Experiment Results: Effort ~ Size?

Effort (hours)

Class-count

Sig. (2-tailed)

.302

.171

There is no significant correlation between class-count and the effort spent in modeling.

Spearman’s Correlation Coefficient

0,00

5,00

10,00

15,00

20,00

25,00

30,00

35,00

40,00

0 10 20 30 40 50 60 70

Class-count

Eff

ort

(h

ou

rs)

class-count is not a good measure for the effort spent on modeling?

Number of classes in design


Measurement of Modeling/Design in RUP Effort Distribution

designing modeling


incep.

elaboration

construction

transition

constr.

transition elaboration

‘traditional’ (avg. of 17 projects)

Model-driven (1 project)

Impact of MDD on Effort Distribution as % of total project effort

architecture & code generator


Does Quality of Modeling Matter? An Industrial Case Study

operations

attributes

associations

Focus on detail in a UML Model

Low detail High detail


Paym ent

-amount

-cashTendered

Cash

+authorized()

-name

-bankID

Check

+authorized()

-number

-type

-expDate

Credit

+calcTax()

+calcTotal()+calcTotalWeight()

-date

-status

Order

+calcSubTotal()

+calcWeight()

-quantity

-taxStatus

OrderDetail

1

1

1* Preprocessor Parser Analyser Inheritance Relator Statistic Filter Statistic CalculatorDB Creator DB Filler DB Checker

SAAT.BAT

Select ‘defects’ in defect DB

Find classes in source code that were

repaired for solving this defects

Find corresponding classes in UML models

Determine LoD for CD and SD

Payment

-amount

-cashTendered

Cash

+authorized()

-name

-bankID

Check

+authorized()

-number

-type

-expDate

Credit

+calcTax()

+calcTotal()+calcTotalWeight()

-date

-status

Order

+calcSubTotal()

+calcWeight()

-quantity

-taxStatus

OrderDetail

1

1

1* Preprocessor Parser Analyser Inheritance Relator Statistic Filter Statistic CalculatorDB Creator DB Filler DB Checker

SAAT.BAT

component C2

requires I2

requires I3

uses I2.g

uses I3.h

component C2

requires I2

requires I3

uses I2.g

uses I3.h

defect density

LoD per class

Relation between UML-LoD and Code Quality


Level of Detail for Sequence Diagrams is significantly (negatively) correlated with defect density.

More detailed model => fewer defects

Relation between Level of Detail and Defect Density

= 1 class


Quality Assurance for Software Design / UML Model

- How to define Quality for design/model?

- How to assess quality?

- How to check conformance in code?


Quality Models Existing models

- Decomposition of characteristics - Bottom level: metrics

- Differences in - Relations between characteristics

- Vocabulary Boehm’s Quality Model

Boehm McCall

ISO 9126 Rombach

…


How to Define UML Model Quality? The quality attributes of a UML model is determined by its purpose...

Frame

work

for qu

ality o

f UML

mod

els (L

ange

& Ch

audro

n, 200

5)


How to measure goodness of UML model? Design Heuristics

- Coupling, Cohesion, absence of anti-patterns

Completeness

- Interaction of classes must be described in SD

- Methods of CD must be called in SD

Consistency

- Messages in SD must correspond to method in class diagram

Correspondence

- Dependencies in the implementation are allowed only if they occur in the design

In many cases it can not be automatically determined

whether a flaw is an incompleteness or an inconsistency


Automatically Checking Quality of Design

Visualization of model + metrics

Quality Metrics/Rules • Completeness • Consistency

UML Analysis Tool

UML model

Quality = absence of ‘bad things’ such as omissions, spaghetti


Screenshot of the MetricView tool

= violation

= ok

Quality Metrics • Completeness • Consistency • … extensible

Leiden University. The university to discover. Dr. Michel R.V. Chaudron, Devnology, June 2012, Leiden Christian Lange 42

MetaView to visualize the (hidden) inter-diagram relations Example: Which classes contribute to a certain use case?

Supporting task-oriented modeling using interactive UML views, C.F.J. Lange, M. A.M. Wijns, and M. R.V. Chaudron, Journal of Visual Languages and Computing

MetricView Tool

http://www.youtube.com/watch?v=G3HJ_QR9EG4

Leiden University. The university to discover. Christian Lange. 43

MetricView The values of metrics are visualized on class diagrams using colors Example: Coupling-Between-Objects (CBO)


MetricView Tool

We will analyse your UML model : mail to [email protected]


Leiden University. The university to discover. Christian Lange. 44

3D visualization of design measurements



Your cases welcome…


Findings from State of the Practice

UML is used in different ways for different purposes

- Incompleteness

- Modelers focus on complex and critical parts

- Disproportion

- Different parts of the system are modeled different in level

of quality

- Inconsistency

- Conflicting information in different views of one model

Quality Assurance for modeling should address these


Industrial Experience with MetricView as Quality Assurance Tool

- Based on 15+ industrial projects

- If there are weak spots in the design, then these are indicated as ‘suspects’ by MetricView

- MetricView indicates many weak spots that do not

require improvement according to project architects (false positives?)

prioritization is needed - The later MetricView is applied, the fewer ‘weak spots’

are removed from the design process issue


Understanding Effectiveness of Modeling through Eyetracking


Improved Quality

UML Modeling

Fewer Defects

Reduced maintenance

effort

Better Understanding

Problem Domain

Reduced testing effort

Reduced rework

More efficient Maintenance

building the right system

building the system right

Improved requirements

Better Risk management

Improved Productivity

More Accurate Estimating

Improved Design Compliance

Hypothetical Benefits of Modeling

Developer

Benefits

Process

Benefits

Product

Benefits

Project Management Benefits

Improved Design Quality

Shared System Model

Common Language

Community Benefits

Improved Communication

Better Understanding Solution Space

More efficient Testing


Concluding Remarks - Modeling is becoming common in software development.

There are payoffs for modeling in: - improved code quality - improved communication

- Empirical studies enhance alignment of research and

practice – your input is needed

- Practical Quality Assurance for UML should use low hanging fruit:

i) Naming and layout-conventions, ii) Reviews & Inspections

iii) Versioning, iv) Metrics


Future Directions - Automated Reasoning about Designs

- Diagnosis & Prioritization

- Severity Assessment of design flaws / technical debt

- Next: Empirical studies

- by Hafeez & Arjan

- by Dave

Hafeez Dave


Comparing “MDD with code generation” & “Modeling + Programming”

MDD with code generation

Requires: - stable domain - very good understanding of domain - Multiple projects

- Don’t introduce in critical

projects

Needs dedicated techniques for Quality Assurance, estimation

Modeling with Programming

Off the shelves tooling Standard language (staff hiring & turnover) Divergence of model and code


How often do design issues lead to problems?

0

10

20

30

40

50

60

70

Inconsistency Understandability Inaccuracy Incompleteness

Pe

rce

nta

ge

of

th

e r

esp

on

de

nts

Never

Sometimes

Often

Very Often

Incompleteness of models was considered most often to lead to implementation problems


How do developers apply detail to UML models?

Developers apply more detail on critical and complex parts of a system

61%

52%

37%

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

Equal details for all parts

Details proportional to complexity

More details for critical parts

Programmers determine details

Percen

tag

e o

f th

e r

esp

on

den

ts

Disagree

Somewhat Disagree

Neutral

Somewhat agree

Agree

Ariad

i Nug

roho,

Miche

l R. V

. Cha

udron

: A Su

rvey o

f the P

ractic

e of D

esign

- Co

de Co

rrespo

nden

ce am

ongst

Profe

ssion

al So

ftware

Engin

eers.

ESEM

2007:

467

-469


Which factor most often drives developers to non-correspondence?

Among other factors, developers considered incomplete model more often drives them to non-correspondence

29%

48%

31%

0

10

20

30

40

50

60

Meeting deadline

Impractical model

Incomplete model

Model doesn’t satisfy

requirements

Perc

enta

ge o

f th

e r

espondents

Never Sometimes

Often Very Often

Ariad

i Nug

roho,

Miche

l R. V

. Cha

udron

: A Su

rvey o

f the P

ractic

e of D

esign

- Co

de Co

rrespo

nden

ce am

ongst

Profe

ssion

al So

ftware

Engin

eers.

ESEM

2007:

467

-469


Does strictness of implementing a model differ for different modeling constructs?

Developers favor to implement dependency and inheritance relations more strictly than other constructs

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

Package structure

Dependency relations

Inheritance relations

Class and method names

The order of method calls

Pe

rce

nta

ge o

f th

e r

esp

on

de

nts

Loosely

Somewhat loosely

Neutral

Somewhat strict

Strictly

31%

46%

36%

Ariad

i Nug

roho,

Miche

l R. V

. Cha

udron

: A Su

rvey o

f the P

ractic

e of D

esign

- Co

de Co

rrespo

nden

ce am

ongst

Profe

ssion

al So

ftware

Engin

eers.

ESEM

2007:

467

-469


Which classes are modeled?

Complexity and Coupling is higher for classes that are modeled than not modeled


Maintenance paradox

- Maintainers of software would like better documentation

- Even if UML models are created during design Too detailed models become outdated Better abstraction in reverse- / round trip-

engineering is needed in the setting of incremental & iterative development


Practical QA for UML modeling Naming and layout-conventions The Elements of UML2.0 Style,

Scott Ambler, Cambridge Univ Press, 2005

Reviews & Inspections Guidelines by e.g. Shull et.al., Biffl,

Version Management Many tools around (e.g. CVS, SVN, …)

Metrics SDMetrics http://www.sdmetrics.com/

Low Hanging Fruit (c) Erin Witzel, 2010

Investigations into effectiveness of software modeling

MetricView contributors Johan Muskens

Christian Lange

Martijn Wijns

Dennis van Opzeeland

Ariadi Nugroho

Werner Heijstek

Maurice Vermeer

Alex Telea

Robin van den Broek

MRV Chaudron Sheet 59

Investigations into effectiveness of software modeling Publications

Journal Publications

Supporting task-oriented modeling using interactive UML views. PDF C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Journal of Visual Languages and Computing (Elsevier), Volume 18, Issue 7, August 2007.

In Practice: UML Software Architecture and Design Description. C.F.J. Lange, M.R.V. Chaudron, J. Muskens. IEEE Software, Volume 23, Issue 2, March 2006.

Conference Publications

A Visualization Framework for Task-Oriented Modeling using UML. C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07). January 2007.

An Experimental Investigation of UML Modeling Conventions. C.F.J. Lange, Bart DuBois, M.R.V. Chaudron, Serge Demeyer. Proceedings of the ACM/IEEE International Conference on Model-driven Engineering Languages and Systems (MoDELS 2006). October 2006.

Effects of Defects in UML Models - An Experimental Investigation. C.F.J. Lange, M.R.V. Chaudron. Proceedings of the International Conference on Software Engineering (ICSE 2006). May 2006. (acceptance rate: 9%)

Improving the Quality of UML Models in Practice. C.F.J. Lange. Proceedings of the International Conference on Software Engineering (ICSE 2006). May 2006.

Visual Exploration of Combined Architectural and Metric Information. M. Termeer, C.F.J. Lange, A. Telea, M.R.V. Chaudron. Proceedings of VISSOFT 2005, IEEE CS Press. September 2005.

Combining Metrics Data and the Structure of UML Models using GIS Visualisation Approaches. C.F.J. Lange, M.R.V. Chaudron. IEEE International Conference on Information Technology 2005. Las Vegas, April 2005.

An Exploratory Study on the Industrial Use of UML: Improving Control over Design Quality. C.F.J. Lange, M.R.V. Chaudron. JACQUARD Conference. February 2005.

Investigations in Applying Metrics to Multi-View Architecture Models. J. Muskens, M.R.V. Chaudron, C.F.J. Lange. In Proceedings of EUROMICRO’04, Rennes, France, September 2004.

An Empirical Assessment of Completeness in UML Design. C.F.J. Lange, M.R.V. Chaudron. Proceedings of the 8th Conference on Empirical Assessment in Software Engineering (EASE04). May 2004.

Konsistenz und Vollständigkeit industrieller UML Modelle. C.F.J. Lange, M.R.V. Chaudron. Proceedings of “Modellierung 2004“, Marburg, Germany, 24-26 March 2004.

Workshop Publications

Towards Task-Oriented Modeling using UML. C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Proceedings of the 10th QAOOSE workshop, co-located with ECOOP2006. July 2006.

Managing Model Quality in UML-based Software development. C.F.J. Lange, M.R.V. Chaudron. Proceedings of IEEE Conference on Software Technology and Engineering Practice 2005 (STEP), in the "Workshop on Software Process Improvement, Quality Assurance and Measurement. September 2005.

Quantitative Techniques for the Assessment of Correspondence between UML Designs and Implementations. Dennis J.A. van Opzeeland, Christian F.J. Lange, Michel R.V. Chaudron. Proceedings of the 9th QAOOSE workshop, co-located with ECOOP2005. July 2005.

An Empirical Investigation in Quantifying Inconsistency and Incompleteness of UML Designs. C. Lange, M.R.V. Chaudron, J. Muskens, L.J. Somers, H.M. Dortmans. 2nd workshop on consistency problems in UML-based software development, October 20, 2003, in San Francisco, USA as part of the “International Conference on Unified Modeling Language 2003”.

60 Michel Chaudron, Christian Lange, Martijn Wijns

http://www.langomat.de/research/papers/LANGE_MVEexperiment_JVLC.pdf

Devnology Back to School: Empirical Evidence on Modeling in Software Development

Technology

Transcript of Devnology Back to School: Empirical Evidence on Modeling in Software Development