Day 2 meet shilpa - measuring software quality-are you up-to-date on what and how you measure to...

www.xbosoft.com

1

Webinar March 19 2014 11.00 AM EST

You are at the right place, it’s almost time….10 minute

countdown

WELCOME

XBOSOFT Software Quality Knowledge Center is proud to partner with and

introduce

Shilpa DadhwalCEO and Founder

SQE Labs, Inc.

Welcome to the session on ‘Measuring Software Quality –

Its Economic Value’

Shilpa DadhwalFounder & DirectorSQE Labs Inc.

Skype ID : shilpa2dEmail ID : [email protected]

Presented By:

On the callShilpa Dadhwal, Philip Lew and Jan Princen

Moderator: Carolyn Choh

Shilpa Dadhwal,Founder and CEO,SQE Labs Inc

Philip LewCEO, XBOSoft

Jan PrincenManaging Director, XBOSoft

Agenda

• Introduction XBOsoft• Software Quality – Quick review of Day 1• Moving on to Day 2

– Looking at Quality – what attributes to measure?– Implications for Senior Management– Implications for Project Management

• Benefits• QA & Next Sessions at 11 am -12 noon EST• Today’s Meeting March 19th

https://www3.gotomeeting.com/register/197591286

• Tomorrow’s meeting March 20th





• Specialize in: Software Quality Improvement

• Quality Process Improvement & Testing

• American Company• Located in: San Francisco, Amsterdam &

Beijing• Over 100 Software Quality Experts

XBOSoft Introduction

8

“Proven Results”

“Our products are very complex. We expected a one year ramp up time, but XBO was able to contribute to our release cycles in just 6 months. ."

- Development Vice President, Hyperion-Oracle

“

"The results are very encouraging. Thanks a lot for all the effort put into this project. We really appreciate your work."

- Manager, Quality Assurance, Autodesk

"Everything you've done is impressive and we are quite pleased with the quality of the test cases, well done!So keep up the good work."

- QA Director, CA

Welcome to the Session 2 on ‘Measuring Software Quality –

Its Economic Value’

Shilpa DadhwalFounder & DirectorSQE Labs Inc.

Skype ID : shilpa2dEmail ID : [email protected]

Presented By:

SQE Labs offers an independent Software Testing and Quality

Assurance services and is committed to improving quality by

providing superior QA solutions to customers

About USA Specialist Independent Software Testing Solutions Provider

Team of Software Quality Engineering professionals having strong

Techno-functional knowledge

Provides services in the areas of Outsourced Software Testing,

Software Testing Consultancy and Software Testing Trainings

Having Offices in Texas (US) and Chandigarh (India)

What is Software Quality?

What is its Economic Value?

How to mange Software Quality?

What are its Attributes?

Why do we need to measure it?

How do we measure it?

Issues while measuring it?

Guidelines for measuring it?

A Specialist Independent Software Testing Solution Provider Questions in Mind

Session 1

Session 2

Session 3

1. Define what quality means to you, for your software? (Give the highest priority

parameter)

‘High Reliability when Operating’

2. How can you measure this factor in your software?

No. of crashes encountered while working on the system for specified no. of

hours

3. Does this require any corrective or preventive action plan?

Yes

4. What action could be taken to improve it?

Testing of Memory Leaks in the application

5. How can you keep track of the same in any future product?

Record the no. of crashes encountered while running the application for

specified no. of hours

Case Study from Session 1

A Specialist Independent Software Testing Solution Provider

1. What is a METRIC?

2. What are DIFFERENT TYPES of Metric?

3. What are the desirable CHARACTERISTICS of a Metric?

4. What are the ATTRIBUTES of Software Quality?

5. How to NORMALIZE Metrics?

A Specialist Independent Software Testing Solution Provider Agenda

“When you can measure what you are speaking

about and express it in numbers you know

something about it;

but when you cannot measure it, when you cannot

express it in numbers, your knowledge is of a

meagre and unsatisfactory kind”.

William Thomson (Lord Kelvin)

QuoteA Specialist Independent Software Testing Solutions Provider

PollA Specialist Independent Software Testing Solutions Provider

How much time will you take to integrate this new module with the existing module?

• Much less

In Project A , 100 defects have been logged but in Project B, 200 defects have been logged. So which Project is better?

• Project B

How is the Software Quality of your product?• Good

Can you as a Manager , make any useful decisions from such subjective information?

1) Yes2) No

Measure: Quantitative indication of the extent, amount, dimension, or size of some attribute of a product or process. A single data point

Metrics: The degree to which a system, component, or process possesses a given attribute. Relates several measures (e.g. average number of errors found per person hour)

Direct Metrics: Immediately measurable attributes (e.g. line of code, execution speed, defects reported)

Indirect Metrics: Aspects that are not immediately quantifiable (e.g. reliability, productivity)

Common Terminology

A Specialist Independent Software Testing Solutions Provider

Products• Quantify the characteristics of the product being developede.g. Size, reliability

Process• Quantify the characteristics of the process being used to

develop the softwaree.g. Efficiency of fault detection, Effectiveness of Defect Removal

Project• Quantify the characteristics of the project and its executione.g. Staffing pattern over the lifecycle of the project, Productivity

Software Metric Classification


Product metrics are measures of the software product at any stage of its development, from requirements to release.

Focus on the quality of deliverables

Product metrics are combined across several projects to produce process metrics

Metrics for the product:o The complexity of the Software designo Size of the final programo Number of pages of documentation producedo Defect densityo Customer Satisfaction index

Product MetricsAn Independent Software Testing Solution Provider

Focus on quality achieved as a consequence of a repeatable or managed process.

Process Metrics are measures of the software development process, such as

o Overall development timeo Average delay of milestone completion

Process metrics are collected across all projects and over long periods of time

Their intent is to provide indicators that lead to long-term software process improvement

It is defined mostly at a Strategic level

Process MetricsAn Independent Software Testing Solution Provider

To improve any process,

Measure specific attributes of the process

Derive meaning metrics from these attributes

Use these metrics to provide indicators

The indicators lead to a strategy for improvement

Process Improvement


Project Metrics are measures of Software project and are used to monitor and control the project

Used by a project manager and software team to adapt project work flow and technical activities:

• Minimize the development schedule by making the necessary adjustments to avoid delays and mitigate problems

• Assess product quality on an ongoing basis and modify the technical approach to improve quality

Used in estimation techniques and other technical work

Project MetricsA Specialist Independent Software Testing Solutions Provider

Metrics collected in the past are used as a basis from which effort and time estimates are made for the current project

Metrics:o Effort or time per SE tasko Errors uncovered per review houro Scheduled vs. actual milestone dateso Number of changes and their characteristicso Distribution of effort on SE tasks

Tactical and Short Term.

Project MetricsAn Independent Software Testing Solution Provider

METRICS should be

– RELIABLE (free from random error)

– REPEATABLE (same entity, same environment, same visitors and same

evaluator)

– REPRODUCIBLE (same entity, same environment, same visitors but

different evaluator)

– AVAILABLE (constraint conditions)

– INDICATIVE OF IMPROVEMENT (of the entity quality)

– CORRECT (objective, impartial and precise)

– MEANINGFUL (about the entity behavior or quality characteristics).

Desirable characteristics of Metrics

An Independent Software Testing Solution Provider

A Metric is a qualitative or quantitative measurement of a specified Software Quality Attribute, for measuring Software Quality

Software Quality Metrics are subset of Software Metrics, that focus on quality aspects of product, process and project

Thus, in order to define Metrics for measuring Software Quality, we must know which Quality Attributes needs to be measured

The Quality Requirement Specification gives quantitative and qualitative values for these attributes that must be satisfied by the executable system

Measuring Software Quality


Software Quality attributes is a property of a software that can have some

qualitative or quantitative value and can be measured or observed.

Categories of Software Quality Attributes are :

1. Design Quality Attributes

2. Runtime Quality Attributes

3. System Quality Attributes

4. User Quality Attributes

5. Non-runtime Quality Attributes

6. Architecture Quality Attributes

7. Business Quality Attributes

Software Quality Attributes


Conceptual Integrity

• Defines the consistency and coherence of the overall design

• Includes the way the components or modules are designed

Metric – i) Mean Time to Change

Maintainability

• Ability of the system to undergo changes with a degree of ease

Metric – i) Mean Time to Change

Reusability

• Defines the capability for components and subsystems to be suitable for

use in other applications

Metric – i) Implementation time

Design Quality Attributes


Interoperability

• Ability of a system to operate successfully by communicating and exchanging

information with other external systems written and run by external parties

Metric – i) Modularity

Manageability

• Defines how easy is it for system administrators to manage the applications

Metric – i) Implementation time

Reliability

• Ability of the system to remain operational over time

Metric – i) Mean Time to Failure

Scalability

• Ability of the system to either handle increases in load without impact on the

performance of the system or the ability to be readily enlarged

Metric – i) Horizontal Scalability

RuntimeQuality Attributes


Performance

• Indication of the responsiveness of a system to execute any action

Metric – i) Throughput

Security

• Capability of a system to prevent malicious or accidental actions outside of the

designed usage

Metric – i) No. of Security defects logged per KLOC

Availability

• Portion of the time the system is functional and working

Metric- i) Total Unavailability = Measured as a percentage of the total system

downtime over total no. of hours of service in a year period

RuntimeQuality Attributes


Supportability

• Ability of a system to provide information helpful for identifying and

resolving issues when it fails to work correctly

Metric – i) Time taken to resolve an issue encountered in the software

Testability

• Measure of how easy it is to create a test criteria for a system and its

componentMetric – i) Modularity

SystemQuality Attributes


Usability

• Defines how well the application meets the requirements of the user

and consumer by being intuitive

Metric - i) Learnability

UserQuality Attributes


Portability

• Ability of a system to run under different computing environments

Metric – i) Install ability

Integrity

• Ability to make the separately developed components of the system work

correctly together

Metric – i) Access Control

Reusability

• Degree to which existing applications can be reused in a new applicationMetric- i) Implementation time

Modifiability

• Ease with which a software system can accommodate changes to its software

Non-RuntimeQuality Attributes


Correctness

• Accountability for satisfying all requirements of the system

Metric - Defects per FP/LOC

Conceptual Integrity

• Integrity of the overall structure that is composed from a number of

small architectural structures

ArchitectureQuality Attributes


Cost

• Cost of a system with respect to time to market, expected project

lifetime

Metric – i) Time to Market

Marketability

• Use of the system with respect to market competitionMetric – i) Price Earning ratio

BusinessQuality Attributes


In Project A , 100 defects have been logged. But in Project B, 200 defects have been logged.

So which Project is better?

Can we compare the two projects, just by looking at the above data?

Additional information required:KLOC for Project A = 1000KLOC for Project B = 10,000

Defect Density of Project A = 100/1000= 0.1defects/KLOCDefect Density of Project B = 200/10000= 0.02 defects/KLOC

Thus, in Project A more defects have been logged per KLOC as compared to Project B

Normalization of Metrics


How does an organization combine metrics that come from different individuals or projects?

Depend on the size and complexity of the project

Normalization: compensate for complexity aspects particular to a product

Normalization approaches:Size oriented (lines of code approach)Function oriented (function point approach)

Normalization of Metrics


Case Study of Metrics Program


• Motorola followed Goal/Question/Metric paradigm

• Goals were identified, questions were formulated in quantifiable terms and metrics were established

• Few Examples are as follows:• Goals:

• Goal 1: Increase Software Reliability• Goal 2: Reduce Cost of Non-Conformance

• GOAL 1: INCREASE SOFTWARE RELIABILITY• Question: What is the rate of software failures and how does it

change over time?• Metric: Failure Rate = Number of failures/Execution time

• GOAL 2: REDUCE COST OF NON-CONFORMANCE• Question: What is the cost to fix post –release problems during

the month?• Metric: Cost of fixing Problems = Dollar cost associated with

fixing post-release problems within a month

To be continued in Session 3

Next…A Specialist Independent Software Testing Solution Provider

What is Software Quality?

What is its Economic Value?

How to mange Software Quality?

What are its Attributes?

Why do we need to measure it?

How do we measure it?

Issues while measuring it?

Guidelines for measuring it?

A Specialist Independent Software Testing Solution Provider Questions Answered

Session 1

Session 2

Session 3

Quality attributes are measurable or observable characteristics of

products and processes.

The quality requirements specification gives quantitative or

qualitative values for these attributes that must be satisfied by the

executable system.

Software Quality can be measured only if you know which

Software Quality Attribute should be measured and which metric

should be used for measuring the same.

A Specialist Independent Software Testing Solution Provider Few Remarks

Thank YouA Specialist Independent Software Testing Solutions Provider

India OfficeSQE Labs Inc.

116/8, Mohali (Chandigarh)India – 160062

www.sqelabs.com

US OfficeSQE Labs LLC,

6533, Preston Rd, Plano,TX - 75024

Contact me : [email protected]

http://www.sqelabs.com/



mailto:[email protected]

Follow: @XBOSoft

Questions: [email protected]

mailto:[email protected]

Thank You!

See you tomorrowTomorrow’s meeting March 20th


Shilpa Dadhwal,Founder and CEO,SQE Labs Inc

Philip LewCEO, XBOSoft

Jan PrincenManaging Director, XBOSoft


Day 2 meet shilpa - measuring software quality-are you up-to-date on what and how you measure to...

Technology

Transcript of Day 2 meet shilpa - measuring software quality-are you up-to-date on what and how you measure to...