1

Software Quality

1

Overview

§ Everything we have been talking about all semester has had something to do with software quality. § This is perhaps the most central and important topic of the

course.

§ We will discuss the meaning of quality, and how to achieve and assure it.

§ We will then turn our attention to defects: where they come from, what to do with them, and how to prevent them.

§ We then discuss the Quality Triangle: People, Processes, and Tools.

§ We conclude with a discussion of software metrics. 2

2

Objectives

§ Here is what you should be able to do upon completion of this module:§ Define Software Quality. § Achieve and assure software quality.§ Discuss the Return on Investment (ROI) of quality processes.§ Explain how to prevent defects.

§ Describe the interrelationship of People, Process and Tools.§ Describe the Quality organization: CM, Testing, and SQA. § Discuss the use of metrics in a software engineering process.

3

Outline

§ Software Quality.§ Definition.§ Assuring and Achieving quality.§ ROI (return on investment).

§ Defects:§ Definition.§ Preventing defects.§ Finding and fixing defects.

§ The Quality Triangle: People, Processes, and Tools.§ Software Metrics.4

3

5

Software Quality

§ Definition of quality: The degree to which a system, component, or process meets (1) specified requirements, and (2) customer or user needs or expectations.1

6

Software Quality Criteria

§ Correctness§ Efficiency§ Flexibility§ Robustness§ Interoperability§ Maintainability§ Performance

§ Portability§ Reliability§ Reusability§ Testability§ Usability§ Availability§ Understandability

4

7

Achieving Quality

§ How do we build quality into a product?

8

Assuring Quality

§ How do we know we have quality?§ How do we demonstrate this to others?

5

9

Return On Investment

§ What is the cost of quality?§ What is the cost of lack of quality?

10

Defects

§ Definitions.§ Preventing defects.§ Finding and fixing defects.

6

11

Some Definitions

§ Error: A conceptual, syntactic or clerical discrepancy which results in one or more faults in the software. 1

§ Fault: A specific manifestation of an error. A discrepancy in the software which can impair its ability to function as intended. An error may be the cause for several faults. 1

12

More Definitions

§ Failure: A software failure occurs when a fault in the computer program is evoked by some input data, resulting in the computer program not correctly computing the required function in an exact manner. 2

§ Defect: Either a fault or discrepancy between code and documentation that results in mischief in the testing, installation, maintenance, or use of software. 3

7

13

References

§ 1 Glossary of Software Engineering Terminology, IEEE Std 610.12-1990§ 2 Lloyd, D.K., and M. Lipow, Reliability,

Management, Methods and Mathematics, 2nd

Edition, published by the authors, 1977.§ 3 Dunn, Robert, and Ullman, Richard, Quality

Assurance for Computer Software, McGraw-Hill, New York, 1982.

14

Categorization of Defects

§ Classified by phase in which defect is introduced:§ Inadequate or incorrect

requirements definition.§ Inadequate or non-

compliant top-level design.§ Errors in detailed

design.§ Coding errors.

§ Generic Faults:§ Computational.§ Logic.§ I/O.§ Data Handling.§ Interface.§ Database & Data

Definition.§ Other.

8

15

Preventing Defects

§ Standards.§ Software Engineering Methods.§ Languages (let the compiler do some of the

checking).§ Formal proofs of correctness.§ Prototyping.§ Continuous Process Improvement.

16

Finding Defects

§ Reviews.§ Walkthroughs.§ Formal Inspections.§ Testing.§ Defect Reporting.

9

17

Removing Defects

§ Configuration Management.§ Baselines.§ Change Control Board.§ Software Configuration.

§ Regression testing.§ Ripple effect.

18

The Quality Triangle

Processes

10

19

Quality Assurance

§ Organization.§ Configuration Management.§ Testing (IV&V).§ SQA.

§ Standards.§ Tools / Support Software.

20

Configuration Management

§ Bill of Materials hierarchy.§ Software Library.§ Each version of each component.

§ Configuration Matrix.§ Version control.

§ Change Control Board (CCB).

11

21

Testing

§ Purpose of testing:§ Defect detection.§ Verification.§ Validation.§ Acceptance.

22

Phases of Testing

System Specification Qualification Test

Software Requirements

Integration Test

Software Design String Test

Code Unit Test

12

23

Software Quality Assurance

§ Independent organization.§ Concerned with the software process.§ Guided by Standards.

24

Standards

§ Software Quality Assurance Program.§ Tools, Techniques

and Methodologies.§ Computer Program

Design.§ Work Certification.§ Documentation.

§ Library Control.§ Reviews and Audits.§ Configuration

Management.§ Testing.§ Corrective Action.§ Subcontractor

Control.Source: Mil-S-52779A, ‘‘Software Quality Assurance Program Requirements’’

13

25

Tools

§ Requirements.§ Analysis and Design.§ Coding.§ Configuration Management.§ Testing.§ Software Quality Assurance.§ Program Management.

26

Metrics

§ Definitions.§ Product Metrics.§ Process Metrics.

§ Use of Metrics.§ Establishing a metrics program.

14

27

Metrics

§ Process Metrics – measure aspects of the process.§ Product Metrics – measure aspects of the

product.

28

Software Product Metrics

§ Examples of software metrics:§ Complexity.§ Reliability.§ Usability.

§ Types of metrics:§ Static – can be counted without running the software.§ Dynamic – must run the software with

instrumentation.

15

29

Reliability -- MTBF

§ MTBF: Mean time between failures.§ As testing proceeds, the rate of defects found

(hopefully) decreases. § As the defect rate decreases, the MTBF

increases.

Test time

Defect rate

MTBFGoal

30

Software Process Metrics

§ Examples:§ Productivity.§ Manageability.§ Scrap & Rework.§ Turnover in Personnel.

§ Types of metrics:§ Static – counting things.§ Dynamic – change over time.

16

31

Use of Metrics

§ Proper uses of metrics:§ Statistical Process Control.§ Continuous Process Improvement.

§ Improper uses of metrics:§ Using product or process metrics to measure people.§ Setting arbitrary measurement targets.

32

Establishing a Metrics Program

§ How do you decide what to measure?§ Measure everything you can think of, and then see

what is useful.or

§ Determine what is useful (i.e., what your goals are) in advance, then see what needs to be measured to support those goals.§ GQM – Goal Question Metric