Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First...

57
Software Quality and Testing (SCSJ3343)

Transcript of Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First...

Page 1: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Software Quality and Testing (SCSJ3343)

Page 2: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Lecturer’s Details

Dr. Rohayanti Hassan(Sec. 03)

Webpage: http://comp.utm.my/rohayanti

Phone: 07-5538719

Email: [email protected]

Page 3: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Course Schedule

• Sec. 3 - Perdana

– Thursday (8.30 am – 11.00 am) @ BK4, N28.

• UTMSPACE

– Refer schedule @ BK4, B11, FAB.

Page 4: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Attendance Policies

• 42 hours of lectures over 14 weeks

• 80% of attendance is compulsory – W4 (16 hrs), W7 (28hrs), W9 (36hrs)

• Warning letter(s) will be issued

• If less than 80% - not allowed to sit for final!

Page 5: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Course Policy

• Be PUNCTUAL !

– Will not be allowed to enter the class after 10 minutes!

• 80% attendance rule applies.

• NO Plagiarism for assignments and project

– 0 mark will be given.

• Etiquette. Please keep all cell phones, pagers, and other electronic devices (including laptops) turned OFF during class. If your activities during class are deemed disruptive, you will be asked to leave. And please dress appropriately.

Page 6: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Course Learning Outcomes

i. Apply established software quality and testing concepts, the fundamental test process, test approaches, and principles to support test objectives.

ii. Design tests by using established techniques

iii. Execute test cases according to test design, and analyze and report on the results of tests.

iv. Ability to do work effectively in a team and lead the team in organising and illustrating the test activities throughout the software testing life cycle

• Please fill in the entry survey at the end of this lecture. URL available on the e-learning

Page 7: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Assessment

Page 8: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Mid-Semester Exam

– Date & Day: Friday ( 3/11/17 )

– Time: 9 am

– Venue

• Sec 01 – BK1

• Sec 02 – BK 2

• Sec 03 (UNSRI) – BK3

Page 9: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

SOFTWARE QUALITY ASSURANCE

INTRODUCTION

Page 10: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Basic Questions

• What is the point of SQA?

• How necessary is SQA? • How good is software?

• Is software production different from producing other products? If so, how and why?

Page 11: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Objectives

• What is Software Quality?

• Why is Software Quality important?

• What is Software Quality Assurance ?

• Software Quality factors

• Elements of Software Quality Assurance

• Development and quality plans

• Process Maturity models

Page 12: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

What is Software?

• Software is:

– Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system.

[IEEE]

Page 13: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Differences between Software and Other Products

1. Product Complexity

2. Product Visibility

3. Product Development Process

Page 14: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

"Software Crisis"

• term coined by DoD years ago

• Problem Today: complexity of problems addressed by software has outpaced improvements in software creation process

Time

supply

demandP

rogr

amm

ers

Page 15: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Quality Products

Page 16: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Software vs. Quality

• Software is:

– Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system.

• Quality:

– The combination of all 4 components is needed to assure the quality of the development process as well as the ensuing long years of maintenance.

Page 17: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Software: Good & Bad

Page 18: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

The Big Question

• Q: How do we assure quality?

• A: We have a good process.

Page 19: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

What is Quality?

• Popular Views about Quality:– quality related to luxury, class and taste

– quality is related to quality of life.

– I know about quality when I see the product.

• Professional Views about Quality:– Conformance to requirements.

– Fitness for use.

© Business Beam (Pvt.) Limited 21

Page 20: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Two aspects of Software Quality

• “q” – Intrinsic product quality, often limited to defect rate and reliability.

• “Q” – Customer Satisfaction, often includes product quality and process quality.

© Business Beam (Pvt.) Limited 22

Page 21: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Pressman's definition of"Software Quality"

Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristicsthat are expected of all professionally developed software.

Page 22: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

IEEE Definition of "Software Quality"

1. The degree to which a system, component, or process meets specified requirements.

2. The degree to which a system, component, or process meets customer or user needs or expectations.

Page 23: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Role of Customer

• Customers don’t buy products, they buy assurances that their expectations with the products would met.

• You also, can only sell assurances – in other words, Quality.

• Customer is in absolute position to judge the quality.

© Business Beam (Pvt.) Limited 25

Page 24: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Quality Assurance

• Quality assurance is a system of activities designed to ensure production that meets pre-established requirements and standards.

© Business Beam (Pvt.) Limited 26

Page 25: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

IEEE Definition of"Software Quality Assurance"

1. A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements.

2. A set of activities designed to evaluate the process by which the products are developed or manufactured. Contrast with quality control.

Page 26: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

SQ vs SQA

• SQ: The degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations. [IEEE]

• SQA: Part of quality management focused on providing confidence that quality requirements will be fulfilled. [ISO 9000]

Page 27: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Quality Assurance vs. Quality Control

• Quality control and quality assurance activities serve different objectives

• Quality control activities are only a part of the total range of quality control assurance activities

Page 28: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

More Definitions

• "software error"

• "software fault"

• "software failure"

• types of errors1. code error

2. procedure error

3. documentation error

4. software data error

Page 29: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Nine causes of Software Errors

• Faulty requirements definition.

• Client-developer communication failures.

• Deliberate deviations from software requirements.

• Logical design errors.

• Coding errors.

• Non-compliance with documentation and coding instructions.

• Shortcomings of the testing process.

• User interface and procedure errors.

• Documentation errors.

Page 30: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

When are defects introduced?

• The majority of defects are introduced in earlier phases.

– Requirements are the top factor in a project’s success or failure.

Phase % of defects

% effort to fix

Requirements 56 82

Design 27 13

Code 7 1

Other 10 4

Page 31: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Cost of fixing defects

• Relative cost of fixing defects

– benchmark: cost at requirements phase = 1

Phase found Cost ratio

Requirements 1

Design 3 – 5

Coding 10

Unit / integration testing 15 - 40

System / acceptance testing 30 – 70

Production 40 - 1000

Page 32: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Importance of Software Quality

• Software is a major component of computer systems (about 80% of the cost) – used for

– Communication (e.g. phone system, email system).

– Health monitoring.

– Transportation (e.g. automobile, aeronautics).

– Economic exchanges (e.g. e-commerce).

– Entertainment.

– etc.

• Software defects are extremely costly in term of

– money

– reputation

– loss of life

Page 33: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Importance of Software Quality

• Notable consequences of software errors:

– 1988 shooting down of Airbus 320 by the USS Vincennes - cryptic and misleading output displayed by tracking software.

– 1991 patriot missile failure - inaccurate calculation of time due to computer arithmetic errors.

– London Ambulance Service Computer Aided Dispatch System – several deaths.

– On June 3, 1980, the North American Aerospace Defense Command (NORAD) reported that the U.S. was under missile attack.

– First operational launch attempt of the space shuttle, whose real-time operating software consists of about 500,000 lines of code, failed -synchronization problem among its flight-control computers.

– 9 hour breakdown of AT&T's long-distance telephone network - caused by an untested code patch.

Page 34: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Importance of Software Quality

• Ariane 5 explosion June 4, 1996

– First flight of the European Ariane 5 launcher crashed about 40 seconds after takeoff.

– Cost was about half a billion dollars.

– Explosion was the result of a software error:

• Uncaught exception due to floating-point error: conversion from a 64-bit integer to a 16-bit signed integer applied to a larger than expected number.

• Module was re-used without proper testing from Ariane 4 .

– Error was not supposed to happen with Ariane 4.

– No exception handler.

Page 35: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Importance of Software Quality

• Mars Climate Orbiter - September 23, 1999

– Mars Climate Orbiter, disappeared as it began to orbit Mars.

– Cost about $US 125-million.

– Failure due to error in a transfer of information between a team in Colorado and a team in California.• One team used imperial units (e.g., inches, feet and pounds) while

the other used metric units for a key spacecraft operation.

Page 36: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Importance of Software Quality

• Mars Polar Lander - December, 1999

– Mars Polar Lander, disappeared during landing on Mars.

– Failure most likely due to unexpected setting of a single data bit.• Defect not caught by testing.

• Independent teams tested separate aspects.

Page 37: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Importance of Software Quality

• Internet viruses and worms– Blaster worm (US$ 525 million)– Sobig.F (US$ 500 million – 1 billion)

• Exploit well known software vulnerabilities– Software developers do not devote enough effort to

applying lessons learned about the causes of vulnerabilities.

– Same types of vulnerabilities continue to be seen in newer versions of products that were in earlier versions.

• Usability problems

Page 38: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Importance of Software Quality

• Monetary impact of poor software quality (Standish group - 1995)

• 175,000 software projects/year - Average Cost per project– Large companies - $US 2,322,000

– Medium companies - $US 1,331,000

– Small companies - $US 434,000

• 31.1% of projects canceled before completed.– cost $81 billion

Page 39: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Importance of Software Quality

• 52.7% of projects exceed their budget - costing 189% of original estimates.– cost $59 billion

• 16.2% of software projects completed on-time and on-budget (9% for larger companies).

• Large companies - delivered systems have approximately only 42% of originally-proposed features and functions.

• 78.4% of smaller companies projects get deployed with at least 74.2% of their original features and functions.

Page 40: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Software Quality Assurance

Page 41: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

SQA Activities

• Assuring, with acceptable levels of confidence, conformance to functional technical requirements.

• Assuring, with acceptable levels of confidence, conformance to manageria requirements of scheduling and budgets.

• Initiating and managing activities for the improvement and greater efficiency of software development and SQA activities.

Page 42: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

SQA Responsibilities for a Project

• Review of documents developed by development team.

• Track the compliance with standards.

• Development of QA Plan (test plan + test cases).

• Implementation of test cases (Black Box or Glass Box Testing).

• Management of bug repository.

• Participating in code and design reviews.

© Business Beam (Pvt.) Limited 44

Page 43: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

SQA Across the Life Cycle

© Business Beam (Pvt.) Limited 45

Requirements Collection

Analysis

Architecture & Design

Development

Testing

Deployment

Requirement Specs

Functional Specs

Design Specs

Code & Executables

Deployment Docs

Typical SDLC phases and relevant Artifacts

Reviews

Reviews

Reviews

Implement Test Cases

Review

SQA Responsibilities

Page 44: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

SQA Artifacts for a Project

© Business Beam (Pvt.) Limited 46

Requirement Specs

Functional Specs

Design Specs

Code & Executables

Deployment Docs

Dev. Artifacts SQA Artifacts

RS Reviews

FS Reviews

DS Reviews

Bug Reports

DD Reviews

Test Plan

Test Cases

More Test Cases

SQA Artifacts (more)

Page 45: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Software Quality Factor

Page 46: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Question

• To know that quality has improved, it would be helpful to be able to measure quality.

• How can we measure quality?

Page 47: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Requirements

• SoftwareRequirements

– A condition or capability needed by a user to solve a problem or achieve an objective that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document. (IEEE)

• RequirementEngineering

– The development and use of cost‐effective technology for the elicitation, specification and analysis of the stakeholder requirements which are to be met by software intensive systems (Berry and Ryan, 1999)

Page 48: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Comprehensive Requirements

• Comprehensive definition of requirements that will cover all aspects of software use throughout all stages of the software life cycle.

• Factor models define the broad spectrum of software requirements.

Page 49: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Software Quality Factors

Different Models:

• McCall’s Factor Model (11 Factors)

• Evans and Marciniak factor model (12 Factors)

• Deutsch and Willis factor model (15 Factors)

• ISO/IEC 9126 (10 Factors)

Page 50: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

McCall’s Factor Model

Page 51: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Deutsch & Willis Factor Model

Page 52: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Deutsch & Willis Factor Model (cont.)

Page 53: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

ISO/IEC 9126: Internal and External Quality

Page 54: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

ISO/IEC9126 Sub-characteristicsQuality Sub-characteristics

Functionality SuitabilityAccuracyInteroperabilitySecurity

Portability InstallabilityCoexistenceAdaptabilityReplaceability

Efficiency Resource UtilizationPerformance

Reliability MaturityFault tolerance (robustness)Recoverability

Usability UnderstandabilityLearnabilityOperabilityAttractiveness

Maintainability AnalyzabilityChangeabilityStabilityTestability

Page 55: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

In-class Exercise

• What quality characteristic and its relevant sub-characteristic that best fits the following requirements:

– If the customer provides a “How Did You Hear About Us” identifier code during the application process, retrieve customer information from GloboRainBQW.

– In case of a disconnection of the Telephone Banker’s workstation from the system while dealing with a customer, the system shall restore the work to the same state when connected.

– All pages shall be self contained with all control information built into the page such as that the user does not have to leave the screen to get help.

Page 56: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Summary

• Relationship of software, software quality & SQA

• The need of software quality factor

• Different software quality factors

Page 57: Software Quality and Testing SCSJ3343 · reported that the U.S. was under missile attack. –First operational launch attempt of the space shuttle, whose real-time operating software

Formation of Group

• Refer to your e-learning

• Ice breaking session

• Problem Solving 1