1

ECE 453 – CS 447 – SE 465 Software Testing & Quality Assurance

InstructorPaulo Alencar

2

Welcome

• Welcome to the Software Testing and Quality Assurance course

• Demanding, challenging and rewarding course• A glimpse of what software testing in theory and

practice is all about• Course website:

www.student.cs.uwaterloo.ca/~cs447/Winter2008

3

How to reach me

• Paulo Alencar– E-mail: palencar@cs.uwaterloo.ca– Phone extension: 33944– Office hours:

• Friday 1:00 – 2:00pm;

2:00 – 3:00pm (by appointment)

• DC 2597B

4

Outline for today

Introduction

• Course description

• Software Testing Basics

5

Course components• Lectures - 3

– SE 465 E2 1303• 11:30-12:20 MWF

– ECE 453 / CS 447 E2 1303• 4:30 – 5:20 MWF

• Tutorials and Discussions – The schedule is on the website

• Project– Evolution– Testing

6

Course Resources

• Schedule• Lecture notes & slides• Recommended book

– Software Testing A Craftsman's Approach2nd edition, Paul C. Jorgensen, CRC Press

• Past exams• Project description• Grade allocation• Course news system

7

Project component

• Design and implement software enhancements and testing

• Groups of 4 • 30% of total grade• Two parts:

– System Enhancements• Requirements• Design• Implementation

– Testing• Black Box• White Box

8

Goals of this course

• Become familiar with software testing and quality assurance concepts and methods

• Learn how to design and plan software testing activities

• Experience software testing in a project that uses several components

Note:– “Scratching the surface of software testing and QA”– “Fitting you to become an amateur software tester”

9

Course outline

Unit 1: Overview of Software Testing (Chapter 1-2)• Unit 2: Mathematics for S/W Testers (Chapters 3-4)• Unit 3: Black Box / Functional Testing (Chapters 5-8)• Unit 4: White Box / Structural Testing (Chapters 9-11)• Unit 5: Integration and System Testing (Chapter 12-13)• Unit 6: Soft. Reliability and Quality Assurance Basics• Unit 7: Software Reengineering and Evolution Basics• Unit 8: Project Planning and Management

10

Course Grade

Component Weight

Project 30%

Midterm Exam 20%

Final Exam 50%

11

What to do now

• (perhaps) Adjust course registration

• UW Bookstore for textbook Software Testing - A Craftsman's Approach

2nd edition, Paul C. Jorgensen, CRC Press

• Start reading chapters 1-2

12

Outline for today

• Introduction

• Course descriptionSoftware testing basics

13

MotivationSoftware problems and related faults:

• Ariane 5 rocket – cost $500M Data conversion (64-bit to 16-bit format)

• Therac 25 – cost 6 livesOverdosing patients during radiation

• Mars missions – orbital explores & polar lander – cost $300MMixture of pounds and kilograms

• NASA Mariner 1 probe –Period instead of a comma in FORTRAN do-loop

• AT&T long distance service fails for 9 hoursWrong break statement in C-code

• USS Yorktow cruiserInput division by zero

14

Software Lifecycle

• Requirements phase• Analysis phase• Design phase (System and Object)• Implementation phase• Testing phase• Integration phase• Maintenance phase• Retirement

15

Software Lifecycle Activities

Subsystems

Structured By

class...class...class...

SourceCode

Implemented By

Solution Domain Objects

Realized By

SystemDesign

ObjectDesign

Implemen-tation

Testing

Application

Domain Objects

Expressed in Terms Of

Test Cases

?

Verified By

class....?

RequirementsElicitation

Use CaseModel

Analysis

...and their models

16

Software Testing: Definition

“The process of devising a set of inputs to a given piece of software that will cause the software to exercise some

portion of its code.

The developer of the software can then check that the results produced by the software are in accord with his or her

expectations.”

www.reactive-systems.com/esda-glossary.msp

17

A Testing CycleAlthough testing varies between organizations, there is a cycle

to testing:

1. Requirements Analysis: Testing should begin in the requirements phase of the software life cycle (SDLC).

2. Design Analysis: During the design phase, testers work with developers in determining what aspects of a design are testable and under what parameter those testers work.

3. Test Planning: Test Strategy, Test Plan(s), Test Bed creation. 4. Test Development: Test Procedures, Test Scenarios, Test Cases,

Test Scripts to use in testing software. 5. Test Execution: Testers execute the software based on the plans

and tests and report any errors found to the development team. 6. Test Reporting: Once testing is completed, testers generate metrics

and make final reports on their test effort and whether or not the software tested is ready for release.

7. Retesting the Defects

18

Some Software Testing Objectives

• Find as many defects as possible.• Find important problems fast.• Assess perceived quality risks.• Advise about perceived project risks.• Advise about perceived quality.• Certify to a given standard.• Assess conformance to a specification

(requirements, design, or product claims).

19

Software Quality Assurance: Definition

“The function of software quality that assures that the standards, processes, and procedures are appropriate for

the project and are correctly implemented.”

www.hq.nasa.gov/office/codeq/software/umbrella_defs.htm

20

Software Maintenance: Definitions

Software Evolution:

“A continuous change from a lesser, simpler, or worse state to a higher or better state [for a software system]”

www.cc.gatech.edu/reverse/glossary.htm

Software Maintenance:

“Software maintenance is one of the activities in software engineering, and is the process of enhancing, porting, and

optimizing deployed software, as well as remedying defects”

21

Average cost distribution(1976–1981 data)

Object-Oriented and Classical Software Engineer, 5th Edition, Schach (2002)

22

Summary

• Testing is an important part of the Software Lifecycle

• Highly technical and challenging• It is affected by the selected process• Quality Assurance is paramount both for mission

critical and non-critical systems• Software Evolution aims to keep systems

operational when environment changes occur