PROGRAMMING TESTING 201300071-1B MODULE 2: SOFTWARE SYSTEMS 22 NOVEMBER 2013.
Module for Software Testing
-
Upload
mohd-khanapi-abd-ghani -
Category
Documents
-
view
245 -
download
1
description
Transcript of Module for Software Testing
1
Software Testing
and
Quality Assurance
Lab module
Mohd Khanapi Abd Ghani Mohd Suffian Sulaiman
Juliana Jaafar
Department of Software Engineering Faculty of Information and Communication Technology
Universiti Teknikal Malaysia Melaka 2010
2
TABLE OF CONTENTS
LAB 1: PROJECT PLANNING .............................................................................. 5
1.1 INTRODUCTION............................................................................................. 5 1.2 STEP-BY-STEP/ EXAMPLES ......................................................................... 8 1.3 EXERCISES ................................................................................................... 13 1.3 SELF REVIEW QUESTION .......................................................................... 17
LAB 2: REQUIREMENT ANALYSIS.................................................................. 18
2.1 INTRODUCTION........................................................................................... 18 2.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 19 2.3 SELF REVIEW QUESTION .......................................................................... 25
LAB 3: SOFTWARE DESIGN .............................................................................. 26
3.1 INTRODUCTION........................................................................................... 26 3.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 27 3.3 SELF REVIEW QUESTION ........................................................................... 29
LAB 4: SOFTWARE TEST PLAN ....................................................................... 30
4.1 INTRODUCTION........................................................................................... 30 4.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 32 4.3 SELF REVIEW QUESTION .......................................................................... 35
LAB 5: SOFTWARE TEST DESCRIPTION....................................................... 36
5.1 INTRODUCTION........................................................................................... 36 5.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 41 5.3 SELF REVIEW QUESTION .......................................................................... 43
LAB 6: STUB AND DRIVER ................................................................................ 44
6.1 INTRODUCTION........................................................................................... 44 6.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 45 6.3 SELF REVIEW QUESTION .......................................................................... 49
LAB 7: RECORDING THE TEST RESULT ....................................................... 50
7.1 INTRODUCTION........................................................................................... 50 7.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 52 7.3 SELF REVIEW QUESTION .......................................................................... 54
LAB 8: UNIT TESTING ........................................................................................ 55
8.1 INTRODUCTION........................................................................................... 55 8.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 57 8.3 SELF REVIEW QUESTION .......................................................................... 60
LAB 9: INTEGRATION / SYSTEM TEST .......................................................... 61
9.1 INTRODUCTION........................................................................................... 61 9.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 62
3
9.3 SELF REVIEW QUESTION ........................................................................... 63 LAB 10: USER ACCEPTANCE TEST................................................................... 64
10.1 INTRODUCTION........................................................................................... 64 10.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 65 10.3 SELF REVIEW QUESTION .......................................................................... 66
LAB 11: QUALITY PLAN....................................................................................... 67
11.1 INTRODUCTION........................................................................................... 67 11.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 68
LAB 12: REVIEW MEETING ................................................................................ 74
12.1 INTRODUCTION........................................................................................... 74 12.2 STEP-BY-STEP/ EXAMPLES ....................................................................... 75 12.3 SELF REVIEW QUESTION .......................................................................... 77
4
Faculty of Information and Communication Technology, Universiti Teknikal Malaysia Melaka, Karung Berkunci 1200, 75450 Melaka, Malaysia. © UTeM 2010 FIRST EDITION 2005 All rights reserved. No part of this publication may be reproduced stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without either the prior written permission of the publisher. The programs in this book have been included for their instructional value. They have been tested with care but are not guaranteed for any particular purpose. The author or publisher does not offer any warranties or representations nor does it accept any liabilities with respect to the programs.
SOFTWARE TESTING &
QUALITY ASSURANCE
Mohd Khanapi Abd Ghani Mohd Suffian Sulaiman
Juliana Jaafar
5
LAB 1: PROJECT PLANNING
By the end of this section, students should be able to: Form a group and develop the team hierarchy with dedicated designation.
Describe task for each dedicated designation.
Identify the deliverables that should be produced for this lab project.
Produce a Software Development Plan (SDP) which focuses mainly on formal
qualification testing and software product evaluation.
1.1 INTRODUCTION
Project is a temporary task that should be finished in a certain period. The total
number of period to complete the project shall be agreed by both parties – contractor and
customer. In other words, project is a specialized task force within specific timeline and
budget which only finish once the task given has been accomplished. A project usually
starts in response to a broad objective or initiates through business requirement. In order
to make sure the project will be successes; the project should have a clear definition and
goal, good teamwork and sufficient resources, facilities, budget and time. Otherwise, it
will be difficult to achieve the targeted goal or fulfill the client’s needs.
Along this course, students will be exposed to an actual environment of software
development life cycle. The student will be assigned with a project and the project will be
done or execute during along twelve sessions of lab. To execute a project, the student will
follow each phases of software engineering process starting from requirement analysis,
analysis modeling, design modeling, implementation and testing to get the gist of running
a real software project. However, this lab module will only emphasize on testing phase
6
and any of testing related activities as the rest of other phases are already covered in
Requirement Analysis and Design subject. Nevertheless, te student shall deliver the
common software engineering documents including Software Development Plan (SDP),
Software Requirements Analysis (SRA), Software Requirements Specification (SRS),
Software Design Description (SDD), Software Test Plan (STP) and Software Test Result
(STR). It was noted that the same documents also being produced in most of the software
development project in industry.
As in real environment, student will need to run the project in team. They have to
form a hierarchical group organization, identify roles such as team leader, programmer,
system analyst and document’s writer, and assign each role and specific task to an
individual. Team leader will be the key person in managing the project starting from
capturing the requirement until the delivering of the product. Whilst, team members will
support the team leader by giving their full commitment and cooperation.
Communication is one of the key factors contributed to the success of the software
project. It is a good practice to have a frequent discussion or meeting in order to discuss
and resolve any arising project issues.
1.1.2 SOFTWARE DEVELOPMENT PLAN (SDP)
Software Development Plan (SDP) is a crucial document to software project
development groups as a means of recording and monitoring the planned project.
Basically, SDP contains chapters such as the Software Development Management,
Software Engineering, Formal Qualification Testing, Software Product Evaluations and
Software Configuration Management.
Software Development Management chapter focuses on the project organization
and resources which includes the contractor facilities and organizational structure. A
detail schedule and key milestones should also be included in the chapter. Each of project
activities can be represented using Gantt chart while the activity network can be
7
represented using Pert chart. In addition, this chapter also cover the risk management
aspect, security, formal review, software development library, corrective action process
and problem change report.
Software Engineering chapter consists of the organization and resources of the
development team, the software standard and the procedures used in software
development phases. The qualification of each team members will be detailed out to
present the capabilities of each personnel in software development. It also describes the
software development environment such as operating system, compilers and code
auditors in order to execute the software development activities. On top of that, it also
documents the software development techniques and methodologies together with the
design and coding standards. In summary, this chapter records all the major and
significance activities in software development life cycle of the project.
The next two remaining chapters are important and will be the major focus in this
subject. They are Formal Qualification Testing and Software Product Evaluation. The
Formal Qualification Testing describes the test organization team, the resources, the test
approach or philosophy planned, test assumption made and any constraints (if applicable)
in the testing phase. An external quality manager will be appointed to audit all the
compliance of the documents.
Lastly, Software Product Evaluation chapter describes the type of review will be
performed along the software project development process such as software specification
review, preliminary design review and test readiness review. Some of common criteria
which will be focused on the review sessions are the internal consistency,
understandability, traceability and consistency with other documents, appropriate use of
analysis, design or coding technique and the requirement covered by appropriate tests.
8
1.2 STEP-BY-STEP/ EXAMPLES
1. Student will form a group of 5 to 6 persons and develop the project organization
hierarchy. Then, propose the group name and the group motto.
2. Student will delegate the task among the team members and describe each role
and its responsibilities.
3. To ensure that each student experience the different role in project development,
the role will be rotated among the team members in the group.
4. The requirements for the project development will be explained accordingly.
5. In managing the project, student will use the filing system to record all the
software project development process by using softcopy and hardcopy.
6. To depict the real software development environment, key project milestones will
be given and for that each project activity needs to be monitored and controlled.
7. Team leader has to ensure all given tasks is distributed accordingly.
8. Each group should deliver the following:
a. Software Development Plan (SDP)
• The document describes the manner how software engineering
activities are carried out from planning stage until the stage of
software configuration management. Particularly, in this course
we only focus on the quality plan and testing plan.
b. Software Requirement Analysis (SRA)
• The document describes the user requirement and the required
characteristics of the ‘to be’ system.
c. Software Requirement Specification (SRS)
• The document describes the details specification of the ‘to be’
system.
d. Software Design Document (SDD)
• The document describes the detail architecture of the software. It
explains how objects, algorithms and data are used in building the
system. In this course we only focus on the design specification.
9
e. Software Test Plan (STP)
• The document tells the manner how the software testing is carried
out.
f. Software Test Description (STD)
• The document focuses on the detail description of each test case.
g. Software Test Result (STR)
• The document records the test result and any arising issues during the implementation of testing phase.
1.2.1 SOFTWARE DEVELOPMENT PLAN (SDP) FRAMEWORK
Cover
Document Approval
Table of Contents
List of Figure
List of Table
1.0 Introduction (Scope)
1.0 Identification
1.2 System Overview (Purpose of the system / CSCI)
1.3 Document Overview (Explain briefly each chapter)
2.0 Referenced Document
2.1 Government Document
2.2 Non Government Document
3.0 Software Development Management
3.1 Organizational Structure (Working Level)
3.2 Personnel
3.3 Project facilities
3.4 Schedule and Milestone
3.5 Risk Management (Not Applicable)
4.0 Software Engineering
4.1 Software / Hardware Items
10
4.2 Design Standards
4.3 Coding Standards
5.0 Formal Qualification Testing
5.1 Organizational Structure (Formal Qualification Testing)
5.2 Personnel (Formal Qualification Testing)
5.3 Test Approach/ Philosophy
6.0 Software Product Evaluation
6.1 Organizational Structure (Software Product Evaluation)
6.2 Personnel (Software Product Evaluation)
6.3 Software Product Evaluations Procedure & Tools
6.3.1 Procedures
6.3.2 Tools
6.4 Activity (Dependent Product Evaluation)
6.4.1 Software Development Plan Review (SDPR)
6.4.2 Software Specification Review (SSR)
6.4.3 Software Design Review (SDR)
6.4.4 Test Readiness Review (TRR)
7.0 Software Configuration Management (Not Applicable)
8.0 Notes (acronym & abbreviation)
9.0 Exhibit (Annex / Appendix)
11
1.2.3 SOFTWARE DEVELOPMENT MODEL In building a software product, it is essential to go through a sequence of phases
which helps in building a timely, budgetary, high quality product. This series of
processes is called software development process. There are several variations of
software development models to be followed such as Waterfall model, Spiral Model,
Incremental Model and RAD model. However, the typical phases of software
development are conceptual, requirements gathering, design and analysis,
implementation, testing and operation and maintenance. The selection of the model
mainly based on the background of the project, the organization and the people involved
in the project. Ones need to wisely select the software development model. Moreover,
the chosen software development model will affect the approach taken in testing. This is
indeed important in planning testing activities of the project.
1.2.2 CODING STANDARDS AND GUIDELINES
In formal reviews, the inspectors will look for problems and omissions in the
code. There are classic bugs where something just won’t work as expected. These can be
found by careful analysis of the code – senior programmers and testers are great at this.
There are also problems where the code may operate properly but may not
comply to the specific standard and guideline. It’s equivalent to writing words that can be
understood and get a point across but don’t meet the grammatical and syntactical rules of
the English language. Standards are the established, fixed, have-to-follow-them rules-the
do’s and don’ts. Guidelines are the suggested best practices, the recommendations and
the preferred way of doing things. Standards are rules that need to comply, have no
exceptions and no short of a structured waiver process. Whilst, guidelines can be a bit
loose, which are not mandatory, yet something valuable to practice.
12
1.2.3 EXAMPLE OF PROGRAMMING STANDARDS
TOPIC 3.05 Control-Restriction on control structures
STANDARD The go to statement (and hence labels as well) should not be used.
The while loop should be used instead of the do-while loop, except where the logic of the problem explicit requires doing the body at least once regardless of the loop condition
If a single if-else can replace a continue, an if-else should be used
JUSTIFICATION The go to statement is prohibited for the empirical reason that its use is highly correlated with errors and hare-to-read code, and for the abstract reason that algorithms should be expressed in structures that facilitate checking the program against the structure of the underlying process.
The standard has four main parts:
i. Title : Describes the topic that the standards covers.
ii. Standard (or guideline) : Describes the standard or guideline explaining
exactly what is allowed and what is not.
iii. Justification : Provides the reasoning behind the standard so that the
programmer understands why it’s good programming practice.
iv. Example: Shows the simple programming samples of how to apply the
standards.
13
1.3 EXERCISES Before you start your project assignment, form a project group consist of 3-4 persons.
Develop organisation structure including project personnel and responsibilities.
a) Below are templates for project title and motto, and basic project organisation. Fill in
the empty boxes or develop your own project organisation.
___________________
[Team Leader]
__________________
[System Analyst]
__________________
[Technical Writer]
__________________
[Programmer]
__________________
[Software Tester / Quality Auditor]
Project:_____________________________________
Motto:____________________________________
14
b) Below is a sample table on how to list down the required human resources and
their job description. Fill in the empty boxes or create your own format for
determining the human resources requirement in your project.
Role Description Quantity _________________
[Team Leader]
_________________
[System Analyst]
_________________
[Technical Writer]
_________________
[Software Tester / Quality Auditor]
_________________
[Programmer]
c) Choose an appropriate software development model for your project and justify your
answer.
15
d) Identify two programming standards and three programming guidelines for your project.
STD 001 <Topic> Standard :
Justification :
Example :
STD 002 <Topic> Standard :
Justification :
Example :
GDL 001 <Topic> Standard :
Justification :
Example :
16
GDL 002 <Topic> Guideline :
Justification :
Example :
GDL 003 <Topic> Guideline :
Justification :
Example :
17
1.3 SELF REVIEW QUESTION
1. What’s wrong with just testing that a program works as expected?
2. How much more does it cost to fix a bug found after the product is released
compared to a bug found at the very start of the project?
3. What’s the goal of a software tester?
4. List reasons why the product specification is usually the largest source of bugs in
a software product?
5. Name several tasks that should be performed before a programmer starts writing
the first line of code.
Comment [MK1]: rephrase
18
LAB 2: REQUIREMENT ANALYSIS
By the end of this section, students should be able to:
Produce the Software Requirement Analysis (SRA) which captures the
functional requirements.
Produce the Software Requirement Specification (SRS) which describes detail
requirements.
2.1 INTRODUCTION
Software is written to fulfill specific needs that a person or a group of people has.
Let’s call them the customer. To ensure that we are building the right software product,
the development team must find out what exactly the customer wants. Some teams
simply guess, but most collect detailed information in the form of surveys, feedback from
previous versions of the software, competitive product information, magazine reviews,
focus groups, and numerous other methods, some are formal, and some are not. The
gathered information is then studied and analyzed, condensed and interpreted to define
exactly what features the software product should have.
The result of the customer requirements studies is just raw data. It doesn’t
describe the detail of the proposed product; it just confirms on what to be created and
what are the features required by customer. The detail will be specified in requirement
specifications. It will utilize all the information and any unstated but mandatory
requirements and define what the product will be, what it will do and how it will look.
The format of specification varies greatly. Some companies-especially those
developing products for the government, aerospace, financial and medical industries –
use a very rigorous process with many checks and balances. The result is an extremely
detailed and through specification that’s locked down, meaning that it can’t change
19
except under very extreme conditions. Everyone on the development team knows exactly
what they are creating.
There are development teams, usually ones who create software for less-critical
applications, who produce specifications on cocktail napkins, if they create them at all.
This has distinct advantage of being very flexible, but there’s lots of risk than not
everyone is “on the same page.” And, what the product finally becomes isn’t known until
it’s released.
2.2 STEP-BY-STEP/ EXAMPLES
1. For each project, construct the Software Requirement Analysis (SRA) which
records all the requirements with its dedicated identification.
2. Conduct an internal review session to review the SRA document. The main
objective of the review session is to identify any bugs in the document. List
all bugs found during the session and update the SRA document accordingly.
3. From the 11 factors in Mc Call’s Quality model, identify the main quality factors
that you think important to be included in your project. Then, match each
functional requirement to the identified quality criteria.
4. Based on the SRA, constructs the Software Requirement Specification (SRS)
which describes in detail the description of each requirements. Specifically in
this course, the SRS should be described using use cases.
5. 3. Once the Software Development Plan (SDP) baseline has been created, the
plan can be sent for a Software Development Plan Review (SDPR). Any
findings/issues found by internal Software Tester / Quality Auditor will be
recorded in Review Report Form (RRF).
2.2.1 SOFTWARE REQUIREMENT ANALYSIS (SRA) FRAMEWORK
Cover
Document Approval
Table of Contents
List of Figure
20
List of Table
1.0 Introduction
1.1 Identification
1.2 System Overview
1.3 Document Overview
2.0 Referenced Document
2.1 Government Document
2.2 Non Government Document
3.0 Functional Requirement
4.0 Non Functional Requirement (Not Applicable)
5.0 Exhibit
Note: Diagram involved is use case diagram and activity diagram
2.2.2 EXAMPLE OF FUNCTIONAL REQUIREMENT IN SRA
FR_No Requirement Description Phase
BDC_1 Format Date Rules - Ability to define the type of dates
-
1
…
21
2.2.3 EXAMPLE OF BUGS IN SOFTWARE REQUIREMENT ANALYSIS
(SRA)
Section Number
Problematic Phrase or Section Bug and corrective actions
X.X.X “Wrong IC format should not be migrated”
• Avoid negative phrases. – To rephrase ““The system should only migrate a correct IC format. ”
• Ambiguous statement - Should states clearly the correct IC format.
X.X.X “The calculation of total price must include service fees.”
• Unclear statement – Should states the amount of the service fees and the calculation of the service fees (if applicable)
X.X.X Abbreviation of IC has been specified under “Terms Acronyms and Abbreviations” but some of the sections still use the word “Identification Card”.
• Inconsistency – Should either remove the abbreviation of IC from the list or make sure all sections use the specified abbreviation.
X.X.X “The browser should respond within acceptable time”
• Not testable – Should specified the duration of acceptable time. E.g “The browser should response within 10 seconds”
• •
22
2.2.4 QUALITY FACTORS AND REQUIREMENTS
ID Version Section Description Priority
000 Suitability 000-001 1.0 3.1 The site is suitable for IE version 5 and
higher. 1
000-001 1.1 3.6 The system shall accept payment in local currencies, US dollars and Pounds.
2
010 Security 010-001 2.0 6.0 Action should be taken within 1 hour
for all reports regarding high priority security event.
1
020 Interoperability 020-001 1.0 1.1 The system should be deployed in any
operating systems. 3
2.2.5 SOFTWARE REQUIREMENT SPECIFICATION (SRS) FRAMEWORK
Cover
Document Approval
Table of Contents
List of Figure
List of Table
1.0 Introduction
1.1 Identification
1.2 System Overview
1.3 Document Overview
1.4 Summary of use case
2.0 Referenced Document
2.1 Government Document
2.2 Non Government Document
23
3.0 Use Case Specification
4.0 Notes (acronym & abbreviations)
5.0 Exhibit / Annex / Appendix
note: diagram involved is sequence diagrams and GUI. 2.2.6 EXAMPLE OF ENGINEERING DOCUMENTS IN SRS
2.1.1 Use Case “Format Rule Date” [SRS_REQ_100]
2.1.1.1 Brief Description
The use case provides the capability to display the format of date rule
according the user needs.
2.1.1.2 Characteristic of Activation
The use case starts when the user runs the format of date rule function
from the main menu.
Pre-Condition(s)
Not applicable
Flow of Events
2.1.1.4.1 Basic Flow
1. This use case begin when the user select the menu which is :
a. Ability to define type of dates [A1]
b. ____________________________
c. ____________________________
2. The use case ends.
2.1.1.4.2 Alternative Flow
24
[A1] Types of dates [SRS_REQ_101]
1. The user key in the type of date based on the format given.
2. The system will display the type of dates [E1]
3. The use case ends.
[A2] ________________________[ ]
1. _________________________
2. _________________________
3. _________________________
[A3] _________________________[ ]
1. __________________________
2. __________________________
3. __________________________
2.1.1.4.3 Exception flow
[E1] – Invalid input
If invalid input accounted, the system should display the following
message ‘Invalid input’ on the message panel
2.1.2.5 Post-Condition(s)
The business date is message is displayed
2.1.2.6 Rule(s)
Not applicable
2.1.2.7 Constraints(s)
Not applicable
25
2.3 SELF REVIEW QUESTION
1. Why can the waterfall methodology method be difficult to use? 2. Why would a software tester like the spiral model better than others? 3. Can a software tester perform white-box testing on a specification? 4. Explain what’s wrong with this specification: When the user selects the Compact
Memory option, the program will compress the mailing list data as small as possible using a Huffman-sparse-matrix approach.
5. Explain what a tester should worry about with this line from a specification: The
software will allow up to 100 million simultaneous connections, although no more than 1 million will normally be used.
26
LAB 3: SOFTWARE DESIGN
By the end of this section, students should be able to: Produce the Software Design Document (SDD).
The students have a basic understanding regarding the configuration
identification.
3.1 INTRODUCTION One common misconception is that when a programmer creates a program, he
simply sits down and starts writing code. That may happen in some small, informal
software project, but for anything other than the smallest programs, there must be a
design process to plan out how the software will be written. Think about this book, which
required an outline before the first words were typed, or a building, which needs
blueprints drawn before the first concrete is poured. The same process should also be
implemented in building software.
The documents produced in design phase vary greatly depending on the company,
the project, and the team, but the main purpose is to plan and organize the codes that are
to be written.
27
3.2 STEP-BY-STEP/ EXAMPLES
1. Student will develop the detail design in Software Design Document (SDD).
2. Student will construct the configuration identification in order to develop the
traceability among design documents.
3. Students should already complete the Software Requirement Analysis (SRA) and
Software Requirement Specification (SRS) baseline and able to establish the
Software Specification Review (SSR) and the internal Software Tester / Quality
Auditor and will state all the arising matter in the Review Report Form (RRF).
3.2.1 SOFTWARE DESIGN DOCUMENT (SDD) FRAMEWORK
Cover
Document Approval
Table of Contents
List of Figure
List of Table
1.0 Introduction
1.1 Identification
1.2 System Overview
1.3 Document Overview
2.0 Referenced Document
2.1 Government Document
2.2 Non Government Document
3.0 Preliminary Design (Not Applicable)
3.1 CSCI Overview
3.2 CSCI Architecture
3.3 CSCI Design Description
4.0 Detail Design (algorithm)
5.0 Requirement Traceability
6.0 Notes (acronym & abbreviations)
28
7.0 Exhibit / Annex / Appendix
note : diagram involved is component diagram, package diagram, class diagrams.
3.2.2 EXAMPLE OF CONFIGURATION IDENTIFICATION
i. Example of document identification
SDD / BITP 3213 / S1 / G1 / 2004
ii. Example of requirement identification
2.1.1 Use case “Format Date Rule” [SRS_REQ_100]
…
2.1.1.4.2 Alternative flow
[A1] Types of dates [SRS_REQ_101]
iii. Example of traceability identification
SRS SDD Description
SRS_BDC_001 SDD_FDR_101 Ability to define types of dates
SRS_BDC_002 SDD_FDR_101 Ability to define types of holiday
SRS_BDC_003 SDD_FDR_101 Ability to configure an observable
holiday
29
3.3 SELF REVIEW QUESTION
1. Due to time constraints, it would be impossible to execute a thorough and complete test. Thus, what information do you think should be considered when deciding to stop the testing?
2. Is it possible to have a high-quality and low-reliability product? Provide an
example to support your answer.
3. Why is it impossible to test a program completely?
4. If you were testing a feature of your software on Monday and finding a new bug every hour, at what rate would you expect to find bugs on Tuesday?
5. If you’re testing a program’s ability to print a document to a printer, what would
be appropriate generic test-to-fail test cases?
30
LAB 4: SOFTWARE TEST PLAN
By the end of this section, students should be able to:
Develop the Software Test Plan (STP) document.
Understand the concept and the significance of Software Test Plan in software
development life cycle.
4.1 INTRODUCTION
The testing process can’t operate in isolation. Performing your testing tasks would
be very difficult if the programmers wrote their code without telling you what it does,
how it works, or when it will be complete. Likewise, if you and the other software testers
don’t communicate what you plan to test, what resources you need, and what your
schedule is, your project will have little chance of succeeding. The software test plan is
the primary means by which software testers communicate their testing schedules,
activities, approaches and measurement to the rest of software development team.
The IEEE Standard 829-1998 for Software Test Documentation states that the
purpose of the software test plan is as follows:
To prescribe the scope, approach, resources, and schedule of the testing
activities. To identify the items being tested, the features to be tested, the testing
tasks to be performed, the personnel responsible for each task, and the risks
associated with the plan.
Given that definition and the rest of the IEEE standard, you will notice that the form the
test plan takes is a written document. That shouldn’t be too surprising, but it’s an
important point because although the end result is a piece of paper (or online document or
web page), that paper isn’t what the test plan is all about.
The test plan is a by product of the detailed planning process that’s undertaken to
create it. It’s the planning process that matters, not the resulting document.
31
The Software Test Plan is used as the basis of project management control
throughout the testing process. It is a living document that guides test execution through
to conclusion, and it must be updated to reflect to any changes.
Depending on your project background and characteristic, you may have multiple
test plans. You may consider having multiple test plans when your test have:-
• Different phases and levels (e.g. unit testing, integration testing, beta
testing, etc)
• Different test types (e.g. performance testing, functional testing, regression
testing, etc)
• Different audience (e.g. hardware and software test)
However, to reduce the risk of redundancy and overlap, ones may have one master plan
which describes the common elements for the overall testing phase.
Basically, a STP should cover information specifying the approach to testing, test
organization, test environment, test strategy and test schedule.
• Test organization
Test organization specifies the staff members who will be expected to involve in the
testing phase, the role that they will perform and their responsibilities.
• Test environment
Test environment describes the setting and context in which the test will be executed.
It also covers the software and hardware to be used in order to run the testing.
• Test strategy
Test strategy specifies the approaches and techniques used in testing software
product.
• Test schedule
32
Test schedule outlines the time frame together with the individual who responsible for
each activity in the testing process. The time frame must be synchronized with the
project plan. Any changes in the schedule might affect the project plan as a whole.
• Test cycle
Test cycle records number of iteration in running the testing. Test cycle can be
counted based on happy or unhappy scenarios, modules, users or locations. It will
ensure the result can be verified.
4.2 STEP-BY-STEP/ EXAMPLES
1. Identify level and the type of testing to be executed for the project. Describes the
objective and goal for each of the testing level or testing type.
2. Based on the types and level of testing identified, each group will develop the
testing plan(s) as accordance to the framework given.
3. Students should already complete the Software Design Document (SDD) baseline
and able to establish the Software Design Review (SDR) and will state all the
arising matter in the Review Report Form (RRF).
4.2.1 SOFTWARE TEST PLAN (STP) FRAMEWORK
Cover
Document Approval
Table of Contents
List of Figure
List of Table
1.0 Introduction
1.1 Test Plan Identifier
1.2 Purpose – Describes the main objective and goal of the test plan
33
1.3 Background – Briefly describes the overall project testing background
information
1.4 System Overview – An overview of the testing and how it relates with
the critical/high risk project functions
1.5 Applicable documents – List all documents pertinent to test program
1.6 Document Overview - Explain briefly on each chapter
2.0 Scope of Testing
2.1 Test Items – Specifies the components or modules together with their
versions that need to be tested
2.2 Features to be tested and Risk issues
2.3 Features not to be tested and Risk issues
3.0 Testing Strategy and Approach
3.1 Overview – Describes the strategies, approaches or methodologies
used for the testing
3.2 Testing Criteria
3.2.1 Entry criteria – A list of criteria to measure that the system
is ready for a particular test phase.
3.2.2 Exit criteria – A list of criteria to measure that the system is
ready to end the testing phase.
3.2.3 Suspension or resumption criteria – A list of criteria to
measure that the system needs to be suspended / resumed.
3.3 Configuration Management – E.g. Version controls, release
management
3.4 Problem / Changed Management
3.4.1 Inspection and Analyze procedures
3.4.2 Problem reporting/tracking procedures – Includes the
procedures for notifications and escalations
3.4.3 Administrative procedures
3.4.4 Review/Meeting procedures
3.5 Testing tools – Describes any testing tools or supporting tools that
will be used in the testing.
34
3.6 Test Measurements and Metrics
4.0 Testing Deliverables – E.g. reports, charts
5.0 Testing Resources, Schedule and Tasks
5.1 Testing Roles and Responsibilities
5.2 Staffing and Training needs
5.3 Test Task Structure – List of all test activities, work breakdown
structure, responsible persons and key milestones
6.0 Environmental and Infrastructure Needs - Includes any configurations
or setup needed for the environment or infrastructure
7.0 Risk and Contingencies
8.0 Approval
9.0 Notes (acronym & abbreviations)
10.0 Exhibit (Annex / Appendix)
note : exhibit shall include the test cases for unit testing, integration & system
testing & user acceptance test.
35
4.3 SELF REVIEW QUESTION
1. What is the purpose of a test plan?
2. Why is it the process of creating the plan that matters, not the plan itself?
3. Names a few typical testing resources that should be considered in test planning phase?
4. What are the four test cases planning?
5. What is ad hoc testing?
36
LAB 5: SOFTWARE TEST DESCRIPTION
By the end of this section, the students should be able to:
Design the test cases and produce Software Test Description (STD).
Understand the concept and the significance of Software Test Description
(STD)
5.1 INTRODUCTION
The overall project test plan is written at very high level. It breaks out the
software into specific features and testable items and assigns them to individual testers,
but it doesn’t specify exactly how those features will be tested. There may be a general
mention of using automation or black-box or white-box testing techniques, but the test
plan doesn’t get into the details of exactly where and how they will be used. This next
level of detail that defines the testing approach for individual software features is the test
design specification.
IEEE 829 states that the test design specification “refines the test approach
[defined in the test plan] and identifies the feature to be covered by the design and its
associated tests. It also identifies the test cases and test procedures, if any, required to
accomplish the testing and specifies the feature pass/fail criteria”
The purpose of the test design specification is to organize and describe the testing
that needs to be performed on a specific feature. It doesn’t, however, give the detailed
cases or the steps to execute to perform the testing. The following topics, adapted from
the IEEE 829 standard, address this purpose and should be part of the test design specs
that you create:
Identifiers. A unique identifier that can be used to reference and locate the
test design spec. The spec should also reference the overall test plan and
contain pointers to any other plans or specs that it references.
37
Features to be tested. A description of the software feature covered by the
test design spec – for example. “the addition function of calculator,” “font size
selection and display in WordPad,” and “Video card configuration testing of
QuickTime.”
Approach. A description of the general approach that will be used to test the
features. It should expand on the approach, if any, listed in the test plan,
describe the technique to be used, and explain how the results will be verified.
For example – A testing tool that used in the testing.
Test case identification. A high level description and references to the
specific test cases that will be used to check the feature. It should list the
selected equivalence partitions and provide references to the test cases and test
procedures used to run them. For example – “Check the highest possible value
– Test Case ID# 15326”.
Pass/Fail Criteria – Describes exactly what constitutes a pass and a fail of
the tested feature. What is acceptable and what is not? This may be very
simple and clear – a pass is when all the test cases are run without finding a
bug. It can also be fuzzy – a failure is when 10 percent or more of the test
cases fail. There should be no doubt, though, what constitutes a pass or a fail
of the feature.
5.1.1 TEST CASES
IEEE 829 states that the test case specification “documents the actual values used
for input along with the anticipated outputs. A test also identifies any constrains on the
test procedure resulting from use of that specific test case”.
Essentially, the details of the test case should explain exactly what values or
conditions will sent to the software and what result is expected. It can be referenced by
one or more test design specs and may reference more than one test procedure. The IEEE
829 standard also lists some other important information that should be included:
Identifiers. A unique identifier is referenced by the test design specs and the
test procedure specs.
38
Test item. Describes the detailed feature, code module and so on that’s being
tested. It should be more specific than the features listed in the test design plan
or specification. If the test design spec said “the addition function of
Calculator,” the test case spec would say “upper limit overflow handling of
addition calculations.” It should also provide references to product
specifications or other design docs from which the test case was based.
Input specification. Specifies a list all inputs or conditions given to the
software to execute the test case. If you’re testing calculator, this may be as
simple as 1+1. If you’re testing cellular telephone switching software, there
could be hundreds or thousands of input conditions. If you’re testing a file
based product, it would be the name of the file and a description of its
contents.
Output specification. Describes the result you expect from executing the test
case. Did 1+1 equal 2? Were the thousands of output variables set correctly in
the cell software? Did all the contents of the file load as expected?
Environmental needs. Environmental needs are the hardware, software, test
tools, facilities, staff, and so on that are necessary to run the test case.
Special procedural requirements. This section describes anything unusual
that must be done to perform the test. Testing WordPad probably doesn’t need
anything special, but testing nuclear power plant software might.
5.1.2 TEST PROCEDURES
After you document the test designs and test cases, what remains are the
procedures that need to be followed to execute the test cases. IEEE 829 states that the test
procedure specification “identifies all the steps required to operate the system and
exercise the specified test cases in order to implement the associated test design”
The test procedure or test script specification defines the step-by-step details of exactly
how to perform the test cases. Here’s the information that needs to be defined:
39
Identifier. A unique identifier that ties the test procedure to the associated test
cases and test design.
Purpose. The purpose of the procedure and reference to the test cases that it will
execute.
Special requirements. Other procedures, special testing skills or special
equipment needed to run the procedure.
Procedure steps. Detailed description of how the tests are to be run:
o Log. Tells how and by what method the results and observations will be
recorded.
o Setup. Explains how to prepare for the test.
o Start. Explains the steps used to run the test.
o Procedure. Describes the steps used to run the test.
o Measure. Describes how the results are to be determined-for example.
With a stopwatch or visual determination.
o Shut down. Explain the steps for suspending the test for unexpected
reasons.
o Restart. Tells the testers how to pick up the test at a certain point if
there’s a failure or after shutting down.
o Stop. Describes the steps for an orderly halt to the test.
o Wrap up. Explains how to restore the environment to its pre-test
condition.
o Contingencies. Explains what to do if things don’t go as planned.
40
5.1.3 TEST DESIGN APPROACH AND TECHNIQUES
Software test description, test cases and test procedures are the result from the
test analysis and design phase. In the activity, different approaches and
techniques will be used to transform the general testing objective defined during
test planning phase into tangible test conditions and test cases. Below are the
summarization of the common approach and common techniques used in
designing the test cases (Dustin, et al., 2000).
White Box Approach Black Box Approach
A structural testing approach focuses on internal structure of the system. Program based.
Functional testing approach focuses on application externals. Specification based.
Type of Test Unit testing Code level Integration testing
Code level Integration Testing System level Integration Testing System Testing User acceptance testing Usability testing
Techniques Fault Insertion String Test Error Handling Statement Coverage Decision Coverage Path Coverage Condition Coverage Data Flow Coverage Memory Leak
Boundary Value Analysis Cause-Effect Graphing Error Guessing Path Analysis Equivalence Partitioning Decision Table Testing State Transition Testing Use Case Testing
Personnel Involved Developers and/or Test Engineers
Developers Test Engineers End Users
41
5.2 STEP-BY-STEP/ EXAMPLES
Prerequisite: The SRA, SRS and SDD should be completed before we proceed
with test design.
1. Within a group, students will discuss and develop the Software Test
Description (STD) based on the framework given. For each testing level/type,
identifies the approach and techniques used.
• Students will utilize the SDD in constructing the unit test cases
• Students will utilize the SDD and/or SRS in constructing the
integration testing
• Students will utilize the SRS in constructing the system testing
• Students will utilize the SRA in constructing the user acceptance
testing
2. The students should start the implementation or able to develop the source
code for the system that had been analyzed and designed. The students should
delegate each functions to develop for each team member.
5.2.1 SOFTWARE TEST DESCRIPTION (STD) FRAMEWORKS
Cover
Document Approval
Table of Contents
List of Figure
List of Table
1.0 Introduction ( Scope )
1.1 Identification
1.2 System Overview ( Purpose of the system / CSCI)
1.3 Document Overview ( Explain briefly each chapter)
2.0 Referenced Document
42
2.1 Government Document
2.2 Non Government Document
3.0 Scope of the tests
3.1 The Software package/item to be tested (name, version and
revision)
3.2 The documents providing the basis for the designed tests (name
and version for each document)
4.0 Test environment (for each test)
4.1 Test identification (the test details are documented in the STP)
4.2 Detailed description of the operating system and hardware
configuration and the required switch settings for the tests
4.3 Instructions for software loading
5.0 Testing process
5.1 Instructions for input, detailing every step of the input process
5.2 Data to be recorded during the tests
6.0 Test cases (for each case)
6.1 Test case identification details
6.2 Input data and system settings
6.3 Expected intermediate results (if applicable)
6.4 Expected results (numerical, message, activation of equipment,
etc)
7.0 Notes (acronym & abbreviations)
8.0 Exhibit (Annex / Appendix)
43
5.3 SELF REVIEW QUESTION
1. What is the purpose of a test design specification?
2. What is a test case specification?
3. Other than a traditional document, what other means of resources can you use in developing you test cases?
4. What is the purpose of a test procedure specification?
5. At what level of detail should test procedures be written?
44
LAB 6: STUB AND DRIVER
By the end of this section, the students should be able to:
Understand the concept and the significance of the stub and driver.
Create the test driver and stub in order to perform unit test.
6.1 INTRODUCTION
Unit testing usually runs in development environment which require the
access to the code. It involves the testing of each function, module, or class and also
the integration of those units. For the integration of the units ones often greatly use
either of these two strategies:-
• Big bang testing
To test all units at one go, once the completed package is available.
• Incremental Testing
To test units in modules, as they are completed (unit tests); then to test
groups of tested modules integrated with newly completed modules
(integration tests). This process continues until all the package modules
have been tested. Once this phase is completed, the entire package is
tested as a whole (system test).
They are three strategies in which to construct the hierarchy of the
integration modules:
o Top-down
Starts testing at the highest level module in the software structure
and the last modules to be tested are the lowest level modules.
45
o Bottom-up
This strategy is the reverse of Top-down. It starts at the lowest
level modules and ends with the highest modules.
o Backbone
Starts with critical modules (according to risks order) and ends
with the least critical modules.
In implementing the above categories, ones should construct stubs and drivers. Stubs
and drivers are software replacement simulators required for modules not available when
performing a unit or an integration test.
• Driver
A driver is a substitute module of the upper level module. The driver
passes the test data on to the tested module and accepts the results from
the tested module. Drivers are required in bottom up testing until the top
level modules are developed.
• Stub
A stub or “dummy module” replaces an unavailable lower level module,
subordinate to the module tested. Stubs are required for top down testing
of incomplete system. In this case, stub provides the results to the module
being tested.
6.2 STEP-BY-STEP/ EXAMPLES
1. Student will create stubs and drivers to perform the unit test based on the
example given. For the unit integration testing, student can choose either to
use the top-down or bottom-up approach.
2. At this stage, students should complete the baseline of Software Test Plan
(STP).
46
6.2.1 EXAMPLE FOR STUB AND DRIVER
The objective of this example is to combine three words into one word called
“SAYA MINAT BELAJAR”. The project manager divides the task into three classes and
each class will be assigned a programmer for coding work. The distributions of task are
arranged as follows:
1. Hafiz will perform task1, which consists of three methods – saya, NamaSaya
and NamaKeluarga
2. Kiah will perform task2, which consists of one method - Minat
3. Radzi will perform task3, which consists of the method - Belajar
Problem:
Unfortunately, Kiah could not complete the task2 as planned in the schedule. The
tester has to complete the unit testing for the whole module in order to perform
the next stage of testing.
Instruction:
Supposed you are the software tester and requested by your manager to run the
test even though the task2 is still not completed.
Problem solving:
1. Create a stub names stubtask2.java to replace the uncompleted unit of task2.
2. In order to test the data contains in the task1’s class, create a driver called
driverTask1.java
3. By using the stubtask2.java, create the main class names motto.java
47
The decomposition view for the example module is
Motto
Task1 Task2 Task3
/* * file name : task1.java * The purpose of this class is to display name * */ public class task1 { public static String saya(String mesg) { mesg = "Saya "; return mesg; } public static String namaSaya(String mesg) { mesg = "Mohd Suffian"; return mesg; } public static String namaKeluarga(String mesg) { mesg = "Sulaiman "; return mesg; } }
/* * file name : stubtask2.java * The purpose of this class is to display Minat */ public class stubTask2 { public static String minat(String mesg) { mesg = "Minat"; return mesg; } }
48
/* * file name : task3.java * the purpose of this class is to display Belajar */ public class task3 { public static String belajar(String mesg) { mesg = "Belajar "; return mesg; } }
/* * file name : task1Driver.java * The purpose of this class is to test the spesific unit * software of task1 */ class task1Driver { public static void main(String[] args) { String mesg = " "; task1 displayName = new task1(); System.out.println(displayName.saya(mesg)+ " " +displayName.namaSaya(mesg)+ displayName.namaKeluarga(mesg)); } }
49
6.3 SELF REVIEW QUESTION
1. What is the difference between a test stub and a test driver?
2. Always design your black box test cases first. True or False?
3. If your product development is in a hurry, you can skip module testing and proceed directly to integration testing. True or False?
4. Provide two reasons why testing in a big bang software development model is
nearly impossible? How can these be addressed?
5. What is the difference between white box testing and debugging?
/* * file name : motto.java * The purpose of this class is to call all related classes * to display the combined words */ class motto { public static void main (String[] args) { String mesg = " "; task1 displayTask1 = new task1(); stubTask2 displayTask2 = new stubTask2(); task3 displayTask3 = new task3(); System.out.println(displayTask1.saya(mesg)+ " "+displayTask1.namaSaya(mesg)+displayTask1.namaKeluarga(mesg)+ " " + displayTask2.minat(mesg)+ " "+displayTask3.belajar(mesg)); } }
50
LAB 7: RECORDING THE TEST RESULT
By the end of this section, the students should be able to:
Write and prepare the Software Test Result (STR).
Understand the concept and the significance of Software Test Result (STR) in
testing phase.
7.1 INTRODUCTION
If you stand back and look at the big picture of software testing, you’ll see that it
has four main phases. They are test planning, test analysis and design, test execution and
test evaluation and reporting, which will be covered by this topic.
On the surface, it may seem as though reporting the problems you discover would
be the easiest compared to the other phases. The skills to efficiently find bugs and telling
the world about the bugs would surely be simpler and less time-consuming job. However,
in reality, it might be the most important - and sometimes most difficult – task for the
software tester.
There some fundamental principles for reporting bug:
Report bugs as soon as possible. The earlier you find the bug, the more time it
remains in the schedule to get it fixed. Suppose that you find an embarrassing
misspelling in a Help file a few months before the software is released. That bug
has a high likelihood of being fixed. If you find the same bug a few days before
the release, odds are it won’t be fixed. This may seem strange – the bug is still the
same bug whether you find it today or three months from now. Ideally, it
shouldn’t matter when it is found, just what the bug is. In reality, however, the
risks of fixing the bugs go up over time and increasingly weigh on the decision-
making process.
Effectively describe the bugs. Suppose you were a programmer, and receiving
the following bug report from a tester: “Whenever I type a bunch of random
51
character in the login box, the software starts to do weird stuff.” How would you
even begin to fix this bug without knowing what the random characters were, how
big a bunch is, and what kind of weird stuff was happening?. The bugs report
should provide detailed information about the incident and it should lead to
development and process improvement. Tester must be aware that bugs reports
will be read by different type of readers for different goals (e.g Developer – who
need to fix the bugs, Managers- who need to make resource allocation,
prioritization decisions about the problem, Technical support – who need to be
aware of deferred/unfixed issues at ship time, Testers – who need to know the
current state of the system). Thus, bugs report must be clear, straight-forward,
condense and focus.
Be nonjudgmental in reporting a bug. It’s easy for testers and programmers to
form an adversarial relationship. Bug reports can be viewed by the programmers
and others on the development team as the tester’s “report card” on their work.
Therefore, the bugs report must be nonjudgmental, non-personal and non-
inflammatory. A bug report that says, “Your code for controlling the printer is
terrible, it just plain doesn’t work. I can’t believe that you even checked it in for
testing,” should be out of line. Bug reports should be written against the product,
not the person, and states only the facts. No gloating, no grandstanding, no
personalizing, no egos, no accusations. Tact and diplomacy are what matters.
Follow up on your bug reports. One thing worse than not finding an important
bug is finding a bug, reporting it, and then forgetting about it or losing track of it.
You’ve learned that testing software is hard work, so don’t let the results of your
labor, the bugs you find, become neglected. From the moment you find the bug,
it’s your responsibility to make sure that it’s reported properly and given the
attention it needs. A good tester finds and logs lots of bugs. A great tester finds
and logs lots of bugs but also continues to monitor them through the process of
getting them fixed.
52
7.2 STEP-BY-STEP/ EXAMPLES
1. Students will construct the Software Test Result (STR) based on the framework
given.
2. Once done, students may proceed to the coding phase.
3. The blank template for Test Incident report for unit testing, integration/system
testing and the user acceptance test should be completed.
7.2.1 SOFTWARE TEST RESULT (STR) FRAMEWORK
Cover
Document Approval
Table of Contents
List of Figure
List of Table
1. Introduction ( Scope )
1.1. Test incident report identifier
1.2. Summary – Describes the summary of the bugs report in one line,
especially impact on stakeholders
1.3. System Overview (Purpose of the system / CSCI)
1.4. Document Overview (Explain briefly each chapter)
2. Test Overview
2.1. The documents providing the basis for the tests (name and version for
each document)
2.2. Test site
2.3. Initiation and concluding times for each testing session
2.4. Test team members
2.5. Other participations
2.6. Hours invested in performing the tests
3. Test environment
53
3.1. Software and Hardware configurations
3.2. Preparations and training prior to testing
4. Summary tables for testing
4.1. Summary of current test
4.2. Summary tables for total number of errors
5. Incident
5.1. Events and unpredicted responses of the software during testing
5.1.1. Inputs
5.1.2. Expected results
5.1.3. Actual results
5.1.4. Anomalies
5.1.5. Date and time of the incident
5.1.6. Reproducibility
5.1.7. Reported by
5.1.8. Priority
5.1.9. Severity of the impact on the system
5.1.10. Risk, costs, opportunities and benefits of fixing/not fixing the bugs
5.1.11. Other details
5.2. Proposals
5.2.1. Changes in the test environment, including test preparations.
5.2.2. Changes or corrections in test procedures and test case files.
6. Notes (acronym & abbreviations)
7. Exhibit (Annex / Appendix)
note: exhibit shall include the test cases for unit testing, integration & system
testing & user acceptance test.
54
7.3 SELF REVIEW QUESTION
1. What is software test result?
2. What is the difference between software test result and software test description? 3. What is test incident report?
4. Basically how to classify the bug?
5. What are the basic principles can you apply in your bug reports to give them the
best chance of getting the bug fixed?
55
LAB 8: UNIT TESTING
By the end of this section, the students should be able to:
To prepare, execute and conduct the unit test.
To record the unit test in the bugs report framework.
8.1 INTRODUCTION
The objective of unit testing is to ensure that reliable program units are produced
meet the specified requirements. The process is also intended to identify errors in
program logic. Typically the software engineer or programmer who coded the unit will
design and run a series of test to verify those units meet their requirements.
The precise definition of a “unit” will depend on the approach to design of testing
process and the implementation technology employed in its development, for example:
• A unit in an application developed using a procedural programming language could
be represented by a function or procedure, or closely related group of functions or
procedures.
• A unit in an application developed using an object-oriented programming language
could be represented by a class or an instance of a class or the functionality
implemented by a method
• A unit in a component based development environment could be a predefined
reusable component.
56
Unit testing is typically a white box testing approach, based primarily on the
functional and data requirements expressed in the design documentation. Example of area
typically covered in unit testing includes:
• Correctness of calculations/manipulations performed by the unit.
• Low-level performance issues (such as performance bottlenecks observed
under repeated invocation of the unit and its functionality)
• Low level reliability issues (such as memory leaks observed under repeated
and extended invocation of the unit and its functionality)
• Screen/window and navigation (for graphical user interface (GUI) units)
including consistency of the “look and feel” of the window, use of “hot keys”,
and function keys and keyboard shortcuts
• File/record creation, update, and deletion.
• Report contents, layout and calculations
• Communication between interoperating units.
Besides than executing the tests, it is also essential to have a walkthrough session to
review the developed code. The source code can be compared against the low level
design description described in the SDD. In addition, we may also compare the source
code with the programming standard/guideline document. Below are few examples of
bugs that can be found during code inspection/review:
• Data reference errors – e.g. uninitialized variables
• Data declaration errors – e.g similar variable names
• Computation error – e.g possibly of null pointer exception
• Control flow error – e.g. possible premature loop exit
• Source code does not comply with standard programming guidelines
57
8.2 STEP-BY-STEP/ EXAMPLES
1. Select a module from your project and perform a code review session to identify
bugs in the code. List all bugs found and rectify the bugs accordingly.
2. Student will analyze the Software Design Document (SDD) and produce the test
cases for unit testing. Students may apply several different techniques learnt
during the lecture sessions. Accordingly, the test cases will then be updated in
Software Test Description (STD).
3. Student will execute the unit testing and report any bugs in the bugs report
document.
4. Students will fix any bugs founds and may proceed with the next coding phases.
8.2.1 EXAMPLE OF UNIT TESTING TECHNIQUES
The specification for a software system for validating the salary payment
according to salary scale as stated in a piece of code below:
Switch (salaryScale) {
Case 45 : basicSalary = 2000;
break;
Case 49 : basicSalary = 2500;
break;
Case 52 : basicSalary = 3000;
break;
Default : system.out.println (“Wrong salary
scale”);
System.exit(0);
}
58
1. Boundary Analysis technique
What is boundary analysis? A boundary value analysis is when you look at a condition to
understand its limitations. It's important to recognize that boundaries have lower limits
as well as upper limits. I point out lower boundaries because I often see testers focusing
on maximum capacity but forgetting about lower boundaries. A simple boundary analysis
would be the review of a single data entry field to determine its limits. In such an
example, a lower limit might be that the data entry field is a required field so tests might
include attempting to leave the field blank or using the space bar to create an entry. The
upper limit tests might include attempting to enter more characters than the field accepts
and attempting to exceed the field value in through any method available.
Figure 8.1: Boundary analysis for salary scale
Test Id Test cases for
salary scale
Boundary Analysis Expected Result
1 44 “wrong salary scale”
2 45 2000
3 46
45
“wrong salary scale”
4 48 49 “wrong salary scale”
5 49 2500
6 50
“wrong salary scale”
7 51 “wrong salary scale”
8 52 3000
9 53
52
“wrong salary scale”
Figure 8.2: Test cases/scripts for salary scale
45 49 52
salary scale 44 46 48 50 51 53
59
2. Path Coverage technique
Path coverage is where the different paths in a software module are created by the choice
in conditional statements such as IF-THEN-ELSE, SWITCH and DO WHILE. The flow
chart is created and the test cases are created based on the conditional statement. Based
on flow chart created in Figure 8.3, there are four paths discovered including:
Path 1 : (1,2,9); Path 2 : (3,4,9); Path 3 : (5,6,9);Path 4 : (7,8,9). Figure 8.4 depicts the
possible test cases to be used for testing the salary scale.
Figure 8.3: Path coverage analysis for salary scale
Test ID Test Case Action Expected Result
1 Path 1 Input salary scale = 45 Basic salary = 2000
2 Path 2 Input salary scale = 49 Basic salary = 2500
3 Path 3 Input salary scale = 52 Basic salary = 3000
4 Path 4 Input salary scale = ▲ “wrong salary scale”
Figure 8.4: Test cases for salary scale
60
3. Equivalence Partition technique What is equivalence partition? Equivalence partitioning is when you examine multiple values to determine where tests overlap or additional tests would not likely provide more value. Well, equivalence partitioning helps you group tests in order to select a manageable number of tests (TechTarget, 2010).
Test ID Salary scale Partition Tested Expected output 1 43 Salary scale < 44 “wrong salary scale” 2 47 46 < salary scale < 49 “wrong salary scale” 3 50 49 <salary scale < 52 “wrong salary scale” 4 53 Salary scale > 52 “wrong salary scale”
8.3 SELF REVIEW QUESTION
1. What is unit testing? 2. What is the biggest problem of white box testing? 3. White box testing can find missing items as well as problems. True or False? 4. If a programmer was told that he could name his variables with only eight
characters and the first character had to be capitalized, would that be a standard or a guideline?
5. What are entrance and exit criteria?
45 49 52
salary scale< 44 46 < salary scale < 49 49 < salary scale < 52 salary scale >52
61
LAB 9: INTEGRATION / SYSTEM TEST
By the end of this section, the students should be able to:
Prepare, execute and conduct the integration / system test.
Record the integration / system test in the test cases framework.
9.1 INTRODUCTION
The purpose of integration/system testing is to demonstrate that the module
interact together in a correct, stable and coherent manner prior to system testing. Testing
should be performed against the functional requirements by using black box testing
techniques. Test case design should demonstrate correct interfacing and interaction
between modules, but should avoid any duplication of unit testing effort.
The precise definition of a module will depend on the approach to the design and
the technology employed, for example:
• A module in an application developed using a procedural programming language
could be represented by a closely related group functions or procedures that perform a
well-defined service and communicate with other component module via strictly
defined interfaces.
• A module with other application developed using an object oriented programming
language could be represented by a collection of objects that perform a well-defined
service and communicate with other component module via strictly defined
interfaces.
62
Integration/system testing is typically a black box testing activity, based primarily
on the functional and data requirements expressed in the specification documentation.
Example of area typically covered in integration/system testing includes:
• Invocation of one module from another interoperating module.
• Correct transmission of data between interoperating modules.
• Nonfunctional issues such as the reliability of interfaces between modules.
9.2 STEP-BY-STEP/ EXAMPLES
1. Student will analyze the Software Requirement Specification (SRS) and produce
the test cases for integration/system testing. Students may apply several different
techniques learnt during the lecture sessions. Accordingly, the test cases will then
be updated in Software Test Description (STD).
2. Student will execute the integration/system testing and report any bugs in the bugs
report document.
3. The students will proceed to coding phase with addition of one functions is
completed.
9.2.1 EXAMPLE TEST CASES FORM OF INTEGRATION/SYSTEM TEST
System : Date Component Cycle : 1(happy scenario)
Module/Unit : Format Date Rule [SRS_REQ_100] Date : 10.11.2005 Tester : Abd. Rahim Bin Ahmad
Test No Action Expected Result Actual Result (OK/NOK)
SRS_REQ_101
To test the types of date
Test_BDC_101
Key in the type of date according the format given.
The format of date is accepted and proceed to
63
-eg. dd/mm/yy dd-mm-yy dd.mm.yyyy
the next process
Test_BDC_102
Display the type of date - eg. - Birthday of Yang dipertuan Agong, - Hari Raya Puasa, - Tahun Baru Cina - Deepavali
The system is able to display the selected type of date.
SRS_REQ_102
… … …
…
…
… …
9.3 SELF REVIEW QUESTION 1. What is integration testing? 2. What is the hidden strategy behind the integration testing? 3. Name several advantages of executing the white box testing? 4. What is code coverage? 5. All software has a user interface and therefore must be tested for usability. True
or False?
64
LAB 10: USER ACCEPTANCE TEST
By the end of this section, the students should be able to:
Prepare, execute and conduct the user acceptance test.
Record the user acceptance test in the bugs report framework.
10.1 INTRODUCTION
Software is written to be used. That sound pretty obvious, but it’s sometimes be
forgotten with the rush to design, developing and testing a complex product. More time
and effort are utilized on the technology aspects of writing the code that the development
team ignores the most important aspect of software – that someone will eventually use
the software product. It really doesn’t matter whether the software is embedded in a
microwave oven, a telephone switching station, or an Internet stock trading website.
Eventually the bits and bytes bubble up to where a live person will interact with it.
Usability is how appropriate, functional and effective that interaction is.
The purpose of user acceptance testing is to confirm that the testing process meets
its business requirements as an indicator that the system works correctly and is useable
before it is formally “delivered” to the end user. User Acceptance Test is conducted by
one or more user representative with the assistance of the Test Team.
65
Typically the user acceptance testing involves the test analyst selecting a
representative subset of the system and asking the user representative to execute them.
The selection of these Test Cases normally based upon:
• Discussion with the user representative
• Review of the requirements to identify any particularly important areas or
functions that should be verified.
User Acceptance Test should employ the Black Box approach and should make
use of thread testing techniques to verify the high level business requirements of the
system. In practice, the user representative will involve in testing process by performing
typical tasks that they would perform during their normal usage of the system.
10.2 STEP-BY-STEP/ EXAMPLES
1. Student will analyze the Software Requirement Analysis (SRA) and produce the
test cases for user acceptance testing. Students may apply several different
techniques learnt during the lecture sessions. Accordingly, the test cases will then
be updated in Software Test Description (STD).
2. Student will execute the user acceptance testing and report any bugs in the bugs
report document.
3. The students will proceed to coding phase with addition of one more functions is
completed.
66
10.2.1 EXAMPLE TEST CASES FORM OF USER ACCEPTANCE TEST
System : Date Component Cycle : 1(happy scenario)
Module/Unit : Format Date Rule [BDC_1] Date : 10.4.2006 Tester : Syamin Bin Hamzi
Test No Action Expected Result Actual Result (OK/NOK)
To test ability to define the types of dates
Test_BDC_101
Key in the type of date according the format given. -eg. dd/mm/yy dd-mm-yy dd.mm.yyyy
The date is accepted and proceed to the next process
Test_BDC_102
Display the type of date - eg. - Birthday of Yang dipertuan Agong, - Hari Raya Puasa, - Tahun Baru Cina - Deepavali
The system is able to display the selected type of date.
…
…
…
…
10.3 SELF REVIEW QUESTION
1. Why does knowing how the software works influence how and what you should test?
2. What are a few advantages of having a beta test program for your software? 3. What are a few cautions to be considered with a beta test program? 4. You can perform the black box testing without a product specification or
requirements document? True or False? 5. Describe the pesticide paradox problem and how does the problem be resolved by
bringing in new people to look at the software?
67
LAB 11: QUALITY PLAN
By the end of this section, the students should be able to:
Understand the concept and the significance of software quality checklist.
Understand and able to prepare for review session
11.1 INTRODUCTION
The purpose of quality plan is to ensure the needed preparation have been
established before the actual implementation of the review session is conducted. An
invitation letter will be sent to all parties involved in the review session. The letter will
specifically state the date, time and venue of the session. Beforehand, we have to identify
the responsible individual and assign the role (e.g the moderator, recorder, reader, author
and inspector to handle the review meeting) accordingly.
Basically there are two types of review session. They are technical and
quality/formal review. The technical review focuses on the functionality of requirement,
architecture and design while the quality review focuses on the software project
documents and how they were written.
The formal review (also called inspection) has a six step process as stated below:
Planning
Identifies work product to be inspected and sets the inspection schedule.
Overview
Optional phase where team members who are not familiar with the work
product will receive an orientation session.
68
Individual Preparation
Team member inspect the work individually looking for detects in the
work product.
Review Meeting
Inspection team members met to discuss possible defects in the work
product.
Rework
The work product is revised to conform to requirements and specifications
Follow-up
The rework is verified, final inspection data is collected and summarized
and the inspection is officially closed.
11.2 STEP-BY-STEP/ EXAMPLES
1. Discuss among the group members to a quality plan.
a. Prepare and produce the invitation letter for conducting a review sessions.
b. Identify the responsible person to involve in the review session.
c. Produce a standard form to record the discussion matter in the review
session.
2. Students should already complete the Software Test Description (STD) and
Software Test Result (STR) baseline and are able to establish the Test Readiness
Review (TRR) and the internal Software Tester / Quality Auditor will state all the
arising matters in the Review Report Form (RRF).
69
11.2.1 EXAMPLE OF CHECKLIST FOR SOFTWARE REQUIREMENT INSPECTIONS
1. Clarity Are the requirements clear and unambiguous? 2. Standards Have all the requirements standards been followed? 3. Completeness Are all the requirements complete? 4. Level Of Detail Are the requirements free from design? 5. Consistency Are requirements consistent?
Are data structures and functions named and used consistently?
6. Functionality Are functions correctly specified? Are input and outputs clearly specified? Are functions logically independent and highly cohesive?
7. Performance Are the performance requirements for timing, memory and resource utilization clearly defined?
8. Testability Are requirements testable and verifiable? 9. Feasibility Can each item be implemented with the available technique,
tools, resources, and personnel and with it the specified cost and schedule constraints?
10. Tractability Are the requirements uniquely identified that they will be able to be traced back from code?
11. Modifiability Are the requirements uniquely structured such that any necessary changes to the requirements can be made easily, completely and consistenly?
70
11.2.2 EXAMPLE OF INVITATION LETTER FOR REVIEW SESSION
Name of writer and address Name of the recipient and address Date salutation, Title of letter (invitation for review) State the purpose of invitation 2. State the venue, date and time of review will be held. 3. State the closing remarks briefly. Yours sincerely, _________________ ( Name of writer ) position and company
71
11.2.3 EXAMPLE OF REVIEW FORM
QUALITY/TECHNICAL/TESTING REVIEW REPORT FORM
Section 1 - Project/Deliverable Details
System Name: Business Date Component Ref. No: (If any)
SDD/BITP3213/S2/G1/2004
Deliverable Name: Software Design Document (SDD) Version: 0.1
Author Id/Name: (Group Name)
Pyan Dot Com Sdn Bhd
Section 2 - Reviewer/Review Details
Reviewer Id/Name: Reviewer’s Role:
Imatera Software Corporation
Quality √ Technical √ Testing
Review Number Review Date Review Outcome
17 Okt 2005 Issues Found (Refer to Review Issues Form)
No Issues Found or Outstanding
1 SDD - format, template - description
6
2
3
4
Section 3 - Sign Off Details (Only when all issues have been resolved)
Signature: Date: 22 Okt 2005
Position: Project Manager
72
PROJECT REVIEW ISSUES FORM
System Name: Business Date Component (BDC)
Deliverable Name: Software Design Document (SDD)
Vers Nbr: 0.1
Date/Nbr of Review: 22 Okt 2005 Page Nbr:
Reviewer Id/Name: Imatera Software Corporation
Review Type: Quality & Technical (Qual, Tech, User, Test or CW)
If there are no issues found during the review, please note accordingly.
Nbr Page/Ref Nbr
DESCRIPTION OF PROBLEMS Outcome A - Accepted R - Rejected
Check Sign.
1, 2, 3 Page numbers should be in roman
4 1.3 whole statement out of content.
6 Figure 1.0 Use case are noun
8 Format Date Rules – Font size different from other paragraph
10 – 21 4.1.1 Algorithm should be simple english
2 Numbering for table of contents is not accordingly
73
11.3 SELF REVIEW QUESTION
1. Why are there testing costs associated with the costs of conformance? 2. The test team is responsible for quality. True or False? 3. Why would being called a QA Engineer be a difficult title to live up to? 4. Why is it good for a test or quality assurance group to report independently to
senior management? 5. Name three ways that you become involved in testing computer software before it
is released to the public.
74
LAB 12: REVIEW MEETING
By the end of this section, the students should be able to:
Conduct the document review meeting.
Record all the issues rose in minutes meeting.
12.1 INTRODUCTION
The review session is very significance to software developer in order to ensure
that they follow the standard stated in product evaluation plan SDP. The purpose of
review is to detect the problems in software work products as close to their insertion
point as possible.
As defined by IEEE (1990), a review process is:
“A process or meeting during which a work product, or set of work products, is
presented to project personnel, managers, users, customers or other interested parties for
comment or approval”
Objective of reviews can be divided into two categories, a direct objective and an
indirect objective. Direct objective is to detect the analysis and design errors as well as
subjects where corrections, changes and completions are required with respect to the
original specifications and approved changes, to identify new risks and to approve the
analysis and design product. Approval allows the team to continue to the next
development phase.
Indirect objective is to provide an informal meeting place for exchange of
professional knowledge about development methods, tools and techniques.
75
The traceability document is to classify the correspondent between the test and the
system requirements. We will able to keep track the similarity and scope of requirement
as guidance to produce the test case.
12.2 STEP-BY-STEP/ EXAMPLES
1. Students will form in group to conduct the review meeting session together with
external Software Tester / Quality Auditor.
2. The external Software Tester / Quality Auditor will audit all the documents
provided which is from the Software Development Plan (SDP), Software
Requirement Analysis (SRA), Software Requirement Specification (SRS),
Software Design Document (SDD), Software Test Plan (STP), Software Test
Description (STD) and the Software Test Result (STR) and the completed system.
3. The software tester / quality auditor will list all the issues in the Review Report
Form (RRF).
4. Students will fix the problems and update the system to ensure that all
requirements are fulfilled before final release of the product.
76
12.2.1 EXAMPLE MINUTE OF MEETING FOR DOCUMENTATION
REVIEWED
KOLEJ UNIVERSITI TEKNIKAL KEBANGSAAN MALAYSIA
FACULTY OF INFORMATION AND COMMUNICATION TECHNOLOGY
MINUTES OF MEETING FOR DOCUMENTATION REVIEW
NO. 1 2005
Date : _____________________
Time : _____________________
Venue : _____________________
Member’s name : Position
1. ____________________ ____________
2. ____________________ ____________
3. ____________________ ____________
Agenda Action
1. Intro from chairman
…
Name of person who
will take action
2. Confirmation Minutes of Meeting
…
3. Matter arising
…
4. Action plan made to improve the documentation
…
5. Other matter arising
…
Delayed of Meeting
Prepared by: Approved by:
…………… …………….
Name and position Lecturer name
77
12.3 SELF REVIEW QUESTION
1. What are key elements that make the formal reviews work? 2. Beside being more formal, what is the big difference between inspection and other
types of reviews? 3. Should you adopt the code review checklist from this chapter as your team’s
standard to verify the code? 4. Why is defining the software’s quality and reliability goals an important part of
test planning? 5. Why quality is free?
78
REREFENCE
Dr. Daniel Gallin, 2004, Software Quality Assurance, From Theory to Implementation.
England : Addison Wesley
Edward Kit. 1995. Software Testing in the Real World. New York : Addison Wesley.
Elfriede Dustin, 2003, Effective Software Testing, 50 Specific Ways To Improve Your
Testing. United States : Pearson Education, Inc.
Elfriede Dustin, Jeff Rashaka, and John Paul. 2000. Automated Software Testing: Introduction, Management and Performance. Maryland : Addison-Wesley.
Ernest Wallmuller, 1994, Software Quality Assurance. Great Britain : Prentice Hall
Glenford J. Myers, 2004, The Art Of Software Testing. Canada : John Willey & Sons Inc.
Hung Q. Nguyen, 2003, Testing Applications on the Web. Indianapolis : Wiley
Publishing, Inc.
Ron Patton, 2006, Software Testing, Second Edition. United States : Sams Publishing
Thomas Muller et. al., (2007) ISTQB Certified Tester: Foundation Course in Software Testing. International Software Testing Qualification Board.
79
BIOGRAPHY
Dr. Mohd Khanapi Abd Ghani
• PhD (Biomedical Computing) Coventry University U.K.
• MSc (Computer Science – Real-Time Software Engineering) Universiti Teknologi Malaysia
• BSc (Computer Science)
Universiti Teknologi Malaysia • Diploma (Computer Science)
Mara Institute of Technology
Dr. Mohd Khanapi Abd Ghani has been in software industry for 16 years and has a
variety of experiences in software projects. He begins its career as a programmer until
appointed as a Project Manager in several companies such as Heitech Padu Berhad,
Permodalan Nasional Berhad, Bank Simpanan Nasional, Dewan Bahasa & Pustaka which
had experienced in developing the software project with Government – Ministry of
Health, PNB and Subsidiaries, BERNAS, Lembaga Pertubuhan Peladang, Universiti
Malaysia Sabah, HCL info System (India) and Accounting Firm – Razalee & Co. The last
attachment with software industry was the Medical Online Sdn. Bhd. as a Software
Development Manager (development of MSC Telemedicine Flagship Application
project) before change the profession into academic line as a Senior Lecturer at UTeM.
Mohd Suffian Sulaiman
• MSc (Computer Science – Real-Time Software Engineering) Universiti Teknologi Malaysia – Thompson Universite
• BSc (Computer Science)
Universiti Teknologi Malaysia
80
Juliana Jaafar
• MSc in Advanced Computer Science and IT Management
University of Manchester, UK
• BSc in Computer Scince, Minor in Mathematics
University of North Carolina at Charlotte, USA
Juliana Jaafar has been in web based software development industry for almost 8 years
and has moved up the career ladder as a Senior Software Engineer. She has profound
background in Object Oriented Analysis/Design and Java development and its
technologies, with experience in full life cycle of software development, and skilled in
problem statements to well-documented designs.
She started her career as an academician in UTEM, Melaka in 2010 and currently pursues
her career as a Lecturer in UiTM, Shah Alam.