Software Testing Life CycleSoftware Testing Life Cycle
Designed and Compiled by:
Udayakumar SreeSenior Test Engineer
Designed and Compiled by:
Udayakumar SreeSenior Test Engineer
STLC - DefinitionThe course of software being tested in a well-planned way is
known as Software test life cycle.
2
ContractSigning
Requirement Analysis
Test Planning
Test Development
TestExecution
DefectReporting
RetestDefects
ProductDelivery
STLC – Stages InvolvedContract Signing:
Process - The project is signed with client for testing the software
Documents involved: SRS Test Deliverables Test Metrics etc.
3
STLC – Stages Involved
Requirement Analysis: Process: Analyzing software for design and implementation
methods and testable aspects are recorded
Documents involved:
Requirement Specification documents
Functional Specification documents
Design Specification documents (use cases, etc)
Use case Documents
Test Trace-ability Matrix for identifying Test Coverage
4
STLC – Stages InvolvedTest Planning:
Process: To plan, how the testing process should flow Test Process Flow
Test Scope, Test Environment Different Test phase and Test Methodologies Manual and Automation Testing Defect Mgmt, Configuration Mgmt, Risk Mgmt. Etc Evaluation & identification – Test, Defect tracking
tools Documents Involved:
Master Test Plan, Test Scenario, SCM
5
STLC – Stages InvolvedTest Development
Process: Test Traceability Matrix and Test coverage Test Scenarios Identification & Test Case preparation Test data and Test scripts preparation Test case reviews and Approval Base lining under Configuration Management
Documents Involved: Test Plan, RTM Test cases
6
STLC – Stages InvolvedTest Execution:
Process: Executing Test cases Testing Test Scripts Capture, review and analyze Test Results Raising the defects and tracking for its closure
Documents Involved: Test Cases Test Execution report Bug report Requirement traceability matrix
7
STLC – Stages InvolvedDefect Reporting
Process: Defect logging Assigning defect and fixing Retesting Defect closing
Documents involved: Test report Bug Report
8
STLC – Stages InvolvedProduct Delivery
Process: After the product had undergone several tests, the
acceptance test is done by the user/client i.e. UAT, wherein the use cases were executed and the product is accepted to go on live
Test Metrics and process Improvements made Build release Receiving acceptance
Documents involved Test summary reports UAT Test Plan, UAT Test cases
9
Test Plan – What?
Derived from Test Approach, Requirements, Project Plan, Functional Spec., and Design Spec
Details out project-specific Test ApproachLists general (high level) Test Case areasInclude testing Risk AssessmentInclude preliminary Test Schedule Lists Resource requirements
11
Test Plan – Why?
Identify Risks and Assumptions up front to
reduce surprises later.
Communicate objectives to all team
members.
Foundation for Test Spec, Test Cases, and
ultimately the Bugs we find.
Failing to plan = planning to fail.
12
Test Plan – Definition
The test strategy identifies multiple test
levels, which are going to be performed for
the project. Activities at each level must be
planned well in advance and it has to be
formally documented. Based on the individual
plans, the individual test levels are carried
out.
13
Test Plan – Consists of…
Unit Testing Tools Required tool to test at unit level
Priority of Program units Module-wise priority
Naming convention for test cases
Status reporting mechanism
Regression test approach
14
Test Plan – Consists of…ETVX Criteria
Entry means the entry point to that phase. For example, for unit testing, the coding must be complete
and then only one can start unit testing Task is the activity that is performed Validation is the way in which the progress and correctness
and compliance are verified for that phase Exit tells the completion criteria of that phase, after the
validation is done. For example, the exit criterion for unit testing is all unit test
cases must pass
15
Risk Analysis
A risk is a potential for loss or damage to an Organization from materialized threats. Risk Analysis attempts to identify all the risks and then quantify the severity of the risks
Risk Identification Software Risks Business Risks Testing Risks Premature Release Risk Risk Methods
16
Risk Analysis continued…
Software Risks Knowledge of the most common risks
associated with Software development, and the platform you are working on
Business Risks Most common risks associated with
the business using the Software
Testing Risks Knowledge of the most common risks
associated with Software Testing for the platform you are working on, tools being used, and test methods being applied 17
Risk Analysis continued…
Premature Release Risk Ability to determine the risk
associated with releasing unsatisfactory or untested Software Products
Risk Methods Strategies and approaches for
identifying risks or problems associated with implementing and operating information technology, products and process; assessing their likelihood, and initiating strategies to test those risks
18
Test Execution
Software Testing FundamentalsTesting objectives include
Testing is a process of executing a program with the intent of finding an error
A good test case is one that has a high probability of finding an as yet undiscovered error
A successful test is one that uncovers an as yet undiscovered error
20
Testing should systematically uncover different classes of
errors in a minimum amount of time and with a minimum
amount of effort.
A secondary benefit of testing is that it demonstrates that
the software appears to be working as stated in the
specifications
When Testing should start? Testing early in the life cycle reduces the
errors. Test deliverables are associated with every phase of development. The goal of Software Tester is to find bugs, find them as early as possible, and make them sure they are fixed
The number one cause of Software bugs is the Specification
The next largest source of bugs is the Design
21
When to Stop Testing?
Some reasons to stop test are: Deadlines (release deadlines, testing deadlines.) Test cases completed with certain percentages
passed Test budget depleted Coverage of code/functionality/requirements
reaches a specified point The rate at which Bugs can be found is too
small Beta or Alpha Testing period ends The risk in the project is under acceptable limit
22
This can be difficult to determine. Many modern software applications are so complex,
and run in such as interdependent environment, that complete testing can never be done.
Test ExecutionTesting of an application includes:
Unit Testing Integration testing System Testing Acceptance testing
23
These are the functional testing strategies and few other
functional, non-functional, performance and other testing methods
can also be applied on the software.
Test Execution – Unit testingThe unit test plan is the overall plan to carry out the unit test
activities. The lead tester prepares it and it will be distributed to
the individual testers
24
Basic input/output of the units along with their basic functionality will be tested
input units will be tested for the format, alignment, accuracy and the totals
The UTP will clearly give the rules of what data types are present in the system, their format and their boundary conditions
Testing the screens, files, database etc., are to be given in proper sequence
Test Execution – Integration testing
The integration test plan is the overall plan for carrying out the activities in the integration test level
25
This section clearly specifies the kinds of interfaces fall under
the scope of testing internal, external interfaces, with request
and response is to be explained
Two approaches practiced are Top-Down and Bottom-Up
integrations
Given this correctly, the testing activities will lead to the
product, slowly building the product, unit by unit and then
integrating them
Test Execution – System testingThe system test plan is the overall plan carrying
out the system test level activities
26
System testing is based on the
requirements
All requirements are to be verified in the
scope of system testing
The requirements can be grouped in
terms of the functionality
Based on this, there may be priorities
also among the functional groups
Apart from this what special testing is
performed are also stated here
Test Execution – Non-functional testingNon-functional testing includes: Installation testing – Installation environment, practical
obstacles etc.
Compatibility testing – compatibility with other system software
Configuration testing - how well the product works with a broad
range of hardware/peripheral equipment configurations as well as
on different operating systems and software
Security testing - secure from mistaken/accidental users,
hackers, and other malevolent attackers
Recovery testing - how well a system recovers from crashes,
hardware failures, or other catastrophic problems
Usability testing - Testing for 'user-friendliness'
27
Test Execution – Performance testingPerformance testing includes:
Load Testing – Testing with the intent of determining how
well the product handles competition for system resources.
The competition may come in the form of network traffic, CPU
utilization or memory allocation
28
Test Execution – Performance testing Stress testing - Testing with the intent of determining how
well a product performs when a load is placed on the system
resources that nears and then exceeds capacity
29
Bug/Defect Management
BUG LIFE CYCLE - What is a bug?A software bug is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended.
31
BUG LIFE CYCLE
In software testing, the
term life cycle refers to
the various stages that a
defect/bug assumes over
its life.
32
What is a Bug Life Cycle?
BUG LIFE CYCLE
The different stages involved in a bug life cycle are as follows:
Finding Bugs
Reporting/ Documentation
Fixing
Retesting
Closing
33
Stages involved in Bug Life Cycle
BUG LIFE CYCLE
34
1. Finding Bugs:
Software Tester finds bug while testing.
It is then logged and assigned to a programmer to be fixed.
2. Reporting/ Documentation:
In software, bugs need to be tracked and managed to
Communicate bug for reproducibility, resolution, and
regression.
Track bug status (open, resolved, closed).
Ensure bug is not forgotten, lost or ignored
Stages explained
BUG LIFE CYCLE3. Fixing:
Once the bug is assigned to the developer, he fixes the
bug.
Once the programmer fixes the code , he assigns it back to
the tester and the bugs enters the resolved state.
4. Retesting:
The tester then performs a regression test to confirm that
the bug is indeed fixed.
5. Closing:
If the bug is fixed, then the tester closes the bug.
Here the bug then enters its final state, the closed state. 35
Stages explained Continued…
Different status of a Bug
36
Description of Various Status of a bugNew: When the bug is posted for the first time, its state will be
“NEW”. This means that the bug is not yet approved.
Open: After a tester has posted a bug, the lead of the tester
approves that the bug is genuine and he changes the state as
“OPEN”.
Assign: Once the lead changes the state as “OPEN”, he assigns
the bug to corresponding developer or developer team. The state of
the bug now is changed to “ASSIGN”.
Test: Once the developer fixes the bug, he assigns the bug to the
testing team for retesting. Before he releases the software with bug
fixed, he changes the state of bug to “TEST”. It specifies that the
bug has been fixed and is released to testing team.37
Description of Various Status of a bug
Deferred: The bug, changed to deferred state means the bug is
expected to be fixed in next releases.
Rejected: If the developer feels that the bug is not genuine, he
rejects the bug. Then the state of the bug is changed to
“REJECTED”.
Duplicate: If the bug is repeated twice or the two bugs mention
the same concept of the bug, then one bug status is changed to
“DUPLICATE”.
38
Description of Various Status of a bugVerified: Once the bug is fixed and the status is changed to
“TEST”, the tester tests the bug. If the bug is not present in the
software, he approves that the bug is fixed and changes the status
to “VERIFIED”.
Reopened: If the bug still exists even after the bug is fixed by the
developer, the tester changes the status to “REOPENED”. The bug
traverses the life cycle once again.
Closed: Once the bug is fixed, it is tested by the tester. If the
tester feels that the bug no longer exists in the software, he changes
the status of the bug to “CLOSED”. This state means that the bug is
fixed, tested and approved. 39
Severity of a Bug It indicates the impact each defect has
on the testing efforts or users and
administrators of the application
under test.
This information is used by developers
and management as the basis for
assigning priority of work on defects.
40
Priority Levels of a Bug Critical :
An item that prevents further testing of the product or function
under test can be classified as Critical Bug. No workaround is
possible for such bugs.
Examples of this include a missing menu option or security
permission required to access a function under test.
Major / High: A defect that does not function as expected/designed or cause
other functionality to fail to meet requirements can be classified
as Major Bug. The workaround can be provided for such bugs.
Examples of this include inaccurate calculations; the wrong field
being updated, etc 41
Priority Levels of a BugAverage / Medium:
The defects which do not conform to standards and
conventions can be classified as Medium Bugs. Easy
workarounds exists to achieve functionality objectives.
Examples include matching visual and text links which lead to
different end points.
Minor / Low: Cosmetic defects which does not affect the functionality of the
system can be classified as Minor Bugs.
42
Various Bug tracking toolsThe various bug tracking tools available are:
Quality Center® – from HP
Bugzilla® - from Mozilla
Dev Track® – from TechExcel
43
Product Delivery
Product Delivery -Test DeliverablesTest Trace-ability Matrix Test Plan Testing Strategy Test Cases (for functional testing) Test Scenarios (for non-functional
testing) Test Scripts Test Data Test Results Test Summary Report Release Notes Tested Build
45
Product Delivery - Test MetricsMeasuring the correctness of the testing process
with measurable is known to be test metrics.
46
Product Delivery - Test MetricsThere are several test metrics identified as
part of the overall testing activity in order to track
and measure the entire testing process.
These test metrics are collected at each phase
of the testing life cycle/SDLC, analyzed and
appropriate process improvements are determined
and implemented. The metrics should be constantly
collected and evaluated as a parallel activity together
with testing, both for manual and automated testing
irrespective of the type of application
47
Product Delivery - Test Metrics - Classification1. Project Related Metrics – such as
Test Size,
# of Test Cases tested per day –Automated (NTTA)
# of Test Cases tested per day –Manual (NTTM)
# of Test Cases created per day – Manual (TCED)
Total number of review defects (RD)
Total number of testing defects (TD) etc.
48
Product Delivery – Test Metrics – Classification
2. Process Related Metrics – such as
Schedule Adherence (SA)
Effort Variance (EV)
Schedule Slippage (SS)
Test Cases and Scripts Rework Effort, etc.
3. Customer related Metrics – such as
Percentage of defects leaked per release (PDLPR)
Percentage of automation per release (PAPR)
Application Stability Index (ASI) etc. 49
Product Delivery – Acceptance testing – UAT
The client at their place performs the acceptance testing. It will
be very similar to the system test performed by the Software
Development Unit
50
There is no specific clue on the way they will carry out the
testing, since the client performs this test
It will not differ much from the system testing
This is just one level of testing done by the client for the overall
product and it includes test cases including the unit and
integration test level details
51
Top Related