Program Slicing for Refactoring Advanced SW Tools Seminar Jan 2005Yossi Peery.
SW Testing MAM Seminar
-
Upload
mohammed-ramadan -
Category
Documents
-
view
216 -
download
0
Transcript of SW Testing MAM Seminar
-
7/31/2019 SW Testing MAM Seminar
1/34
Mohammed A. Ramadan
Computer Science Department
Modern Academy in Maadi
Software Testing
-
7/31/2019 SW Testing MAM Seminar
2/34
Agenda
22-Apr-12Software Testing2
Testing -What Is it ?
Testing lifecycle.
Types of Testing
Test Strategy
Orthogonal Arrays Testing Strategy
(OATS)
-
7/31/2019 SW Testing MAM Seminar
3/34
Testing What Is it ???
22-Apr-12Software Testing
An activity in which a system or component is executedunder specified conditions, the results are observed orrecorded, and an evaluation is made of some aspect of thesystem or component
From IEEE Std 610 Software Engineering Terminology:test
The process of executing computer software in order to
determine whether the results it produces are correct
Glass 79
The process of executing a program with the intent of findingerrors, Myers 79
Testing is the measure of software quality, Hetzel 85
3
-
7/31/2019 SW Testing MAM Seminar
4/34
Objectives of Testing
22-Apr-12Software Testing4
What are we looking for?
Errors, faults, failures and incidents. Defect, Problem,Incident, Anomaly, Variance, Inconsistency, Feature,Bugs !
A test is the act of exercising software with test cases withan objective of Finding failure Demonstrate correct execution
Goal To find if the product is
Not operating as Intended
-
7/31/2019 SW Testing MAM Seminar
5/34
Matrix of Specification andSoftware
22-Apr-12Software Testing5
Specification Software
Should do Doesnt
Shouldnt do Does
No mention but should Doesnt
No mention Does
Non-Functional Hard to use,Complex, slow
-
7/31/2019 SW Testing MAM Seminar
6/34
Source of Defects
22-Apr-12Software Testing6
56%27%
7%
10%
Requirements
OtherCode
Design
Source: Software Testing in the Real
World Edward Kit, 1995
-
7/31/2019 SW Testing MAM Seminar
7/34
Cost of Repair
22-Apr-12Software Testing7
The sooner you find bugs, the cheaper
Defec
tsremoved
PROJECT PHASES
Coding Testing Maintenance
Repair
Cost 1
Repair
Cost 4
Repair
Cost 10
-
7/31/2019 SW Testing MAM Seminar
8/34
Testing lifecycle
22-Apr-12Software Testing8
Development Lifecycle
ProjectPlanning
RequirementsCapture
Implementation
Build 0 Build 1 Build 2
Test Lifecycle Plan
Test
Analysis &Design
Design
Test
Setup Test
Bed
ExecuteTest
EvaluateTest
Defect Tracking/Enhancement Request
Testing in the V Model Approach
-
7/31/2019 SW Testing MAM Seminar
9/34
Testing lifecycle ,contd
22-Apr-12Software Testing9
Requirements
Capture
Analysis
Test planning &Scenario Design
Test CaseDevelopment
Test Execution
Test ResultAnalysis
Test CycleClosure
Defect FixingCycle
Defects
New Version
The Scenarios Design and the Test Case Developmentcould normally start in parallel with the DevelopmentCycle.
Test Execution Synchs up with the Development Cycleduring the functional Testing Phases
-
7/31/2019 SW Testing MAM Seminar
10/34
Testing in the V Model Approach
22-Apr-12Software Testing10
Verification Validation
Requirements
(Business Needs)
Requirements
(Business
Functions)Design
Code
AcceptanceTesting
System Testing
IntegrationTesting
Unit Testing
Writetests
Run
tests
The test phase of the lifecycle which ensures thatthe end product meets theusers needs.
All QC activities through outthe life cycle that ensure thatinterim deliverables meet theirinput specification.
-
7/31/2019 SW Testing MAM Seminar
11/34
Types of Testing
22-Apr-12Software Testing11
SoftwareSoftware
Black Box Testing
Static
Dynamic
White Box Testing
Static
Dynamic
-
7/31/2019 SW Testing MAM Seminar
12/34
Types of Testing
22-Apr-12Software Testing12
Black Box
White Box
Static
Dynamic
Dynamic
Static
Specifications
Test to Pass
Test to Fail
Regression
EquivalencePartitioning
Boundary Conditions
Invalid Data
State Transitions
Data Testing
Data Coverage
Data Flow
Code Coverage
Data Reference Error
Computation Errors
Control Flow Errors
Input/Output Errors
Review & Analysis
Formal Review
Peer Review
Walkthrough
Inspections
Coding Standards
Generic Code ReviewLine Coverage
Branch Coverage
Condition Coverage
Unit Testing
Structure Testing
-
7/31/2019 SW Testing MAM Seminar
13/34
Black Box Testing
22-Apr-12Software Testing13
Static Black Box TestingTesting the specification
Dynamic Black Box TestingBehavioral Testing
Testing a running program without anyknowledge of how the software behaves.
Software
-
7/31/2019 SW Testing MAM Seminar
14/34
White Box Testing
22-Apr-12Software Testing14
Static White Box Testing
Code review and analysis
(1)Formal Reviews (2)Peer Reviews (3) Walkthroughs
Dynamic Black Box Testing Structural testing
using information from what the code does and how it worksto determine what to test, what not to test.
Unit Testing
Data Coverage
Divide the code into data and its states
Data flow
Track a piece of data completely through the software
Code coverage
Program and Line Coverage
Branch Coverage
Condition Coverage
Software
-
7/31/2019 SW Testing MAM Seminar
15/34
Other Types of Testing
22-Apr-12Software Testing15
User Interface Testing Test the User Interface for operability, ease of use
Performance
Testing the performance of the computations
Load Testing the performance of the system under varying load
conditions
Stress
Testing the performance and accuracy of system under
resource crunch or any other kind of stress Volume
Testing the accuracy of system when system has a highvolume of data
-
7/31/2019 SW Testing MAM Seminar
16/34
Other Types of Testing ,contd
22-Apr-12Software Testing16
Configuration Testing the system under various hardware/software configurations
Compatibility
Testing the system for ability to interact with other system
Installation Testing the installation of system under one or more configurations
Documentation
Testing the system for documentation clarity and ease ofapprehension
Ad-Hoc Testing
Randomly testing the system for errors with no background on thesystem
-
7/31/2019 SW Testing MAM Seminar
17/34
Why Have a Test Strategy?
22-Apr-12Software Testing17
Test Strategy Describes the overall approach that the test team will
use to test, in each phase and in all.
A Test Plan The game plan to implement
what is to be tested, when, how, how long and till what happens?
Effort Estimation Test Strategy helps the overall test effort estimation
-
7/31/2019 SW Testing MAM Seminar
18/34
What to Consider?
22-Apr-12Software Testing18
Product Requirements
Nature of product
Product Lifecycle
Milestones
Repeatability considerations
Environment
Feedback
Cost
-
7/31/2019 SW Testing MAM Seminar
19/34
Optimal Testing
22-Apr-12Software Testing19
Number ofMissed Bugs
Optimal Amountof Testing
Cost of Testing
Over Testing
Under Testing
Amount of Testing
-
7/31/2019 SW Testing MAM Seminar
20/34
22-Apr-12Software Testing20
-
7/31/2019 SW Testing MAM Seminar
21/34
Orthogonal Arrays Testing Strategy(OATS)
22-Apr-12Software Testing21
(OATS) is a systematical, statistical way of testing pair-wise interactions by deriving suitable small set of testcases from a large number of scenarios.
can be used to reduce the number of test combinationsand provide maximum coverage with a minimum numberof test cases.
(OATS) utilizes an array of values representing variablefactors that are combined pair-wise rather thanrepresenting all combinations of factors and levels.
-
7/31/2019 SW Testing MAM Seminar
22/34
Overview
22-Apr-12Software Testing22
Orthogonal arrays are two dimensional arrays of
numbers by choosing any two columns in the array youreceive an even distribution of all the pair-wisecombinations of values in the array.
-
7/31/2019 SW Testing MAM Seminar
23/34
Overview
22-Apr-12Software Testing23
An L9
(34) orthogonal array with 9 runs,
4 factors, 3 levels, and strength of 2.
An L9 (34 )
orthogonal arraywith 9 runs,
4 factors, 3levels, andstrength of 2.
Runs: the number of rows in the array. This directly translates to the number of testcases that will be generated by the OATS technique.
Factors: the number of columns in an array. This directly translates to the maximumnumber of variables that can be handled by this array.
Levels: the maximum number of values that can be taken on by any single factor. An
orthogonal array will contain values from 0 to Levels-1.
Strength: the number of columns it takes to see each of the Levelstrengh possibilitiesequally often.
Orthogonal arrays are most often named following the pattern L runs (LevelsFactors ).
-
7/31/2019 SW Testing MAM Seminar
24/34
Simple OAT Example
22-Apr-12Software Testing24
Consider three parameters : A, B, and C.
Parameters A, B, and C each have positive values :1, 2, and 3.
Testing all combinations of the three parameters would
involve executing a total of 27 test cases.
-
7/31/2019 SW Testing MAM Seminar
25/34
Simple OAT Example (contd)
22-Apr-12Software Testing25
With the way programming works, a fault will most likelydepend on the values of two parameters, not three.
The fault might occur for each of these three test cases:
A=1, B=1, C=1, A=1, B=1, C=2, and A=1, B=1, C=3.
Not necessary to run all 27 scenarios, 9 test cases issuffice.
The 9 scenarios outlined in Table 1 (on the next screen)address all possible pairs within the three parameters.
-
7/31/2019 SW Testing MAM Seminar
26/34
-Parameters
22-Apr-12Software Testing26
All possible pair-wise combinations between parameters Aand B, B and C, and C and A .
-
7/31/2019 SW Testing MAM Seminar
27/34
Example 2
22-Apr-12Software Testing27
Orthogonal Array has 3 columns, representing 3 variables.
Utilize a three level-alphabet Red, Green, and Blue.
Next, organize combinations into a table (next slide).
-
7/31/2019 SW Testing MAM Seminar
28/34
Example 2 (contd)
22-Apr-12Software Testing28
All Combinations for Three Variables of Three Levels Each
A B C
1 Red Red Red
2 Red Red Green3 Red Red Blue
4 Red Green Red
5 Red Green Green
6 Red Green Blue
7 Red Blue Red
8 Red Blue Green
9 Red Blue Blue
10 Blue Red Red
11 Blue Red Green
12 Blue Red Blue
13 Blue Green Red
14 Blue Green Green
15 Blue Green Blue
16 Blue Blue Red
17 Blue Blue Green
18 Blue Blue Blue
19 Green Red Red20 Green Red Green
21 Green Red Blue
22 Green Green Red
23 Green Green Green
24 Green Green Blue
25 Green Blue Red
26 Green Blue Green
27 Green Blue Blue
-
7/31/2019 SW Testing MAM Seminar
29/34
Example 2 (contd)
22-Apr-12Software Testing29
For each pair of columns, AB, AC, and BC, each pair of colorsappears exactly three times.
To save testing effort, the appearance of each pair has been reducedto once.
All-Pairs Array, Three Variables of Three Levels Each
A B C
2 Red Red Green
4 Red Green Red
9 Red Blue Blue
12 Blue Red Blue
14 Blue Green Green
16 Blue Blue Red
19 Green Red Red
24 Green Green Blue
26 Green Blue Green
-
7/31/2019 SW Testing MAM Seminar
30/34
Advantages of OATS
22-Apr-12Software Testing30
Utilizes variable pair combination
Provides uniformly distributed coverage
Useful for integration testing of software components
Useful for testing configurable options on an application
web page allowing font style
background color
page layout selections
-
7/31/2019 SW Testing MAM Seminar
31/34
Advantages of OATS
22-Apr-12Software Testing31
Conventional Test Cases:
Variables:3Input: 4
Possible cases: 81=34
Variables: 3
Input: 5Possible Cases: 243 = 35
Variables: 5
Input: 5
Possible Cases: 3125 = 55
Orthogonal Test Cases
Variables:3Input: 4
Possible cases: 9
Variables: 3
Input: 5
Possible Cases: 11
Variables: 5Input: 5
Possible Cases: 21
-
7/31/2019 SW Testing MAM Seminar
32/34
Disadvantages of OATS
22-Apr-12Software Testing32
Can only be applied at the initial stage of the product/processdesign system.
There are some situations whereby OA techniques are not
applicable, such as a processes involving influencing factorsthat vary in time and cannot be quantified exactly.
Can be overwhelming to use the first time; improves with use
on each project.
Arrays can be difficult to construct.
-
7/31/2019 SW Testing MAM Seminar
33/34
22-Apr-12Software Testing33
Questions
-
7/31/2019 SW Testing MAM Seminar
34/34
Thank You
22 A 1234