Post on 19-May-2015
ISBN 0-13-146913-4Prentice-Hall, 2006
Program Testing + Integration Testing
Zeeshan Rana
Types of faults and how to clasify them The purpose of testing Unit testing Integration testing strategies Test planning When to stop testing
Chapter 8 Objectives
Wrong requirement: not what the customer wants
Missing requirement Requirement impossible to implement Faulty design Faulty code Improperly implemented design
8.1 Software Faults and FailuresWhy Does Software Fail?
Objective of testing: discover faults A test is successful only when a fault is
discovered◦ Fault identification is the process of determining
what fault caused the failure◦ Fault correction is the process of making changes
to the system so that the faults are removed
8.1 Software Faults and Failures Objective of Testing
Purpose Input Expected Output Actual Output Sample Format:
Elements of a Test Case
Algorithmic fault Computation and precision fault
◦ a formula’s implementation is wrong Documentation fault
◦ Documentation doesn’t match what program does Capacity or boundary faults
◦ System’s performance not acceptable when certain limits are reached
Timing or coordination faults Performance faults
◦ System does not perform at the speed prescribed
8.1 Software Faults and FailuresTypes of Faults
An algorithmic fault occurs when a component’s algorithm or logic does not produce proper output◦ Branching too soon◦ Branching too late◦ Forgetting to initialize variable or set loop
invariants◦ Comparing variables of inappropriate data types
8.1 Software Faults and FailuresTypical Algorithmic Faults
Module testing, component testing, or unit testing
Integration testing System Testing
◦ Function testing◦ Performance testing
Acceptance testing Installation testing
Levels of Testing
Levels of TestingTesting Organization Illustrated
Egoless programming: programs are viewed as components of a larger system, not as the property of those who wrote them
8.2 Testing IssuesAttitude Toward Testing
Independent test team◦ avoid conflict◦ improve objectivity◦ allow testing and coding concurrently
8.2 Testing IssuesWho Performs the Test?
Closed box or black box: functionality of the test objects◦ Equivalence Class, Boundary Value Analysis,
Scenario-based, Decision Table based, State Machine based…
Clear box or white box: structure of the test objects ◦ Control Flow
Basis Path, Branch, Statement, Decision…◦ Data Flow
Du Path, All-uses Path
8.2 Testing IssuesViews of the Test Objects
Black box: external behavior description State box: black box with state information White box: state box with a procedure
8.2 Testing IssuesSidebar 8.2 Box Structures
Code walkthrough Code inspection
8.3 Unit TestingCode Review
Testing the unit for correct functionality Testing the unit for correct execution
8.3 Unit Testing
Determining test objectives Selecting test cases Executing test cases
8.3 Unit TestingSteps in Testing
Statement testing Branch testing Path testing …
8.3 Unit TestingWhitebox Test Thoroughness
Whitebox TestingControl Flow Graph
Cyclomatic Complexity = E – N + 2 = 9 – 9 + 2 = 2
8.3 Unit TestingComparing Techniques
• Effectiveness of fault-discovery techniques
Requirements Faults Design Faults
Code Faults
Documentation Faults
Reviews Fair Excellent Excellent Good
Prototypes Good Fair Fair Not applicable
Testing Poor Poor Good Fair
Correctness Proofs Poor Poor Fair Fair
Big-bang Bottom-up Top-down Sandwich testing
8.4 Integration Testing
Component Driver: a routine that calls a particular component and passes a test case to it
Stub: a special-purpose program to simulate the activity of the missing component
8.4 Integration TestingTerminology
System viewed as a hierarchy of components
8.4 Integration TestingView of a System
All components integrated at once Locating faults?
8.4 Integration Testing Bing-Bang Integration Example
Only A is tested by itself Stubs of B, C and D are used at first level N-1 stubs required (N=Number of nodes) Locating faults?
8.4 Integration TestingTop-Down Integration Example
Drivers are used to call the child functions Drivers are relatively intelligent N-leaves drivers Locating faults?
8.4 Integration TestingBottom-Up Integration Example
Viewed system as three layers Employ BU where writing drivers is not costly
Employ TD wherestubs are easier toWrite Locating faults?
8.4 Integration Testing Sandwich Integration Example
Integration TestingCall-Graph Based Testing
• Adjacency Matrix• NxN matrix that tells which components call
the other components• Pairwise Integration
• Test each pair (i.e. each edge)• E testing sessions
• Neighborhood based Integration• Integrate each neighborhood
• The nodes at one edge distance from the node to be integrated
• N-sink nodes sessions
Establish test objectives Design and Write test cases Test test cases Execute tests Evaluate test results
8.6 Test Planning
Test plan explains◦ who does the testing◦ why the tests are performed◦ how tests are conducted◦ when the tests are scheduled
8.6 Test PlanningPurpose of the Plan
What the test objectives are How the test will be run What criteria will be used to determine
when the testing is complete
8.6 Test PlanningContents of the Plan
Automated Testing Tools Testing Management Tools Bug Tracking/Configuration Management
Tools
Software Testing Tools
No time left No money left Statistical Criteria
◦ Number of defects found per week becomes lower than a set threshold
8.8 When to Stop Testing
The Ariane-5’s flight control system was tested in four ways◦ equipment testing◦ on-board computer software testing◦ staged integration◦ system validation tests
The Ariane-5 developers relied on insufficient reviews and test coverage
8.10 Real-Time ExampleThe Ariane-5 System
It is important to understand the difference between faults and failures
The goal of testing is to find faults, not to prove correctness
8.11 What this Chapter Means for You
UCF Slides Software Testing, A Craftsman’s Approach
by Jorgensen Software Testing Tools by Prasad
References