3-White Box Testing
-
Upload
mani-kantan -
Category
Documents
-
view
229 -
download
0
Transcript of 3-White Box Testing
-
8/12/2019 3-White Box Testing
1/47
No part of this presentation may be reproduced without prior written permissionfrom authors, copyright owner and publisher of the book
White Box Testing(Chapter 3)
-
8/12/2019 3-White Box Testing
2/47
3/2/2014 White Box Testing Slide : 2
Objectives
To review the V Model and set the context for differenttypes of testsTo focus on white box testing
To distinguish between different types of white boxtestingTo place special emphasis on Code reviews / inspections
Methods of code coverage and relationship toquality Code complexity
-
8/12/2019 3-White Box Testing
3/47
3/2/2014 White Box Testing Slide : 3
Bending the Waterfall
High-LevelDes ign
Low-LevelDes ign
Development
Testing
Unit Testing
ComponentTesting
IntegrationTesting
Sys temTesting
AcceptanceTesting
SoftwareRequi rements
OverallBus inessRequirements
-
8/12/2019 3-White Box Testing
4/47
3/2/2014 White Box Testing Slide : 4
OverallBus inessRequirements
SoftwareRequi rements
High-LevelDes ign
Low-LevelDes ign
Development Unit TestExecut ion
ComponentTestExecut ion
IntegrationTest Execution
Sys temTest Execution
AcceptanceTest Execution
AcceptanceTest Design
Sys temTest Design
IntegrationTest Design
ComponentTest Design
Unit TestDes ign
V Model
-
8/12/2019 3-White Box Testing
5/47
3/2/2014 White Box Testing Slide : 5
Principles of the V ModelDo an early test design.Involve the right people in the design of the right typeof tests.Make the tests reflect closely the steps on the left-hand side of the V.Unit testing White box testing (focus of this chapter) Black box testing (focus of the next chapter)
-
8/12/2019 3-White Box Testing
6/47
3/2/2014 White Box Testing Slide : 6
White Box Testing
Requires access to codeLooks at the program code and performs testing bymapping program code to functionality
-
8/12/2019 3-White Box Testing
7/473/2/2014 White Box Testing Slide : 7
Why White Box Testing?The program code truly represents what the programactually does and not just what it is intended to do!It minimizes delay between defect injection and defectdetection (i.e., does not postpone detection to be
done by someone else).Can catch obvious programming errors that do notnecessarily map to common user scenarios (e.g.,divide by zero).
-
8/12/2019 3-White Box Testing
8/473/2/2014 White Box Testing Slide : 8
Types of White Box Testing
-
8/12/2019 3-White Box Testing
9/473/2/2014 White Box Testing Slide : 9
Static TestingInvolves only the source code and not the executables orbinariesDoes not involve executing the program on a machine butrather humans going through it or the usage of specializedtools
Some of the things tested by static testing: Whether the code works according to the functional
requirement Whether the code has been written in accordance with
the design developed earlier in the project life cycle
Whether the code follows all applicable standards Whether the code for any functionality has been missed
out Whether the code handles errors properly
-
8/12/2019 3-White Box Testing
10/473/2/2014 White Box Testing Slide : 10
Static Testing by Humans
Humans can find errors that computers cant.Multiple humans can provide multiple perspectives.
A human evaluation can compare the code against thespecifications more thoroughly.It can detect multiple defects at one go.It reduces downstream, inline pressure.
-
8/12/2019 3-White Box Testing
11/473/2/2014 White Box Testing Slide : 11
Static Testing TypesDifferent types Desk checking of the code Code walkthrough Code review
Code inspectionQA vs QC argumentIncreasing the involvement of peopleMore variety of perspectivesIncreasing formalismIncreasing the likelihood of identifying more complexdefects
-
8/12/2019 3-White Box Testing
12/473/2/2014 White Box Testing Slide : 12
Desk Checking
Author informally checks the code against thespecifications and corrects defects found.No structured method or formalism is required toensure completeness.No log or checklist is maintained.It relies only on the thoroughness of the author.It may suffice for obvious programming errors butmay not be effective for incomplete / misunderstoodrequirements.
-
8/12/2019 3-White Box Testing
13/473/2/2014 White Box Testing Slide : 13
Advantages and Disadvantages ofDesk Checking
Advantages The programmer knows the code and
programming language well and hence is bestsuited to read the program
Less scheduling and logistics overheads Reduces delay in defect detection and correctionDisadvantages
Person dependent, not scalable or reproducible Tunnel visioning of developers Developers prefer writing new code and dont like
testing
-
8/12/2019 3-White Box Testing
14/473/2/2014 White Box Testing Slide : 14
Code Walkthrough
Group oriented (as against desk checking)Brings in multiple perspectivesMultiple specific roles (author, moderator, inspector,etc.), as discussed in Fagan Inspection
-
8/12/2019 3-White Box Testing
15/473/2/2014 White Box Testing Slide : 15
Formal Inspection / FaganInspection
Group-oriented activityHighly formal and structuredHas specific roles
Requires thorough preparation
-
8/12/2019 3-White Box Testing
16/473/2/2014 White Box Testing Slide : 16
Roles in a formal inspection Author
Author of the work product Makes available the required material to the reviewers Fixes defects that are reported
Moderator Controls the meeting(s)
Inspectors (reviewers) Prepare by reading the required documents Take part in the meeting(s) and report defects
Scribe Takes down notes during the meeting Assigned in advance by turns Can participate to review to the extent possible Documents the minutes and circulates them to participants
-
8/12/2019 3-White Box Testing
17/473/2/2014 White Box Testing Slide : 17
Process in a Fagan Inspection
Typical documents circulated: Program code Design / program specifications
SRS (if needed) Any applicable standards (e.g., coding standards) Any necessary checklists (e.g., code review
checklist)
-
8/12/2019 3-White Box Testing
18/473/2/2014 White Box Testing Slide : 18
Meetings in a Fagan InspectionPreliminary meeting (optional) Author explains his / her perspective Makes available the necessary documents Highlights concern areas, if any, for which review comments
are sought
Defect Logging Meeting All come prepared! Moderator goes through the code sequentially Each reviewer comes up with comments Comments / defects categorized as defect / observation,
major / minor, systemic / mis -execution Scribe documents all the findings and circulates them
Follow-up meeting (optional) Author fixes defects If required, a follow-up meeting is called to verify
completeness of fixes
-
8/12/2019 3-White Box Testing
19/47
-
8/12/2019 3-White Box Testing
20/473/2/2014 White Box Testing Slide : 20
Combinations of various methods
Prioritizing various programs High priority / important codes subject to formal
inspection, medium ones for walkthrough and lowones for desk checking
Deciding between 100% coverage and partialcoverage
-
8/12/2019 3-White Box Testing
21/473/2/2014 White Box Testing Slide : 21
Static Analysis ToolsHelp identify / are used:
Whether there are unreachable codes (usage of gotostatements sometimes creates this situation; there could beother reasons too)Variables declared but not usedMismatch in definition and assignment of values to variablesIllegal or error-prone type-casting of variablesUse of non-portable or architecture-dependent programmingconstructsMemory allocated but not having corresponding statements for
freeing up memoryFor calculation of cyclomatic complexity
As an extension of compilers (lint, compiler flag drivenchecking)
-
8/12/2019 3-White Box Testing
22/473/2/2014 White Box Testing Slide : 22
Use of a Code Review Checklist
Never leave home without it! The code review checklist aids in organizationallearning by indicating what to look for.It should be kept current as we learn.
A discussion of a sample checklist is given onPage 54.
-
8/12/2019 3-White Box Testing
23/47
3/2/2014 White Box Testing Slide : 23
Structural TestingDone by running the executable on the machineEntails running the product against some predefinedtest cases and comparing the results against theexpected results
Designing test cases and test data to exercisecertain portions of codeTypes of structural testing Unit / code functional testing
Code coverage testing Code complexity testing
-
8/12/2019 3-White Box Testing
24/47
3/2/2014 White Box Testing Slide : 24
Unit / Code Functional Testing
Initial quick checks by developerRemoves obvious errors Done before more expensive checks
Building debug versionsRunning through an IDEDebugging or testing? ( Who cares?! )
-
8/12/2019 3-White Box Testing
25/47
-
8/12/2019 3-White Box Testing
26/47
3/2/2014 White Box Testing Slide : 26
Types of Code Coverage
Statement coveragePath coverageCondition coverage
Function coverage
-
8/12/2019 3-White Box Testing
27/47
3/2/2014 White Box Testing Slide : 27
Programming Constructs
Sequential instructionsTwo-way decision statements (if then else)Multi-way decision statements (switchstatements)Loops, like while/do, for, repeat/until, etc
-
8/12/2019 3-White Box Testing
28/47
3/2/2014 White Box Testing Slide : 28
Testing sequential instructions
Generate test data to make the program enter thesequential block, to make it go through the entireblock (Is this valid?) Asynchronous exceptions Multiple entry points, in non-structured
programmingStatement coverage as a metric: # of statements exercised / Total # of statements
-
8/12/2019 3-White Box Testing
29/47
3/2/2014 White Box Testing Slide : 29
Testing IF THEN ELSEHave data to test the THEN partHave data to test the ELSE partRelevance of statement coverage?
Total = 0; /* set total to zero */if (code == M) {
stmt1;stmt2;Stmt3;stmt4;Stmt5;stmt6;
Stmt7;}else percent = value/Total*100; /* divide by zero */
-
8/12/2019 3-White Box Testing
30/47
3/2/2014 White Box Testing Slide : 30
Path Coverage
Statement Coverage may not indicate truecoverage Path Coverage provides better representation Previous example: Having data go through the
THEN part and ELSE part gives only 50%coverage, irrespective of number of statements ineach path
More detailed example of path coverage in next slidePath coverage = # of paths exercised / total # ofpaths in the program
-
8/12/2019 3-White Box Testing
31/47
3/2/2014 White Box Testing Slide : 31
-
8/12/2019 3-White Box Testing
32/47
3/2/2014 White Box Testing Slide : 32
Condition Coverage
Further refinement of path coverageMakes sure each constituent condition in a booleanexpression is coveredProtects against compiler optimizationsCondition coverage = # of conditions covered /number of conditions in the program
-
8/12/2019 3-White Box Testing
33/47
3/2/2014 White Box Testing Slide : 33
Function Coverage
Finds how many functions are covered by test casesHigher level of abstraction; hence possible to achieve100% coverageMore logical than the other types of coverage
Can be derived from RTM Easy to prioritize as they are based on requirementsLeads more naturally to black box testsCan also be a natural predecessor to performance
testing
-
8/12/2019 3-White Box Testing
34/47
3/2/2014 White Box Testing Slide : 34
Other Uses of Code CoverageMethods
Performance analysis and optimizationResource usage analysisChecking of critical sections
Identifying memory leaksDynamically generated code (e.g., dynamicgeneration of URLs and security checking)
-
8/12/2019 3-White Box Testing
35/47
3/2/2014 White Box Testing Slide : 35
Code Complexity Testing
Which of the paths are independent? (If paths are notindependent, we may be able to minimize the numberof tests.)Is there an upper bound on the number of tests to beexecuted to ensure that all the statements have beenexecuted at least once?Cyclomati c com plex i ty provides answers to someof these questions.
-
8/12/2019 3-White Box Testing
36/47
3/2/2014 White Box Testing Slide : 36
Intuitive Meaning of CyclomaticComplexity
Provides an indication of how many independentpaths are there in a program What gives birth to new paths?How can one count the new paths that are born?
-
8/12/2019 3-White Box Testing
37/47
3/2/2014 White Box Testing Slide : 37
Steps in Determining CyclomaticComplexity
Start with a conventional flow chart.Identify the decision points. Convert complex predicates to simplepredicates (see next slide).
-
8/12/2019 3-White Box Testing
38/47
3/2/2014 White Box Testing Slide : 38
-
8/12/2019 3-White Box Testing
39/47
3/2/2014 White Box Testing Slide : 39
Steps in Determining CyclomaticComplexity
Start with a conventional flow chart.Identify the decision points.
Convert complex predicates to simple predicates (see nextslide).
If there are loops, convert the loop conditions to simplepredicates.Combine all the sequential statements into a single node in theflow graph.
When a set of sequential statements are followed by a simplepredicate, combine all the sequential statements and thepredicate check into one node and have two edges emanatingout of this node. Nodes with two emanating edges are calledpredicate nodes.
Make sure that all edges terminate in some node, adding a nodeto represent all the statements at the end of the program.
-
8/12/2019 3-White Box Testing
40/47
3/2/2014 White Box Testing Slide : 40
Conv ent ional Flow Char tFlow Diagram fo r
Calcula t ing Com plexi ty
-
8/12/2019 3-White Box Testing
41/47
3/2/2014 White Box Testing Slide : 41
Formal and Intuitive Interpretationof Cyclomatic Complexity
Formal definitions CC = # of predicate nodes + 1 CC = E N + 2 (E = edges, N = nodes)
Intuitive definitions When a condition is introduced, adds a new path See the next two slides
-
8/12/2019 3-White Box Testing
42/47
3/2/2014 White Box Testing Slide : 42
End
Case (i): A hypothetical program with no decision node
# of ind ependent paths = 1 # of no des, N = 2; # of edg es, E = 1; Cyc lom atic com plex i ty = E N + 2 = 1 # of pred icate no des, P = 0; Cyclo m at ic com plexi ty = P + 1 = 1
-
8/12/2019 3-White Box Testing
43/47
3/2/2014 White Box Testing Slide : 43
End
Case (ii): Adding one decision node
# of independent paths = 2 # of no des, N = 4;
# of edg es, E = 4; Cyc lom atic com plex i ty = E N + 2 = 2 # of pr edicate no des, P = 1; Cyclo m at ic com plexi ty = P + 1 = 2
-
8/12/2019 3-White Box Testing
44/47
3/2/2014 White Box Testing Slide : 44
Independent Paths and Basis Sets
An independent path is a path in the flow graph thathas not been traversed before in other paths.
A set of independent paths that cover all the edges iscalled a basis set.Find the basis set and write the test cases to executeall the paths in the basis set.
-
8/12/2019 3-White Box Testing
45/47
3/2/2014 White Box Testing Slide : 45
Meaning and Interpretation ofCyclomatic Complexity
Complexity What it means
1-10 Well-written code, testability is high,cost / effort to maintain is low
10-20 Moderately complex, testability ismedium, cost / effort to maintain is
medium
20-40 Very complex, testability is low, cost /effort to maintain is high
>40 Not testable, any amount of money /effort to maintain may not be enough
-
8/12/2019 3-White Box Testing
46/47
3/2/2014 White Box Testing Slide : 46
Challenges in White Box TestingDevelopers not finding time for testing They develop blind spots for their own defectsFully covered code may not correspond to realisticscenarios
-
8/12/2019 3-White Box Testing
47/47
Objectives (Recap)
To review the V Model and set the context for differenttypes of testsTo focus on white box testingTo distinguish between different types of white boxtestingTo place special emphasis on Code reviews / inspections Methods of code coverage and relationship to
quality Code complexity