STM-U4

Prof. N. Prasanna Balaji

SOFTWARE TESTING METHODOLOGYTerm:2009-2010

B.Tech III/IT Semester - I

Unit-IV PPT Slides

Text Books: 1.Software Testing Techniques: Boris Beizer 2. Craft of Software Testing: Brain Marrick

2

Sub Topic No’s

Sub Topic name Lecturer No

Slide No’s

1 Introduction L1 3

2 Domains & Paths L2 22

3 Nice Domains L3 28

4 Ugly Domains L4 42

5 Domain Testing L5 51

6 Domains & Interfaces Testing L6 65

7 Domains & Testability L7 70

ref boris beizer 3

Domain Testing - Domains & Paths

Domain Testing ModelDomain Testing Model

INPUT CLASSIFY DO CASE 1 OUTPUT

DO CASE 2

DO CASE 3

DO CASE n

D1

D3

(x,y,z,…)

(1, 2, 3, 4, 5,…)

{ Outcome }

D2

Dn

ref boris beizer 4

Domain Testing Domain TestingDomain Testing

Views Programs as input data classifiers

Verifies if the classification is correct

ref boris beizer 5



Two ViewsTwo Views

• Based on SpecsBased on Specs

• Functional TestingFunctional Testing

• Based on Implementation informationBased on Implementation information

• Structural Technique

ref boris beizer 6



• Variables• Simple combinations of two variables• Numbers from - to +

• Input variables as numbers

• Loop-free Programs

ref boris beizer 7



• Structural Knowledge is not needed. Only Specs.

• For each Input Case,

• A Hypothetical path for Functional testing

• An Actual path for Structural testing

ref boris beizer 8


DomainDomain (simplified) (simplified)

• A Single Connected Set of numbers

• No arbitrary discrete sets

• Defined by the boundaries

• One or more boundaries

• Specified by predicates

• Bugs likely at the boundaries

• One or more variables

D1

ref boris beizer 9


PredicatesPredicates

• Interpretation

• Structural Testing - CFG

• Functional Testing - DFG

• Specifies the Domain Boundary

• n Predicates in Sequence 22n domains

• Or, just two domains A .AND. B .AND. C

ref boris beizer 10


PathsPaths

• At least One PATH thru the Program

• More paths for disconnected domains

ref boris beizer 11


Summary:Summary:

• Domain for a loop-free program corresponds to a set of numbers defined over the input vector

• For every domain, there is at least one path thru the routine, along which that domain’s processing is done

• The set of interpreted predicates traversed on that path (ie., the path’s predicate expression) defines the domain’s boundaries.

ref boris beizer 12

Domain Testing - Domains & Paths - Domain Closure

Boundary: Closed / Open

MIN MAX

D1D2

D3X < MAXX >= MIN

MIN MAX

D1 D2 D3

X >= MIN X <= MAX

Closed

Closed Open

ref boris beizer 13

Domain Testing - Domains & Paths - Domain Closure

Boundary: Closed / Open

MIN MAX

D1 D2 D3

X > MIN X < MAXOpen

ref boris beizer 14

Domain Testing - Domains & Paths - Domain Dimensionality

• One dimension per variable

• At least one predicate• Slices thru previously defined Domain

• Slicing Boundary

• N-spaces are cut by Hyperplanes

ref boris beizer 15

Domain Testing - Domains & Paths – Bug Assumptions

• Processing is OK. Domain definition may be wrong.

Boundaries are wrong.

Predicates are wrong.

• Once input vector is set on the right path, it’s correctly processed.

• More bugs causing domain errors …

ref boris beizer 16

Domain Testing - Domains & Paths – Bug Assumptions..

• Double-zero representation

• Floating point zero check

• Contradictory domains

• Ambiguous domains defined

undefined

ref boris beizer 17

Domain Testing - Domains & Paths – Bug Assumptions..

• Over-specified domains

• Boundary errors

Boundary closure

Shifted

X > 3 .AND. X < 2 .AND. Y > 3

ref boris beizer 18

Domain Testing - Domains & Paths - Bug Assumptions

• Boundary errors….

Tilted

Missing

Extra boundary

ref boris beizer 19

U3

• Closure Reversal

• Faulty Logic

Simplification of compound predicates

X >= 3


ref boris beizer 20


Strategy for domain testing in 2-dimStrategy for domain testing in 2-dim

ref boris beizer 21

Domain Testing - Domains & Paths - Restrictions

In General…

1.1. Coincidental CorrectnessCoincidental Correctness

DT cannot detectExample

• Representative outcomeRepresentative outcome

Partition testing

Input Equivalence

D1

D2

F1(x) : +1

F2(x) : -1

ref boris beizer 22



INPUT CLASSIFY DO CASE 1 OUTPUT

DO CASE 2

DO CASE 3

DO CASE n

D1

D3

(x, y, z, …)

(1, 2, 3, 4, 5,…)

{ Outcome }D2

Dn

Function f1

f2

f3

f4

ref boris beizer 23

Domain Testing - Domains & Paths - - Restrictions

3.3. Simple boundaries & Compound predicatesSimple boundaries & Compound predicates

X0 16

X >=0 .AND. X <= 16

Y = 1 .AND. X >=0 .AND. X <= 16

X0 16

Y

D1

D1

ref boris beizer 24


Compound predicates….Compound predicates….

• Impact of Impact of .OR..OR.

• Concave, Disconnected

• Adjacent domains with same function

• Example

A B C + D E FA B C + D E F

• Eliminate compound predicates

ref boris beizer 25


4. Functional Homogeneity of Bugs

• Functional form still Retained

a x + b y >= c

• Bugs are only in a, b, c

ref boris beizer 26


5. Linear Vector Space

• Linear boundary predicate, Interpreted• Simple relational operators

• Conversion to linear vector space

• 2-d Polar co-ordinates

• Polynomials

• Problems with Non-linear Boundaries

ref boris beizer 27


6. Loop-free Software

Predicate for each iteration

Loop over the entire transaction

Definite loop

ref boris beizer 28

Domain Testing - Domains & Paths – Nice & Ugly domains

Nice DomainsNice Domains

RequirementsRequirements

• Bugs ill-defined domains

Before DTBefore DT

• Analyze specs

• Make the Boundary Specs Consistent & Complete

ref boris beizer 29

Domain Testing - Domains & Paths – Nice domains

Implemented Domains

• Complete, Consistent & Process all inputs

Specified Domains

• Incomplete, Inconsistent

Programmer’s / Designer’s Effort

ref boris beizer 30


Nice Domains

• Linear, Complete, Systematic, Orthogonal, Consistently Closed, & Convex

U1 U2

V1

V2

D11 D12

D21 D22

ref boris beizer 31


Nice Domains

Advantages

• Ease of DT

• Fewer Bug Occurrences

ref boris beizer 32


Nice Domains

Boundaries are

1. Linear

2. Complete

3. Systematic

4. Orthogonal

5. Closure consistency

6. Convex

7. Simply connected

ref boris beizer 33


1. Linear Boundaries

Interpreted linear inequalities

n-dim Hyperplane:

n+1 Points

n+1 + 1 Test Cases

Non-Linear

• Transform

ref boris beizer 34


2. Complete Boundaries

• Span the total number space (-, +)

• One set of Tests

Incomplete…

• Reasons

• Tests

ref boris beizer 35

Domain Testing - Domains & Paths - Nice domains

3.3. Systematic BoundariesSystematic Boundaries

• Linear Inequalities differing by a constant

fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c

• Parallel lines• Identical Sectors in a Circle

• DTDT• Test a domain Tests for other Domains

ref boris beizer 36


4.4. Orthogonal BoundariesOrthogonal Boundaries

• Two boundaries or, boundary sets

• Parallel to axes

• DTDT

• Each Set Independently

• # Tests O (n)

Uj

Vj

ref boris beizer 37


Orthogonal BoundariesOrthogonal Boundaries

• Tilted sets• transformation• Test cases: O(n)

• Concentric circles with radial lines

• Rectangular coordinates• Polar

r ≥ aj .AND. r < aj+1 .AND.

≥ j .AND. < j+1

ref boris beizer 38


Non-Orthogonal BoundariesNon-Orthogonal Boundaries

• Test Intersections O ( n2 ) + O (n)

ref boris beizer 39


5.5. Closure ConsistencyClosure Consistency

• A Simple pattern in all boundary closures

• Example

• Same relational operator for systematic boundaries

D1 D2D3 D4

ref boris beizer 40


6.6. Convex DomainConvex Domain

• Line joining any two points lies with in the domain

• DT• n on-points & 1 off-pt

ConcaveConcave

• “ But, However, Except, Or … “ in Specs• Handle with special care

ref boris beizer 41


7.7. Simply Connected DomainSimply Connected Domain

• In a single piece

• 2 complementary domains

• D1: Convex D2: Concave, not-Connected

• Programmer / Designer

• Convex part First

D1D2

D1D2

D3

ref boris beizer 42

Domain Testing - Domains & Paths – Ugly Domains

Generally,Generally,

From Bad Specs

Programmer / Designer Simplifies => Ugly to Good

possibility of Introduction of more bugs ?!?

ref boris beizer 43


CausesCauses

1.1. Non-linear BoundariesNon-linear Boundaries

1.1. Ambiguities & ContradictionsAmbiguities & Contradictions

1.1. Simplifying the topologySimplifying the topology

1.1. Rectifying boundary closuresRectifying boundary closures

ref boris beizer 44


1.1. Non-linear BoundariesNon-linear Boundaries

• Transform

ref boris beizer 45


2.2. AmbiguitiesAmbiguities

• Holes in input vector space.

• Missing boundary

• Detected by Specification languages & tools.

ref boris beizer 46


2.2. Contradictions..Contradictions..

• Overlapping of

• Domain Specs

• Closure Specs

D1 D2

D3

ref boris beizer 47


3.3. Simplifying the Topology….Simplifying the Topology….

• Complexity !

• Concavity, Holes, Disconnectedness

ref boris beizer 48



• Smoothing out concavity

• Filling in Holes

ref boris beizer 49



• Joining the pieces

Correct:

• Connect disconnected boundary segments

• Extend boundaries to infinity

ref boris beizer 50


4.4. Rectifying Boundary Closures.Rectifying Boundary Closures.

• make closures in one direction for parallel boundaries with closures in both directions

• Force a Bounding Hyperplane to belong to the Domain.

Consistent Direction Inclusion / Exclusion Consistency

ref boris beizer 51

Domain Testing - Domains & Paths – Domain Testing

General DT StrategyGeneral DT Strategy

1. Select test points near the boundaries.

2. Define test strategy for each possible bug related to boundary

3. Test points for a domain useful to test its adjacent domain.

4. Run the tests. By post test analysis determine if any boundaries are faulty & if so how?

5. Run enough tests to verify every boundary of every domain

ref boris beizer 52


DT for Specific Domain BugsDT for Specific Domain Bugs

Generally,

• Interior point

• Exterior point

• Epsilon neighborhood

• Extreme point

• On point

ref boris beizer 53


DT for Specific Domain BugsDT for Specific Domain Bugs

Domain D1

Epsilon neighborhood

Boundary point

Extremepoint

On Points

Off Points

ref boris beizer 54


1.1. 1-d Domains1-d Domains

2.2. 2-d Domains 2-d Domains

3.3. Equality & inequality PredicatesEquality & inequality Predicates

4.4. Random TestingRandom Testing

5.5. Testing n-dimensional DomainsTesting n-dimensional Domains

ref boris beizer 55


Testing 1-d Domains : Testing 1-d Domains : Bugs with open boundariesBugs with open boundaries

AABB

xx

AABB

xx

AABB

xxx1x1

AABB

xx x1x1

Closure Bug

Shift left Bug

Shift Right Bug

ref boris beizer 56


AABB

xxMissing Boundary

AABB

xxxx

Extra Boundary AABB

x1x1xx

CCxx xx

Testing 1-d Domains : Testing 1-d Domains : Bugs with open boundariesBugs with open boundaries

Bugs with Closed Boundaries : Similar to the aboveBugs with Closed Boundaries : Similar to the above

ref boris beizer 57


2.2. Testing 2-d DomainsTesting 2-d Domains

• Closure bug

• Boundary Shift : up / down

• Tilted Boundary

• Extra Boundary

• Missing Boundary

ref boris beizer 58


Strategy for domain testing in 2-dimStrategy for domain testing in 2-dim

2 n : Domains share tests 3 n : no sharing of tests by domains

ref boris beizer 59


3.3. Equality & Inequality PredicatesEquality & Inequality Predicates

• An Equality predicate defines a line in 2-d

c’c’cc

cc

AA

BB

aa

dd

bb

To avoid bugs

ref boris beizer 60


4.4. Random TestingRandom Testing

• A Point in the center : verifies computation

ref boris beizer 61


5.5. Testing n-Dimensional Domains (strategy)Testing n-Dimensional Domains (strategy)

n-dimensions, p boundary segments

• (n+1)*p test cases : n on points & 1 off point

• Extreme pt shared : 2 * p points

• Equalities over m-dimensions create a subspace of n-m dimensions

• Orthogonal domains with consistent boundary closures, orthogonal to the axes & complete boundaries

• Independent testing

ref boris beizer 62

Domain Testing - Domains & Paths – Domain Testing L5

Procedure for solving for valuesProcedure for solving for values

Simple procedure. Need tools.

1. Identify input variables

2. Identify variables which appear in domain-defining predicates, such as control-flow predicates

ref boris beizer 63

Domain Testing - Domains & Paths – Domain Testing L5

ProcedureProcedure

3. Interpret all domain predicates in terms of input variables:• Transform non-linear to linear• Find data flow path

4. Predicate expression with p # predicates. Find # domains : < 2 p

3. Solve inequalities for extreme points

4. Use extreme points to solve for nearby on points …

ref boris beizer 64

Domain Testing L5

EffectivenessEffectiveness

• Cost effective

• Bugs on boundaries, extreme points

• Hardware logic testing – tool intensive

ref boris beizer 65

Domain Testing L6

Domains & Interface TestingDomains & Interface Testing

• Domain

• Range

Function / Routine

Classify

DomainVariable Range

ref boris beizer 66

Domain Testing L6

Domains & Interface TestingDomains & Interface Testing

• Span compatibility

Routine 2Routine 1Domain

VariableRange for

Routine2

Domain for

Routine2

Range for

Routine1

ref boris beizer 67

Domain Testing L6

Interface Range/Domain Compatibility TestingInterface Range/Domain Compatibility Testing

• Test each var independently

• Find an inverse function

• Build a super domain

ref boris beizer 68

Domain Testing L7

Domains and TestabilityDomains and Testability

• Linearizing transformationsLinearizing transformations

• Polynomial• Rectangular to polar• Generic & rational => Taylor series

ref boris beizer 69

Domain Testing L7


Perform transformations for better testing.Perform transformations for better testing.

• Co-ordinate transformationsCo-ordinate transformations

ref boris beizer 70

Domain Testing L7


• Canonical Program FormCanonical Program Form

ref boris beizer 71

Domain Testing – Questions from the previous year’s examsU4

1. Explain Nice & Ugly Domains.

2. What is domain testing? Discuss applications of domain testing

3. Explain the domain boundary bugs for two dimensional domains

4. What is the purpose of Domain Testing? Give its schematic representation

5. Define the following concepts.1. Domains 2. Domain closure 3. Domain

dimensionality 4. Bug assumptions for Domain Testing

6. Explain simple domain boundaries & compound predicates.

7. Classify what can go wrong with boundaries, then define a test strategy for each case in domain testing.

STM-U4

Documents

Transcript of STM-U4