Test Case Design: Strategies, Techniques & Models Robin Brennan, Senior Consultant Quality Assurance...

Test Case Design:Strategies, Techniques & Models

Robin Brennan, Senior ConsultantRobin Brennan, Senior ConsultantQuality Assurance InstituteQuality Assurance Institute

Quality Assurance Institute 2

Objectives

• Present test strategies, techniques and models to support test case design

• Particular focus on Equivalence Partitioning, Boundary Analysis and Decision Tables


Why Test Case Design?• We cannot test everything• We need test strategies to design test cases for today’s

complex systems• We need systematic techniques to design input/output,

combinations, states, relationships and special testing• The design of the test models and test cases often

reveals defects from earlier phases• Models support test design at any level – method,

unit/class, integration, subsystem, system, user acceptance


Designing Test Cases

• Has a high probability of detecting errors

• Reduces the number of necessary test cases by more than one

• Covers additional test case possibilities

A GOOD TEST CASE…


Designing Test Cases

• Three components of a test case:

– Condition to be tested

– Expected result

– Procedure


Test Case Design – Strategies, Techniques & Models

• Equivalence Partitioning– Represents the equivalent classes of input & tests

a representative of each class

• Boundary Analysis– Determines the boundary conditions on, above

and below the edges of inputs and outputs

• Decision Tables– Represents the relationship between input

conditions and output actions


What Is Equivalence Partitioning

• A black box testing technique– Originally from “The Art of Software Testing”

by Glenford Myers (1979)

• Determines categories of inputs

• Used for both valid and invalid variables

• Can be utilized at any level of testing

• Can be utilized early in the life cycle


What is Equivalence Partitioning

• Incorporates, through inference, additional test cases– Reveals information beyond the set of input

values– Can reasonably assume that if the test uncovers

an error, the other items in the same equivalence class would result in the same error


How to Perform Equivalence Partitioning

• Develop a set of conditions to be tested– Identify each input condition from requirement

or design specifications– Partition each into two or more groups

(equivalence classes)• One Valid

• One or more Invalid


How to Perform Equivalence Partitioning (continued)

• Review all equivalence classes. If any components are not processed in the same manner, split the class into smaller classes

• Develop a minimal set of test cases– Test a representative of each category


Create an Equivalence Class Table• External input condition on the left

• A column each for valid and invalid classes

• Unique number assigned to each equivalence class

EQUIVALENCE CLASSES

EXTERNAL INPUT CONDITION

VALID EQUIVALENCE

CLASSES

INVALID EQUIVALENCE CLASSES


Defining the Equivalence Classes

1. Input descriptions that specify a range of values:

• Identify one valid equivalence class• Identify two invalid equivalence classes, one that

lies above the range and one that lies below the range



2. Input descriptions that define a number of values:

• Identify one valid equivalence class• Identify two invalid equivalence classes outside

each of the ends of the allowed number of values



3. Input descriptions that define a set of values and each is handled differently (discrete input):

• Identify one valid equivalence class for each discrete input in the set

• Identify one additional invalid equivalence class for a value not included in the set



4. Input descriptions that are “data typed”:• Identify one valid equivalence class for the

correct data type• Identify at least one invalid equivalence class

for a different data type



5. Input descriptions with specific mandatory conditions:

• Identify one valid equivalence class for the specific mandatory condition

• Identify one invalid equivalence class for a condition that is not mandatory


EQUIVALENCE CLASS TABLE


VALID EQUIVALENCE

CLASSES

INVALID EQUIVALENCE CLASSES

Job Code (1) 10-60 (2) <10 (3) >60 Office Location (4) Location GO1 (9) something else (5) Location S1 (6) Location S2 (7) Location S3 (8) Location S4 Employee Number (10) numeric (11) something other than numeric (12) special characters Telephone Number (13) one to five

telephone numbers (14) less than one telephone number

(15) more than five telephone numbers Catalog Number (16) first character

numeric (17) first character not numeric

SAMPLE EQUIVALENCE CLASS TABLE


Construct the Test Cases

• Construct as few as possible while still maintaining a reasonable level of confidence

• Avoid “error masking” by not combining invalid values in one test case– If more than one invalid condition is in a test case, an

earlier value may result in never processing a subsequent value


Guidelines

• Write a new test case including as many valid equivalence classes as possible, until all valid conditions have been covered

• Write a separate test case for each invalid equivalence class


TEST CASES DERIVED FROM EQUIVALENCE CLASSES


EQUIVALENCE CLASS

TEST CASE NO.

TEST CASE

Job Code, Office Location, Employee Number, Telephone Number, Catalog Number

1, 4, 10, 13, 16 1 41,GO1,53176,31037498457145639056,9A82437

Office Location 5 2 41,S1,53176,31037498457145639056,9A82437




Job Code 2 6 3,GO1,53176,31037498457145639056,9A82437

Job Code 3 7 87,GO1,53176,31037498457145639056,9A82437


Employee Number 11 9 41,S3,abcde,31037498457145639056,9A82437

Employee Number 12 10 41,S3,%&$#@,31037498457145639056,9A82437

Telephone Number 14 11 41,S3,53176,,9A82437 Telephone Number 15 12 41,S3,53176,31037498457145639056

714575493031035478922134347862213327903531037465897148725430,9A82437

Catalog Number 17 13 41,GO1,53176,31037498457145639056,A47236D


What Is Boundary Analysis• Another black box testing technique

• Complimentary with Equivalence Partitioning

• Higher return-on-investment than random selection

• Can be used at any level, early in the life cycle

• Sounds simple but is often used ineffectively

• Includes both input and output domains

• Includes values on, above and below the edges


Guidelines for Input ValuesOne or more elements are selected from the

edges• For continuous input, select the highest and

lowest value within the range, and select the values just above and below the edges

• For discrete ordered sets of input, select the first and last elements in the set

• For a specific number of input values, select the minimum and maximum number of values, and select a number of values one below and one beyond the valid number of values


Guidelines for Output ValuesOne or more elements are generated from the

edges• For continuous output, generate the highest and

lowest value within the range, and generate the values just above and below the edges

• For a discrete ordered set of output, ensure the first and last elements in the set are processed

• For a specific number of output values, generate the minimum and maximum number of values, and generate a number of values, one below and one beyond the valid number of values


TEST CASES DERIVED FROM BOUNDARY ANALYSIS


EQUIVALENCE CLASS & BOUNDARY

ANALYSIS

TEST CASE NO.

TEST CASE


1, 4, 10, 13, 16; highest valid Job Code, one Telephone No.

*1 60,GO1,53176,3103749845,9A82437


1, 4, 10, 13, 16, lowest valid Job Code, five Telephone No’s.

**2 10,GO1,53176,31037498457145639056714575493031035478922134347862,9A82437





Job Code 2; just below edge *7 9,GO1,53176,31037498457145639056,9A82437

Job Code 3; just above edge *8 61,GO1,53176,31037498457145639056,9A82437


Employee Number 11 10 10,S2,abcde,31037498457145639056,9A82437

Employee Number 12 11 10,S2,%&$#@,31037498457145639056,9A82437

Telephone Number 14 12 10,S2,53176,,9A82437 Telephone Number 15; one beyond valid

number of values *13 60,S2,53176,3103749845714

56390567145754930310354789221343478622133279035,9A82437

Catalog Number 17 14 41,GO1,53176,31037498457145639056,A47236D

* Changed values of Test Case ** Additional Test Case

Boundary

Analysis

Examples


Decision Tables• Graphical representation of conditional logic

• More effective than narratives for communicating information

• Can be used at any level

• Used in many software engineering methodologies as a design tool

• Constructed from specifications, use cases, class diagrams, behavior diagrams, source code

• Handles combinations of Equivalence Classes


Decision Tables for Testing

• Forces consideration of all combinations of conditions and their resultant actions

• Helps develop all variations avoiding omission

• Helps determine irrelevant test conditions

• Results in fewer test cases while retaining good test coverage


Use a Decision Table when…

• Distinct input conditions result in a distinct output action

• Conditions can be modeled by:– Mutually exclusive Boolean expressions– Binary response (Yes/No; True/False)

• Results do not depend on the order of input conditions

• Results do not depend on prior input or output


Decision Table Format

TITLE RULES CONDITIONS

CONDITION ENTRIES

ACTIONS

ACTION ENTRIES


Decision Table Format - Vertical

TITLE RULES CONDITIONS

CONDITION ENTRIES

ACTIONS

ACTION ENTRIES

TITLE: TITLE: Unique Name or NumberUnique Name or Number

to Identify the Tableto Identify the Table

RULES: RULES: Each Unique CombinationEach Unique Combination

of Conditions & Actionsof Conditions & Actions(single column for each Rule)(single column for each Rule)

CONDITIONS: CONDITIONS: All ConditionsAll Conditionsto be Examinedto be Examined

CONDITION ENTRIES: CONDITION ENTRIES: Decision Entry for the ConditionDecision Entry for the Condition

(Yes/No; True/False)(Yes/No; True/False)

ACTIONS: ACTIONS: All Actions that CanAll Actions that Can

Occur for the ConditionsOccur for the Conditions

ACTION ENTRIES: ACTION ENTRIES: Resultant ActionResultant Action

Based on the ConditionsBased on the Conditions


Steps to Create Decision Tables

1. Identify the conditions and their entries

2. Identify the actions (results)

3. Determine the maximum number of Rules using the following formula:

2n …where n is the number of conditionse.g., 3 conditions: 23 = 8 rules

4 conditions: 24 = 16 rules


Steps to Create Decision Tables4. Divide the “Entries” side into the # of Rules

Title

1 2 3 4 5 6 7 8

Conditions

Actions


Steps to Create Decision Tables5. For maximum number of rules:

– The entries for condition 1 will be 50% “Yes” and 50% “No”

– For condition 2, the entries will be a pattern of 25% “Yes”and 25% “No”, repeated across the table

– This pattern continues for each subsequent condition…for example, condition 3 would have a pattern of 12.5% “Yes” and 12.5% “No”

6. Determine the action entry for each column



Title

1 2 3 4 5 6 7 8

Condition1 Y Y Y Y N N N N

Condition2 Y Y N N Y Y N N

Condition3 Y N Y N Y N Y N

Action1 x

Action2 x x x x x x x



7. Collapse the table by eliminating conditions that don’t need to be tested, as follows:

– When two rules have the same action entry and their condition entries only vary by one answer, eliminate the condition where it varies

– Next, analyze the table for redundancies and combine any rules that are now the same

– Repeat this process until all unnecessary tests have been eliminated



8. Confirm the accuracy of the table with the following calculation:

Number of Rules = n + 1

where n is the number of conditions


Sample Decision Table

Automobile Product Reorder Criteria:

“Reorder the model if the current inventory of the model is less than 20 automobiles, the average sales is four automobiles a week or more, and the average sales profit on those sales is 7% or more”


Sample Decision TableAutomobile Product Reorder:

1 2 3 4 5 6 7 8

Have 20 Units Y Y Y Y N N N N

Sold Average of 4/week

Y Y N N Y Y N N

Profit 7% Y N Y N Y N Y N

Reorder Auto X

No Reorder X X X X X X X



1 2 3 4 5 6 7 8

Have 20 Units Y Y Y Y N N N N


Y Y N N Y Y N N

Profit 7% Y N Y N Y N Y N

Reorder Auto X

No Reorder X X X X X X X



1 2 3 5 7

Have 20 Units Y Y Y N N


Y Y N Y N

Profit 7% Y N Y Y Y

Reorder Auto X

No Reorder X X X X



1 2 3 5

Have 20 Units Y Y Y N


Y Y N Y

Profit 7% Y N Y Y

Reorder Auto X

No Reorder X X X


Decision Tables to Test Cases… After creating the decision table…

• Verify the decision table• Create one test case for each unique Rule• Cross-reference with Equivalence Partitioning

test cases and eliminate redundancy• Use boundary-analysis to determine values

for the condition entries• Vary the order of the input variables• Mix the sequence of test orders on different

test executions


Test Case Design – Wrap Up• Evaluate risk, complexity, frequency of use,

inheritance, dependencies, variables, combinations, test time and budget

• Prioritize critical & complex components

• Use equivalence partitioning, boundary analysis, decision tables and other test design strategies and models to determine test cases

• Combine several test design strategies to ensure acceptable test suite coverage


Test Case Design Strategies

• Risk-based– Results of risk analysis

• State-based– State of the objects– Transition paths– State of the environment


Test Case Design Strategies

• Interaction-based– Interactions between components

• Specification-based– Functional; requirements; black box

• Implementation-based– Structural; code-based; white box


Test Case Design Strategies• Use Case Scenarios

– Primary flow– Alternate flows– Exceptions– Extensions

• CRUD– Create, Read, Update, Delete


In Summary• Designing good test cases is hard work

• With a systematic approach, we can design fewer test cases while still ensuring a high probability of detecting errors

• A combination of strategies, techniques and models must be utilized

• Utilize checklists to support various types of testing


In Summary• Use Equivalence Partitioning and Boundary

Analysis for input and output variations • Use Decision Tables for combinations of

conditions• Use State Charts for testing combinations of

message sequencing & instance variables• Use UML diagrams to identify relationships• Use Cases are an excellent source of

information for functional/system testing

Thank You

Test Case Design: Strategies, Techniques & Models Robin Brennan, Senior Consultant Quality Assurance...

Documents

Transcript of Test Case Design: Strategies, Techniques & Models Robin Brennan, Senior Consultant Quality Assurance...