Test Case Design Methods

8/6/2019 Test Case Design Methods

1/51

TEST CASE DESIGNTECHNIQUES


2/51

OUTLINETest Case

Definition

Test case criteria Test case essentials

Test case syntax

Equivalent Partitioning

Boundary Analysis

Constraint Analysis

State Transition Condition Combination

2


3/51

TEST CASE

Definition:A test that (ideally) executes a single

well defined test objective (Testing ComputerSoftware Kaner, Faulk, Nguyen)

Definition: A specific set of test data and

objective (IEEE 729-1983)

3


4/51

WHY WRITE TEST CASE?

Accountability

Reproducibility Tracking

Automation

o n ugs

To verify that tests are being executed correctly

For compliance

To measure test coverage

4


5/51

TEST CASE CRITERIAAn excellent test case satisfies the following

criteria:

Reasonable probability of catching error

Neither too simple nor too complex

Best of its breed

Makes program failures obvious

5


6/51

How is a good test ?

Refers and maintains tight control over specific testdata

Has detailed enough Test Design Steps.

Is aware of the Testers ex erience

TEST CASE CRITERIA

Has clear criteria for Pass or Fail

6


7/51

A Not-so-Good Test ?

Leaves it up to the user to find test data Gives very high level instructions that leave too much

room for artistic interpretation

Does not consider the Testers ex erience

TEST CASE CRITERIA

Leaves out follow-up verification steps which make itdifficult to determine Pass or Fail criteria.

7


8/51

Checklist to review test cases

Is the test procedure accurate and explicit enough for anon-expert user?

Is any supporting documentation required (i.e.

TEST CASE CRITERIA

, ,

Do any test cases test multiple conditions?

Are test cases duplicated?

Are the scripts in a logical testing order?

8


9/51

TEST CASE ESSENTIALS

Things usually to include in a test case: Tracking Information

Test Case ID

Test Description

Purpose/Objective/Title

Procedures/Steps

Script

Parameters/Data

Input

Output Default

Call to system (printer, system clock, available RAM, APIs)

Expected Result

Observed Result

Pass/Fail/Blocked/Skipped Bug ID

Notes/Comments

Environment 9


10/51

Test case objective

The most important part of a test case is

the 1-line title describing the objective ofthe test

That 1-line title can be called


Test Name

Test Case

Test Objective

Test Goal/Purpose

10


11/51

Test case objective

It is most important because

It gives the reader a description and idea ofthe test

A good test name makes review easier


Easier to pass to another person Easier to pass to automation team

In many cases, may be the only part of a test

case documented

11


12/51

TEST CASE OBJECTIVE SYNTAX

Action + Function + Operating Condition

In which: Function may be function, feature, validation point Operating Condition may be data

VerifyTestValidateProveExecutePrintCalculateRunany action verb

12


13/51

TEST CASE EXAMPLES

Action Function Operating ConditionRun annual report from standard data (file location)

Run annual report on Day 1 of fiscal year

Run annual report from empty spreadsheetRun annual report on last day of fiscal year

13


14/51

VALIDATION POINTS

Tests need validation points

This is the expected result.Write it.

Define clearly state: what behavior, result or point

you are attempting to validate.

14


15/51

TEST CASE TEMPLATE

TC ID Description StepsExpected

Result

Observed

resultStatus

TC00

1

/*Test

objective*/

/*Very clear and

specific steps*/

Pre-condition:

/*you need to

pre determine

what your

/* write down

the result that

you get when

Passed/Failed

15

Steps:

1. Action 1

2. Action 2

..

program s

supposed to

do*/

excecute t e

test case*/


16/51

Test Case


Test case syntax

OUTLINE

16

Equivalent Class Partitioning

Boundary Analysis

Constraint Analysis



17/51

Test Case


Test case syntax

OUTLINE

17


Boundary Analysis

Constraint Analysis



18/51

EXAMPLE

Specification:

-This program isdesigned to add two

numbers, which you

18

will enter.-Each number

should be one or

two digits


19/51

There are 199 x 199 = 39,601 test cases for

valid values: definitely valid: 0 to 99

might be valid: -99 to -1

EQUIVALENCE CLASS AND

BOUNDARY ANALYSIS

19

There are infinitely many invalid cases: 100 and above

-100 and below

anything non-numeric


20/51


BOUNDARY ANALYSIS

Equivalence Class: Two tests belong to the

same equivalence class if the expected result ofeach is the same..

Boundaries: Mark the point or zone of

another. Generally, each class is partitioned by the

boundary values.

It is important to recognize that two tests are equivalentonly with respect to a specific risk.

20


21/51


22/51


BOUNDARY ANALYSIS

General Steps

Identify the classes Identify the boundaries

Identify the expected output(s) for valid

nput s Identify the expected error-handling (EH)

for invalid inputs

Generate a table of test cases (maximum, 9test cases for each partition of each class).

22


23/51


BOUNDARY ANALYSIS

Other Equivalence Classes and Special Cases

Negative valuesNumber of digit or characters

0

Non-printable characters or Upper ASCII (128-254)

Space

Nothing

Etc.

23


24/51

Specification: text

box with valid

values from 1 to9999 or 1 to 4-digit

value

EXAMPLE 2

24


25/51

Test Case


Test case syntax

OUTLINE

25


Boundary Analysis

Constraint Analysis



26/51

CONSTRAINT ANALYSIS

General Steps

Identify the dependent variables and studytheir respective constraints.

Identify all possible input and output

List the variables in a tabular format withtheir input/output interfaces and theirrespective constraints.

26


27/51

CONSTRAINT ANALYSIS

DATA RELATIONSHIPS

27

Result???


28/51

This table shows how the variables are related.

CONSTRAINT ANALYSIS

DATA RELATIONSHIPS

Here are the columns: Field: End Date, and Start Date are examples of

fields.

Entry Source: What dialog boxes can you use to

enter data into this field? Can you import data intothis field? Can data be calculated into this field? Listevery way to fill the field -- every screen, etc.

28


29/51

Display: List every dialog box, error messagewindow, etc., that can display the value ofthis field.

Print: List all the reports that print thevalue of this field (and any other functions

CONSTRAINT ANALYSIS

DATA RELATIONSHIPS

that print the value).Constrained By: List every variable that

constrains the value of this variable.

Constraint: List every variable that this one

constrains.

29


30/51

CONSTRAINT ANALYSIS

FUNCTIONALITY RELATIONSHIPS

The Let Windows

manage my virtualmemory settings

radio buttonInactive

Features

availability of thefeatures in the Let

me specify my own

virtual memory

settings group.

Active

Features30


31/51

Test Case


Test case syntax

OUTLINE

31


Boundary Analysis

Constraint Analysis



32/51

STATE TRANSITIONDefinition

Involves an analysis of the relationship among the

states and the events or actions that cause thetransitions from one state to another.

General Steps Identify all supported states

For each test case, define:The starting state

The input events that cause the transitions

The output results of each transitionThe end state

32


33/51

EXAMPLE 1Navigation Command

Current View Mode


34/51

Navigation CommandCurrent View Mode

34


35/51

TRANSITIONING EXAMPLE

35


36/51

Test Case


Test case syntax

OUTLINE

36

Equivalent Class Partitioning Boundary Analysis

Constraint Analysis



37/51

CONDITION COMBINATION

Condition Combination Involves an analysis of the combination relationship

of the variables.

Each combination represents a condition to be tested

Identify the variables. Identify the number of possible unique values of each

variable.

Create a table illustrating the complete uniquecombination of conditions.

37


38/51

COMBINATION

TEST CASE DESIGN


39/51

39


40/51

TEST METHODS

Some Common Test Methods and Test

Types Requirement Testing

Exploratory Testing

Regression Testing

40


41/51

TTESTEST MMETHODETHOD

DefinitionsDefinitions: A definitive procedure for the identification,

measurement, and evaluation of a material, product,

system, or service that produces a test result.

(ASTM American Society for Testing and Materials)

41


42/51

CCOMMONOMMON TTESTEST MMETHODSETHODS

ANDAND TTESTEST TTYPESYPES

Test approaches: Requirement-based Testing (Passive Testing)

Exploratory Testing (Ad Hoc Testing) .

The best practice is a combination of a few methods tosatisfy the project needs

Test Types Regression Testing

Smoke Testing

Stress Testing

Load Testing Performance Testing

42


43/51

RREQUIREMENTEQUIREMENT--BASEDBASED TTESTINGESTING

Requirement-based Testing is:

Testing the requirement

Testing the specifications

requirement and spec documentationThe primary goal of Requirement-based

Testing is RequirementValidation.

43


44/51

RREQUIREMENTEQUIREMENT--BASEDBASED TTESTINGESTING

The Mechanical Approach for Test development:

Analyze requirements

Make at least a test case for every requirement

Execute the test cases

44


45/51

EEXPLORATORYXPLORATORY TTESTINGESTING

Exploratory Testing can have many names: Ad hoc,

Discovery, Error Guessing

The primary goal of exploratory testing is to uncover

new defects in the product on-the-fly.

The task is finding ways to creatively design

experiments, exploring, experimenting, going from a

known place to an unknown place.

45


46/51

EEXPLORATORYXPLORATORY TTESTINGESTING

Goal:probe for weak areas of the program. Some

various error conditions should be checked are: Out of boundary

Null input and overflow (storage of the sum)

Non-numeric

Hardware and memory error handling

Exploratory Testing is the testing approach which

can produce the highest number of bugs

46


47/51

RREGRESSIONEGRESSION TTESTINGESTING

Most common type of work we do!Re-run

-

ExpensiveFirst to automate

47


48/51


Regression Testing: performed after making a

functional improvement or repair to the program.

Regression Purpose:

- eri y t at t e mo i ications ave not cause any

unintended effects

- Make sure that the bugs marked as fixed by

developers are really fixed.

Re-test all test cases can be done using automatedtesting tools.

48


49/51


Some of the most popular strategies for selecting

regression test suites:

Retest all: Rerun all test cases. Simple but impossible in the

time that we have in our everyday practice.

Retest Risk Use Cases: Choose aseline tests to re-run risk

heuristics.

Retest by Profile: Choose baseline tests to re-run by allocating

time in proportion to operational profile.

Retest Changed Segment: Choose baseline tests to re-run by

comparing code changes.

49


50/51

SSMOKEMOKE TTESTINGESTING

Definition: Asmoke test is a collection of written teststhat are performed on a system prior to being accepted

for further testing, also known as build verificationtest.

s s a s a ow an w e approac o e

application.

These written tests can either be performed manuallyor using an automated tool.

50


51/51

SUMMARYSUMMARY

Different test methods have different goals:

Projects with lots of documentation tend of userequirements based testing as the test

development methods.

ro ec s w e ocumen a on re y on a

hoc/exploratory testing. Projects in later phases of release (iteration,

functional increments) rely more on regression

testing.

51

Test Case Design Methods

Documents

Transcript of Test Case Design Methods