What Makes a Good Test Case

8/7/2019 What Makes a Good Test Case

1/19

4/22/2011

1

WHAT MAKES A GOOD TEST CASE

By

Shine K Raj

System Analyst Software testing

[email protected]

A set of test inputs, execution conditions, andexpected results developed for a particularobjective, such as to exercise a particularprogram path or to verify compliance with a

specific requirement. (IEEE)

In my view, a test case is a question that you askof the program. For example whether theprogram will pass or fail the test.

WHATS A TEST CASE?


2/19

4/22/2011

2

TEST CASES: OBJECTIVES

Detect as many defects as possible

Ensure good test coverage

Block premature product releases

Help managers make ship / no-ship decisions

Assess conformance to specification

Conform to regulations

Minimize safety-related lawsuit risk

Find safe scenarios for use of the product

Assess quality Assure quality

Minimize test development, execution andmaintenance costs

CONTENTS IN TEST CASE

test case ID

test case description

test step or order of execution number

related requirement(s) test category

author

check boxes for whether the test is automatableand has been automated.

result: Pass/ fail/ blocked/ untested/ invalid

remarks


3/19

4/22/2011

3

TESTCASES

InformationObjectives

Test

Attributes

Factors Involved in Test Case Quality

FACTORS IN DETAIL

For These Objectives:

Find bugs that will be considered relevant

Get these bugs fixed

Attributes: One test is better than another if it is:

More powerful

More likely to yield significant (more motivating, morepersuasive) results

More credible

Representative of events more likely to be encountered bythe user

Easier to evaluate.

More useful for troubleshooting

More informative

More likely to help the tester or the programmer developinsight into some aspect of the product.


4/19

4/22/2011

4

Type Source

1.Requirement Based Specifications2.Design based Logical system3.Code based Code

4.Extracted Existing files or testcases5.Extreme Limits and boundaryconditions

SOURCES

1. Identify the basic cases that indicateprogram functionality.

2. Create a minimal set of tests to cover allinputs and outputs.3. Breakdown complex cases into single

cases.4. Remove unnecessary or duplicate

cases.5. Review systematically and thoroughly.

Steps for selecting test cases:

Requirement based test cases


5/19

4/22/2011

5

Looks for exceptional conditions,extremes, boundaries, and abnormalities.

Requires experience, creativity of the Test Engineer

Need:

Extreme cases

Extracted cases involved extractingsamples of real data for the testingprocess.

Randomized cases involved using toolsto generate potential data for the testingprocess.

Extracted and Randomized cases


6/19

4/22/2011

6

1. Equivalence Partitioning

2. Boundary Value Analysis

3. Error Guessing

Test case Design methods

Equivalence classis a subset of data that isrepresentative of a larger class

Divide input domain into equivalence classes

Attempt to cover classes of errors

One test case per equivalence class, to reduce total

number of test cases needed

InputInputDomainDomain

Equivalence partitioning


7/19

4/22/2011

7

A program which accepts credit limitswith a given range Say,

$10,000 $15,000

This would have three equivalence classes:-

1. Less than $10,000 (Invalid)

2. Between $10,000 and $15,000 (Valid)

3. Greater than $15,000 (Invalid)

Example

Less than 10,000 Between 10 and 15 More than 15,000

9800 12500 18000

Equivalence Partitions


8/19

4/22/2011

8

Complements equivalence partitioning , selects thetest cases at the edge of equivalence classes.

In practice, more errors found at boundariesofequivalence classes than within the classes

Divide input domain into equivalence classes

A technique that consists of developing test casesand data that focus on the input and outputboundaries of a given function

Boundary Value Analysis

In the same credit limit example:the boundary analysis would test

1. low boundary plus or minus one ($ 9,999 and $10,001)

2. On the boundary ($10,000 and $15000)

3. Upper boundary plus or minus one ($14999 and $15001)

Example


9/19

4/22/2011

9

1. Value immediately below range

2. First value of range

3. Second value of range

4. Value immediately below last value of range

5. Last value of range

6. Value immediately above range

Range of Boundary Values

Based on the theory that test cases can be

developed based upon the intuition and

experience of the Test Engineer.

In an example where one of the inputs

is the date and the Test Engineer

may try February 29, 2000 or 9/9/9999

Error Guessing


10/19

4/22/2011

10

TEST CASE DESIGN TECHNIQUES

Special value testing-

Selecting data based on the basis of the features of thefunction to be computed is called special value testing.This is applicable mainly to mathematical computations

TEST CASE DESIGN-BEST PRACTICE Best practice is to develop at least two test cases for

each requirement for test:

a test case to demonstrate the requirement has beenachieved, often referred to as a positive test case,

another test case, reflecting an unacceptable, abnormal, orunexpected condition or data, to demonstrate that therequirement is only achieved under the desired condition,referred to as a negative test cases.


11/19

4/22/2011

11

To Capture Test Case Details:1.Testcase ID (should be unique, eg: c_01.1, c_01.1a,c_01.2,)

2.Feature functionality to be tested (eachRequirement/feature could be from Usecase)

3.Test Description/ test input details (test input, testdata, action to be performed to test the feature,complex test cases be split to more than one)

4.Expected behavior ( in messages, screens, data, tobe with correct details)

5.Actual and Status

SUBJECTIVE or OBJECTIVE


12/19

4/22/2011

12

23

TEST CASE LIFE CYCLE

Main idea: test cases as products. Therefore testcases have a life cycle.

Create Draft Review

Deleted Deprecated

Released

Update

24

Create: The create phase enters the followinginformation: Test case ID Requirements IDs Title Originator group Creator Test category

Draft: A test engineer enters the followinginformation: Author of a test case Test objective Environment Test steps Clean up Pass/Fail criteria Candidate for automation Automation priority


13/19

4/22/2011

13

25

Review: Here, the creator is the owner

The owner invites test engineers and developers toreview the test case Ensure that the test case is executable and

Pass/Fail criteria are clearly stated Changes may occur Once the test case is approved, it is moved to

Released state.

Released The test case is ready for execution Here, the owner is the test organization

Review the test case for re-usability If there is a need to update the test case, move it to

update state.

26

Update Strengthen the test case as the system functionality or

environment changes

By executing the test case 2/3 times, one may update the testcase to improve reliability

One gets an idea about its automation potential

A major update calls for a review of the test case.

Deleted: If the test case is not a valid one.

Deprecated: If a test case is obsolete, move it to thisstate. A test case becomes obsolete for severalreasons: System functionality has changed, but test cases have not

been properly maintained

Test cases have not been designed with re-usability in mind

Test cases have been carried forward due to carelessness;long after their original justification has disappeared.


14/19

4/22/2011

14

27

CREATION OF A TEST SUITE

Test suite: this is a set of test cases organized in ahierarchal

manner for a certain software release. Two characteristics ofa test suite:

Test cases in a test suite relate to a certain release of thesystem. This is important because not all functionalities aresupported in all the different versions of a system.

Test cases are organized in a hierarchal manner for threereasons: To obtain a balanced test suite To prioritize their execution To monitor the progress of system testing.

28

TEST SUITE ORGANIZATION

Basic Functionality Robustness

Test Suite

Performance

User Admin Maint.

Local Long Dist. Pre-paid Credit CardIntl.

Local

.


15/19

4/22/2011

15

29

TEST CASE RESULT

Untested

Passed

Failed

Blocked

Invalid

Execution is successful and thepass criteria is verified.

Test case successfully executed;Pass criteria not achieved.Report the bug.

A bug prevents the execution of

the test case. Report the bug.

Test case not applicable for thisrelease.

30

DRAFTING A TEST CASE

There are more than 3 classes techniques for generatinginput/output behavior of test cases for functional testing: Analyzing the requirement specification (internal description) and

functional specification (formal description)

Analyze the input and output domains Analyze the source code (for completeness).

For the other categories of test cases, test engineers have to becreative. And, to be creative, remember the basic structure of atest objective: Verify that the system performs X correctly.

Now you can modify the above objective to identify anotherobjective: verify that the system performs X correctly, in a givenenvironment Y.

Here is a pictorial view of the idea of test generation


16/19

4/22/2011

16

31

TEST GENERATION

Requirement

InputExpectedOutcomeSystem

Environment

ERROR-GUESSING

always worth including

after systematic techniques have been used

can find some faults that systematic techniques can

miss a mopping up approach

supplements systematic techniques

Not a good approach to start testing with


17/19

4/22/2011

17

ERROR GUESSING: DERIVING TEST CASES

Consider:

past failures

intuition

experience

brain storming

What is the craziest thing we can do?

HOW DO YOU RECOGNIZE A GOOD TESTCASE?

A test case must be:

Clear of purpose

Well organized

Reviewable Maintainable

Complete

Re-usable

Useful to other testers

Easily Scheduled

Efficient


18/19

4/22/2011

18

CONCLUDING NOTES Theres no simple formula or prescription for

generating good test cases. The space ofinteresting tests is too complex for this.

There are tests that are good for yourpurposes, for bringing forth the type ofinformation that youre seeking.

Given a purpose, we can evaluate tests as betteror worse along several dimensions, in terms ofhow they advance that purpose.

Test types differ along the dimensions. Many test groups stick with a few types of tests. To

achieve the broad range of value from our tests, wehave to use a broad range of techniques,consciously selected to help us achieve ourinformation goals.

DOUBTS PLEASE.......


19/19

4/22/2011

THANKS

What Makes a Good Test Case

Documents

Transcript of What Makes a Good Test Case