Test Design Made Easy (and Fun!)

Rik Marselis 5 November 2015

Let’s walk the testing talk

Question 1

Applying Test design techniques is

easy

Question 2

Applying Test design techniques is

fun

Question 3

How many Test Design Techniques exist?

Test design techniques & coverage types TMap NEXT • Equivalence classes • Boundary value analysis • Decision points (CC, DC, CDC, MCDC,

MCC) • Orthogonal arrays • Pairwise testing • Data cycle test (CRUD) • Statistical usage (op.prof. • Right paths / Fault paths • Checklist • Decision table • Data Combination Test • Elementary comparison tst • Process Cycle test • Real-life test • Semantic test • Syntactic test • Use case test • Error Guessing • Exploratory testing • Reviews • Walkthroughs • Inspections

ISTQB • Equivalence partitioning • Boundary value analysis • Decision table testing • Cause Effect graphing • State transition testing • Use case testing • User story testing • Structure based (SC, CC, DC, CDC,

MCDC, MCC) • Error Guessing • Exploratory testing • Orthogonal arrays • Pairwise testing • Domain analysis • Defect based techniques • Checklist • Combinatorial / Class. tree • Informal review • Technical review • Walkthrough • Inspection • Static analysis

Torbjorn Ryber • Exploratory testing

(this is an approach, not a technique) • Inspection • Walkthrough • Technical review • Informal review • Modelling • Equivalence partitions • Boundary values • Domain tests • Business process testing • Use case testing • State based testing • Decision tables & trees • Elementary comparison • Combinatorial testing • Data cycle testing • Syntax testing • Time cycle testing • Program Logic (SC, BC, PC) • Data flow testing

Boris Beizer • Control-flow testing

• Loop testing

• Data-flow testing

• Transaction-flow testing

• Domain testing

• Syntax testing

• Finite-state testing

And more…

Question 4

How many Test Design Techniques do you apply in your day-to-day work??

Question 5:

Which test design techniques do you use in your day-to-day work?

TMap HD test approaches

Why coverage?

What’s your testing challenge? Select from 4 groups of Coverage types

Better know & use one technique of each group

than

all techniques of only one group

Why experience based?

So it needs a skilled tester !

Use results of previous tests to create new tests

Always use a mix

What’s your testing challenge? Select an experience based approach

Largely intuitive & ad hoc

Structured, chartered

Experience brought together

Checklist: Test Heuristics Cheat Sheet • Abstract (remove details) • Never and Always • Interrupt • Reverse • Some, none, all • Starve • Too few, too many

Source: “Explore It!” by Elisabeth Hendrickson, appendix 2

• Useful approximations • Violate data format rules • Zero, one, many • Zoom • Back, forward, history • Bookmark it

More checklists from Explore it!

Numbers:

• Zero

• One

• Many

• Too many

• Too little

Selecting:

• None

• One

• Some

• Many

• All

Timing:

• Always

• Sometimes

• Never

CRUD:

• Create

• Read

• Update

• Delete

Source: “Explore It!” by Elisabeth Hendrickson

What is the goal of testing?

• To report about quality and risks

• To build confidence

• So you need to log and report

Example of a test log

How to mix test approaches

• Create an Exploratory Testing charter (this can be done long before the test object is available)

• Use coverage based techniques as “test ideas” (thus your start-set of test cases can be made long before …)

• Execute your test based on prepared test cases complemented with exploratory testing

Charter

Explore <target>

With <resources>

To discover <information>

simple

Extensive:

Source: “Explore It!” by Elisabeth Hendrickson, page 13

Use a tool to get a start

Source: the test design tool “COVER” of Sogeti

Calculate Public Transport tariff

• Based on age, profession and discount card the tariff for public transport is calculated.

SPRINT 1: Calculate Transport tariff

• What kind of testing challenge is this?

Assignment: Test the public transport tariff decision

• Think of some test idea’s

• Use the DCT test cases as a starting point

• Execute the test

• Use the logsheet Groups of 2-3

20 minutes

Retrospective

• What is your conclusion about quality & risks?

• How much confidence do you have?

• Have you found defects?

Coffee break

What is the goal?

• With preparing test ideas and some test cases; You have a good start for exploring

– Most people get better coverage than with purely experience based test cases

– Proof of coverage at the spots where you need it

– Faster progress of testing compared to only exploring

– It fuels your imagination for the real exploration

Workshop: The Triangle exercise

In his 1979 book “The art of software testing” Glenford J. Myers starts with a simple test to see if you are suited as a tester.

The challenge is to test an application that gets 3 integers as input which represent the lengths of the sides of a triangle. Then the program tells what type of triangle it is.

So we start thinking…

• What is/are our challenge(s)?

• What possible inputs?

• What possible outputs?

• What’s the process?

• How can we test this?

• How can we be efficient?

Process for triangle application

Input check

start

A = B ?

A = C ?

A = C ? B = C ?

A + B < C OR A + C < B OR B + C < A ?

Input numeric

?

Not a triangle

yes

yes

yes yes yes

yes

no

no

no no no

no

Inspired by Glenford J. Myers

Isosceles Equilateral Scalene Not a triangle

Exercise 2a: Create a charter

• Include at least 3 test design techniques

Explore <target> With <resources> To discover <information>

Example of an extensive charter

Source: “Explore It!” by Elisabeth Hendrickson, page 13

Example of a simple charter

Groups of 2-3 5 minutes

Use MCDC for the complex decision

FALSE (0) (scalene triangle) X Y Z

TRUE (1) (not a triangle) X Y Z

(X) A + B < C

(Y) A + C < B

1 . . . 1 . . . 1

0 . . . 0 . . . 0 (Z) B + C < A

0 0

0 0

0 0

0 0

0 0

0 0

D2.1

D2.2

D2.3

D2.4

Result = Condition X OR Condition Y OR Condition Z

Neutral value (AND = 1, OR = 0 ) 0 0

MCDC = Modified Condition Decision Coverage

Exercise 2b: Execute your charter

Test the triangle application using:

• Your charter

• Your test ideas and tools

• The logsheet

Groups of 2-3

20 minutes Note: application and handouts will be provided

Retrospective

• What is your conclusion about quality & risks?

• How much confidence do you have?

• Have you found defects?

• How many of the tests that Glenford Myers has listed, did you do?

Tooling: Of course ! But beware…

Source: the test design tool “COVER” of Sogeti

Conclusion

• Exploratory testing can be very well combined with test design techniques

• Exploratory tests can be prepared before test execution starts

• Combining scripted and exploratory tests can be both easy and fun!!

Presentation will be available at EuroSTAR-site and on Slideshare.

Rik.Marselis@Sogeti.com

The end. Thank you !!

Happy testing

This whole presentation copyright © Sogeti 2015

Rik.Marselis@Sogeti.com