What is Test-Driven Development?A Primer on Testable Specifications…

Traditional Testing Approach

Requirements &

Design Specs

Develop Code,Compile Executable

Develop Test Scripts

ExecuteTest Scripts

Error Found!

Test Expectation≠Executable Implementation

WTFWe’re supposed to be releasing next week and you tell me this isn’t working? Get that bug fixed!

The objective is

NOT to kill bugs quickly,

but to

AVOID them altogether!

What if…Requirements Specs = Test Specs = Acceptance Criteria

Design Specs = Test Specs = Quality Criteria

We built the right thing.

We built the thing right.

Gre

ater

Det

ail

“A Constructive ApproachTo the Problem of Program Correctness”, 1968, Edsger Wybe Dijkstra

1968How do we know our programs are correct?

1) Construct a proof showing what the program should do

2) Develop code that meets the proof

REPEAT until doneREPEAT until done

In 1968, in fact not until the late 90s,

the tools

didn’t exist to implement Dr. Dijkstra’s vision.

Today, the Constructive Approach = Test Driven Development (TDD)

(Specs = Tests = Proofs)

Specs Tests CodeDevelopment

TestExecution

Gre

ater

Det

ail Acceptance Tests Stories/Functions

xUnit Tests Specific Detailed Methods/Functions

Mark as ready for test when completed; based on Product Owner priority

Test completed code

Write Test(s)

Execute Test to

Show Failure

Create Mock Data To

Show Passing Test(s)

Write Code to

Pass Tests

Why? Know test will show fail

Why? Tests = Specs

X Unit Tests

Recurse over this pattern!

Why? Know test will show pass (expected results)

Say you've got a method,

getName(String path)

that parses a file name from a file path.

You can write a JUnit test class to feed it a path and ensure that the results are what you expected.

What does an xUnit Test look like?

Simple example from jGuru.com

import java.io.File; import junit.framework.*; import junit.extensions.*;

public class ThingTester extends TestCase {

public ThingTester (String name) {

super (name); }

public static void main(String[] args) {

junit.textui.TestRunner.run(ThingTester.class); }

public void testGetName() throws Exception {

File myFile = new File("c:\\xxx\\yyy\\zzz.txt"); assertEquals("zzz.txt", myFile.getName());

} }

To exercise the test:

Compile & run ThingTester.

• Little Dot = Passing Test; • Assertion Failure & Stack Trace = Failure & location • Write test code BEFORE program code used to make it pass.

No problems? Add more test cases. Examples: long paths, files with spaces in the name, etc.Stop when you feel you have enough edge cases covered.

Tests should cover all critical methodsTests should cover all critical edge casesTests should cover all critical error detection

Goal ≠ 100% coverage, but Goal = 100% passing for what is covered

Understand Your Test Coverage

-in your working code!

Don’t

Forget!

(Including your tests!)

Write Test

Execute Test

to Show Failure

Create FeatureTo

Pass Test

Why? Know test will show fail

Why? Tests = Specs

Acceptance Tests

DescribeDesired

Behavior

DescribeFeature

Test Steps

Recurse over this pattern!

Underneath… …are Unit Tests

What does an Acceptance Test1 look like?

Say you've got a feature you want to implement to compute a factorial…

You can write a feature that is the proof of this function or story (including edge cases)

You also implement “steps” that poke at the implementation. As the implementation changes the steps change, but it is less common for the feature to change.

1aka Executable Specification

Example from tutorial at lettuce.it

Feature: Compute factorial In order to play with Lettuce As beginners We'll implement factorial

Scenario: Factorial of 0 Given I have the number 0 When I compute its factorial Then I see the number 1

Business Users can get this!

computefactorial.feature

from lettuce import *

@step('I have the number (\d+)') def have_the_number(step, number):

world.number = int(number)

@step('I compute its factorial') def compute_its_factorial(step):

world.number = factorial(world.number)

@step('I see the number (\d+)') def check_number(step, expected):

expected = int(expected) assert world.number == expected, \

"Got %d" % world.number

computefactorialsteps.py

To exercise the test: Run Feature (duh!)

Add more code to fix problems and re-run….

computefactorial.py

def factorial(number): return 1

by definition, we know that the factorial of 0 is 1So as a mock, we created a function that just returned 1

To exercise the test: Re-run Feature (duh!)

Need more than one case of course, so let’s add more test cases; i.e. feature scenarios

Feature: Compute factorial In order to play with Lettuce As beginners We'll implement factorial

Scenario: Factorial of 0 Given I have the number 0 When I compute its factorial Then I see the number 1

Scenario: Factorial of 1 Given I have the number 1 When I compute its factorial Then I see the number 1

Scenario: Factorial of 2 Given I have the number 2 When I compute its factorial Then I see the number 2

} In this case my steps & asserts were OK, sometimes you have to create more

To exercise the test: Rerun Feature (duh!)

Change code to fix problems and re-run….

computefactorial.py

def factorial(number): number = int(number) if (number == 0) or (number == 1):

return 1 else:

return number

To exercise the test: Rerun Feature (duh!)

Add a few more cases so we get realistic…

Feature: Compute factorial In order to play with Lettuce As beginners We'll implement factorial

Scenario: Factorial of 0 …

Scenario: Factorial of 1 …

Scenario: Factorial of 2 … Scenario: Factorial of 3 Given I have the number 3 When I compute its factorial Then I see the number 6

Scenario: Factorial of 4 Given I have the number 4 When I compute its factorial Then I see the number 24

} Again my steps & asserts were Ok this time, sometimes you have to create more

To exercise the test: Rerun Feature (duh!)

Change code to fix problems…

computefactorial.py

def factorial(number): number = int(number) if (number == 0) or (number == 1): return 1 else: return number*factorial(number-1)

To exercise the test: Rerun Feature (duh!)

Acceptance Tests work best when:

• Driven off of methods (functions) or several functions• Perform below the UI level

If needed, Acceptance Tests can:

• Be driven off of the UI (usually performs a little slower; must be done this way for something like PowerBuilder)

• Can be done manually

The specification independent of the flow or implementation. The steps that poke at underlying code may need modification over time.

-in your working code!

Don’t

Forget!

(Including your tests!)

How often?

For unit tests…

Every time code (which includes a test) is changed… This is probably multiple times a day and should be at a minimum daily.

For acceptance tests…

Whenever a test is completed and then the code is completed. The test may be completed on Tuesday, then code that pokes at the test will be completed as the skeletal object is completed say around Wednesday, and then perhaps Friday is when all the code for that poke is completed.

Test to Specification

Test to Failure

Product Design

Technical Design

Interaction Design

Acceptance Tests

xUnit Tests

UITests

ExploratoryTesting

Stress Testing

Usability Testing

Finite, Repeatable,Automated

Environmental, Less Automated

Diagram concept from p.76, Leading Lean Development, Mary & Tom Poppendieck, 2010

A Full Complement of Tests

The Customer Satisfaction Viewpoint…

Or why it matters…

For More Info…References (Books)• Specification by Example by

Godjo Adzic• Clean Code by Robert Martin• Ship It! By Jared Richardson

and William Gwaltney• JUnit in Action by Vincent

Massol

Google:Unit Testing <language of interest>

Unit Test Harnesses• JUnit (java)• Jasmine (JavaScript)• FlexUnit (Flex)• pyUnit (Python)• utPLSQL (PL/SQL)BDD (Acceptance Testing)• JBehave (Java)• Cucumber (Ruby)• Lettuce (Python)

UI Testing• Selenium