Tutorial: Implementing Specification-By-Example with Gherkin

CHRISTIAN HASSA ([email protected])

TWITTER: @CHRISHASSA

XP 2013 Vienna, June 3rd 2013

Specification-By-Example with Gherkin

About me

• Managing Partner at TechTalk• Working as/coaching Product Owners

TechTalk office Vienna/Austria

• Agile consulting and delivery• Offices: Vienna, Budapest, Zurich

TechTalk Team

About my Team

4

RefiningUser Stories

5

Impact Mapping

Story Mapping

Specification-By-Example

Establishing a shared understanding

Why?

How?

Code

AcceptanceCriterion

Epic

Capability

Impact, Goal

Easier to define upfront Harder to define upfront

Bugreport

Isolated,formalizedexample

Feature

User Story

Example

Reminderfor adiscussion

6

Collecting Acceptance Criteria

“I would try to put a book into the shopping cart …”

“I would try to remove a book from the shopping cart…”

“I’d check whether the shopping cart is empty, when I enter the shop …”

“I would try to add the same book again to the shopping cart …”

Books can be placed into shopping cart.

Books can be removed from shopping cart.

Shopping cart should be empty when entering the shop.

Adding same book again to shopping cart should increase quantity.

As a potential customerI want to collect books in a shopping cartSo that I can order several books at once.

“Imagine this story is already implemented:

how would you verify it?”

7

Example:Mobile Ticket for Wiener Linien

8

Exercise: Smart Alarm

• What would you try out when the following user story was done?

• Collect and write down the list of discussed acceptance criteria• Work in groups of 2-3

As a regular travelerI want to setup a smart alarmso that I can be warned about service disruptions

I want to setup multiple smart alarmsso that I can plan ahead for different travel plans I have

9

Usingexamples

10

UI wire frames,existing UI

rules, key examples

existing artifacts,samples

Examples for user stories

11

Specification-by-Example

Examples …• make abstract descriptions

better understandable

However …• examples are usually not formally

exchanged or documentedBrian Marick

Examples Tests

Requirements

consist of

describe verifyfulfillment of

12

Discussion of acceptance criteria

public void TestInitialOrderDiscount()

{

Customer newCustomer = new Customer();

Order newOrder = new Order(newCustomer);

newOrder.AddBook(

Catalog.Find(“ISBN-0955683610”)

);

Assert.Equals(33.75,

newOrder.Subtotal);

}

Register as “bart_bookworm”Go to “/catalog/search”Enter “ISBN-0955683610”Click “Search”Click “Add to Cart”Click “View Cart”Verify “Subtotal” is “$33.75”

We would like to encourage new users to buy in our shop.Therefore we offer 10% discount for their first order.

Original idea for the illustration: George Dinwiddiehttp://blog.gdinwidiee.com

13

… illustrated with formalized examples

Given the user has not ordered yet

When the user adds a book with the price of EUR 37.5

into the shopping cart

Then the shopping cart sub-total is EUR 33.75.


14

Discover hidden assumptions

Actually, this is not quite right:Books on sale should be excluded.


15

Collaboration: 3 amigos

“HappyPath”

Technical feasability

Exceptions, border cases


16

Abstract acceptance criteria

As a shop visitorI want to collect books in my shopping basketso that I can purchase multiple books at once.

Books can be added to the shopping basket

Books can be removed from the shopping basket

Shopping basket is initially empty

The same book can be added multiple times to the shopping basket

17

Examples for acceptance criteria



Given my shopping basket is empty

When I add the book “Harry Potter” to my shopping basket

Then my shopping basket should contain 1 copy of “Harry Potter”

18



Examples for acceptance criteria

Given my shopping basket contains 1 copy of “Harry Potter”


Then my shopping basket should contain 2 copies of “Harry Potter”


19


Structure of examples




Title: Describes intention/abstract acceptance criterion

Arrange: Context, initial state of the system

Act: Execution of the feature

Assert: Assertion of observable behaviour

And I should see the warning: “Book already existed in basket”

Triple-Aconstraint“Checks”

Chainingup steps

20

Gherkin

21

Feature: Description of feature or user story

Szenariogrundriss: Beschreibung des Akzeptanzkriteriums

Szenario: Beschreibung des Akzeptanzkriteriums

Gherkin Feature Files

Background: context for all scenarios in the feature file

Scenario: Description of acceptance criterion

Angenommen/Wenn/Dann: Automatisierte Szenario SchritteGiven/When/Then: automated scenario steps

Scenario Outline: Description of acceptance criterion

Angenommen/Wenn/Dann: Automatisierte Szenario SchritteGiven/When/Then: automated scenario steps with <place holder>

Examples: table with examples for <place holders>Examples: table with examples for <place holders>

Given: automated scenario steps

@tagname

@tagname

@tagname

@tagname

22

Feature

Feature: Description of feature or user story

#language:en-en@doneFeature: Shopping cartAs visitor of the web shopI want to collect books in a shopping cartso that I can- prepare a short list of books I want to order- combine multiple books in one shipment

@Tagname

23

Scenario

Scenario: Description of acceptance criterion

Angenommen/Wenn/Dann: Automatisierte Szenario SchritteGiven/When/Then: automated scenario steps

@Tagname

@inprogressScenario: The same book can be added multiple times to shopping cartGiven my shopping cart contains 1 copy of "Harry Potter"

When I add the book "Harry Potter" to the shopping cart

Then my shopping cart contains 2 copies of "Harry Potter"And a warning is displayed: "You have added the same book again"

24

Scenario OutlineScenario Outline: Description of acceptance criterion

Angenommen/Wenn/Dann: Automatisierte Szenario Schritte

Examples: table with examples for <place holders>

Given/When/Then: automated scenario steps with <place holder>

Examples: table with examples for <place holders>

@Tagname

@Tagname

Scenario Outline: Simple searchWhen I search for books by the phrase '<search phrase>'Then the list of found books should contain only: <books>

Examples:| search phrase | books | Explanation || Domain | 'Domain …' | whole words are matched || Analysis Communication | 'Bridging…', 'Analysis …' | multiple words are matched with OR |

Scenario Outline: Candidate profile can only be saved with <mandatory field>

When I enter a valid candidate profile, without filling in '<mandatory field>'Then I see the following validation error: 'field <mandatory field> missing'

Examples: Main fields| mandatory field || email || name || first name |

Examples: Extended fields| mandatory field || birthdate |

25

Background

Background: context for all scenarios in the feature file

Given: automated scenario steps

Background: Given the catalog contains the following books| Title | Author || Specification-By-Example | Gojko Adzic || Agile Testing | Lisa Crispin, Janet Gregory || Scrum Field Guide | Mitch Lacey |

26

Scenario step arguments

Given the catalog contains the following books| Title | Author || Specification-By-Example | Gojko Adzic || Agile Testing | Lisa Crispin, Janet Gregory || Scrum Field Guide | Mitch Lacey |

When I add the book "Harry Potter" to the shopping cart

Scalar arguments:

Table arguments:

27

Exercise

• Work in same previous groups of 2-3• Pick an acceptance criteria from the

previously compiled list• Outline usage examples with the following

structure• Given: Context, initial state• When: utilization of the feature• Then: assertions of observable behaviour

• Validate that scenario title (acceptance criterion) still describes intention of formalized scenario

28

Life time of

examples

29

Purpose of the examples

• Shared understanding:acceptance criteria

•Documentation:system details

• Regression-tests:violated assumptions

30

Continuous validation with automation




System

„Step Definitions“ are binding individual stepsto an automatable interface of the application.

Automatableinterface

UIAutomation

Automation does not necessarily have to bind to the UI.

Automatability of system is supported/evolving with development.

31

Demo

http://www.specflow.org

Gherkin automation for .NET• Visual Studio plugin (VS-Gallery)•NuGet Package

32

Some more complex exercises

Choose between• Best-fare option ticket• Smart Alarm

33

Best-fare option ticket

• Assumptions• Ticket price is always adjusted during

validation to the best option (considering all previous trips)

• Feedback about cost of travel and ticket option (type) after each validation

• Valid in the core zone 100

• Scope of the story• Different travel scenarios and how the

price adjusts• Billings resulting from the user’s travel• Feedback to traveler (cost of travel, status

of ticket) when validating ticket

• Out of scope for this story• Credit limit of traveler• Authentication of ticket (validity)

As an ad-hoc travelerI want my ticket to adjust to the best available fare whenever I validateSo that I don’t need to study upfront which ticket I need

Ticket Type Price

Single trip 2,10

24 hours 7,10

48 hours 12,40

72 hours 15,40

8 days 35,80

Week 15,00

Month 45,00

Year 365,00

34

Smart Alarm

• Assumptions• User can specify route to monitor for

disruptions• User can specify when disruptions

should trigger the alarm earlier (minutes before alarm)

• Scope of the story• Different possibilities of disruptions

provided by GTFS and their impact on activating the alarm

• Out of scope for this story• Configuring routes (the user is

selecting an already defined route)• Setting up alarms (the user has

already setup an alarm)

As a regular travelerI want to have a smart alarm that activates earlier when there are traffic disruptions on my routeSo that I can travel earlier to be still on time for my appointment

35

Exercise

Choose between• Best-fare option ticket• Smart Alarm

Collect and discuss scenarios•What would you try out?• Collect examples• Limit scope of the story

36

Exercise

Refine scenarios and formalizeinto Gherkin• Start with a “warm-up” scenario• Increase complexity with additional

scenarios• Reconsider structure of scenarios as you

refine them• Validate scenario title/description after

refinement

37

Impact on

testing

38

Test automation becomes expensivewhen …

• trying to automate

manual tests

• making tests

unreadable when

automating them

• automating after

completing

development

structure

readability

point in time

39

StructureManual tests

Asserts Multiple combined features

Structure ACT-ASSERT-ACT-ASSERT-ACT-ASSERT-…

Dependent featuresLong test path with high chance to breakCause and impact of error hard to trace

Automated Check

Single aspect of a single feature

ARRANGE –ACT –ASSERT

Independent featuresShort test path with lower chance to breakCause and impact of error easy to relate

40

Test automation pyramid

Userjourneys

Acceptance-criteria

Units

exploratory testing

Source: Mike Cohn

many

few

hard

easy

Automatability

41

// Go to web page 'http://localhost:40001/' using new browser instanceBrowserWindow localhostBrowser = BrowserWindow.Launch(

new System.Uri(this.RecordedMethod1Params.Url));

// Click 'Register found item' linkMouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9));

// Click 'Save' buttonMouse.Click(uISpeichernButton, new Point(44, 14));

int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9));

// Click 'Register found item' linkMouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7));

// Click 'Save' buttonMouse.Click(uISpeichernButton, new Point(34, 11));

int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9));

Assert.IsTrue(fundNr1 + 1 == fundNr2);

// Click 'Close' buttonMouse.Click(uICloseButton, new Point(26, 11));

Readability

42

A readable test caseScenario: New found items should receive a consecutive number for the current year

Given the previous found item of thecurrent year had the number 145

When I register a new found item

Then the last found item of thecurrent year should have the number 146

43

When to test (point in time)

Acceptance criteria(ATDD, BDD)

Unit Tests(TDD)

business view

technical view

Exploratory tests Workflow tests

Performance, Scalability, Usability,Security, …

def

inin

g th

e p

rod

uct

criticizing th

e p

rod

uct

New dimension: defining the productSynergy: Specification of requirements and tests

Agile Testing Quadrants: Brian Marick

44

Manual Testing is always necessary!

Userjourneys

Acceptance-criteria

Units

exploratory testing

Source: Mike Cohn

many

few

harder

easier

Automatability

ManualCheckafterStoryDone

Mainsuccesspathes

Undiscoveredacceptancecriteria

No/(few)manualregressionchecks

Few pathesare enough

More timefor exploration

47

Impact on

development

48

Workflow - TDD

49

Extending TDD for business

50

Transparency for stakeholders

In Progress

51

Current sprint report: all sprint scenarios

52

Starting with first scenario (AC)

53

Finishing the first scenario (AC)

54

Progressing scenario after scenario

55


56


57

Implementing user stories in parallel

58

First user story ready for testing

59

Manual testing can start even earlier

60

Work done already can break again

61

Work done already can break again

62

See what is temporarily not working

64

Livingdocumentation

65

Living documentation

Drill into Details(Gherkin scenarios)

Overview(Story Map)

66

User Stories vs. Features

Product/Sprint Backlog

User Story 1AccCrit 1

AccCrit 2

User Story 2AccCrit 3

AccCrit 4

Living Documentation

Feature 1

AccCrit 1

AccCrit 2

Feature nAccCrit 4

AccCrit mUser Story n

AccCrit 5

AccCrit m

AccCrit 3AccCrit 5

„Done“

• Future options of the system• Organized/refined according to

priority, value, effort, risk, ...• Next possible increments of

the product (units of work)

• Current state of the system• Organized/refined for

functional overview• Versioned and maintained

together with source code

67

Lessonslearned

68

Collaboration over specification

Documentation after conversation• Collaborative discovery• Should trigger new questions• Infinite number of examples• Readability and ability to automate

69

Collaboration smells

• Silos and too formal hand-overs• Formalization cannot follow implementation• Scenarios replace communication• Scenarios inhibit consideration of solutions• Scenarios inhibit discovery of new things

•Duty after development is done• Unreadable scenarios without value• High cost for creation and maintenance• Technical problems with automation

70

Level of automation

Controller

Business Layer

Data Layer

Model

View

Browserautomation

Trigger behaviourthrough controller

Assert behaviouron model, db, ..

Setup pre-conditionsthrough serviceinterfaces

Out-of-process

In-process

71

Test execution performance

• Grouping tests

• Current WIP

• Completed work• Database

• In-memory

• Templates for setup• Parallel execution• Smart execution order

72

Internal vs. external DSL

Example Source:Liz Keoghhttps://github.com/lunivore/tictactoe-java/blob/master/scenarios/com/lunivore/tictactoe/scenarios/Three_in_a_row_wins.java

73

Non-functional acceptance criteria

Given there are 100,000 users registered on the systemWhen I create a new accountThen I should be taken to my dashboard within 5ms

Given 1000 users are hitting the homepage simultaneouslyThen each user should get a response within 2ms

Matt Wynnehttp://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing

74

Tools

75

Gherkin automation tools

www.cukes.info

www.behat.org

Ruby, Java, JavaScript, C++

www.specflow.org

.NET, Mono, Silverlight, WP7

PHP

77

Books

Gojko AdzicBridiging theCommunication Gap

Gojko AdzicSpecification byExample

Elisabeth HendricksonExplore IT!

78 Christian Hassa: [email protected] - @chrishassa

Tutorial: Implementing Specification-By-Example with Gherkin

Technology

Transcript of Tutorial: Implementing Specification-By-Example with Gherkin