BDD for APIs

Jason Harmon

Jason Harmon• Blogger on apiux.com• API Architect @uShip

• Likes Python• Works in everything else

• Funny mustache• Loves weiner dogs

Some things to keep in mind…About 80s movies/tv

and software

Humans > Robots

Terminator taught us…

Agile

• INDIVIDUALS AND INTERACTIONS over processes and tools–Inside our organization and with our

users

Automationcan

be ourfriend too

Johnny Fivetaught us…

Agile

• WORKING SOFTWARE over comprehensive documentation–Not only in our product but our

testing & regression

Sometimesthere isno timeto plan

Always havea plan

Agile

• RESPONDING TO CHANGE over following a plan– APIs will change for your business,

better have tests

Work together!

Agile

• CUSTOMER COLLABORATION over contract negotiation– Get good at sharing ideas

What is BDD?

• “Behavior Driven Development”• English readable description of how our product

behaves• Agreed upon by the team: not Dev, QA or

product owned• Used to verify the acceptance criteria for

features in a product• Conceptually like TDD (test-first), but for

functional testing (not code architecture)

* Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley, 2009)

http://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/

Q2 Automated & Manual Tests

Functional TestsExamplesStory TestsPrototypesSimulations

Supp

ortin

g th

e Te

amBusiness Facing

BDD

Polyglot friendly

• Cucumber– Ruby-based, the granddaddy of BDD

• Cucumber-JVM– Java implementation, plays nice with Junit and all JVM

languages (Groovy is quite nice here)• Specflow

– .NET, Visual Studio integrated• Lettuce

– Python-based, very similar to Cucumber• Behat

– PHP-based

GherkinOne Description to Rule Them All

• https://github.com/cucumber/gherkin • Business Readable• DSL• Not code

Gherkin ExampleImperative Style

Scenario: Get nearby place nameGiven I use the geonames hostWhen I access the resource url

"/findNearbyPlaceNameJSON" And I provide parameter "username" as "jharmon" And I provide parameter "lat" as "30.4754724" And I provide parameter "lng" as "-98.1564068"

And I retrieve the JSON resultsThen the status code should be 200And it should have a list "geonames"And the list should have at least 1 item

Gherkin ExampleDeclarative Style

Scenario: Get nearby place nameGiven I use the geonames hostWhen I use the “Find Nearby Place by JSON” endpoint

And I identify my “geonames” user credentials And I provide coordinates 30.4754724, -98.1564068

And I retrieve the JSON resultsThen it should have at least 1 valid "geonames" item

Behavior Driven

• User experience trumps all• API is no exception• Treat API consumers like first class citizens• Describe how your API behaves from a

consumer perspective

Step Definitions

Scenarios“…until it’s green like a cuke”

Scenario outlines

ErrorsRed means stop

Tags

Lots of power to selectively run groups of tests

DEVE

LOPE

RS

TEST

ERS

STAK

EHOL

DER

S

PROD

UCT

Agreement

• Decide during planning how you will test for Acceptance Criteria

• …AS A TEAM…• Gherkin/feature definitions will allow a

ubiquitous language• Technically, this can get easier as the critical

mass of your API interactions is in your testing framework, and scenario steps are predictable

Productivity

• Stakeholders know what they’re getting• Devs know how to build it• Testers know how to test it• Product knows how to sign off• Documentation has a head start

Prod

uct

Dev

elop

ers

Test

ers

Stak

ehol

ders

Real world silos teach us…

Humans die inside silos!!!

Domain Language

• Focus on how you describe everything• Find agreement on how to describe behavior• Recognize when behavior is different than

before– “Wait, we’ve never deleted before!”

• Homework: Domain Driven Design

Share!

• All BDD tests can be committed to SCM, including branches

• Provide process which allows collaborative coding on building test framework

• Devs provide expertise on building a scalable test framework

• Testers utilize existing step definitions to quickly build out tests

Continuous Integration

• Regression is running all the time• Use tags to select the right depth of regression• Very fast identification of what’s broken

Samples

• Python/Lettuce– https://github.com/jasonh-n-austin/api-bdd-tests

• Groovy/Cucumber-JVM– https://github.com/jasonh-n-austin/TwitterRestTests

• .NET/SpecFlow– https://github.com/jasonh-n-austin/TwitterRestTests

Thank YouJason Harmon

@jasonh_n_austinhttp://jhr.mn