Iqnite 2012 Agile Test Automation - A Behaviour Driven Approach

7/30/2019 Iqnite 2012 Agile Test Automation - A Behaviour Driven Approach

1/27

Agile Test Automation

a Behaviour Driven Approach

Ray Hua

2


2/27

1. A journey of test automation

2. Some lessons learned


3/27

2 years ago

No automation

NowBDD

Skilled testers

Manual

testers


4/27

A commercial tool

Some contractors

A project delivery approach

Half a million dollars

Some automation


5/27

A mix of open-source & commercial tools

In-house R&D

One automation engineer

BAU delivery

Some automation

that didnt work

An in house solution


6/27

Start redesigning test framework

Start training testers

Some automation

that didnt work

In house solution QA team on board


7/27

7 projects

90% functional coverage

Migrated to Selenium/Grid 2, more VMs

Code migration completed by testers

Some automation

that didnt work

In house solution QA team on board Grow regression

suite and upgrade

technology


8/27

Executable requirements

BDD engine for .net SpecFlow

BA/PM on board

BDD automation, scripted by testers

Some automation

that didnt work

In house solution QA team on board Grow regression

suite and upgrade

technology

Behaviour

Driven

Development


9/27

C#

Visual Studio

TFS

Nunit

Resharper

Selenium/Grid

Team City

Vmware


10/27

Behaviour Driven?

A specialized version of TDD

Name of test methods should besentences

Readable when test fails

A ubiquitous language for analysisExecutable acceptance criteria


11/27

Tools

Java JBehave.Net SpecFlow

Ruby Cucumber/RSpec


12/27

A BDD scenario:

Given the account is overdrawn

And the card is validWhen the customer requests cash

Then ensure a rejection message is displayed

And ensure cash is not dispensed

And ensure the card is returned.


13/27

BDD engine generates stubs (SpecFlow):

[Given(TheAccountIsOverdrawn)]

Public void TheAccountIsOverdrawn (){}

[And(TheCardIsValid)]

Public void TheCardIsValid (){}

[When(TheCustomerRequestsCash)]

Public void TheCustomerRequestsCash (){}

[Then(EnsureARejectionMessageIsDisplayed)]

Public void EnsureARejectionMessageIsDisplayed (){}


14/27

[Given

(TheAccountIsOverdrawn)]Public void TheAccountIsOverdrawn (){

setupAccount(overdrawn);}

[And(TheCardIsValid)]

Public void TheCardIsValid (){

setupCard(valid);}

[When(TheCustomerRequestsCash)]

Public void TheCustomerRequestsCash (){aCustomer.Setup(standard);

customer.requestCash();}


15/27

The team lovesReadable test report for PM with colours

indicating pass, fails and in-progress

IntelliSense for BA to write stories

Tool generates method stubs to help

Testers focusing on what to test next.


16/27

Challenges

Unskilled testers

Test framework

Unstable tests


17/27

Unskilled testersStart training with motivated testers

Set examplesLearn by following, then creating

Give enough time

Always have helps available


18/27

Test framework

Use established patternsLeverage application code

Make it earlier to understand

Naming and conventions


19/27

Unstable tests

Decouple tests from build pipeline

Categorize tests

Leverage application codeWait and retry and verify


20/27

Catches

An additional layer = more code

More code = more maintenance

Given/When/Then format can be

too verbose


21/27

TipsCategorize tests

Stabilize tests

Speed up tests

Design testable architecture


22/27

Categorize tests

Warm up tests < 1 minute

Smoke tests < 5 minutes

Regression tests < 15 minutes

Daily regression tests < 1 hourUnstable tests


23/27

Stabilize tests

Ensure atomicityControl the environment (system, application

and data)

Run tests in random orderWait, retry and verify


24/27

Speed up tests

Run tests in parallel [Selenium Grid] [SauceLabs]

One code base, use application code where

necessary

Smaller tests


25/27

Design testable architectureBuilt-in testability

Automation hooksE.g. Client side MVC/API/server side MVC

Lots of unit tests

Less integration testsEven less end-to-end tests


26/27

Keep tests

Green(its the heartbeat)

and

fix them when they fail

Most Importantly


27/27

[email protected]

Iqnite 2012 Agile Test Automation - A Behaviour Driven Approach

Documents

Transcript of Iqnite 2012 Agile Test Automation - A Behaviour Driven Approach