Writing Acceptance Tests Using Fitnesse

facundo.farias@gmail.com

Facundo Farias

Agenda

1. Acceptance Testing2. Acceptance Tests3. How To: Acceptance tests4. FitNesse and RESTful Web Services5. Samples6. How does Acceptance Test fit in our process?

Acceptance Testing

• Acceptance Testing is a high-level testing procedure used to verify that a software system behaves as specified by customer.

• The term Acceptance Testing itself is strongly related to the agile software development method Extreme Programming (XP).

Acceptance Tests by definition

• Acceptance tests are created from user stories. During an iteration the user stories selected during the iteration planning meeting will be translated into acceptance tests. The customer specifies scenarios to test when a user story has been correctly implemented. A story can have one or many acceptance tests, what ever it takes to ensure the functionality works.

• Acceptance tests are black box system tests.• Each acceptance test represents some expected result

from the system.• A user story is not considered complete until it has

passed its acceptance tests.• Acceptance tests should be automated so they can be

run often.

Benefits of Acceptance Tests

Some of the benefits using Acceptance Tests:1. Helps to clarify requirements,2. Tests are written using customer language,3. Shows working software in a real environment,4. Increase communication within the team,5. Can be considered as an absolute criteria for

deciding when a feature is complete,6. Eliminates ambiguity from users stories,7. Can be seen as a partial replacement for

documentation.

Many tools to implement it

• Fitnesse • Cucumber• Concordion• Selenium• GreenPepper• Jbehave• Others

Which one should I use?

• It depends on:1. The Programming Language

2. The Development Infrastructure and Tools,

3. The OS

We did a choice: Fitnesse for REST

• After a brief analysis of each tool, and an small PoC, we found Fitnesse the best tool for our project. Why:

1. The tests are written in a wiki format,2. The implementation was very easy,3. We didn’t need to touch anything in our code, since with

a single table we can write complex tests,4. Fitnesse exposes a REST interface to run the tests in a

suite automatically returning XML results,5. The received XML can be transformed using an XSLT file

to convert the results to JUnit format, in this way, it will be integrated to Bamboo,

6. It has a big community in the Open Source world.

FitNesse & RESTful Web Services• One thing that acceptance testing can perform very well is the

validation of RESTful web services, but unfortunately FitNesse does not provide native support for testing RESTful web services. Fortunately if you search the Internet you'll quickly find a custom fixture called RestFixture that does most of what you will need to test your web services. In short, the RestFixture allows you to execute various HTTP verbs against a service, passing the appropriate data, and then validate the results.

• The RestRixture is an ActionFixture, therefore all the ActionFixture goodies are available. On top of that it contains the following 7 methods:

• header: to be able to set the headers for the next request (a CRLF separated list of name:value pairs),

• body: to allow request body input, essential for PUT and POST,• let: to allow data from the response headers and body to be

extracted and assigned to a label that can then be passed around. • GET, POST, PUT, DELETE to execute requests.

Getting started with RestFixture• Each test is a row on a RestFixture table and it has the following

format:

• |VERB|uri|?ret|?headers|?body|• VERB is one of GET, POST, PUT, DELETE• uri is the resource URI• ?ret is the expected return code of the request. it can be

expressed as a regular expression, that is you can write 2\d\d if you expect any code between 200 and 299.

• ?headers is the expected list of headers. In fact, the expectation is checked by verifying that *all* the headers in here are present in the response. Each header must go in a newline and both name and value can be expressed as regular expressions to match in order to verify inclusion.

• ?body it's the expected body in the response. This is expressed as a list of XPath expressions to allow greater flexibility.

GET sample

GET sample result

POST sample

POST sample result

PUT sample

PUT sample result

DELETE sample

DELETE sample result

How are we using it?

• We’ve implemented FitNesse with this specific RestFixture to validate the our REST interfaces.

• We have a suite for the interfaces already developed, called Regression. This suite will be executed as an smoke test after each code commit, since it tests the sunny day of each API e2e.

• We have another suite, to test the Current sprint. This should be RED at the very beginning of the sprint, but should start be GREEN when the sprint is reaching the end.

• Both test suites can be executed thru a Bamboo Plan. The first stage (Regression) will run always, and the second stage (Current) will be optional (we choose when to run it).

References

• Acceptance Tests [extremeprogramming.org]• Test Driven .NET Development with FitNesse• Acceptance Testing [University of Helsinki]• FitNesse Web Page• Fit: Framework for Integrated Test• RestFixture: Validation of RESTful Web Services• ResFixture Documentation Tests

Q&A

Thanks!