Internal DSLs For Automated Functional Testing
29
Boise Code Camp 2010 John Sonmez http://simpleprogrammer.com
-
Upload
john-sonmez -
Category
Technology
-
view
4.060 -
download
3
description
Automated functional testing is one of the hardest things to get right on a project. Many people with high hopes set out to develop fully automated regression tests, only to be caught up in the tangle of fragile tests which are always broken. In this session I will present a clean way to build a framework using free tools to develop an internal Domain Specific Language custom to the application being testing. I will show the benefits of this approach versus the record and modify methods of most automated testing tools. In addition, I will talk about a real working practical example I have developed for the Dept of Health in Welfare in testing their new case management system.
Transcript of Internal DSLs For Automated Functional Testing
Boise Code Camp 2010John Sonmez
http://simpleprogrammer.com
Automated regression testing is hard Fragile tests Difficult to write tests Tools are expensive Recording is not flexible enough Tests failures don’t indicate code problem
#1 Problem with automated tests is the tests break easily.
Screen changes cause multiple tests to break.
Functionality or text changes cause tests to break.
Why? Multiple tests are dependant on parts of the user interface.
Recording is easy at first, until you need variables or reuse.
Scripting the tests requires knowledge of HTML, CSS, Javascript and a high level programming language such as C# or Java.
QA or business person thinks in terms of domain knowledge, not button clicks and HTML divs.
Most of the record tools have license costs from several hundred to several thousand dollars.
Cost prohibitive, so only certain people in organization get license.
Support contracts. Upgrade costs. “Automation Experts” trained in the
vendor specific tool are expensive.
Recording seems to work at first, but then the scenarios get more complicated.
When the UI changes scripts have to be re-recorded.
Recording limits the kinds of testing you can do. Test result interactions Check against external data source
False failures reduce the confidence in the tests.
Start treating test code like real code. Instead of building “test scripts”, build a
“testing application.” Apply the same tools of good software
craftmanship to test code craftmanship. Bridge the gap between domain
knowledge and test writing.
User: Add a person to the case. Tester:
1. Click button with id=“loginPage123”. 2. Set focus userName field. Add username to
field. 3. Set focus passWord field. Add password to field. 3. Click button with id=“login”. 4. Check HTML title for text containing
“successful”. 5. Click div in top navigation with id of addPerson. 6. blah blah blah… blah blah blah
User: Add a person to the case. Tester:
1. Case.AddPerson(“Joe Smith”); 2. AssertTrue(Case.isPersonOnCase(“Joe
Smith”));
Abstraction… Business language becomes testing language.
Internal Domain Specific Language (DSL) A language to write tests which uses the
same or similar vocabulary as the business or user.
Framework built specific to the application, not a general purpose testing framework.
Framework developed by developers and testers collaboratively. User of framework is test automation resources.
Domain Specific Language According to Wikipedia: A programming
language or specification language dedicated to a particular problem domain, a particular problem representation technique, and/or a particular solution technique.
Basically a custom programming language designed for a specific problem or set of problems.
A DSL that is a set of APIs still in the native language used to approximate a full DSL.
Writing a language is hard. Writing an API is easier. Objected oriented programming techniques
already lead us in this direction. We use internal DSLs in English all the time.
Company acronyms and language. Programmer speak Slang
Remember the gap? Bridge the gap by building an application
specific layer that exposes a DSL for writing tests.
Hide the details about web pages, HTML, CSS, Javascript from the tests.
Easy to use, fluent design.
The web browser provides us a layer we can use to abstract away the notion of parsing HTML, CSS and Javascript.
Without the browser, we would have to write code to open HTTP connections and GET web pages.
We would have to parse CSS, and Javascript, to verify functionality.
Web testing framework interacts with the browser for us.
Operates on the DOM the browser provides.
Without the Web Testing Framework, we would have to write code to manipulate the browser, and change it’s state through COM or some other interface.
We build this on top of the Web Testing Framework.
Abstracts away all concepts having to do with browsers and HTML.
Uses the Web Testing Framework to translate business language into web functionality.
Contains all our changing things about the application.
Without this layer our code would contain hard coded ids of elements and would have to deal with page layouts.
Uses only the application framework. Uses the internal DSL. Simple Written as close to business language as
possible.
A good starting place is to start writing the test.
Pretend like you don’t know anything about the web, but you do know about the application.
Think “fluent.” Think easy to use.
Web based store. Write a test to create a new account and
buy bananas. Account.Create(“Joe Smith”, “123 Main Street,
Boise ID 83713”, “[email protected]”); Category.Fruits.GoTo(); Cart.AddItem(Items.BANANAS, 1); Cart.CheckOut(PaymentType.CreditCard);
Simple, business logic based syntax.
Similar to Test Driven Development when unit testing.
Build the business objects and methods needed to make the test syntax possible.
Don’t worry about all the variations, only add what is needed for the test you are working on.
Here the identifiers are hard coded in the business objects, NOT in the tests.
Account.Create Make an account class with a static method
called create, that takes in a name, address, and email.
In the method use the web testing framework to navigate to the account creation page and enter the information into the fields.
After the information is entered create the new account.
Default non-important selections. (You can make them specific later if needed.)
Actual test layer. Think about how to achieve the same
amount with fewer lines of code. The simpler and fewer lines of code to
write the tests, the easier to write the tests.
Eliminate variables that are not important to the test.
Eliminate any steps that cannot vary.
Do we are about the account name, address, etc? Account.Create(“Joe Smith”, “123 Main Street,
Boise ID 83713”, “[email protected]”); becomes Account.Create(People.Base_Case);
Do we need to explicitly go to the fruits page to add a banana? Category.Fruits.GoTo(); Cart.AddItem(Items.BANANAS, 1); becomes Cart.AddItem(Items.BANANAS, 1);
Created for Dept of Health and Welfare IBES System to automate web testing.
Creates test cases in very few lines of code.
Able to write a test cases which goes through 30+ screens in a few minutes.
Test frameworks Watij (http://watij.com/) Watin (http://watin.sourceforge.net/) Watir (http://watir.com/)
My Blog, posts on automation and Watij. http://simpleprogrammer.com
Contact me I can help you build your framework [email protected] Twitter jsonmez
