BDD Testing and Automating from the trenches - Presented at Into The Box June 14, 2016
-
Upload
gavin-pickin -
Category
Internet
-
view
157 -
download
0
Transcript of BDD Testing and Automating from the trenches - Presented at Into The Box June 14, 2016
BDD Testing & Automation from the Trenches Into the Box 2016
Gavin Pickin
★Who Am I?★Gavin Pickin – developing Web Apps since late 90s
○Ortus Solutions Software Consultant
○ContentBox Evangelist
★What else do you need to know?
○CFMLRepo.com http://www.cfmlrepo.com
○Blog - http://www.gpickin.com
○Twitter – http://twitter.com/gpickin
○Github - https://github.com/gpickin
★Lets get on with the show.
★State of the Room
★ A few questions for you guys★ If you have arms, use them.
★State of the RoomTesting? What’s testing?
★State of the Room
Yeah,
I’ve heard of it.
Why do you
think I’m here?
★State of the RoomYes I know I should be testing, but I’m not sure how to do it
★State of the RoomMy Boss and my Customers wouldn’t let me
★State of the Room
I’m a tester
★State of the RoomI’m a test writing ninja
Call me Majano,
Luis Majano
★Ways to Test your Code★Click around in
the browser yourself
★Setup Selenium / Web Driver to click around for you
★Structured Programmatic Tests
★Types of Testing
★Types of Testing
★Black/White Box
★Unit Testing
★Integration Testing
★Functional Tests
★System Tests
★End to End Tests
★Sanity Testing
★Regression Test
★Acceptance Tests
★Load Testing
★Stress Test
★Performance Tests
★Usability Tests
★+ More
★Levels of Testing
★Cost of a Bug
The bug will cost one way or another
★Integration Testing
★Integration Testing
★Integration Tests several of the pieces
together
★Most of the types of tests are variations of an Integration Test
★Can include mocks but can full end to end tests including DB / APIs
★Unit Testing
★Unit Testing
“unit testing is a software verification and validation method in which a programmer tests if individual units of source code are fit for use. A unit is the smallest testable part of an application”- wikipedia
★Unit Testing★Can improve code quality -> quick error
discovery
★Code confidence via immediate verification
★Can expose high coupling
★Will encourage refactoring to produce > testable code
★Remember: Testing is all about behavior and expectations
★Styles – TDD vs BDD
★TDD = Test Driven Development
○Write Tests
○Run them and they Fail
○Write Functions to Fulfill the Tests
○Tests should pass
○Refactor in confidence
★Test focus on Functionality
★Styles – TDD vs BDD★BDD = Behavior Driven Development
Actually similar to TDD except:
○Focuses on Behavior and Specifications
○Specs (tests) are fluent and readable
○Readability makes them great for all levels of testing in the organization
★Hard to find TDD examples in JS that are not using BDD describe and it blocks
★TDD Example
Test( ‘Email address must not be blank’, function(){
notEqual(email, “”, "failed");
});
★BDD ExampleDescribe( ‘Email Address’, function(){
It(‘should not be blank’, function(){
expect(email).not.toBe(“”);
});
});
★Matchers
expect(true).toBe(true);
expect(true).toBe(true);
expect(true).toBe(true);
expect(true).toBe(true);
★Matchers
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
★Matcher Samples
expect(true).toBe(true);
expect(a).not.toBe(null);
expect(a).toEqual(12);
expect(message).toMatch(/bar/);
expect(message).toMatch("bar");
expect(message).not.toMatch(/quux/);
expect(a.foo).toBeDefined();
expect(a.bar).not.toBeDefined();
★CF Testing Tools
★MxUnit was the standard
★TestBox is the new standard
★Other options
★TestBoxTestBox is a next generation testing framework for ColdFusion (CFML) that is based on BDD (Behavior Driven Development) for providing a clean obvious syntax for writing tests.
It contains not only a testing framework, runner, assertions and expectations library but also ships with MockBox, A Mocking & Stubbing Framework,.
It also supports xUnit style of testing and MXUnit compatibilities.
★TestBox TDD Examplefunction testHelloWorld(){
$assert.includes( helloWorld(), ”world" );
}
★TestBox BDD Exampledescribe("Hello world function", function() {
it(”contains the word world", function() {
expect(helloWorld()).toContain("world");
});
});
★TestBox New BDD Examplefeature( "Box Size", function(){
describe( "In order to know what size box I need
As a distribution manager
I want to know the volume of the box", function(){
scenario( "Get box volume", function(){
given( "I have entered a width of 20
And a height of 30
And a depth of 40", function(){
when( "I run the calculation", function(){
then( "the result should be 24000", function(){
// call the method with the arguments and test the outcome
expect( myObject.myFunction(20,30,40) ).toBe( 24000 );
});
});
});
});
});
});
★Using Testing in your Workflow
★Using HTML Test Runners○Keep a Browser open
○F5 refresh tests
★Command Line Tests
★Run testbox – manual
○Run tests at the end of each section of work
★Run Grunt-Watch – automatic
○Runs testbox on every file change
○Grunt can run other tasks as well,
minification etc
★Testing in your IDE
★Browser Views
○Eclipse allows you to open files in web view – uses HTML Runner
★Run testbox/ Grunt / in IDE Console
○Fairly Easy to setup
○See Demo– Sublime Text 3 (if we have time)
★Installing Testbox★Install Testbox – Thanks to Commandbox
○box install testbox
★Decide how you want to run Testbox
★Create a runner.cfm<cfsetting showDebugOutput="false"><!--- Executes all tests in the 'specs' folder with simple reporter by default ---><cfparam name="url.reporter" default="simple"><cfparam name="url.directory" default="tests.specs"><cfparam name="url.recurse" default="true" type="boolean"><cfparam name="url.bundles" default=""><cfparam name="url.labels" default="">
<!--- Include the TestBox HTML Runner ---><cfinclude template="/testbox/system/runners/HTMLRunner.cfm" >
★Create a Test Suite// tests/specs/CFCTest.cfc
component extends="testbox.system.BaseSpec" {
function run() {
it( "will error with incorrect login", function(){
var oTest = new cfcs.userServiceRemote();
expect( oTest.login( '[email protected]', 'topsecret').result ).toBe('400');
});
}
}
★Create a 2nd Test Suite// tests/specs/APITest.cfccomponent extends="testbox.system.BaseSpec" { function run() { describe("userService API Login", function(){ it( "will error with incorrect login", function(){ var email = "[email protected]"; var password = "topsecret”; var result = ""; http url="http://127.0.0.1:8504/cfcs/userServiceRemote.cfc?method=login&email=#email#&password=#password#" result="result”;
expect( DeserializeJSON(result.filecontent).result ).toBe('400'); }); }); }}
★ Running Testbox with runner.cfm
*Install Testbox Runner – Thanks Sean Coyne
*npm install testbox-runner
*Install Grunt Shell
*npm install grunt-shell
*Add Grunt Configuration
★Running Testbox with Grunt Watch
★Install Testbox Runner – Thanks Sean Coyne
○npm install testbox-runner
★Install Grunt Shell
○npm install grunt-shell
★Add Grunt Configuration
★Adding TextBox Config 1
module.exports = function (grunt) {
grunt.loadNpmTasks('grunt-shell');
grunt.initConfig({ … })
}
★Adding TextBox Config 2
Watch: {
…
cfml: {
files: [ "tests/*.cfc"],
tasks: [ "testbox" ]
}
}
★Adding TextBox Config 3
shell: {
testbox: {
command: "./node_modules/testbox-runner/index.js --colors --runner http://127.0.0.1:53874/tests/runner.cfm --directory /tests/specs --recurse true”
}
}
★Adding TextBox Config 4
grunt.registerTask("testbox", [ "shell:testbox" ]);
grunt.loadNpmTasks('grunt-contrib-jasmine');
grunt.loadNpmTasks('grunt-contrib-watch');
★GruntFile.js Gists
Simple Jasmine + Testbox Example
https://gist.github.com/gpickin/9fc82df3667eeb63c7e7
★Testbox output with Grunt
★Testbox Runner JSON
★Testbox has several runners, you have seen the
HTML one, this Runner uses the JSON runner and then formats it.
★http://127.0.0.1:53874/tests/runner.cfm?
reporter=json
★Testbox Runner JSON{"totalSuites":3,"startTime":1465879026042,"bundleStats":[{"TOTALSUITES":1,"STARTTIME":1465879026042,"TOTALPASS":0,"TOTALDURATION":60,"TOTALSKIPPED":0,"TOTALFAIL":0,"TOTALSPECS":0,"PATH":"tests.specs.integration.api.BaseAPITest","ENDTIME":1465879026102,"DEBUGBUFFER":[],"TOTALERROR":0,"NAME":"tests.specs.integration.api.BaseAPITest","ID":"1DDCA037-FF86-4E9B-B62231A290304E9F","SUITESTATS":[{"STARTTIME":1465879026102,"TOTALPASS":0,"TOTALDURATION":0,"TOTALSKIPPED":0,"TOTALFAIL":0,"TOTALSPECS":0,"BUNDLEID":"1DDCA037-FF86-4E9B-B62231A290304E9F","STATUS":"Skipped","PARENTID":"","SPECSTATS":[],"ENDTIME":1465879026102,"TOTALERROR":0,"NAME":"tests.specs.integration.api.BaseAPITest","ID":"231E5335-408A-41A4-AFC0A1199989F05E","SUITESTATS":[]}],"GLOBALEXCEPTION":""},{"TOTALSUITES":2,"STARTTIME":1465879026102,"TOTALPASS":4,"TOTALDURATION":2395,"TOTALSKIPPED":0,"TOTALFAIL":0,"TOTALSPECS":4,"PATH":"tests.specs.integration.api.UsersAPITest","ENDTIME":1465879028497,"DEBUGBUFFER":[],"TOTALERROR":0,"NAME":"tests.specs.integration.api.UsersAPITest","ID":"8B3B6F5D-C7F2-4E77-B701A9F646061CB3","SUITESTATS":[{"STARTTIME":1465879026571,"TOTALPASS":1,"TOTALDURATION":472,"TOTALSKIPPED":0,"TOTALFAIL":0,"TOTALSPECS":1,"BUNDLEID":"8B3B6F5D-C7F2-4E77-B701A9F646061CB3","STATUS":"Passed","PARENTID":"","SPECSTATS":[{"ERROR":{},"STARTTIME":1465879026571,"TOTALDURATION":472,"FAILORIGIN":{},"STATUS":"Passed","SUITEID":"60F5FB8B-3E97-46C0-9165FDD893DF08B4","ENDTIME":1465879027043,"NAME":"Tests the ability to create a user","ID":"EBF1E06B-80A9-476E-9AA4889200FD48DC","FAILMESSAGE":""}],"ENDTIME":1465879027043,"TOTALERROR":0,"NAME":"CRUD API Methods and Retrieval","ID":"60F5FB8B-3E97-46C0-9165FDD893DF08B4","SUITESTATS":[]},{"STARTTIME":1465879027043,"TOTALPASS":3,"TOTALDURATION":1354,"TOTALSKIPPED":0,"TOTALFAIL":0,"TOTALSPECS":3,"BUNDLEID":"8B3B6F5D-C7F2-4E77-B701A9F646061CB3","STATUS":"Passed","PARENTID":"","SPECSTATS":[{"ERROR":{},"STARTTIME":1465879027043,"TOTALDURATION":501,"FAILORIGIN":{},"STATUS":"Passed","SUITEID":"4CBC4C98-BD18-4C0D-BE187715F5C1BFAB","ENDTIME":1465879027544,"NAME":"Tests list methods","ID":"0503EA96-88EA-4D5B-AF96773612180709","FAILMESSAGE":""},{"ERROR":{},"STARTTIME":1465879027544,"TOTALDURATION":392,"FAILORIGIN":{},"STATUS":"Passed","SUITEID":"4CBC4C98-BD18-4C0D-BE187715F5C1BFAB","ENDTIME":1465879027936,"NAME":"Tests the ability to retrieve the created user record","ID":"C96F286C-09F4-4082-A8083A594B0EA277","FAILMESSAGE":""},{"ERROR":{},"STARTTIME":1465879027936,"TOTALDURATION":461,"FAILORIGIN":{},"STATUS":"Passed","SUITEID":"4CBC4C98-BD18-4C0D-BE187715F5C1BFAB","ENDTIME":1465879028397,"NAME":"Tests the ability to update a user record","ID":"28DDD96E-24E5-4081-BF77E11444F8067F","FAILMESSAGE":""}],"ENDTIME":1465879028397,"TOTALERROR":0,"NAME":"Runs tests against created user","ID":"4CBC4C98-BD18-4C0D-BE187715F5C1BFAB","SUITESTATS":[]}],"GLOBALEXCEPTION":""}],"totalPass":4,"totalDuration":2455,"totalSkipped":0,"totalFail":0,"totalSpecs":4,"labels":[],"endTime":1465879028497,"totalError":0,"totalBundles":2}
★Running in Sublime Text 2
★Install PackageControl into Sublime Text
★Install Grunt from PackageControl
○https://packagecontrol.io/packages/Grunt
★Update Grunt Sublime Settings for paths
{
"exec_args": { "path": "/bin:/usr/bin:/usr/local/bin” }
}
★Then Ctrl / Command Shift P – grunt
★Running in Sublime Text 2
★Continuous Integration
Travis CI
Travis CI
Jenkins
Jenkins
★Q&A
★Any questions?