Post on 15-Apr-2017
Acceptance & Functional Testing with Codeception
Joe Ferguson February 6th 2015
Who Am I?Joe Ferguson
PHP Developer
Twitter: @JoePFerguson
Organizer of @MemphisPHP
@NomadPHP Lightning Talks
Passionate about Community
One late night coding session of fighting with tests…
I found
hey, I’m everyone…
You gotta be…
…me!
Codeception• Selenium WebDriver integration• Elements matched by name, CSS, XPath• Symfony2, Laravel, Yii, Phalcon, Zend Framework• PageObjects and StepObjects included• BDD-style readable tests• Powered by PHPUnit• API testing: REST,SOAP,XML-RPC• Facebook API testing• Data Cleanup• HTML, XML, TAP, JSON reports• CodeCoverage and Remote CodeCoverage• Parallel Execution
Types of Testing
Unit
Functional
Acceptance
Unit TestingIn computer programming, unit testing is a software testing method by which individual units of source
code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to
determine whether they are fit for use.
https://en.wikipedia.org/wiki/Unit_testing
Functional TestingFunctional testing is a quality assurance (QA) process and a type
of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal
program structure is rarely considered (not like in white-box testing). Functional testing usually describes what the system
does.
Functional testing does not imply that you are testing a function (method) of your module or class. Functional testing tests a slice
of functionality of the whole system
https://en.wikipedia.org/wiki/Functional_testing
Acceptance TestingIn engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met.
In software testing the ISTQB defines acceptance as: formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.
Acceptance testing is also known as user acceptance testing (UAT), end-user testing, operational acceptance testing (OAT) or field (acceptance) testing.
https://en.wikipedia.org/wiki/Acceptance_testing
How I think about types
Unit tests test your functions and methods.
Functional tests test parts of your application.
Acceptance tests test your interactions.
Isn’t unit testing enough?
It might be, how high is your coverage?
How do you know methods work together?
Functional tests allow you to test individual scenarios in
your application
Acceptance tests allow you to test the interactions in your
application
Where do we start?
Install Codeception
Eh…wait a minute…
Better way to install*
*http://blog.doh.ms/2014/10/13/installing-composer-packages
Add to PATH
Bootstrap Codeception
Configure Codeception
Run Codeception
Add Laravel5 Helper
Codecept build
Our First Test
Login Form
Login Form Template
Generate Functional Test
Writing our test
Click on what?
Click on ‘Copy CSS Path’
How to run testscodecept run // runs all tests
codecept run functional // run functional tests
codecept run functional tests/functional/AdminCanLoginCept.php // run single test
Can also add -vv or -vvv for detailed output
Run our test
What are we testing?
The route ‘auth/login’ shows us a login form
Submitting login form performs some action
Dashboard (after successful login) displays
Some success text exists “Hello admin!”
What are we NOT testing?
Navigation on the page
“Remember Me” functionality works
Password Reset works
User is actually an admin
Functional testing tests a slice of functionality
of the whole system
We don’t care about
Navigation on the page
“Remember Me” functionality works
Password Reset works
User is actually an admin
We care about
$I->wantTo('ensure an admin can log in');
What if our test fails?
Update our password
Run Our test
Failed tests output
Test will pass when…
the user exists in the database
the routes (get & post) work
login form view renders properly
dashboard view renders properly
Framework Modules
Using Laravel5 Module
amOnAction is translated to the /auth/login url
Test Passes
Check the DB for user
seeRecord() will fail if the user does not exist
User is in the database
Check for errors
Useful for checking that error messages are returned
dontSee() ensures text does not exist on view
Testing your API
Generate API Suite
api.suite.yml
Test API Create User
http://codeception.com/docs/10-WebServices
Now we know slices of our application work
Acceptance Testing
Testing Real(ish) things
Using a real(ish) browser
Add PhantomJS
Configure acceptance.suite.yml
Export your Database
php artisan migratephp artisan db:seed
mysqldump -u homestead -psecret homestead > tests/_data/dump.sql
Our 1st acceptance test
Test passes
That looks a lot like our functional test!
Functional Test runs via PhpBrowser
Acceptance Test runs via PhantomJS
Testing Elements via XPath
Use XPath to submit form
/html/body/div/div/div/div/div[2]/form/div[4]/div/button
Even more specific
Checking specific elements
Using XPath for form fields isn’t very easy to read.
XPath isn’t very easy to read…
When to use XPath?
When you can’t add an ID or Class on an element
When you want to ensure styles/formatting (error messages, flash messages, etc)
When testing navigation elements
When testing any dynamic elements
Acceptance failures come with screenshots!
Log in and look for content that doesn’t exist
Test Failed!
We see our content does not exist on the page
Some issues…
PhantomJS pretty broken on Travis-CI
Acceptance tests are SLOW (and memory hogs)
Acceptance tests aren’t unit tests, it can be hard to find WHAT is causing the tests to fail
Functional & Acceptance tests aren’t replacements for unit tests, but can be easier to write if your app isn’t written to be testable
Codeception Resources
http://www.codeception.com
http://chrislema.com/acceptance-testing-wordpress-codeception
Writing Acceptance Tests:
https://vimeo.com/113466213
Testing Resources
https://leanpub.com/grumpy-testinghttps://leanpub.com/grumpy-phpunit
http://grumpy-learning.com
Questions?
Feedback!
https://joind.in/talk/4c449
Joe FergusonTwitter: @JoePFergusonEmail: joe@joeferguson.meFreenode: joepferguson
Contact Info: