Acceptance & Functional Testing with Codeception - Devspace 2015

83
Acceptance & Functional Testing with Codeception Joe Ferguson October 10th 2015

Transcript of Acceptance & Functional Testing with Codeception - Devspace 2015

Page 1: Acceptance & Functional Testing with Codeception - Devspace 2015

Acceptance & Functional Testing with Codeception

Joe Ferguson October 10th 2015

Page 2: Acceptance & Functional Testing with Codeception - Devspace 2015

Who Am I?Joe Ferguson

PHP Developer

Twitter: @JoePFerguson

Organizer of @MemphisPHP

@NomadPHP Lightning Talks

Passionate about Community

Page 3: Acceptance & Functional Testing with Codeception - Devspace 2015

One late night coding session of fighting with tests…

Page 4: Acceptance & Functional Testing with Codeception - Devspace 2015

I found

Page 5: Acceptance & Functional Testing with Codeception - Devspace 2015

hey, I’m everyone…

Page 6: Acceptance & Functional Testing with Codeception - Devspace 2015

Example Test

Sample taken from http://codeception.com

Page 7: Acceptance & Functional Testing with Codeception - Devspace 2015

You gotta be…

…me!

Page 8: Acceptance & Functional Testing with Codeception - Devspace 2015

Codeception• Selenium WebDriver integration• Elements matched by name, CSS, XPath• Symfony2, Laravel, Yii, Phalcon, Zend Framework• PageObjects and StepObjects included• BDD-style readable tests• Powered by PHPUnit• API testing: REST,SOAP,XML-RPC• Facebook API testing• Data Cleanup• HTML, XML, TAP, JSON reports• CodeCoverage and Remote CodeCoverage• Parallel Execution

Page 9: Acceptance & Functional Testing with Codeception - Devspace 2015

Types of Testing

Unit

Functional

Acceptance

Page 10: Acceptance & Functional Testing with Codeception - Devspace 2015

Unit TestingIn computer programming, unit testing is a software testing method by which individual units of source

code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to

determine whether they are fit for use.

https://en.wikipedia.org/wiki/Unit_testing

Page 11: Acceptance & Functional Testing with Codeception - Devspace 2015

Functional TestingFunctional testing is a quality assurance (QA) process and a type

of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal

program structure is rarely considered (not like in white-box testing). Functional testing usually describes what the system

does.

Functional testing does not imply that you are testing a function (method) of your module or class. Functional testing tests a slice

of functionality of the whole system

https://en.wikipedia.org/wiki/Functional_testing

Page 12: Acceptance & Functional Testing with Codeception - Devspace 2015

Acceptance TestingIn engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met.

In software testing the ISTQB defines acceptance as: formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.

Acceptance testing is also known as user acceptance testing (UAT), end-user testing, operational acceptance testing (OAT) or field (acceptance) testing.

https://en.wikipedia.org/wiki/Acceptance_testing

Page 13: Acceptance & Functional Testing with Codeception - Devspace 2015

How I think about types

Unit tests test your functions and methods.

Functional tests test parts of your application.

Acceptance tests test your interactions.

Page 14: Acceptance & Functional Testing with Codeception - Devspace 2015

Isn’t unit testing enough?

Page 15: Acceptance & Functional Testing with Codeception - Devspace 2015

It might be, how high is your coverage?

Page 16: Acceptance & Functional Testing with Codeception - Devspace 2015

How do you know methods work together?

Page 17: Acceptance & Functional Testing with Codeception - Devspace 2015

Functional tests allow you to test individual scenarios in

your application

Page 18: Acceptance & Functional Testing with Codeception - Devspace 2015

Acceptance tests allow you to test the interactions in your

application

Page 19: Acceptance & Functional Testing with Codeception - Devspace 2015

Where do we start?

Page 20: Acceptance & Functional Testing with Codeception - Devspace 2015

Install Codeception

Page 21: Acceptance & Functional Testing with Codeception - Devspace 2015

Eh…wait a minute…

Page 22: Acceptance & Functional Testing with Codeception - Devspace 2015

Better way to install*

*http://blog.doh.ms/2014/10/13/installing-composer-packages

Page 23: Acceptance & Functional Testing with Codeception - Devspace 2015

Add to PATH

Page 24: Acceptance & Functional Testing with Codeception - Devspace 2015

Bootstrap Codeception

Page 25: Acceptance & Functional Testing with Codeception - Devspace 2015

Configure Codeception

Page 26: Acceptance & Functional Testing with Codeception - Devspace 2015

Run Codeception

Page 27: Acceptance & Functional Testing with Codeception - Devspace 2015

Add Laravel5 Helper

Page 28: Acceptance & Functional Testing with Codeception - Devspace 2015

Codecept build

Page 29: Acceptance & Functional Testing with Codeception - Devspace 2015

Our First Test

Page 30: Acceptance & Functional Testing with Codeception - Devspace 2015

Login Form

Page 31: Acceptance & Functional Testing with Codeception - Devspace 2015

Login Form Template

Page 32: Acceptance & Functional Testing with Codeception - Devspace 2015

Generate Functional Test

Page 33: Acceptance & Functional Testing with Codeception - Devspace 2015

Writing our test

Page 34: Acceptance & Functional Testing with Codeception - Devspace 2015

Click on what?

Click on ‘Copy CSS Path’

Page 35: Acceptance & Functional Testing with Codeception - Devspace 2015

How to run tests

codecept run // runs all tests

codecept run functional // run functional tests

codecept run functional tests/functional/AdminCanLoginCept.php // run single test

Can also add -vv or -vvv for detailed output

Page 36: Acceptance & Functional Testing with Codeception - Devspace 2015

Run our test

Page 37: Acceptance & Functional Testing with Codeception - Devspace 2015

What are we testing?

The route ‘auth/login’ shows us a login form

Submitting login form performs some action

Dashboard (after successful login) displays

Some success text exists “Hello admin!”

Page 38: Acceptance & Functional Testing with Codeception - Devspace 2015

What are we NOT testing?

Navigation on the page

“Remember Me” functionality works

Password Reset works

User is actually an admin

Page 39: Acceptance & Functional Testing with Codeception - Devspace 2015

Functional testing tests a slice of functionality of the whole system

Page 40: Acceptance & Functional Testing with Codeception - Devspace 2015

We don’t care about

Navigation on the page

“Remember Me” functionality works

Password Reset works

User is actually an admin

Page 41: Acceptance & Functional Testing with Codeception - Devspace 2015

We care about

$I->wantTo('ensure an admin can log in');

Page 42: Acceptance & Functional Testing with Codeception - Devspace 2015

What if our test fails?

Page 43: Acceptance & Functional Testing with Codeception - Devspace 2015

Update our password

Page 44: Acceptance & Functional Testing with Codeception - Devspace 2015

Run Our test

Page 45: Acceptance & Functional Testing with Codeception - Devspace 2015

Failed tests output

Page 46: Acceptance & Functional Testing with Codeception - Devspace 2015

Test will pass when…

the user exists in the database

the routes (get and post) work as expected

login form view renders properly

dashboard view renders properly

Page 47: Acceptance & Functional Testing with Codeception - Devspace 2015

Framework Modules

Page 48: Acceptance & Functional Testing with Codeception - Devspace 2015

Using Laravel5 Module

amOnAction is translated to the /auth/login url

Page 49: Acceptance & Functional Testing with Codeception - Devspace 2015

Test Passes

Page 50: Acceptance & Functional Testing with Codeception - Devspace 2015

Check the DB for user

seeRecord() will fail if the user does not exist

Page 51: Acceptance & Functional Testing with Codeception - Devspace 2015

User is in the database

Page 52: Acceptance & Functional Testing with Codeception - Devspace 2015

Check for errors

Useful for checking that error messages are returned

dontSee() ensures text does not exist on view

Page 53: Acceptance & Functional Testing with Codeception - Devspace 2015

Testing your API

Page 54: Acceptance & Functional Testing with Codeception - Devspace 2015

Generate API Suite

api.suite.yml

Page 55: Acceptance & Functional Testing with Codeception - Devspace 2015

Test API Create User

http://codeception.com/docs/10-WebServices

Page 56: Acceptance & Functional Testing with Codeception - Devspace 2015

Now we know slices of our application work

Page 57: Acceptance & Functional Testing with Codeception - Devspace 2015

Acceptance Testing

Page 58: Acceptance & Functional Testing with Codeception - Devspace 2015

Testing Real(ish) things

Page 59: Acceptance & Functional Testing with Codeception - Devspace 2015

Using a real(ish) browser

Page 60: Acceptance & Functional Testing with Codeception - Devspace 2015

Add PhantomJS

Page 61: Acceptance & Functional Testing with Codeception - Devspace 2015

Configure acceptance.suite.yml

Page 62: Acceptance & Functional Testing with Codeception - Devspace 2015

Export your Database

php artisan migratephp artisan db:seed

mysqldump -u homestead -psecret homestead > tests/_data/dump.sql

Page 63: Acceptance & Functional Testing with Codeception - Devspace 2015

Our 1st acceptance test

Page 64: Acceptance & Functional Testing with Codeception - Devspace 2015

Test passes

Page 65: Acceptance & Functional Testing with Codeception - Devspace 2015

That looks a lot like our functional test!

Page 66: Acceptance & Functional Testing with Codeception - Devspace 2015
Page 67: Acceptance & Functional Testing with Codeception - Devspace 2015

Functional Test runs via PhpBrowser

Page 68: Acceptance & Functional Testing with Codeception - Devspace 2015

Acceptance Test runs via PhantomJS

Page 69: Acceptance & Functional Testing with Codeception - Devspace 2015

Testing Elements via XPath

Page 70: Acceptance & Functional Testing with Codeception - Devspace 2015

Use XPath to submit form

Page 71: Acceptance & Functional Testing with Codeception - Devspace 2015

/html/body/div/div/div/div/div[2]/form/div[4]/div/button

Page 72: Acceptance & Functional Testing with Codeception - Devspace 2015

Even more specific

Page 73: Acceptance & Functional Testing with Codeception - Devspace 2015

Checking specific elements

Page 74: Acceptance & Functional Testing with Codeception - Devspace 2015

Using XPath for form fields isn’t very easy to read.

XPath isn’t very easy to read…

Page 75: Acceptance & Functional Testing with Codeception - Devspace 2015

When to use XPath?

When you can’t add an ID or Class on an element

When you want to ensure styles/formatting (error messages, flash messages, etc)

When testing navigation elements

When testing any dynamic elements

Page 76: Acceptance & Functional Testing with Codeception - Devspace 2015

Acceptance failures come with screenshots!

Page 77: Acceptance & Functional Testing with Codeception - Devspace 2015

Log in and look for content that doesn’t exist

Page 78: Acceptance & Functional Testing with Codeception - Devspace 2015

Test Failed!

Page 79: Acceptance & Functional Testing with Codeception - Devspace 2015

We see our content does not exist on the page

Page 80: Acceptance & Functional Testing with Codeception - Devspace 2015

Some issues…

PhantomJS pretty broken on Travis-CI

Acceptance tests are SLOW (and memory hogs)

Acceptance tests aren’t unit tests, it can be hard to find WHAT is causing the tests to fail

Functional & Acceptance tests aren’t replacements for unit tests, but can be easier to write if your app isn’t written to be testable

Page 81: Acceptance & Functional Testing with Codeception - Devspace 2015

Codeception Resources

http://www.codeception.com

http://chrislema.com/acceptance-testing-wordpress-codeception

Writing Acceptance Tests: https://vimeo.com/113466213

Page 82: Acceptance & Functional Testing with Codeception - Devspace 2015

Testing Resources

https://leanpub.com/grumpy-testinghttps://leanpub.com/grumpy-phpunit

http://grumpy-learning.com

Page 83: Acceptance & Functional Testing with Codeception - Devspace 2015

Questions?