Best Practice Testing with Lime 2

Folie 1

Best Practice Testing with Lime 2

Bernhard Schussek

Who am I?

Web Developer since 7 years

Lead Developer at 2bePUBLISHED

Development of symfony applications

Search Engine Marketing & Optimization

since 2007

Developer of Lime 2

Recently: Symfony Core Contributor

Part IThe Interactive Stuff

What is important?Brainstorming

Things to Keep in Mind

Write Tests


Write Tests

Test Frequently!


Write Tests

Test Frequently!

Performance


Write Tests

Test Frequently!

Performance

ReliabilityMake sure they do actually work...


Write Tests

Test Frequently!

Performance

ReliabilityMake sure they do actually work...

Readability

Part IITesting Strategies

4 Phase Test

The Test Fixture

Test Isolation

How to Write Testable Code?

Fake Objects

Functional vs. Unit Tests

Creation and Helper Functions

Cukeet

Testing Strategies - Cukeet

Cukeet

Web 3.0 recipe sharing website

Business Model

Recipes

Categories

Images

...

// @Test: resize() resizes the thumbnail

copy('data/fixtures/test.png', 'web/uploads/test.png'); $thumbnail = new CukeetThumbnail('web/uploads/test.png');

$thumbnail->resize(100, 100);

$size = getimagesize($thumbnail->getPath()); $t->is($size, array(100, 100), 'The image has been resized');

unlink('web/uploads/test.png'); unset($thumbnail);

Our Tested ClassTesting Strategies 4 Phase Test






Fixture SetupTesting Strategies 4 Phase Test






Test ExecutionTesting Strategies 4 Phase Test






Result VerificationTesting Strategies 4 Phase Test






Fixture TeardownTesting Strategies 4 Phase Test

Testing Strategies 4 Phase Test

4 Phase Test

Fixture setup

Test execution

Result Verification

Fixture teardown

Goals:Leave the room how you entered it

Make it obvious what you are testing


// setup copy('data/fixtures/test.png', 'web/uploads/test.png'); // test $thumbnail = new CukeetThumbnail('web/uploads/test.png'); $thumbnail->resize(100, 100); // assertions $size = getimagesize($thumbnail->getPath()); $t->is($size, array(100, 100), 'The image has been resized'); // teardown unlink('web/uploads/test.png'); unset($thumbnail);Use Comments!Testing Strategies 4 Phase Test

The Test Fixture

Testing Strategies The Test Fixture

Everything that a test depends on

Data in the database, objects, files,

Global Fixture

Fresh Fixture


Fixture Setuptest.png, new CukeetThumbnail()Test 1Test 2Test 3Test 4Fixture Setuptest.png, new CukeetThumbnail()Test 1Test 2

Fixture Setuptest.png, new CukeetThumbnail()Fixture SetupGlobal Fixture

Fresh Fixture

Test Isolation

Lime 2Annotations

Testing Strategies Annotations

Annotations

Control the program flow in the test

Supported Annotations:

@Test

@Before

@After

@BeforeAll

@AfterAll

// @Before


// @After



$thumbnail->resize(100, 100); $size = getimagesize($thumbnail->getPath()); $t->is($size, array(100, 100), 'The image has been resized');Testing Strategies Annotations

Testing Strategies - Annotations

Annotations

Improve test isolation

Fixtures can be built and destroyed automatically

The chance of interacting tests is reduced

class CukeetThumbnailTest extends LimeTestCase{ private $thumbnail;

public function setup() { copy('data/fixtures/test.png', 'web/uploads/test.png'); $this->thumbnail = new CukeetThumbnail('web/uploads/test.png'); } public function teardown() { unlink('web/uploads/test.png'); unset($this->thumbnail); } public function testTheThumbnailCanBeResized() {}}Same As With AnnotationsTesting Strategies xUnit Style Test Class

How to Write Testable Code?

class CukeetRecipe{ public function save() { $user = sfContext::getInstance()->getUser();

if (!$user->hasCredential('admin')) { throw new RuntimeException('No permission!'); } }}Testing Strategies How to Write Testable Code

class CukeetRecipe{ public function save() { $user = sfContext::getInstance()->getUser();

if (!$user->hasCredential('admin')) { throw new RuntimeException('No permission!'); } }}Uh oh...Testing Strategies How to Write Testable Code

Expensive Operation

Unexpected Side-Effects

Lose Control

Testing Strategies How to Write Testable Code?

TestTested ClassDependency

?

Dependency Injection

Testing Strategies Dependency Injection

Dependency InjectionAll dependencies of an object should be passed through method arguments

Constructor InjectionFor required dependencies

Setter InjectionFor optional dependencies

Parameter Injection

class CukeetRecipe{ private $user;

public function __construct(sfBasicSecurityUser $user) { $this->user = $user; }

public function save() { if (!$this->user->hasCredential('admin')) { throw new RuntimeException('No permission!'); } }Testing Strategies Constructor Injection

// @Before

$user = new sfBasicSecurityUser(); $recipe = new CukeetRecipe($user);

// @Test: Users with credential 'admin' can save

$user->addCredential('admin'); $recipe->save($user);

// @Test: Normal users cannot save

$t->expect('RuntimeException'); $recipe->save($user);

Testing Strategies Dependency Injection





sloooowwww

sfContext, Database, ...

How to Replace Slow Dependencies?

Fake Objects


TestTested ClassFake Object

class CukeetRecipe{ public function save(CukeetCategoryTable $categoryTable) { if (is_null($this->Category)) { $this->Category = $categoryTable->findDefault(); } }}Testing Strategies Slow Dependencies

// @Test: The default category is assigned, if empty

// setup $defaultCategory = new CukeetCategory(); $categoryTable = new FakeCategoryTable($defaultCategory); // test $recipe = new CukeetRecipe(); $recipe->Category = null; $recipe->save($categoryTable); // assertions $t->is($recipe->Category, $defaultCategory, 'The default...');Testing Strategies Fake Objects

class FakeCategoryTable extends CukeetCategoryTable{ protected $defaultCategory;

public function __construct(CukeetCategory $defaultCategory) { $this->defaultCategory = $defaultCategory; }

public function findDefault() { return $this->defaultCategory; }}Testing Strategies Fake Objects

Testing Strategies Stub Objects

StubProvides fake output

Acts as if it was the real object

Does not have any logic inside


// setup $defaultCategory = new CukeetCategory(); $categoryTable = $t->stub('CukeetCategoryTable'); $categoryTable->findDefault()->returns($defaultCategory); $categoryTable->replay(); // test $recipe = new CukeetRecipe(); $recipe->Category = null; $recipe->save($categoryTable); // assertions $t->is($recipe->Category, $defaultCategory, 'The default...');

Stub Generation in Lime 2Testing Strategies Stub Objects


// setup $defaultCategory = new CukeetCategory(); $categoryTable = $t->stub('CukeetCategoryTable'); $categoryTable->findDefault()->returns($defaultCategory); $categoryTable->replay(); // test $recipe = new CukeetRecipe(); $recipe->Category = null; $recipe->save($categoryTable); // assertions $t->is($recipe->Category, $defaultCategory, 'The default...');Testing Strategies Stub Objects

Stub Configuration


Attention!Don't replace entities (recipe, category, )

Replace services (table, ...)


TestTested ClassStub

Test state after test execution

Testing Strategies Mock Objects

TestTested ClassMock

Test behaviour during test execution


MockBehaviour verification

Monitors indirect input

Does the tested object call the right methods?

// @Test: Upon destruction all data is saved to the session

// setup $storage = $t->mock('CukeetSessionStorage'); $storage->write('Foo', 'Bar'); $storage->replay(); $user = new CukeetUser($storage); // test $user->setAttribute('Foo', 'Bar'); $user->__destruct(); // assertions $storage->verify();Testing Strategies Mock Objects


Attention!Verifying behaviour leads to unflexible classes

Implementation changes break tests

Overuse of mocks can harm your health

$browser->info('1 - Recipes are displayed in each category')

->info(' 1.1 - The two most recent recipes are displayed') ->click('Desserts') ->with('response')->begin() ->checkElement('#recipes li:first', '/Deluxe Apples/') ->checkElement('#recipes li:last', '/Quatre Quarts/') ->checkElement('#recipes li', 2) ->end() ->click('Burgers') ->with('response')->begin() ->checkElement('#recipes li:first', '/Kamikaze Burgers/') ->checkElement('#recipes li:last', '/Surprise Burgers/') ->checkElement('#recipes li', 2) ->end()Testing Strategies Functional Tests


Testing Strategies Functional vs. Unit Tests

Functional Tests

Acceptance tests

Integration Tests

Test the system as a whole

Collaboration between classes/components

Are slow!

They do not

Thoroughly test a system for correctness

Test edge cases!

Testing Strategies Functional vs. Unit Tests

Unit Tests

Test classes/components
in isolation

Test edge cases

Are fast!

The biggest part of an application should be tested in unit tests


->info(' 1.1 - The two most recent recipes are displayed') ->click('Desserts') ->with('response')->begin() ->checkElement('#recipes li:first', '/Deluxe Apples/') ->checkElement('#recipes li:last', '/Quatre Quarts/') ->checkElement('#recipes li', 2) ->end() ->click('Burgers') ->with('response')->begin() ->checkElement('#recipes li:first', '/Kamikaze Burgers/') ->checkElement('#recipes li:last', '/Surprise Burgers/') ->checkElement('#recipes li', 2) ->end()Testing Strategies Functional vs. Unit Tests

// CukeetRecipeTableTest.phprequire_once dirname(__FILE__).'/../bootstrap/doctrine.php';

$t->comment('findRecent() returns recent recipes of a category');

// setup ... // test $actual = $table->findRecent(2, $category); // assertions ... $t->is($actual, $expected, 'The correct recipes were returned');Testing Strategies Test Edge Cases in Unit Tests

// CukeetRecipeTableTest.phprequire_once dirname(__FILE__).'/../bootstrap/doctrine.php';

$t->comment('findRecent() returns recent recipes of a category');

// setup ... // test $actual = $table->findRecent(2, $category); // assertions ... $t->is($actual, $expected, 'The correct recipes were returned');

Bootstrap FileTesting Strategies Test Edge Cases in Unit Tests

// bootstrap/doctrine.phprequire_once dirname(__FILE__).'/unit.php';

$database = new sfDoctrineDatabase(array( 'name' => 'doctrine', 'dsn' => 'sqlite::memory:',));

// load missing model files and create tablesDoctrine::createTablesFromModels(ROOT_DIR.'/lib/model');

// load fixture dataDoctrine::loadData('data/fixtures/fixtures.yml');

Load YAML FixturesTesting Strategies Test Edge Cases in Unit Tests


Fixture Setuptest.png, new CukeetThumbnail()Test 1Test 2Test 3Test 4Global Fixture

// @Before

reload(); $table = Doctrine::getTable('CukeetRecipe');

// @Test: findRecent() returns recent recipes of a category

// setup $category = new CukeetCategory(); $category->fromArray(array('name' => 'Desserts', ...)); $recipe1 = new CukeetRecipe(); $recipe1->fromArray(array('name' => 'Quatre Quarts', ...)); $recipe1->Category = $category; $recipe1->save(); ...Testing Strategies Use Inline Fixtures


// @Test: findRecent() returns recent recipes of a category

// setup $category = createCategory(); $recipe1 = createRecipeInCategory($category); $recipe2 = createRecipeInCategory($category); $recipe3 = createRecipe(); $recipe4 = createRecipeInCategory($category); save($recipe1, $recipe2, $recipe3, $recipe4); // test $actual = $table->findRecent(2, $category); // assertions $expected = createCollection($recipe4, $recipe2); $t->is($actual, $expected, 'The correct recipes were returned');Testing Strategies Creation and Helper Functions

// bootstrap/doctrine.php

function createRecipe(array $properties = array()){ static $i = 0;

$recipe = new CukeetRecipe(); $recipe->fromArray(array_merge(array( 'name' => 'Recipe ' . ++$i, 'created_at' => date('Y-m-d H:m:i', $i), ), $properties));

return $recipe;}Testing Strategies Creation and Helper Functions


->info(' 1.1 - The two most recent recipes are displayed') ->click('Desserts') ->with('response')->begin() ->checkElement('#recipes li:first', '/Deluxe Apples/') ->checkElement('#recipes li:last', '/Quatre Quarts/') ->checkElement('#recipes li', 2) ->end() ->click('Burgers') ->with('response')->begin() ->checkElement('#recipes li:first', '/Kamikaze Burgers/') ->checkElement('#recipes li:last', '/Surprise Burgers/') ->checkElement('#recipes li', 2) ->end()Testing Strategies Functional Test Revisited (1)


->info(' 1.1 - The two most recent recipes are displayed') ->click('Desserts') ->with('response')->begin() ->checkElement('#recipes li:first', '/Deluxe Apples/') ->checkElement('#recipes li:last', '/Quatre Quarts/') ->checkElement('#recipes li', 2) ->end() ->click('Burgers') ->with('response')->begin() ->checkElement('#recipes li:first', '/Kamikaze Burgers/') ->checkElement('#recipes li:last', '/Surprise Burgers/') ->checkElement('#recipes li', 2) ->end()

Testing Strategies Functional Test Revisited (2)

Let's Summarize

Testing Strategies - Summary

Summary4 Phase Test

Test Isolation

Annotations

Dependency Injection

Stubs

Mocks



Part IIIWhatz Nu in Lime 2

Annotation Support

// @Before


// @After



$thumbnail->resize(100, 100); $size = getimagesize($thumbnail->getPath()); $t->is($size, array(100, 100), 'The image has been resized');What Nu in Lime 2 Annotation Support

Stub & Mock Generation

// @Test: Upon destruction all data is saved to the session

// setup $storage = $t->mock('SessionStorage'); $storage->write('Foo', 'Bar')->returns(true)->once(); $storage->flush()->atLeastOnce(); $storage->any('read')->throws('BadMethodCallException'); $storage->replay(); // test ...Whatz Nu in Lime 2 Mock Objects

Improved Error Messages

# got: object(CukeetRecipe) (# ...# '_data' => array (# ...# 'name' => 'Quatre Quarts',# ...# ),# )# expected: object(CukeetRecipe) (# ...# '_data' => array (# ...# 'name' => 'Kamikaze Burger',# ...# ),# )What Nu in Lime 2 Improved Error Messages

Multi-Processing Support

Whatz Nu in Lime 2 Multi-Processing Support

Test Suite in 1 ProcessTotal 3:20 minutes

One CPU core at 100%

One CPU core at 20%

Test Suite in 8 ProcessesTotal 1:40 minutes

Both CPU cores at 100%

Up To 100% Faster!

Extensibility

Whatz Nu in Lime 2 - Extensibility

Extensible Testersoverride is(), like() etc. for specific types

class LimeTesterException extends LimeTesterObject{ // don't compare stack trace protected $properties = array('message', 'code', 'file', 'line');}

LimeTester::register('Exception', 'LimeTesterException');

$exception1 = new RuntimeException('Foobar', 10);$exception2 = new RuntimeException('Foobar', 10);$t->is($exception1, $exception2, 'Hurray!')

Whatz Nu in Lime 2 - Extensibility

Extensible OutputsImplement custom outputs for the tests

Log-Output

Email-Output

...

Integration With CI-Tools

sismo

phpUnderControl

?

Bernhard Schussek: Best Practice Testing with Lime 2

Bernhard Schussek: Best Practice Testing with lime

Bernhard Schussek: Best Practice Testing with Lime 2

Best Practice Testing with Lime 2

Technology

Transcript of Best Practice Testing with Lime 2