Phing101 or How to staff a build orchestra

Phing 101 / How to staff a build orchestra#phpuceu11

Sonntag, 20. Februar 2011

Testing in build land

Bending Phing to your needs

Introduction

Phing in Action

Refactoring buildfiles

Best practices

Session agenda

Alternatives in PHP land


Extensible build framework

Tool that can take care of your repetitive and boring tasks

Heavily influenced by Apache Ant

External build DSL and it‘s interpreter

Cli Adapter

What‘s that Phingy thingy?

Empowers CI and CD

Enables one button push automation


Let‘s you use PHP and PHP land API‘s when extending it

Let‘s you stay in PHP land

Ships with currently ≈ 95 tasks

When to pick Phing (over Ant / Rake / ...)

No additional dependencies added (Java runtime, Ruby interpreter)


Installing the conductor

$ pear channel-discover pear.phing.info

$ pear install phing/phing

$ pear install -a phing/phing

$ phing -v


Setting the stage

<?xml version="1.0" encoding="UTF-8"?><project name="phpuceu" default="localize" basedir="."> <property name="city.name" value="Manchester" override="true"/> <target name="localize" depends="hello" description="Localize city"> <echo msg="to ${city.name}!"/> </target> <target name="hello" description="Say hello"> <echo msg="Phing says hello"/> </target></project>


Phing in Action

<?xml version="1.0" encoding="UTF-8"?><project name="phpuceu" default="localize" basedir="."> <property name="city.name" value="Manchester" override="true"/> <target name="localize" depends="hello" description="Localize city"> <echo msg="to ${city.name}!"/> </target> <target name="hello" description="Say hello"> <echo msg="Phing says hello"/> </target></project>


How Phing empowers CI and CD

<?phpexec('phing -f faulty-build.xml 2>&1', $output, $returnCode);var_dump($returnCode);

exec('phing -f successful-build.xml 2>&1', $output, $returnCode);var_dump($returnCode);


How Phing empowers CI and CD


Avoid repeated man tool and tool --help look ups

Identify repetitive, error-prone, and boring tasks

Why / when to automate

Have a simple Cli Adapter, to avoid the ‘bash hustle™‘

Save time, increase efficiency


Picking your orchestra staff

Get to know the official Phing guide

Pick the tasks suitable for your identified problem domain

Bundle them in callable targets

Define the orchestration flow


<?xml version="1.0" encoding="UTF-8"?><project name="phpuceu" default="setup-mongodb-test-data" basedir="."> <property name="mongo.test.db" value="orchestra-test" override="true" /> <property name="mongo.test.data.file" value="${project.basedir}/fixtures/orchestra.json" /> <property name="mongo.test.data.collection" value="testcollection" /> <target name="setup-mongodb-test-data" description="Setting up MongoDb test data"> <exec command="mongoimport --db ${mongo.test.db} --collection ${mongo.test.data.collection} --file ${mongo.test.data.file}" logoutput="true" /> </target></project>

Wrap Cli commands with the exec task



<?xml version="1.0" encoding="UTF-8"?><project name="phpuceu" default="overview" basedir="."> <target name="adhoc-setup" description="Adhoc task(s) setup"> <adhoc-task name="git-commits-by-date"><![CDATA[ class GitCommitsByDate extends Task { private $date; function setDate($date) { $this->date = $date; } function main() { $gitCommand = sprintf( 'git log --since="%s" --oneline', $this->date ); exec($gitCommand, $logs); foreach ($logs as $log) { $this->log($log); } } }]]> </adhoc-task> </target> <target name="todays-commits" depends="adhoc-setup" description="Gather todays Git commits"> <git-commits-by-date date="2011-02-20" /> </target></project>

Adhoc tasks



<?phprequire_once 'phing/Task.php';require_once 'phing/BuildException.php';class GitTagTask extends Task { private $annotation;

public function setAnnotation($annotation) { $this->annotation = $annotation; } public function init() {} public function main() { $this->_validateProperties(); $gitCommand = sprintf("git tag -a %s -m ''", $this->annotation); $this->log('Executing "%s"', $gitCommand); exec($gitCommand); $project = $this->getProject(); $logMsg = sprintf('Tagged %s with %s', $project->getName(), $this->annotation); $this->log($logMsg); } /** * @throws Phing/BuildException */ private function _validateProperties() {}}

Custom tasks



<?xml version="1.0" encoding="UTF-8"?><project name="phpuceu" default="build" basedir="."> <taskdef name="git-tag" classname="phing-tasks.GitTagTask" /> <property name="tagable.build" value="false" override="true" /> <property name="tag.annotation" value="" override="true" />

<target name="-tag-project"> <if> <equals arg1="${tagable.build}" arg2="true" /> <then> <git-tag annotation="${tag.annotation}"/> </then> <else> <echo message="Skipping project tagging" /> </else> </if> </target> <target name="build" depends="inspect,test,-tag-project"> <echo msg="Building..." /> </target>

<target name="inspect"> <echo msg="Inspecting..." /> </target> <target name="test"> <echo msg="Testing..." /> </target></project>

Using custom tasks



Connect with Michiel Rook to get Svn commit privileges

Transforming custom tasks into official ones

Add task and entourage to $PHING_TRUNK/task/ext/*Add task to $PHING_TRUNK/tasks/defaults.properties

Add PEAR packages depen-dencies to $PHING_TRUNK/build/BuildPhingPEARPackageTask.phpDocument task & commit


Make it known to the Phing Cli at runtime with the -logger option

Implement Phing‘s BuildListener interface

To use it permanently add it to the Phing bash script

Crafting custom build listeners


Crafting custom build listeners

<?phpclass BuildhawkLogger extends DefaultLogger { private $_gitNotesCommandResponse = null;

public function buildFinished(BuildEvent $event) { parent::buildFinished($event); if ($this->_isProjectGitDriven($event)) { $error = $event->getException(); if ($error === null) { $buildtimeForBuildhawk = $this->_formatBuildhawkTime( Phing::currentTimeMillis() - $this->startTime); if (!$this->_addBuildTimeAsGitNote($buildtimeForBuildhawk)) { $message = sprintf("Failed to add git note due to '%s'", $this->_gitNotesCommandResponse); $this->printMessage($message, $this->err, Project::MSG_ERR); } } } } private function _isProjectGitDriven(BuildEvent $event) { $project = $event->getProject(); $projectGitDir = sprintf('%s/.git', $project->getBasedir()->getPath()); return file_exists($projectGitDir) && is_dir($projectGitDir); } private function _formatBuildhawkTime($micros) { return sprintf("%0.3f", $micros); } private function _addBuildTimeAsGitNote($buildTime) { $gitCommand = sprintf("git notes --ref=buildtime add -f -m '%s' HEAD 2>&1", $buildTime); $gitNotesCommandResponse = exec($gitCommand, $output, $return); if ($return !== 0) { $this->_gitNotesCommandResponse = $gitNotesCommandResponse; return false; } return true; }}


Using custom build listeners

$ phing -logger phing.listener.BuildhawkLogger

$ buildhawk --title 'Examplr' > examplr-build-times.html


Unit tests for custom tasks

Testing tasks and buildfiles

Structure and execution tests for the buildfile / build itself


Integration, formal tests for the buildfile itself


class GitHubIssuesTaskTest extends PHPUnit_Framework_TestCase { private $task; private $testReportDirectory; public function setUp() { $this->task = new GitHubIssuesTask; $this->task->init(); $this->testReportDirectory = TEST_DIR . DIRECTORY_SEPARATOR . 'test-ghi-dir'; } /** * @test * @expectedException BuildException * @dataProvider nonAcceptedReportTypesProvider */ public function taskShouldThrowExceptionOnNonAcceptedReportTypes($type) { $this->task->setReportType($type); $this->task->main(); } * @return array */ public function nonAcceptedReportTypesProvider() { return array( array('doc'), array('vid'), array('pdf'), ); }}


Integration, formal tests for the buildfile itself


class GitHubIssuesTaskTest extends PHPUnit_Framework_TestCase{ private $task; private $testReportDirectory; public function setUp() { // omitted } /** * @test */ public function taskShouldCreateReportDirectoryWhenNotExisting() { Phing::setProperty('host.fstype', 'UNIX'); $this->assertFalse(file_exists($this->testReportDirectory)); $this->task->setReportType('txt'); $this->task->setRepository('name:repos'); $this->task->setReportDirectory($this->testReportDirectory); $project = $this->getMock('Projekt', array('logObject'));

$project->expects($this->once()) ->method('logObject') ->with($this->anything());

$this->task->setProject($project); $this->task->main();

$this->assertTrue(file_exists($this->testReportDirectory)); $this->assertTrue(is_dir($this->testReportDirectory)); $this->assertTrue(rmdir($this->testReportDirectory)); }}Sonntag, 20. Februar 2011



/** * @test * @group structure */public function buildfileShouldContainACleanTarget() { $xml = new SimpleXMLElement($this->_buildfileXml); $cleanElement = $xml->xpath("//target[@name='clean']"); $this->assertTrue(count($cleanElement) === 0, "Buildfile doesn't contain a clean target");}




/** * @test * @group structure */public function targetBuildShouldDependOnCleanTarget() { $xml = new SimpleXMLElement($this->_buildfileXml); $xpath = "//target[@name='build']/@depends"; $dependElement = $xml->xpath($xpath); $this->assertTrue(count($dependElement) > 0, 'Target build contains no depends attribute' ); $dependantTasks = array_filter(explode(' ', trim($dependElement[0]->depends)) ); $this->assertContains('clean', $dependantTasks, "Target build doesn't depend on the clean target" );}




/** * @test * @group execution */public function initTargetShouldCreateInitialBuildArtifacts() { $this->_isTearDownNecessary = true; $this->_buildfileRunner->runTarget(array('init')); $expectedInitArtifacts = array( "{$this->_buildfileBasedir}/build", "{$this->_buildfileBasedir}/build/logs/performance/", "{$this->_buildfileBasedir}/build/doc", "{$this->_buildfileBasedir}/build/reports" );

foreach ($expectedInitArtifacts as $artifact) { $this->assertFileExists( $artifact, "Expected file '{$artifact}' doesn't exist" ); }}


Extract target


<target name="test-mongodb-daos" description="Testing MongoDb based Daos"> <exec command="mongoimport --db ${mongo.test.db} --collection ${mongo.test.data.collection} --file ${mongo.test.data.file}" logoutput="true" /> <exec command="phpunit --configuration mongodb-test.xml" dir="${test.directory}" logoutput="true" /></target>

<target name="setup-mongodb-test-data" description="Setting up MongoDb test data"> <exec command="mongoimport --db ${mongo.test.db} --collection ${mongo.test.data.collection} --file ${mongo.test.data.file}" logoutput="true" /></target>

<target name="test-mongodb-daos" depends="setup-mongodb-test-data" description="Testing MongoDb based Daos"> <exec command="phpunit --configuration mongodb-test.xml" dir="${test.directory}" logoutput="true" /></target>Sonntag, 20. Februar 2011

Extract target

Introduce property file


<property name="mongo.test.db" value="orchestra" override="true" /><property name="mongo.test.data.file" value="${project.basedir}/fixtures/orchestra.json" /><property name="mongo.test.data.collection" value="testcollection" />

<property file="mongodb-test.properties" />

[mongodb-test.properties]mongo.test.db=orchestramongo.test.data.file=fixtures/orchestra.jsonmongo.test.data.collection

+


Replace comment with description

Extract target



<target name="inspect" depends="lint"> <phpcodesniffer ... /></target>

<target name="inspect" depends="lint" description="Sniff code base against PHP_CodeSniffer ruleset"> <phpcodesniffer ... /></target>

Enforce internal target


Extract target



<target name="deploy" depends="test-setup, test, smoke-test" description="Deploys the project into ..."> </target>

<target name="-deploy" depends="test-setup, test, smoke-test" description="Deploys the project into ..."> </target>

Enforce internal target


Extract target


Introduce distinct target naming


<property name="deployment-mode" value="staging" override="true" /><target name="cloneGitRepos"> </target>

<property name="deployment.mode" value="staging" override="true" /><target name="clone-git-repos"> </target>

Threat buildfiles as ‘first-class‘ code citizens

Describe your targets

Follow an iterative approach

Apply the Martin Fowler‘s / Don Roberts “Rule of Three“

Best practices

Establish a common buildfile coding standard


phake

taskman

pake

Alternatives in PHP land

<?phpdesc("Say hello");task('hello', function() { echo 'Phake says hello '; });

desc("Localize city");task('localize','hello', function($app) { $cityName = 'Manchester'; if (isset($app['city.name']) && $app['city.name'] !== '') { $cityName = $app['city.name']; } echo 'to ' . $cityName . '!' . PHP_EOL; });

task('default', 'localize');


@raphaelstolt

Brought to you by


[5] http://www.flickr.com/photos/rodrigodavid/4074212668

[9] http://www.flickr.com/photos/roland/1924450950

[8] http://www.flickr.com/photos/crazytales562/3013076129

[7] http://www.flickr.com/photos/52569650@N00/4310444273

[6] http://www.flickr.com/photos/melitron/4923829202

[4] http://www.flickr.com/photos/clarazamith/3816864822

[3] http://www.flickr.com/photos/rotterdamsphilharmonischorkest/

[2] http://www.flickr.com/photos/shoppingdiva/3695515673

[1] http://www.flickr.com/photos/seosmh/1352680740

Image credits


http://www.flickr.com/photos/rodrigodavid/4074212668


http://www.flickr.com/photos/roland/1924450950

http://www.flickr.com/photos/roland/1924450950

http://www.flickr.com/photos/crazytales562/3013076129

http://www.flickr.com/photos/crazytales562/3013076129

http://www.flickr.com/photos/52569650@N00/4310444273

http://www.flickr.com/photos/52569650@N00/4310444273

http://www.flickr.com/photos/melitron/4923829202




http://www.flickr.com/photos/rotterdamsphilharmonischorkest/5206448834

http://www.flickr.com/photos/rotterdamsphilharmonischorkest/5206448834

http://www.flickr.com/photos/22280677@N07/2503527571/


http://www.flickr.com/photos/seosmh/1352680740


[5] https://github.com/jaz303/phake

[4] https://github.com/indeyets/pake

[3] http://code.google.com/p/lib-taskman/

[2] ‘Refactoring Ant buildfiles‘ essay, Julian Simpson in The ThoughtWorks® Anthology by Pragmatic Programmers

[1] http://phing.info

Links & Resources




https://github.com/indeyets/pake

https://github.com/indeyets/pake

http://code.google.com/p/lib-taskman/

http://code.google.com/p/lib-taskman/







Phing101 or How to staff a build orchestra

Technology

Transcript of Phing101 or How to staff a build orchestra