Gradle Plugin Best Practices by Example

Unless otherwise indicated, these s l ides are © 2013-2015 Pivotal Software, Inc. and l icensed under a Creat ive Commons Attr ibut ion-NonCommercial l icense: ht tp: / /creat ivecommons.org/ l icenses/by-nc/3.0/

SPRINGONE2GXWASHINGTON, DC

Gradle plugin best practices by example

By Benjamin Muschko Gradle Inc.


The dark past of build logic

2

Unstructured spaghetti code

Copy & paste of code snippets

The build tool tells you how to structure code

Build can only be understood by build guru™

Testing through manual execution


Build logic “-ilities”

Typical non-functional software requirements also apply to build code…

• Reusability

• Testability

• Maintainability

• Extensibility

• Configurability

3


Reusability

Avoid copy/paste! Allow code to be shared among independent projects.

4


Testability

Build code is no different from application code. Test it on multiple levels!

5


Maintainability

Avoid the big ball of mud! Cohesion and separation of concerns are important.

6


Extensibility

Extend Gradle's build language by your own declarative & expressive language constructs.

7


Configurability

Don't box in your users! Implement convention over configuration with ease.

8


The end goal

9

Maintainable, reusable & tested code

Consumers only configure the code

Complex implement. details are hidden


How to get there?

Concepts that help implement these requirements…

• Good software engineering practices

• Custom tasks

• Plugins

• Gradle's extension mechanisms

• Testing capabilities

10


Techniques covered in talk

• Declarative vs. imperative logic

• Convention over configuration

• Capabilities vs. conventions

• Gradle's extension mechanisms

• Testing plugin logic

• Publishing the plugin artifacts

• Writing and generating documentation

• Setting up Continuous Integration/Delivery

11


Case study: Gradle Docker plugin

Serves as showcase plugin project.

• Plugin for managing Docker images and containers through Docker remote API

• API communication via Docker Java library

• Written in Groovy

• Source code available on GitHub (https://github.com/bmuschko/gradle-docker-plugin)

12

https://github.com/bmuschko/gradle-docker-plugin


Always check in the Wrapper

Project becomes instantly buildable for every developer and on the CI machine.

13

Check in Wrapper files into VCS repository.


Hiding complex, imperative logic

Concept applies to tasks and plugins.

• Reusable and configurable

• Easy to structure, refactor and test

• Avoid global properties and methods

14


Task type > ad-hoc task

Prefer implementing task types to implementing ad-hoc tasks.

15

task createDockerfile(type: Dockerfile) { destFile = file("$buildDir/mydockerfile/Dockerfile") from 'ubuntu:12.04' maintainer 'Benjamin Muschko "[email protected]"' }

task buildImage(type: DockerBuildImage) { dependsOn createDockerfile inputDir = createDockerfile.destFile.parentFile tag = 'bmuschko/myimage' }


Binary plugin > script plugin

Use script plugins to organize build logic based on functional boundaries in project.

16

apply from: "$rootDir/gradle/dependencies.gradle" apply from: "$rootDir/gradle/test-setup.gradle" apply from: "$rootDir/gradle/integration-test.gradle" apply from: "$rootDir/gradle/functional-test.gradle" apply from: "$rootDir/gradle/additional-artifacts.gradle" apply from: "$rootDir/gradle/publishing.gradle" apply from: "$rootDir/gradle/documentation.gradle" apply from: "$rootDir/gradle/release.gradle"


Convention over configuration

Provide sensible defaults and standards. Expose a way to re-configure them to user’s needs.

Examples:

• src/main/java for Java-based applications

• Project name derived from directory name

• Output directory is build

17


Convention

Plugin consumers will be most comfortable with using convention plugins. Pick sensible defaults for the user.

Good indicator:

The less a consumer has to re-configure defaults the better.

18


Configuration

Real-world project might need to re-configure the defaults. Make it convenient to change defaults.

Reason:

Their view of the world might look different than yours.

19


Users only declare the “what”

Expose a custom DSL from your binary plugin to configure runtime behavior. The “how” is the responsibility of the plugin implementation.

20

apply plugin: 'com.bmuschko.docker-remote-api'

docker { url = 'https://192.168.59.103:2376' certPath = new File(System.properties['user.home'], '.boot2docker/certs/boot2docker-vm') }


Example: Providing default dependencies

Plugins often rely on external libraries. Automatically resolve default version, but make it configurable.

21


Customizing Docker Java library

Introduce custom configuration in plugin.

22

project.configurations.create('docker') .setVisible(false) .setTransitive(true) .setDescription('The Docker Java libraries

to be used.') }

Setting dependency from consuming build.

dependencies { docker 'com.github.docker-

java:docker-java:2.0.0' }


Providing default (but customizable) dependency

Only use defaults, if no dependencies are assigned to custom configuration.

23

Configuration config = project.configurations.create('docker')

project.tasks.withType(AbstractDockerRemoteApiTask) { config.defaultDependencies { deps -> deps.add(project.dependencies .create('com.github.docker-java:docker-java:2.1.0')) deps.add(project.dependencies .create('org.slf4j:slf4j-simple:1.7.5')) } conventionMapping.classpath = { config } } }


Capabilities vs. conventions

Separating general-purpose functionality from pre-configured, opinionated functionality.

Finding the right balance between both aspects is key.

24


Capabilities

• Un-opinionated functionality

• Provide general-purpose concepts

Examples:

• Custom task types without creating an actual task instance

• Add new concepts without configuring them (e.g. source sets, build types and flavors)

25


Conventions

• Opinionated functionality

• Instantiate and/or pre-configure concepts

Examples:

• Standardized directory layouts

• Enhanced tasks created by a plugin

• Adding default values to extensions

• Declaring task dependencies to form a lifecycle

26


Plugin composition

Plugins can build upon other plugins. This is a common pattern.

27

A base plugin provides generic capabilities

Another plugin builds on the base, adding opinionated conventions


Example: Docker plugins

Allows users to pick functionality they need in their projects.

28

Plugin that adds task types for interacting with Docker remote API

Plugin for creating & pushing Docker image for Java applications


Example: Docker Java application plugin

Applying plugin by identifier or type.

29

import org.gradle.api.Plugin import org.gradle.api.Project

class DockerJavaApplicationPlugin implements Plugin<Project> { @Override void apply(Project project) { project.apply(plugin: DockerRemoteApiPlugin) } }


Build for further extension

• Anticipate more specific extensions to your plugin

• Implemented through plugin composition

• Base plugin act as enablers for custom convention plugins

• Enterprise build convention plugins re-configure your plugins

30


Put yourself in the shoes of consumer

Do not automatically apply plugins in your plugin!

Reason:

• Makes your plugin highly opinionated.

31


Reacting to plugins

React to plugins that might be applied in consuming build script. Only then logic is executed.

The same concept applies to tasks.

32


Using configuration rules

33

project.plugins.withType(IdeaPlugin) { // configure Idea plugin in the

context of your plugin }

Reacting to task types

project.tasks.withType(War) { // configure all War tasks in the

context of your plugin }

Reacting to plugins


Testing build logic

34

Build logic needs to be testable on multiple levels.


Implementing test types with Gradle

35

Unit testing:

No Gradle tooling needed

Integration testing:

Use ProjectBuilder to create pseudo Project instance

Functional testing:

Nebula Test, Gradle TestKit


Example: Writing unit test with Spock

36

package com.bmuschko.gradle.docker.tasks.image import spock.lang.Specification import static com.bmuschko.gradle.docker.tasks.image.Dockerfile.*

class DockerfileTest extends Specification { def "Instruction String representation is built correctly"() { expect: instructionInstance.keyword == keyword instructionInstance.build() == builtInstruction where: instructionInstance | keyword | builtInstruction new FromInstruction('ubuntu:14.04') | 'FROM' | 'FROM ubuntu:14.04' new FromInstruction({ 'ubuntu:14.04' }) | 'FROM' | 'FROM ubuntu:14.04' } }


Separating test type source code

37

Default unit test directory

Custom integration test directory

Custom functional test directory


Example: Integration test source set and task

38

sourceSets { integrationTest { groovy.srcDir file('src/integTest/groovy') resources.srcDir file('src/integTest/resources') compileClasspath += sourceSets.main.output + configurations.testRuntime runtimeClasspath += output + compileClasspath } }

task integrationTest(type: Test) { description = 'Runs the integration tests.' group = 'verification' testClassesDir = sourceSets.integrationTest.output.classesDir classpath = sourceSets.integrationTest.runtimeClasspath mustRunAfter test }


Example: Writing integration test with Spock

39

import org.gradle.api.Project import org.gradle.testfixtures.ProjectBuilder

class DockerJavaApplicationPluginIntegrationTest extends Specification { @Rule TemporaryFolder temporaryFolder = new TemporaryFolder() Project project = ProjectBuilder.builder().withProjectDir(temporaryFolder.root).build()

def "Creates tasks out-of-the-box when application plugin is applied"() { when: project.apply(plugin: DockerJavaApplicationPlugin) project.apply(plugin: 'application') then: project.tasks.findByName(DockerJavaApplicationPlugin.DOCKERFILE_TASK_NAME) } }


Functional testing

40

Testing the build logic from the end user’s perspective.

• Exercising build logic as part of a programmatically executed build.

• Make assertions about build outcome.


Using the Gradle TestKit

41

Functional testing support in Gradle core.

• Uses Tooling API as test execution harness.

• Agnostic of test framework.

• Assertions made based on build output, build logging or test of tasks + their result.


TestKit usage

42

Declaring TestKit dependency

Declaring the Spock dependency

dependencies { testCompile gradleTestKit() }

dependencies { testCompile

'org.spockframework:spock-core:1.0-groovy-2.3'

}


Example: Writing functional tests with TestKit

43

def "can successfully create Dockerfile"() { given: buildFile << """ import com.bmuschko.gradle.docker.tasks.image.Dockerfile task dockerfile(type: Dockerfile) { from 'ubuntu:14.04' maintainer 'Benjamin Muschko "[email protected]"' } “”"

when: def result = GradleRunner.create().withProjectDir(testProjectDir.root).withArguments('dockerfile').build() then: result.task(":dockerfile").outcome == SUCCESS testProjectDir.file('Dockerfile').exists() }


DEMO

44

Functional testing with TestKit


Roadmap for TestKit

45

Gradle 2.6

Mechanics for executing tests

Functional tests can query build result

Gradle 2.7

Isolation of test environment, dedicated daemons

Gradle 2.8

Convenient injection of code under test


Cross-version compatibility tests

46

Forward and backward compatibility independent of Gradle version used to build plugin artifact.

2.4 2.5 2.62.32.2

Version used to build plugin


Implementing compatibility tests

47

No built-in support in Gradle yet. On the roadmap for TestKit.

Intermediate options:

Custom implementation using Tooling API.

Community plugins like https://github.com/ysb33r/gradleTest

https://github.com/ysb33r/gradleTest


Importance of documentation

48

Plugins are not self-documenting.

Thorough documentation is crucial for plugin consumers.


Answer questions for consumers

49

How can I use the plugin and configure it?

What tasks & extensions are provided?

What impact does the plugin have on my project?


Impact on consumers

50

De-mystify added functionality.

Give consumers the feeling that they are under control.


What should be documented?

51

• Purpose of plugin, the repository that hosts the plugin

• Plugins: Identifier, type and description bundled in a JAR

• Enhanced tasks: their name, type and dependencies

• Custom task types: Javadocs, Groovydocs

• Conventions, custom extensions (e.g. DSLs)

• Extension properties/methods

• Usage examples in textual form

• A good set of functional tests that demonstrate the use of the plugin


Example: Plugin description & usage

52


Example: Extension usage and properties

53


Example: Textual usage & functional tests

54


Example: Linking custom tasks to API docs

55


Example: Publishing API docs to GitHub pages

56

Javadocs/Groovydocs are essential to allow users discover API classes exposed by plugin.

• Create new branch gh-pages

• Remove all files from working tree and index

• Push new branch

Available under

http(s)://<username>.github.io/<projectname>


Example: Publishing Javadocs to GitHub pages

57

Use gradle-git plugin to push to automatically publish Javadocs.

apply plugin: 'org.ajoberstar.github-pages'

githubPages { repoUri = '[email protected]:bmuschko/gradle-docker-plugin.git' pages { from(javadoc.outputs.files) { into 'docs/javadoc' } } }


DEMO

58

Publishing changed API documentation


Publishing the plugin binaries

59

Make artifact(s) available to consumers by uploading them to one or many binary repositories.


Provide appropriate metadata

60

apply plugin: 'maven-publish'

publishing { publications { mavenJava(MavenPublication) { pom.withXml { def root = asNode() root.appendNode('name', 'Gradle Docker plugin') root.appendNode('description', 'Gradle plugin for managing Docker images & containers.') root.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin') root.appendNode('inceptionYear', '2014') } } } }


Publishing doesn’t stop with plugin JAR

61

task sourcesJar(type: Jar) { classifier 'sources' from sourceSets.main.allSource }

task groovydocJar(type: Jar, dependsOn: groovydoc) { classifier 'groovydoc' from groovydoc.destinationDir }

task javadocJar(type: Jar, dependsOn: javadoc) { classifier 'javadoc' from javadoc.destinationDir }


Share Open Source plugins on the portal

62

https://plugins.gradle.org/

https://plugins.gradle.org/


Publishing to the Gradle plugin portal

63

Requires use of the “Plugin Publishing Plugin”buildscript { repositories { maven { url 'https://plugins.gradle.org/m2/' } } dependencies { classpath 'com.gradle.publish:plugin-publish-plugin:0.9.1' } }

apply plugin: 'com.gradle.plugin-publish'


Declaring required portal plugin metadata

64

Requires use of the “Plugin Publishing Plugin”pluginBundle { website = 'https://github.com/bmuschko/gradle-docker-plugin' vcsUrl = 'https://github.com/bmuschko/gradle-docker-plugin.git' description = 'Gradle plugin for managing Docker images and containers.' tags = ['gradle', 'docker', 'container', 'image', 'lightweight', 'vm', 'linux'] plugins { dockerRemoteApi { id = 'com.bmuschko.docker-remote-api' displayName = 'Provides custom tasks for interacting with Docker remote API.' } } }


Continuous Integration

65

A plugin might start small but will grow in complexity over time. Avoid integration hell!

• Fast feedback with every code integration

• Always execute (an assorted set of) tests

• Use CI cloud-based product or host in-house


Adding a CI badge

66

Give consumers and contributor a chance to see the build status.


Working towards Continuous Delivery

67

Make sure your plugin is always production ready.

• End goal: avoid manual steps

• Build and visualize suitable delivery pipeline

• Go the extra mile - model push-button release capabilities


Example: Potential release steps

68

PubliAssemble artifact(s)

Tag VCS repository

Publish artifact(s)

Plugin JAR Sources JAR

Docs JAR

Create tag Push tag

Create metadata Upload artifact(s) Update API docs


DEMO

69

Implementing a plugin release process


Example: Build pipeline with Snap CI

70

Build, test and deploy in the cloud.

• Free of charge for public Github repos

• Model build pipelines with automatic and/or manual execution steps

• Execute build for pull requests


Example: Snap CI pipeline for Docker plugin

71

Compilation Unit tests

Integration tests

Functional tests

Release

https://snap-ci.com/bmuschko/gradle-docker-plugin/branch/master

https://snap-ci.com/bmuschko/gradle-docker-plugin/branch/master


DEMO

72

Showcasing build pipeline on Snap CI


Other aspects

73

• Be mindful of the performance impact your plugin might have on consuming build

• Avoid using internal Gradle APIs as much as possible


Thank You!

74

Please ask questions…

https://www.github.com/bmuschko

@bmuschko

https://www.github.com/bmuschko

Gradle Plugin Best Practices by Example

Technology

Transcript of Gradle Plugin Best Practices by Example