Gradle Plugin Best Practices by Example

Unless otherwise indicated, these s l ides are © 2013-2015 Pivotal Software, Inc. and l icensed under a Creat ive Commons Attr ibut ion-NonCommercial l icense: ht tp: / /creat ivecommons.org/ l icenses/by-nc/3.0/

SPRINGONE2GXWASHINGTON, DC

Gradle plugin best practices by example

By Benjamin Muschko Gradle Inc.

The dark past of build logic

Unstructured spaghetti code

Copy & paste of code snippets

The build tool tells you how to structure code

Build can only be understood by build guru™

Testing through manual execution

Build logic “-ilities”

Typical non-functional software requirements also apply to build code…

• Reusability

• Testability

• Maintainability

• Extensibility

• Configurability

Reusability

Avoid copy/paste! Allow code to be shared among independent projects.

Testability

Build code is no different from application code. Test it on multiple levels!

Maintainability

Avoid the big ball of mud! Cohesion and separation of concerns are important.

Extensibility

Extend Gradle's build language by your own declarative & expressive language constructs.

Configurability

Don't box in your users! Implement convention over configuration with ease.

The end goal

Maintainable, reusable & tested code

Consumers only configure the code

Complex implement. details are hidden

How to get there?

Concepts that help implement these requirements…

• Good software engineering practices

• Custom tasks

• Plugins

• Gradle's extension mechanisms

• Testing capabilities

Techniques covered in talk

• Declarative vs. imperative logic

• Convention over configuration

• Capabilities vs. conventions

• Gradle's extension mechanisms

• Testing plugin logic

• Publishing the plugin artifacts

• Writing and generating documentation

• Setting up Continuous Integration/Delivery

Case study: Gradle Docker plugin

Serves as showcase plugin project.

• Plugin for managing Docker images and containers through Docker remote API

• API communication via Docker Java library

• Written in Groovy

• Source code available on GitHub (https://github.com/bmuschko/gradle-docker-plugin)

Always check in the Wrapper

Project becomes instantly buildable for every developer and on the CI machine.

Check in Wrapper files into VCS repository.

Hiding complex, imperative logic

Concept applies to tasks and plugins.

• Reusable and configurable

• Easy to structure, refactor and test

• Avoid global properties and methods

Task type > ad-hoc task

Prefer implementing task types to implementing ad-hoc tasks.

task createDockerfile(type: Dockerfile) { destFile = file("$buildDir/mydockerfile/Dockerfile") from 'ubuntu:12.04' maintainer 'Benjamin Muschko "benjamin.muschko@gmail.com"' }

task buildImage(type: DockerBuildImage) { dependsOn createDockerfile inputDir = createDockerfile.destFile.parentFile tag = 'bmuschko/myimage' }

Binary plugin > script plugin

Use script plugins to organize build logic based on functional boundaries in project.

apply from: "$rootDir/gradle/dependencies.gradle" apply from: "$rootDir/gradle/test-setup.gradle" apply from: "$rootDir/gradle/integration-test.gradle" apply from: "$rootDir/gradle/functional-test.gradle" apply from: "$rootDir/gradle/additional-artifacts.gradle" apply from: "$rootDir/gradle/publishing.gradle" apply from: "$rootDir/gradle/documentation.gradle" apply from: "$rootDir/gradle/release.gradle"

Convention over configuration

Provide sensible defaults and standards. Expose a way to re-configure them to user’s needs.

Examples:

• src/main/java for Java-based applications

• Project name derived from directory name

• Output directory is build

Convention

Plugin consumers will be most comfortable with using convention plugins. Pick sensible defaults for the user.

Good indicator:

The less a consumer has to re-configure defaults the better.

Configuration

Real-world project might need to re-configure the defaults. Make it convenient to change defaults.

Reason:

Their view of the world might look different than yours.

Users only declare the “what”

Expose a custom DSL from your binary plugin to configure runtime behavior. The “how” is the responsibility of the plugin implementation.

apply plugin: 'com.bmuschko.docker-remote-api'

docker { url = 'https://192.168.59.103:2376' certPath = new File(System.properties['user.home'], '.boot2docker/certs/boot2docker-vm') }

Example: Providing default dependencies

Plugins often rely on external libraries. Automatically resolve default version, but make it configurable.

Customizing Docker Java library

Introduce custom configuration in plugin.

project.configurations.create('docker') .setVisible(false) .setTransitive(true) .setDescription('The Docker Java libraries

to be used.') }

Setting dependency from consuming build.

dependencies { docker 'com.github.docker-

java:docker-java:2.0.0' }

Providing default (but customizable) dependency

Only use defaults, if no dependencies are assigned to custom configuration.

Configuration config = project.configurations.create('docker')

project.tasks.withType(AbstractDockerRemoteApiTask) { config.defaultDependencies { deps -> deps.add(project.dependencies .create('com.github.docker-java:docker-java:2.1.0')) deps.add(project.dependencies .create('org.slf4j:slf4j-simple:1.7.5')) } conventionMapping.classpath = { config } } }

Capabilities vs. conventions

Separating general-purpose functionality from pre-configured, opinionated functionality.

Finding the right balance between both aspects is key.

Capabilities

• Un-opinionated functionality

• Provide general-purpose concepts

Examples:

• Custom task types without creating an actual task instance

• Add new concepts without configuring them (e.g. source sets, build types and flavors)

Conventions

• Opinionated functionality

• Instantiate and/or pre-configure concepts

Examples:

• Standardized directory layouts

• Enhanced tasks created by a plugin

• Adding default values to extensions

• Declaring task dependencies to form a lifecycle

Plugin composition

Plugins can build upon other plugins. This is a common pattern.

A base plugin provides generic capabilities

Another plugin builds on the base, adding opinionated conventions

Example: Docker plugins

Allows users to pick functionality they need in their projects.

Plugin that adds task types for interacting with Docker remote API

Plugin for creating & pushing Docker image for Java applications

Example: Docker Java application plugin

Applying plugin by identifier or type.

import org.gradle.api.Plugin import org.gradle.api.Project

class DockerJavaApplicationPlugin implements Plugin<Project> { @Override void apply(Project project) { project.apply(plugin: DockerRemoteApiPlugin) } }

Build for further extension

• Anticipate more specific extensions to your plugin

• Implemented through plugin composition

• Base plugin act as enablers for custom convention plugins

• Enterprise build convention plugins re-configure your plugins

Put yourself in the shoes of consumer

Do not automatically apply plugins in your plugin!

Reason:

• Makes your plugin highly opinionated.

Reacting to plugins

React to plugins that might be applied in consuming build script. Only then logic is executed.

The same concept applies to tasks.

Using configuration rules

project.plugins.withType(IdeaPlugin) { // configure Idea plugin in the

context of your plugin }

Reacting to task types

project.tasks.withType(War) { // configure all War tasks in the

context of your plugin }

Reacting to plugins

Testing build logic

Build logic needs to be testable on multiple levels.

Implementing test types with Gradle

Unit testing:

No Gradle tooling needed

Integration testing:

Use ProjectBuilder to create pseudo Project instance

Functional testing:

Nebula Test, Gradle TestKit

Example: Writing unit test with Spock

package com.bmuschko.gradle.docker.tasks.image import spock.lang.Specification import static com.bmuschko.gradle.docker.tasks.image.Dockerfile.*

class DockerfileTest extends Specification { def "Instruction String representation is built correctly"() { expect: instructionInstance.keyword == keyword instructionInstance.build() == builtInstruction where: instructionInstance | keyword | builtInstruction new FromInstruction('ubuntu:14.04') | 'FROM' | 'FROM ubuntu:14.04' new FromInstruction({ 'ubuntu:14.04' }) | 'FROM' | 'FROM ubuntu:14.04' } }

Separating test type source code

Default unit test directory

Custom integration test directory

Custom functional test directory

Example: Integration test source set and task

sourceSets { integrationTest { groovy.srcDir file('src/integTest/groovy') resources.srcDir file('src/integTest/resources') compileClasspath += sourceSets.main.output + configurations.testRuntime runtimeClasspath += output + compileClasspath } }

task integrationTest(type: Test) { description = 'Runs the integration tests.' group = 'verification' testClassesDir = sourceSets.integrationTest.output.classesDir classpath = sourceSets.integrationTest.runtimeClasspath mustRunAfter test }

Example: Writing integration test with Spock

import org.gradle.api.Project import org.gradle.testfixtures.ProjectBuilder

class DockerJavaApplicationPluginIntegrationTest extends Specification { @Rule TemporaryFolder temporaryFolder = new TemporaryFolder() Project project = ProjectBuilder.builder().withProjectDir(temporaryFolder.root).build()

def "Creates tasks out-of-the-box when application plugin is applied"() { when: project.apply(plugin: DockerJavaApplicationPlugin) project.apply(plugin: 'application') then: project.tasks.findByName(DockerJavaApplicationPlugin.DOCKERFILE_TASK_NAME) } }

Functional testing

Testing the build logic from the end user’s perspective.

• Exercising build logic as part of a programmatically executed build.

• Make assertions about build outcome.

Using the Gradle TestKit

Functional testing support in Gradle core.

• Uses Tooling API as test execution harness.

• Agnostic of test framework.

• Assertions made based on build output, build logging or test of tasks + their result.

TestKit usage

Declaring TestKit dependency

Declaring the Spock dependency

dependencies { testCompile gradleTestKit() }

dependencies { testCompile

'org.spockframework:spock-core:1.0-groovy-2.3'

Example: Writing functional tests with TestKit

def "can successfully create Dockerfile"() { given: buildFile << """ import com.bmuschko.gradle.docker.tasks.image.Dockerfile task dockerfile(type: Dockerfile) { from 'ubuntu:14.04' maintainer 'Benjamin Muschko "benjamin.muschko@gmail.com"' } “”"

when: def result = GradleRunner.create().withProjectDir(testProjectDir.root).withArguments('dockerfile').build() then: result.task(":dockerfile").outcome == SUCCESS testProjectDir.file('Dockerfile').exists() }

Functional testing with TestKit

Roadmap for TestKit

Gradle 2.6

Mechanics for executing tests

Functional tests can query build result

Gradle 2.7

Isolation of test environment, dedicated daemons

Gradle 2.8

Convenient injection of code under test

Cross-version compatibility tests

Forward and backward compatibility independent of Gradle version used to build plugin artifact.

2.4 2.5 2.62.32.2

Version used to build plugin

Implementing compatibility tests

No built-in support in Gradle yet. On the roadmap for TestKit.

Intermediate options:

Custom implementation using Tooling API.

Community plugins like https://github.com/ysb33r/gradleTest

Importance of documentation

Plugins are not self-documenting.

Thorough documentation is crucial for plugin consumers.

Answer questions for consumers

How can I use the plugin and configure it?

What tasks & extensions are provided?

What impact does the plugin have on my project?

Impact on consumers

De-mystify added functionality.

Give consumers the feeling that they are under control.

What should be documented?

• Purpose of plugin, the repository that hosts the plugin

• Plugins: Identifier, type and description bundled in a JAR

• Enhanced tasks: their name, type and dependencies

• Custom task types: Javadocs, Groovydocs

• Conventions, custom extensions (e.g. DSLs)

• Extension properties/methods

• Usage examples in textual form

• A good set of functional tests that demonstrate the use of the plugin

Example: Plugin description & usage

Example: Extension usage and properties

Example: Textual usage & functional tests

Example: Linking custom tasks to API docs

Example: Publishing API docs to GitHub pages

Javadocs/Groovydocs are essential to allow users discover API classes exposed by plugin.

• Create new branch gh-pages

• Remove all files from working tree and index

• Push new branch

Available under

http(s)://<username>.github.io/<projectname>

Example: Publishing Javadocs to GitHub pages

Use gradle-git plugin to push to automatically publish Javadocs.

apply plugin: 'org.ajoberstar.github-pages'

githubPages { repoUri = 'git@github.com:bmuschko/gradle-docker-plugin.git' pages { from(javadoc.outputs.files) { into 'docs/javadoc' } } }

Publishing changed API documentation

Publishing the plugin binaries

Make artifact(s) available to consumers by uploading them to one or many binary repositories.

Provide appropriate metadata

apply plugin: 'maven-publish'

publishing { publications { mavenJava(MavenPublication) { pom.withXml { def root = asNode() root.appendNode('name', 'Gradle Docker plugin') root.appendNode('description', 'Gradle plugin for managing Docker images & containers.') root.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin') root.appendNode('inceptionYear', '2014') } } } }

Publishing doesn’t stop with plugin JAR

task sourcesJar(type: Jar) { classifier 'sources' from sourceSets.main.allSource }

task groovydocJar(type: Jar, dependsOn: groovydoc) { classifier 'groovydoc' from groovydoc.destinationDir }

task javadocJar(type: Jar, dependsOn: javadoc) { classifier 'javadoc' from javadoc.destinationDir }

Share Open Source plugins on the portal

https://plugins.gradle.org/

Publishing to the Gradle plugin portal

Requires use of the “Plugin Publishing Plugin”buildscript { repositories { maven { url 'https://plugins.gradle.org/m2/' } } dependencies { classpath 'com.gradle.publish:plugin-publish-plugin:0.9.1' } }

apply plugin: 'com.gradle.plugin-publish'

Declaring required portal plugin metadata

Requires use of the “Plugin Publishing Plugin”pluginBundle { website = 'https://github.com/bmuschko/gradle-docker-plugin' vcsUrl = 'https://github.com/bmuschko/gradle-docker-plugin.git' description = 'Gradle plugin for managing Docker images and containers.' tags = ['gradle', 'docker', 'container', 'image', 'lightweight', 'vm', 'linux'] plugins { dockerRemoteApi { id = 'com.bmuschko.docker-remote-api' displayName = 'Provides custom tasks for interacting with Docker remote API.' } } }

Continuous Integration

A plugin might start small but will grow in complexity over time. Avoid integration hell!

• Fast feedback with every code integration

• Always execute (an assorted set of) tests

• Use CI cloud-based product or host in-house

Adding a CI badge

Give consumers and contributor a chance to see the build status.

Working towards Continuous Delivery

Make sure your plugin is always production ready.

• End goal: avoid manual steps

• Build and visualize suitable delivery pipeline

• Go the extra mile - model push-button release capabilities

Example: Potential release steps

PubliAssemble artifact(s)

Tag VCS repository

Publish artifact(s)

Plugin JAR Sources JAR

Docs JAR

Create tag Push tag

Create metadata Upload artifact(s) Update API docs

Implementing a plugin release process

Example: Build pipeline with Snap CI

Build, test and deploy in the cloud.

• Free of charge for public Github repos

• Model build pipelines with automatic and/or manual execution steps

• Execute build for pull requests

Example: Snap CI pipeline for Docker plugin

Compilation Unit tests

Integration tests

Functional tests

Release

https://snap-ci.com/bmuschko/gradle-docker-plugin/branch/master

Showcasing build pipeline on Snap CI

Other aspects

• Be mindful of the performance impact your plugin might have on consuming build

• Avoid using internal Gradle APIs as much as possible

Thank You!

Please ask questions…

https://www.github.com/bmuschko

@bmuschko

Gradle Plugin Best Practices by Example

Technology

Transcript of Gradle Plugin Best Practices by Example

The simple & securechallenge-hacktion.innocherche.com/.../09/Appaloosa-Presentation-1… · Jenkins CI Plugin Version history Cloud build Fastlane, Gradle Beta campaigns Crashlytics

Elegant Builds at Scale - Jfokus · Introduction to Gradle Gradle for Android Gradle C/C++ Workshop Advanced Gradle Fundamentals Advanced Extending Gradle Mastering Dependencies and

Rocking the Gradle! - java.ociweb.comjava.ociweb.com/javasig/knowledgebase/2013-04/gradle-intro.pdf · Rocking the Gradle! Friday, April 12, 13. 2 Why Gradle? Friday, April 12, 13.

openTCS: Developer’s Guide · 2019-12-10 · In case you use NetBeans, install the Gradle Support plugin from the NetBeans plugin portal. You should then be able to open the source

Family Nessus Plugin TENABLE NETWORK SECURITYstatic.tenable.com/oldsite/blog/files/example---nessus-plugin... · DNS ... Tenable Network Security 1 Plugin Family Summary ... Mandriva

Jenkins user plugin This time it's user plugin This time it's #jenkinsconf. Jenkins User Conference Israel #jenkinsconf About me ... Why Gradle? Build on top of ...

gradle user to addict sf - Meetupfiles.meetup.com/1715787/gradle_user_to_addict.pdf · build.gradle apply plugin: 'android' android { compileSdkVersion 19 buildToolsVersion "19.0.3"

openTCS: Developer’s Guide · 2019-10-22 · In case you use NetBeans, install the Gradle Support plugin from the NetBeans plugin portal. You should then be able to open the source

gradle user to addict sf - files.meetup.comfiles.meetup.com/1715787/gradle_user_to_addict_notes.pdf · build.gradle apply plugin: ‘com.android.application’ * Ok, so the ﬁrst

The Gradle build system · Note! The Android plugin for Gradle in the /build.gradle file must be synced with the distributionUrl gradle version in the gradle-wrapper.properties and

Droidcon Create Your Gradle Plugin

Mastering the NDK with Android Studio 2.0 and the gradle-experimental plugin

Releasing with gradle (gradle summit 2014)

Setting up Sudoku example on Android Studio · project to Android Gradle as Android Studio uses Gradle build. 6. It will display a summary of the changes done as Android Gradle projects

Gradle presentation

Extending Gradle

Idiomatic Gradle Plugin Writing - GradleSummit 2016

GRADLE 4 - JUG · PDF fileAndroid • Gradle tasks of Android plugin will become cacheable

with Gradle Android Development - Rich Web Experience · PDF fileAndroid Development with Gradle Using the Android Plugin for Gradle

Gradle Userguide