@quidryan @quidryan http://www.slideshare.net/quidryan

Netflix Build Language

Justin Ryan <jryan@netflix.com>

@quidryan @quidryan http://www.slideshare.net/quidryan

Justin Ryan <jryan@netflix.com>

Why Netflix Uses Gradle

• Better Dependency Management story

• Flexible lifecycle

• Groovy

How Netflix Uses Gradle

• JVM Languages

• Resolution

• Code Quality

• Publishing

• Deployment Orchestration

How Netflix Sets up Gradle

• Patched Gradle

• Custom Distribution

• Custom Wrapper

apply plugin: ‘nebula’!apply plugin: ‘java’!!nebula {! readyForJava7 = true!}!!dependencies {! compile ‘netflix:platform:latest.release’!}

build.gradle

@NetflixOSS

github.com/Netflix

netflix.github.io

Unite two builds

• Model a responsible project

• Componentize via plugins

nebula-plugins

• Infrastructure

• Use Github

• Use CloudBees

• Use Bintray

• Mailing List

• nebula-plugin-plugin

Github Repositoriesnebula-* or gradle-*-plugin

Ensuring Github• https://github.com/nebula-plugins/ensure

• Ensure repository has

• Description

• Web Hooks

• Ensure “contrib" team has all repositories in it

• Ensure a contrib team exists for every repository

Continuous Integration

• Job DSL to create jobs

• Per branch

• Snapshot job

• Release job

• Lock job

CloudBees jobsRelease, snapshot, and pull request per branch

Job DSL

Bintray Packages

Ensuring Bintray• https://github.com/nebula-plugins/ensure

• Ensure every repository has a package

• Ensure package has the description as Github

• Ensure license is set to Apache 2.0

• Ensure labels are “gradle" and “nebula”

nebula-plugin-plugin

nebula-test• ProjectSpec

• PluginProjectSpec

• IntegrationSpec

• Thanks to Marcin and Luke

• Runs with Tooling API or GradleLauncher

class PluginExampleSpec extends PluginProjectSpec {! @Override! String getPluginName() { return 'plugin-example' }!! def ‘run task’() {! when:! project.plugins.apply(PluginExample)!! then:! def t = project.tasks.get(‘example’)!! when:! t.run()!! then:! new File(projectDir, ‘build/example.txt’).exists()! }!}

PluginProjectSpec

!def 'setup and run build'() {! buildFile << '''! apply plugin: 'java'! '''.stripIndent()!! when:! writeHelloWorld('nebula.hello')!! then:! fileExists('src/main/java/nebula/hello/HelloWorld.java')!! when:! def result = runTasksSuccessfully(‘build’, ‘-v’)!! then:! fileExists(‘build/classes/main/nebula/hello/HelloWorld.class')! result.wasExecuted(':compileTestJava')! def output = result.standardOutput! output.contains('Skipping task \’:compileTestJava\'')!}

IntegrationSpec

nebula-core• Collection of tasks

• Download

• Untar

• Unzip

• AlternativeArchiveTask

• CopySpecHelper

• GradleHelper

class GradleHelper {!! def beforeEvaluate(Closure beforeEvaluateClosure)!! def getTempDir(String taskBaseName)!! def addDefaultGroup(String defaultGroup)!}!!class CopySpecHelper {!! def visitCopySpec(CopySpecInternal copySpec, Closure closure)!! def findCopySpec(CopySpecInternal delegateCopySpec, closure)!}!!class ClassHelper {!! String findSpecificationVersion(Class clazz)!! Manifest findManifest(Class clazz)!! def findManifestValue(Class clazz, String key, defaultValue)!}

nebula-core helpers

nebula-publishing-plugin• Artifact plugins

• nebula-javadoc-jar

• nebula-source-jar

• nebula-test-jar

• Publishing plugins

• resolved-ivy

• resolved-maven

• nebula-sign

apply plugin: ‘nebula-maven-publishing‘!apply plugin: ‘nebula-source-jar'!apply plugin: ‘nebula-javadoc-jar'!apply plugin: ‘nebula-test-jar'!apply plugin: ‘nebula-sign'!apply plugin: 'java'

nebula-publishing.gradle

my-plugin-1.12.0-javadoc.jar!my-plugin-1.12.0-javadoc.jar.md5!my-plugin-1.12.0-javadoc.jar.sha1!my-plugin-1.12.0-javadoc.jar.asc!my-plugin-1.12.0-sources.jar!my-plugin-1.12.0-sources.jar.md5!my-plugin-1.12.0-sources.jar.sha1!my-plugin-1.12.0-sources.jar.asc!my-plugin-1.12.0-tests.jar!my-plugin-1.12.0-tests.jar.md5!my-plugin-1.12.0-tests.jar.sha1!my-plugin-1.12.0-tests.jar.asc!my-plugin-1.12.0.jar!my-plugin-1.12.0.jar.md5!my-plugin-1.12.0.jar.sha1!my-plugin-1.12.0.jar.asc!my-plugin-1.12.0.pom!my-plugin-1.12.0.pom.md5!my-plugin-1.12.0.pom.sha1!my-plugin-1.12.0.pom.asc

gradle-info-plugin• Collects meta data

• ‘info-java'

• ‘info-ci'

• 'info-scm'

• Reports in key/value pairs

• ‘info-jar'

• ‘info-props'

buildscript {!! repositories { jcenter() }!! dependencies { !! ! classpath 'com.netflix.nebula:gradle-info-plugin:1.12.+'!! ! classpath ‘org.eclipse.jgit:org.eclipse.jgit:3.2.0.201312181205-r'!! }!}!apply plugin: 'info'

info.gradle

Manifest-Version=1.0!Implementation-Title=com.netflix.nebula#my-plugin;1.12.1-SNAPSHOT!Implementation-Version=1.12.1-SNAPSHOT!Built-Status=integration!Built-By=jryan!Build-Date=2014-06-10_13:30:44!Gradle-Version=1.12-20140608201532+0000!Module-Source=!Module-Origin=git@github.com:nebula-plugins/my-plugin.git!Change=976292c!Build-Host=localhost!Build-Job=LOCAL!Build-Number=LOCAL!Build-Id=LOCAL!Created-By=1.7.0_45-b18 (Oracle Corporation)!Build-Java-Version=1.7.0_45!Module-Owner=justin@halfempty.org!Module-Email=justin@halfempty.org!X-Compile-Target-JDK=1.7!X-Compile-Source-JDK=1.7

info.gradle output

gradle-contacts-plugin• Express people involved in the project

• Make people and roles available to other plugins

• contacts-manifest

• contacts-pom

apply plugin: 'contacts'!contacts 'minnie@disney.com', ‘mickey@disney.com'!!contacts {! 'club@disney.com'! 'bobby@company.com' {! roles 'notify', 'owner'! }! 'billy@company.com' {! role 'techwriter'! }! 'downstream@netflix.com'! role 'notify'! }!}!

contacts.gradle

gradle-scm-plugin• Attempt to provide SCM abstraction for other plugins

• E.g. gradle-dependency-lock-plugin and gradle-info-plugin

gradle-dependency-lock-plugin

• Developer declare their ideal situation

• Save resolved version

• If tests pass, commit to SCM

{! "com.github.townsfolk:gradle-release": { !! ! "locked": "1.2", "requested": "1.2" },! "com.jfrog.bintray.gradle:gradle-bintray-plugin": { !! ! "locked": "0.3", "requested": "0.3" },! "com.netflix.nebula:nebula-project-plugin": { !! ! "locked": "1.12.0", "requested": "1.12.+" },! "com.netflix.nebula:nebula-test": { !! ! "locked": "1.12.0", "requested": "1.12.+" },! "org.codehaus.groovy.modules.http-builder:http-builder": { !! ! "locked": "0.7.1", "requested": “latest.release" },! "org.jfrog.buildinfo:build-info-extractor-gradle": { !! ! "locked": "2.2.4", "requested": "2.2.+" }!}

apply plugin: ‘gradle-dependency-lock'!

lock.gradle

./gradlew generateLock

nebula-integtest-plugin• Sets up integTest source set

• Adds integTestCompile and integTestRuntime configurations

• Creates integrationTest task

nebula-project-plugin

• Pull together other plugins

• Responsible projects

nebula-plugin-plugin

• Used by plugins

• Strong opinions on how to publish

• Force nebula-project-plugin on projects

gradle-ospackage-plugin• Merging ubuntu-packager-plugin and gradle-rpm-

plugin

• Uses CopySpec definition

• Via just Java, generates RPMs and DEBs

ospackage {!! os = LINUX!! into '/opt/foo'!! from ('dist') {!! ! user 'builds'!! ! exclude '**/*.md'!! }!! postInstall file('scripts/postInstall.sh')!}!!buildRpm {!! requires('bar', '2.2', GREATER | EQUAL)!! from (‘build/metadata.properties’)!! link(‘/etc/init.d/foo’, '/opt/foo/bin/foo.sysv',)!}!!buildDeb {!! link('/etc/init/foo', '/opt/foo/bin/foo.upstart')!}

ospackage.gradle

gradle-override-plugin• Take command line arguments

• Intelligently apply in afterEvaluate

• E.g. -Nfindbugs.enabled=false

Internal Plugins

• netflix-repos

• nebula-ospackage

• nebula-grails

• nebula-findbugs, etc

• ivyimport

• nebula-fixexcludes

• nebula-intellij

buildscript {!! repositories { jcenter() }!! dependencies { !! ! classpath ‘com.netflix.nebula:nebula-plugin-plugin:1.12.+'!! ! classpath ‘org.eclipse.jgit:org.eclipse.jgit:3.2.0.201312181205-r'!! }!}!!description ‘Example Plugin'!apply plugin: ‘nebula-plugin'!!contacts {! ‘jryan@netflix.com’ {! moniker 'Justin Ryan'! github 'quidryan'! }!}

plugin.gradle

curl -s get.gvmtool.net | bash!gvm install lazybones!cd ~/Projects/github/nebula-plugins!lazybones create nebula-plugin <name-of-project>!cd <name-of-project>!git init!git remote add origin git@github.com:nebula-plugins/<name-of-project>.git!./gradlew clean build!git add -A!git add -f gradle/wrapper/gradle-wrapper.jar!git commit -m "Initial template”!git push --set-upstream origin master

Making a plugin

https://github.com/nebula-plugins/nebula-plugins.github.io/wiki/New-Plugins

Getting it out the door • Let “ensure” run

• Run <name-of-project>-release

• Link package to jcenter

• Link package to gradle-plugins

Unopinionated Plugins

Opinionated Plugins

Gotchas

• NamedDomainObjectSet

• Debugging Tooling

• File as @Output

• afterEvaluate

Outstanding• CloudBees permissions

• Bot to create repositories

Participating

• Use individual plugins

• Get on nebula-plugins Google Group

• Move your plugin to nebula-plugins

• Start a new plugin in nebula-plugins

@quidryan @quidryan http://www.slideshare.net/quidryan

We’re hiring

Justin Ryan <jryan@netflix.com>

HOUSE of GRADLE