Configuration as Code in Jenkins. What's new? Nov 2016
-
Upload
oleg-nenashev -
Category
Software
-
view
21 -
download
2
Transcript of Configuration as Code in Jenkins. What's new? Nov 2016
Configuration as Code in Jenkins What's new?Oleg Nenashev (@oleg_nenashev),Jenkins Developer
CloudBees, Inc.
DevOps StockholmNov 28, 2016
2© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
PhD, Hardware EngineeringR&D/Automation in HW and Embedded
Core Team Member, Jenkins projectJenkins Developer, CloudBees
• CloudBees Jenkins Platform• Support and consulting
Contributor, LibreCores project
About me
@oleg_nenashevoleg-nenashevonenashev
3© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps 3
Oleg’s “Hall of Shame”(c)
Also:• Jenkins Core• Remoting –
Communication Layer for Agents
• Windows Service Wrapper
4© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Do know about Jenkins?
About you
5© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
1. The most popular CI automation server in the world
2. It’s a framework• Swiss Army Knife of Automation
3. It’s open source• Can be customized
4. Commercial support is available
Who is Mr. Jenkins?
6© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Jenkins is not only about Continuous Integration
Continuous Integration
Continuous Delivery
DevOps, System Administration
Accounting automation
Home Automation
CRON + WebUI
7© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Do you use Jenkins?
About you
8© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Jenkins is Popular
http://stats.jenkins-ci.org/jenkins-stats/
• Common: ~70% of teams use Jenkins
• Almost 150k installations• ~300 active committers
(over last 6 months)• ~3 new plugins every week• ~50 plugin releases every
week
Hudson Jenkins
9© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
NEW WEBSITE
NOW IT’S A CD AUTOMATION SERVER
FOCUS ON USER EXPERIENCE
UI IMPROVEMENTS
SECURITY IS ENABLED BY DEFAULT
PIPELINE AS CODE
Jenkins 2
http://bit.ly/jenkins20Stats:• Released on April 2016• LTS line – July 2016• 50% adoption over six months
10© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
CI – de-facto standardNew buzzwords
• Continuous Delivery• DevOps• Microservices
Cloud-native systemsCentralization/StandardizationAutomation becomesmission-critical
Automation System trends
No test engineers
What was wrong?
12© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Classic Approaches• Shell script steps to execute custom logic• Matrix Project – multi-configuration jobs• Delivery Pipeline, MultiJob – defining build flows• Parameterized Trigger / Copy Artifact - parallelize
tests and to collect results• Custom Tools plugin to manage tools• Throttle Concurrent Builds and Lockable
Resources to dispatch the access to shared resources• Naginator plugin to restart the builds on-demand• …
13© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
What if we combine them?
13
14© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Classic Approaches
Spaghetti Automation
• Many job dependencies
• Difficult build flow tracking
• Complex configurations
• Duplication
• Expensive maintenance
15© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
ü Storage together with the projectü Tracking together with the projectü Tests together with the project
…. as code?Configuration…Documentation…Pipeline… Everything…
16© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Configuration as Code in Jenkins
Job ConfigurationJenkins system configuration
17© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Jenkins Configuration… as Code
External Tools
Jenkins CLI and REST
API
python-jenkins
jenkins-client (java)
Configuration Management
systems
Ansible, Chef
Docker
Solutions within Jenkins
project
Groovy Boot Hooks
Scriptler Plugin
SCM Sync Configuration
UnsafeJust examples…
18© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Jenkins 2 & External Configuration Management
19© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Security by default in Jenkins 2Known issues…• chef-cookbooks/jenkins (Issue #466)• puppet-jenkins (Issue #575 )• python-jenkins• …
Fixed• ansible-jenkins (not all implementations)• jenkinsci/docker (for a standard image)
Jenkins 2 & External Configuration Management
20© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
So, what’s new?
21© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Something in progressPipeline-alike Configuration as Code plugin• Groovy DSL• Source: ▸ https://github.com/jenkinsci/system-config-dsl-plugin
• Started by Kohsuke Kawaguchi in 2015• Status: pre-alpha
22© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
What about System-level?Jenkins system from a single definition
23© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
What’s new?No **standard** solution…
But t
24© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
What about Jenkins Jobs?
25© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Configuration as Code for jobs
Job DSL
Job Builder Plugin
Jenkins PipelinePipeline is offered by default in Jenkins 2.0
…
Groovy
YAML
Groovy
26© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Pipeline is a…
ØGroovy DSL for automatic flow description *• Alternatives – Job DSL, Jenkins Job Builder
Ø Job type in Jenkins (Pipeline Job)• Job definitions can be stored in SCM• Automatic job creation from repositories and organizations
ØEcosystem for automation as code• ~20 dedicated plugins• Integration in many other plugins• Suggested by default in Jenkins 2
27© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Example
// Run on a node having the “my_fpga” labelnode("linux && ml509") { git url:"http://github.com/myorg/myproject.git"sh "make all"
} http://bit.ly/pipeline-tutorial
28© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Pipeline. Useful FeaturesRobustness against restarts of Jenkins master
Robustness against network disconnects• sh() steps are based on the Durable Task plugin• Jenkins continues execution once the node reconnects
Parallelization across multiple nodes:• It can be achieved via combination of parallel() and node() steps• No need to use the trigger/copy chain (but you can do it)
Ability to store the shared logic in standalone Pipeline libraries
Visualization: Pipeline Stage View, Blue Ocean
29© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Stage View
30© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Blue Ocean
31© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps 31
32© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Example 2. Running on two nodes in parallel…stage ‘test'parallel ‘board1' : {node(‘linux && board1') {unstash 'source'sh "./bin/runTest --target=board1"step([$class:'JUnitResultArchiver', testResults:'**/test-results/*.xml'])
}}, ’board2': {node(‘linux && board2') ) {unstash 'source'sh "./bin/runTest --target=board2"step([$class:'JUnitResultArchiver', testResults:'**/test-results/*.xml'])
}}
33© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Lockable Resources plugin is integrated with PipelineThrottle Concurrent Builds plugin is not integrated, license limitations may be a problem
Resource management
echo 'Starting'node('board1') {stage(’Compile')// Build project// ...stage(’Run tests')lock(env.NODE_NAME + "-board1") {// Run something on the board// ...
}}
34© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
FailoverPipeline allows re-running unreliable stages on other nodes.
System failure
Node #1 Node #2. . .
. . .
for (def board : boards) {echo "trying board " + board;try {node(board) {checkout scmsh ‘./bin/run.sh’ // Call passed => DONEbreak;
}} catch (Exception ex) {if (ex.message.contains
("exit code 255")) {// Fatal errorfail("Test run failed")
}}
}
Try all possible boards:
35© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Docker PluginDocker Pipeline PluginDocker Custom Build Environment Plugin…Yet Another Docker Plugin
Example: Docker plugins
36© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Example. Docker-based tools and PipelineDocker Pipeline Plugin:
• You can share workspaces• External Workspace Manager
• You can create build chains
See also: Kubernetes Pipeline Plugin
37© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Example. CD for Jenkins IRC Botdef imageName = 'jenkinsciinfra/ircbot’
node('docker') {
checkout scm
sh 'git rev-parse HEAD > GIT_COMMIT'
commit = readFile('GIT_COMMIT').take(6)
def imageTag = "build${commit}"
stage('Build ircbot’) {
withMavenEnv
(["BUILD_NUMBER=${env.BUILD_NUMBER}:${commit}"]) {
sh 'make bot' // Make invokes Maven
}}
stage ('Build container’) {
def whale = docker.build("${imageName}:${imageTag}”)
}
stage 'Deploy container') {
whale.push()
} }https://github.com/jenkins-infra/ircbot
Docker Registry
Server
Jenkins
SCM
Puppet
38© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Pipeline features. Interactive step// Buildtheproject,blastittotheboards//Preparetheruninthetestmanagementsystem
//Sendthenotificationtotesters.Thenwaitfortheinputecho’Automatictestspassed’hipchatSend('@QATeam Thebuild' + buildVersion + 'hasbeenstaged:' + stagingBoard
+ '.PleaseTestitandreporttheresultbackto' + build.url+ '.TestLink runishere:' + testLinkRunURL)
inputmessage:'Havemanualspot-checkspassed?Readytorelease?'
//Proceedonthesuccessfulpath
mailto:"[email protected]",subject:"DreamProduct- buildisreleased",body:"Hithere,Thenewbuildoftheproducthasbeenreleased.Link:" +artifactoryLink
~~~~~~Code~~~~~~
~~~~~~Code~~~~~~
Too Complex?• More syntax sugar• Pipeline Model Definition• Snippet Generator
40© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Syntax Sugar - @Symbol
$class is not required anymore… almost•Plugins need to be adapted
It simplifies syntax of Pipeline and JobDSL
41© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Syntax Sugar - @SymbolWas:
Now:
42© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Syntax sugar on the top of Pipeline• “pipeline { }” closure
Provides common configuration sections in Pipeline• Some static analysis of the configuration• Less flexibility• Simplified case like error handling
More info:• https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki
42
Pipeline Model Definition
43© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps 43
Example. Pipeline Model Definitionpipeline { agent label:"generic-linux"tools { maven "Maven 3.3.9"jdk "Oracle JDK 8u40"
} stages { stage("build") { steps { sh 'mvn clean install -Dmaven.test.failure.ignore=true' }
} }post { always { junit "path/to/xml" } failure { mail to:"[email protected]", subject:"FAILURE:${currentBuild.fullDisplayName}",
body: "${currentBuild.url}" } }
}
Jenkins Pipeline Library Management
45© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
load() – loading of Groovy code from a fileCPS Global Library•Comes as a part of Pipeline plugin set•Creates Git repo within Jenkins•Defines global variables and classes
Pipeline Remote Loader•https://github.com/jenkinsci/workflow-remote-loader-plugin•Loads Pipeline from Git/GitHub•The plugin logic is implemented as Pipeline
Was
45
46© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Now. Pipeline Shared Libraries /
Library Management in Pipeline
47© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Pipeline Shared Libraries• Libs are located in SCM• Versioning• Definition for scopes:▸Jenkins system-wide▸Folder▸Job
48© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Pipeline Library Definition [1/2]
49© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Pipeline Library Definition [2/2]
50© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Using Pipeline Libraries
51© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
More examples:https://github.com/jenkinsci/pipeline-
examples
52© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Pipeline. What’s next?
• Stability / Fault tolerance• UI/UX, Blue Ocean• Syntax improvements• (?) Freestyle => Pipeline Converter
53© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Pipeline. Dev toolsAvailable: •Minimal IDE: Auto-completion•Minimal static analysis•Documentation browser for steps•Library manager
Missing:•Debugger•Real-time code deployment from IDE•Real-time approval of Script Security•Freestyle => Pipeline Converter
54© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
• Pipeline greatly simplifies task parallelization and failover within a single build
• Pipeline saves much time on automation system maintenance• Pipeline is a programming languages, high barrier to entry• Pipeline still lacks Dev Tools and integrations, but it evolves
Pipeline. Lessons learnt
55© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Summary
1. Jenkins is an effective automation framework
2. Jenkins offers Configuration-as-Code for System configs and Jobs
3. C-as-C for Jenkins System configurations is far from ideal
4. C-as-C for Jobs is not ideal as well, but it evolves quickly
• Especially Pipeline ecosystem
56© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
1. Use configuration as code where it is possible2. If you use Jenkins…
1. Consider upgrading to Jenkins 22. Try Jenkins Pipeline3. Organize your system management scripts
3. If you use Pipeline…1. Try Pipeline model definition2. Try new Pipeline libraries
Takeaways
57© 2016 CloudBees, Inc. All Rights Reserved.@oleg_nenashev, @cloudbees, #SthlmDevOps
Jenkins project:• Website: https://jenkins.io
Pipeline:• Tutorial: https://github.com/jenkinsci/workflow-plugin/blob/master/TUTORIAL.md• Compatibility: https://github.com/jenkinsci/pipeline-plugin/blob/master/COMPATIBILITY.md• Examples: https://github.com/jenkinsci/pipeline-examples• Model definition: https://github.com/jenkinsci/pipeline-model-definition-plugin
CloudBees website:• Website: https://www.cloudbees.com• CloudBees Jenkins Platform: https://www.cloudbees.com/products/cloudbees-jenkins-platform
Useful Links
Software at the speed of ideas
THANK YOU!www.cloudbees.com
Questions?Offline: [email protected]
oleg-nenashev
oleg_nenashev