Yale Build and Deploy
description
Transcript of Yale Build and Deploy
![Page 1: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/1.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Yale Build and Deploy
E Camden FisherYale University ITS
![Page 2: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/2.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Who am I?
E Camden Fisher
Technical Lead
Unix Infrastructure and Virtualization
@fishnix
![Page 3: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/3.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
What is Jenkins?
Jenkins is an application and a framework that manages and monitors the execution of
repeated tasks.
![Page 4: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/4.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Who Uses Jenkins @Yale?
![Page 5: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/5.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Overview
Before Jenkins After Jenkins• "fire and forget", consistent!• Testing is automated!• Code coverage is easy!• Bugs caught early and often!• Devs worry about code!• Change control in the right
places for deployments!• Rapid progress! Agility!• Identical artifact per
environment!• Identical configuration per
environments!• Deployments are “click”
easy• Integration Nirvana
• Building is slow, error prone• Testing is onerous• Code coverage is onerous• Bugs caught later• Devs worry about servers• No change control for
deployments• Slow progress• Different artifact per
environment• Inconsistent configuration
per environment• Deployments are "hard"• Integration Hell
![Page 6: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/6.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Let’s Not Forget Continuous Integration!
Maintain a Single Source RepositoryAutomate the BuildMake Your Build Self-TestingEvery Commit Should Build the Mainline on an Integration MachineKeep the Build FastTest in a Clone of the Production EnvironmentMake it Easy for Anyone to Get the Latest ExecutableEveryone can see what's happeningAutomate DeploymentAvoid "Integration Hell!!”
![Page 7: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/7.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Why did we chose Jenkins?
Easy!
Extensible
Scalable
Flexible
Open Source
Supported!
![Page 8: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/8.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Yale History Lesson
Application lifecycle is a progression
Source Code Management
Maven and Artifactory
Building and Testing with Jenkins
Container Configurable artifacts
Runtime Configurable Containers
Managed deployments with Jenkins
![Page 9: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/9.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
SCM
![Page 10: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/10.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
MVN
Maven
Project Object Model (POM)
Simplifies dependency resolution ("oops I forgot
that .jar!")
Makes the build process easy and uniform
Artifactory (Maven Repository)
Where do I put my built artifacts?
Makes it easy for everyone to get the latest build!
![Page 11: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/11.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Builds: Before Jenkins
We have a single source repo + a place
to store built artifacts, but…..
– Builds still take a long time
– Testing takes longer!
– Code coverage take even longer!
– Build environments are not standardized
– Mistakes are caught later, hard to debug
![Page 12: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/12.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Builds: Enter Jenkins
SCM commits automatically kick off a build
Testing and code coverage is automated and is run on
every commit.
Broken builds immediately notify the team and the
committer
Tests run in a Clone of the Production Environment!
Everyone can easily see what's happening!
Developers can get back to coding instead of building
and testing.
Releases are quick and easy
![Page 13: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/13.jpg)
![Page 14: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/14.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Deployment: The problem.
Configuration must be different per environmentChoice between loss of control or loss of agility– Often devs configure the container– Either devs can edit deployables or they can't
If they can... things are bad– Code is deployed, edited and removed w/out Change
Control– Moving quickly… "I'll clean it up later!"– Dev environments quickly diverge from Production– Security is compromised
If they can't... things are worseChange requests are "slow”Systems groups must do everything
![Page 15: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/15.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Deployments: How do we fix them?
Standardize the process!
Promote SANE change control
Normalize deployables/artifacts
Single location for app configuration data
Standardize the container
Externalize what makes an environment unique and special
![Page 16: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/16.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Yale Application Installer Plugin
![Page 17: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/17.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Container Configurable Artifacts
Artifacts were built with embedded configuration.– ie. datasources, CAS service endpoints, etc
– Artifacts are different per environment! What!?
– "Oops, I forgot to update that parameter!"
Externalization of configuration parameters– deployable XML
– Apps self configure with JNDI
Now we have Container Configurable artifacts– The SAME artifact migrates between environments
– XML configuration data (stored as build parameters in Jenkins) is all that differentiates environments
![Page 18: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/18.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Yale Standard Java Container
JBoss EAP (5.1 now)
Minimal customization to externalize configuration into runtime
Packaged in RPM
Configuration Management to install + manage what’s “special”
Meets 100% of use cases (so far)
Runtime configurable container is key!
![Page 19: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/19.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Yale Standard Java Container
$JBOSS_HOME/deploy/jbossweb.sar/server.xml
<!-- Connector for SSL 20100624 ECF --><Connector protocol="HTTP/1.1" SSLEnabled="true" port="${jboss.bind.httpsport}" address="${jboss.bind.address}” secure="true" clientAuth="false" scheme="https” proxyName="${jboss.proxyname}" proxyPort="${jboss.proxyport}” SSLCertificateFile="${jboss.server.home.dir}/conf/server.crt" SSLCertificateKeyFile="${jboss.server.home.dir}/conf/server.pem” URIEncoding="UTF-8” SSLProtocol="TLS”>
/etc/init.d/jboss_nodexx
# JBoss variablesexport JBOSS_OPTS=“-Djboss.bind.httpport=${HTTPPORT} \ -Djboss.bind.httpsport=${HTTPSPORT} ${ADDLJBOPTS}”
${JBOSS_HOME}/bin/run.sh ${JBOSS_OPTS} -c ${JBOSS_NODE} -g ${JBOSS_PARTITION} -b ${JBOSS_BIND} -u ${CLSTR_ADDR}
![Page 20: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/20.jpg)
![Page 21: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/21.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
![Page 22: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/22.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
![Page 23: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/23.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
![Page 24: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/24.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
![Page 25: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/25.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
![Page 26: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/26.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
![Page 27: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/27.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Deployments: The result.
Consistency!
Jenkins writes configuration XML: $human_error--
Eliminate shells, and elevated privileges on servers
Container is managed by infrastructure with the O/S
Developers can deploy to DEV at will
Empowers developers to GTD
Puts gates at appropriate places
Changes to the Jenkins jobs, containers + deploys to Test/Prod require change control
Frees Systems folks to work on more interesting things
Standard Container + Cloudy IaaS + Container Configurable artifacts = Vendor-lock-in-less PaaS!
![Page 28: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/28.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Drupal Continuous Deployment
Yale has a large shared Drupal
infrastructure
Shared = needs change control
Migration process looks similar to Java
Some end users want to edit themes!
Jenkins to the rescue!
![Page 29: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/29.jpg)
![Page 30: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/30.jpg)
![Page 31: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/31.jpg)
![Page 32: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/32.jpg)
![Page 33: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/33.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
What’s New?
Workflow integration through web servicesBuild + Release of Apache Servicemix bundles and “features”Spawning and Destroying Servicemix child instancesDeployment of Features and OSGI bundles to Apache ServicemixDeploying Drupal 7– git + pull
![Page 34: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/34.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Other Languages
Php– Code coverage and unit testing available
Ruby– Automated unit testing– Automated code coverage– Automated deployment coming soon
.Net– Build, unit test, archive creation
![Page 35: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/35.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
The Future
RBAC + folders– delegate responsibility to other systems
groups
More ruby– unit testing and code coverage– deployments
Enterprise Service BusCentralized SSH mangagement– the tools are better now
Managing/Provisioning with cfg mgmtCloud deployments
![Page 36: Yale Build and Deploy](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813c92550346895da6421c/html5/thumbnails/36.jpg)
Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf
Questions?
E Camden Fisher
@fishnix