Evolving Search Relevancy: Presented by James Strassburg, Direct Supply
Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct...
-
Upload
lucidworks -
Category
Technology
-
view
48 -
download
1
Transcript of Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct...
![Page 1: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/1.jpg)
O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A
![Page 2: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/2.jpg)
Building a Solr Continuous Delivery Pipeline With Jenkins James Strassburg
Senior Software Architect, Direct Supply
![Page 3: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/3.jpg)
http://bit.ly/SolrCD
Links to Repositories
![Page 4: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/4.jpg)
• Overview of Continuous Delivery
• Jenkins Delivery Pipeline as Code
• Packaging / Deploying Solr with a Pipeline
• Deploying Solr Configuration with a Pipeline
Objectives
![Page 5: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/5.jpg)
Solr – Where We’re Going
![Page 6: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/6.jpg)
Solr Configuration – Where We’re Going
![Page 7: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/7.jpg)
• Deployments are predictable and routine
• Every change is deployable
• Reduce work-in-progress (lean)
• Poka-Yoke – mistake proofing
Continuous Delivery
![Page 8: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/8.jpg)
Plugins: Build Pipeline, Copy Artifacts
Jenkins < 2.x
![Page 9: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/9.jpg)
• Required separate builds for each stage
• Not version controlled
• Complicated build configuration
• More work for me ;)
Jenkins < 2.0
![Page 10: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/10.jpg)
• Jenkinsfile
• Groovy script that defines your pipeline
• Versioned in your repository
• Vagrantfile, Dockerfile, or .travis.yml (Travisfile?)
Jenkins >= 2.x Pipelines as Code
![Page 11: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/11.jpg)
node { stage ‘build’ sh ‘echo building...’ stage ‘test’ sh ‘echo running tests...’ stage ‘deploy’ sh ‘echo deploying...’ }
Jenkinsfile Example
BUILD
TEST
DEPLOY
![Page 12: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/12.jpg)
• Fork of lucene-solr repo
• A repo for Solr configuration
• A repo with config for a Jenkins Docker Image
Example Implementation
![Page 13: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/13.jpg)
• Clone my solr-jenkins-cicd repo and…
$ ./start_jenkins_and_solr.sh Then browse to http://localhost:8080 admin:admin
Starting Jenkins
![Page 14: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/14.jpg)
• Apply bug fixes (from JIRA) without waiting for an official release
• Building helps you understand the code
Why Build / Package Solr
![Page 15: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/15.jpg)
stage “Build Solr Package” env.PATH = "${tool 'ant'}/bin:${env.PATH}” checkout scm sh 'ant ivy-bootstrap’ sh 'ant -Dversion.suffix=${BUILD_NUMBER}\ create-package -f solr/build.xml’ archive 'solr/package/**/*.tgz'
Building Solr
![Page 16: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/16.jpg)
stage "Deploy Solr - Stage Env” sh 'scripts/deploy_server.sh 10.0.0.11\ solr /usr/local/solr 8983 ~/stage_private_key'
Deploy Solr
![Page 17: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/17.jpg)
Deploy Solr – There will be no demo
![Page 18: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/18.jpg)
${BUILD_NUMBER}
![Page 19: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/19.jpg)
• Collections
• Core properties
• Configsets
• Extra libs
• solr.xml
• solr.in.sh
Solr Configuration – Artifacts in Version Control
![Page 20: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/20.jpg)
• Different files for different environments
• Deploy script takes an environment parameter
• Similar to Chef’s data bags or Puppet’s Hiera
Environmental Differences
![Page 21: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/21.jpg)
Environmental Differences - DIH
![Page 22: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/22.jpg)
Configuration Deployed / Collections and DIH
![Page 23: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/23.jpg)
• This groovy statement will pause the pipeline until an admin approves it:
input message: 'Approve this release', ok: 'Approve', submitter: 'admin'
Manual Intervention
![Page 24: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/24.jpg)
Manual Intervention
![Page 25: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/25.jpg)
• Clone/build lucene-solr repo (ant server)
• Clone configuration repo
• Use scripts/start_solr.sh and stop_solr.sh
Developing Configuration
![Page 26: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/26.jpg)
Deploy Configuration Demo
![Page 27: Building a Solr Continuous Delivery Pipeline with Jenkins: Presented by James Strassburg, Direct Supply](https://reader036.fdocuments.us/reader036/viewer/2022070516/5870680e1a28ab48378b560f/html5/thumbnails/27.jpg)
• http://bit.ly/SolrCD - Links to Repositories
• http://bit.ly/JenkinsPipeline - Pipeline as Code
• http://bit.ly/FowlerCD - More on CD
• @jstrassburg
Links / Questions