Seacon Continuous Delivery Pipeline Tools Track
-
Upload
mark-rendell -
Category
Technology
-
view
7 -
download
0
Transcript of Seacon Continuous Delivery Pipeline Tools Track
![Page 1: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/1.jpg)
SEACON:UK 2017 Breakout Track
Exploring different approaches to Continuous Delivery Pipelines
![Page 2: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/2.jpg)
Diversion into Inclusion & Diversity1.It should be down to all of us to champion this at every
opportunity.2.We all suffer from Unconscious Bias and even if this has just a
0.1% on the opportunities that we give others, cumulatively this stacks up. (Take a test: https://implicit.harvard.edu/implicit/selectatest.html)
3.Mirco aggression is a term for behaviour that intimidates others and make it harder for them to perform a their best in the working environment. Be vigilant and call it out. (Real examples here: http://www.microaggressions.com/)
Other links:• https://www.fastcompany.com/3044738/hit-the-ground-running/7-simple-methods-to-fight-against-your-
unconscious-biases• https://codeascraft.com/2016/10/19/being-an-effective-ally-to-women-and-non-binary-people/• http://weprinciples.org/files/attachments/
WHITE_PAPER_Nudging_the_Unconscious_Mind_Nielsen_&_Kepinski_Oct_2014.pdf• https://managingbias.fb.com/• http://www.cmu.edu/news/stories/archives/2015/june/bias-blind-spot.html
![Page 3: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/3.jpg)
Agenda• Introduction to Continuous Delivery In Practice
• This Exercise
• XebiaLabs - XL Deploy and XL Release
• ADOP and Jenkins
• Thoughtworks GoCD
• General Q&A
![Page 4: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/4.jpg)
The Objective: Continuous Delivery of Ideas
FEEDBACK
![Page 5: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/5.jpg)
BuildStatic Code Unit Tests
Source CodeRepo
CI pipeline
Deploy Code
Run Test Harness
Deploy CodePPE
Perf,Security
TestDeploy
Prod
CD pipeline(rest of the way to Prod)
Automating the Workflow from Commit to Release
![Page 6: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/6.jpg)
The Workflow needs to hold up
How do we control flow?
What versions
were tested
together?
What should
we release?
![Page 7: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/7.jpg)
Tests configurationBuild tools configuration
Deployment software configurationMiddleware configuration
Operating System configuration Virtual infrastructure configuration
Hardware configuration
Business
Application
Business
Application
Business
Application
Business
Application
Business
Application
Compile Code AnalysisCommitter: jdoe
Story:25
Commit ID: 113 Unit Test Deploy PPE Functional Test Security and Perf Test
Compile Code AnalysisCommitter: jdoe
Story:25
Commit ID: 113 Unit Test Deploy PPE Functional Test Security and Perf Test
Compile Code AnalysisCommitter: jdoe
Story:25
Commit ID: 113 Unit Test Deploy PPE Functional Test Security and Perf Test
Compile Code AnalysisCommitter: jdoe
Story:25
Commit ID: 113 Unit Test Deploy PPE Functional Test Security and Perf Test
Compile Code AnalysisCommitter: jdoe
Story:25
Commit ID: 113 Unit Test Deploy PPE Functional Test Security and Perf Test
We should probably do this all the way down!
![Page 8: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/8.jpg)
Commits of any Source Code
triggers a pipeline
???
One pipeline won’t work - the tools need handle complexity
BuildStatic Code Unit Tests
Deploy Code
Run Test
Harness
Deploy CodePPE
Perf,Securit
y Test
Deploy Prod
![Page 9: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/9.jpg)
Luckily in 2017 they all do now support a lot things e.g.:
• Agents for secure remote execution on different platforms
• Application security
• Role Based Access Control
• Reporting and metrics
• Automated and manual triggers and flow control
• Commercial support
• Extensibility
![Page 10: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/10.jpg)
But how do they differ?
THE EXCERCISE
![Page 11: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/11.jpg)
Application DesignReference Application Stack• Frontend App• Backend App (tightly coupled to frontend – both must be released
together)• Configuration Management = Chef or Puppet• Provisioning = TerraformOrchestration & Automation Scripts = (you choose) Tools• Software configuration Management = (you choose but not of
concern)• Binary storage = (you choose but not of concern)• Testing = (you choose and but of concern)• Monitoring = (you choose but not of concern)• Automation of build = (you choose but not of concern)• IaaS platform = (you choose but not of concern)Automation of deployment = (you choose)Continuous Delivery Pipeline = (you choose)
![Page 12: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/12.jpg)
Example PipelineFrontend
Code, Config
BackendCode, Config
Config MgmtScripts
ProvisioningScripts
Orchestration& Automation
Scripts
Build & Package
Unit TestCode Analysis
Build & Package
Unit TestCode Analysis
Unit Test &Code Analysis
Sandbox Testing(various)
Install &ConfigureNon-Prod
Runtime Testing(various)
Install &ConfigureProduction
Unit Test &Code Analysis
Sandbox Testing(various)
ProvisionNon-Prod
Runtime Testing(various)
ProvisionProduction
Unit Test &Code Analysis
Sandbox Testing(various)
UpdateJobs /Flows
MonitoringBackupHealingetc.
System TestDeploy
System TestDeploy
Install &ConfigureSystem Test
Runtime Testing(various)
Non-ProdDeploy
Runtime Testing(various)
ProductionDeploy
Runtime Testing(various)
Runtime Testing(various)
ProvisionSystem Test
Runtime Testing(various)
![Page 13: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/13.jpg)
FrontendCode, Config
BackendCode, Config
Config MgmtScripts
ProvisioningScripts
Orchestration& Automation
Scripts
Build & Package
Unit TestCode Analysis
Build & Package
Unit TestCode Analysis
Unit Test &Code Analysis
Sandbox Testing(various)
Install &ConfigureNon-Prod
Runtime Testing(various)
Install &ConfigureProduction
Unit Test &Code Analysis
Sandbox Testing(various)
ProvisionNon-Prod
Runtime Testing(various)
ProvisionProduction
Unit Test &Code Analysis
Sandbox Testing(various)
UpdateJobs /Flows
MonitoringBackupHealingetc.
System TestDeploy
System TestDeploy
Install &ConfigureSystem Test
Runtime Testing(various)
Non-ProdDeploy
Runtime Testing(various)
ProductionDeploy
Runtime Testing(various)
Runtime Testing(various)
ProvisionSystem Test
Runtime Testing(various)
Visualisation and release Tracking1. How can “what is deployed where” be visualised and tracked (over time) and how can tested combinations of components be tracked and locked-in?
2. How can it be enforced that a consistent use of automation is used across deployments to different environments?
![Page 14: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/14.jpg)
FrontendCode, Config
BackendCode, Config
Config MgmtScripts
ProvisioningScripts
Orchestration& Automation
Scripts
Build & Package
Unit TestCode Analysis
Build & Package
Unit TestCode Analysis
Unit Test &Code Analysis
Sandbox Testing(various)
Install &ConfigureNon-Prod
Runtime Testing(various)
Install &ConfigureProduction
Unit Test &Code Analysis
Sandbox Testing(various)
ProvisionNon-Prod
Runtime Testing(various)
ProvisionProduction
Unit Test &Code Analysis
Sandbox Testing(various)
UpdateJobs /Flows
MonitoringBackupHealingetc.
System TestDeploy
System TestDeploy
Install &ConfigureSystem Test
Runtime Testing(various)
Non-ProdDeploy
Runtime Testing(various)
ProductionDeploy
Runtime Testing(various)
Runtime Testing(various)
ProvisionSystem Test
Runtime Testing(various)
Developing and Testing Automation3. What does the activity of creating automation and orchestration entail (Coding, DSL, Drag and Drop etc.)?
4. How is automation and orchestration tested and new versions released?
5. How can people share and re-use automation packages?
![Page 15: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/15.jpg)
FrontendCode, Config
BackendCode, Config
Config MgmtScripts
ProvisioningScripts
Orchestration& Automation
Scripts
Build & Package
Unit TestCode Analysis
Build & Package
Unit TestCode Analysis
Unit Test &Code Analysis
Sandbox Testing(various)
Install &ConfigureNon-Prod
Runtime Testing(various)
Install &ConfigureProduction
Unit Test &Code Analysis
Sandbox Testing(various)
ProvisionNon-Prod
Runtime Testing(various)
ProvisionProduction
Unit Test &Code Analysis
Sandbox Testing(various)
UpdateJobs /Flows
MonitoringBackupHealingetc.
System TestDeploy
System TestDeploy
Install &ConfigureSystem Test
Runtime Testing(various)
Non-ProdDeploy
Runtime Testing(various)
ProductionDeploy
Runtime Testing(various)
Runtime Testing(various)
ProvisionSystem Test
Runtime Testing(various)
Build & Package
Unit TestCode Analysis
System TestDeploy
Operational Concerns7. How are the Continuous Delivery Pipeline tools upgraded / patched and typically how often does this happen and why?
6. How can the platform be extended e.g. Via Plugins?
![Page 16: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/16.jpg)
The main event!
• Ben Hählen – Xebia Labs – XL Deploy and XL Release
• Hitesh Joshi – Accenture – ADOP and Jenkins
• Marius Ciotlos – Betfair – Thoughtworks GoCD
![Page 17: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/17.jpg)
![Page 18: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/18.jpg)
XL Release and XL Deploy
![Page 19: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/19.jpg)
XL Release
![Page 20: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/20.jpg)
XL Release
![Page 21: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/21.jpg)
XL Release
![Page 22: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/22.jpg)
XL Release
![Page 23: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/23.jpg)
XL Deploy
![Page 24: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/24.jpg)
XL Deploy
![Page 25: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/25.jpg)
XL Deploy
![Page 26: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/26.jpg)
To facilitate comparison we would like to propose the following scenario. The objective of the exercise is to design (no need to implement) a solution for achieving continuous delivery of the following.
Reference Application Stack• App 1 = Front-End• App 2 = Back-End• Configuration Management = Chef or Puppet• Provisioning = Terraform Orchestration & Automation Scripts = XL Release Tools• Software configuration Management = Git, SVN• Binary storage = Nexus, Artifactory, etc• Testing = Sonar, Cucumber, etc• Monitoring = ELK, Splunk, etc• Automation of build = Jenkins, VSTS, TFS, etc• IaaS platform = On-premise, Azure, Kubernetes, Docker, etc Automation of deployment = XL Deploy Continuous Delivery Pipeline = XL Release
Application Design
![Page 27: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/27.jpg)
1. How can “what is deployed where” be visualised and tracked over time and how can tested combinations of components be tracked and locked-in?
![Page 28: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/28.jpg)
1. How can “what is deployed where” be visualised and tracked over time and how can tested combinations of components be tracked and locked-in?
![Page 29: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/29.jpg)
1. How can “what is deployed where” be visualised and tracked over time and how can tested combinations of components be tracked and locked-in?
![Page 30: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/30.jpg)
1. How can “what is deployed where” be visualised and tracked over time and how can tested combinations of components be tracked and locked-in?
![Page 31: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/31.jpg)
2. How can it be enforced that a consistent use of automation is used across deployments to different environments?
![Page 32: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/32.jpg)
3. What does the activity of creating automation and orchestration entail (coding, DSL, drag and drop etc.)?
![Page 33: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/33.jpg)
3. What does the activity of creating automation and orchestration entail (coding, DSL, drag and drop etc.)?
![Page 34: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/34.jpg)
4. How is automation and orchestration tested and new versions released?
![Page 35: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/35.jpg)
5. How can people share and re-use automation packages?
![Page 36: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/36.jpg)
6. How can the platform be extended e.g. via plugins?
![Page 37: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/37.jpg)
7. How are the Continuous Delivery Pipeline tools upgraded / patched and typically how often does this happen and why?
Short Term Support : Every 2 months
Long Term Support : Every 6 monthsMaintenance Release : Every 2 months
![Page 38: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/38.jpg)
THANKS
![Page 39: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/39.jpg)
About Me
RAPID MOBILISATION
GLOBAL SHARED SERVICES TEAM
SUPPORTING 24 / 7
Software Configuration Management
Release ManagementAssessments
Environment Management
DevOps TransformationBuild & Deploy Automation Continuous Delivery
Rational and Jira
Infrastructure as code CloudPaaS Tooling
![Page 40: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/40.jpg)
Jenkins
![Page 41: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/41.jpg)
Jenkins with ADOP• All open source• Commercial Jenkins option though CloudBees• Accenture created ADOP so we can help
1: December 2016: https://www.cloudbees.com/jenkins/about
Stats• 150,000 Jenkins active installations, 1M users, 12.5M jobs (1)
• Jenkins plugins: 1200+• Jenkins Pipeline installations: 77,000+• ADOP: ~550 forks + ~200 stars• ADOP plugins (cartridge) > 50
![Page 42: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/42.jpg)
On Premise
Docker Machine
Docker Compose
ADOP Platform Extension
ADOP Cartridge
AWS Cloud
AWS regionAvailability
Zone
Internet
Selenium
Platform for Reusable Automation
Cartridges
![Page 43: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/43.jpg)
Solution ComponentsReference Application Stack• App 1 = Hybris• App 2 = SOLR• Configuration Management = Chef or Puppet• Provisioning = Terraform Orchestration & Automation Scripts = ADOP, Jenkins, Ansible
Tools• Software configuration Management = Git• Binary storage = Nexus (with Lifecycle)• Testing = Cucumber, Selenium Grid• Monitoring = Sensu and ELK• Automation of build = Ant and Maven• IaaS platform = AWS Automation of deployment = Ansible Continuous Delivery Pipeline = ADOP, Jenkins
![Page 44: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/44.jpg)
Visualisation and release tracking1. How can “what is deployed where” be visualised and tracked over time and how can tested combinations of components be tracked and locked-in?
Components
Environments
![Page 45: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/45.jpg)
Visualisation and release tracking2. How can it be enforced that a consistent use of automation is used across deployments to different environments?
![Page 46: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/46.jpg)
Developing and Testing Automation3. What does the activity of creating automation and orchestration entail (coding, DSL, drag and drop etc.)?
![Page 47: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/47.jpg)
$ tree.|-- README.md|-- jenkins| `-- jobs| |-- dsl| | |-- environment_provisioning.groovy| | `-- hybris_jobs.groovy`-- src `-- urls.txt
Developing and Testing Automation5. How can people share and re-use automation packages?
A Cartridge encapsulates a reference complete implementation for a particular technology or product, typically consisting of:• Reference application• Build/Deploy Pipeline• Environment Creator – depending on technology
Hybris Cartridge
CreateEnvironment(AWS EC2,
RDS) Base Hybris Build
(Platform + Code) Deploy Hybris
Docker image with config +
code
![Page 48: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/48.jpg)
Developing and Testing Automation4. How is automation and orchestration tested and new versions released?
Same Continuous Delivery Pipeline for Jenkins and Ansible
![Page 49: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/49.jpg)
Operational Concerns6. How can the platform be extended e.g. via plugins?
![Page 50: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/50.jpg)
Operational Concerns7. How are the Continuous Delivery Pipeline tools upgraded / patched and typically how often does this happen and why?
![Page 51: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/51.jpg)
THANKS
![Page 52: Seacon Continuous Delivery Pipeline Tools Track](https://reader036.fdocuments.us/reader036/viewer/2022062902/58ef245d1a28ab4a428b4577/html5/thumbnails/52.jpg)
THANKS