CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User...
Transcript of CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User...
![Page 1: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/1.jpg)
Jenkins User Conference Israel #jenkinsconf
Gidi Samuels
www.liveperson.com
July 16, 2014
CI / CD @ Liveperson
#jenkinsconf
![Page 2: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/2.jpg)
About Myself
Lead the CI Team at Liveperson
3 years in Liveperson
![Page 3: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/3.jpg)
About Myself
Lead the CI Team at Liveperson
3 years in Liveperson
![Page 4: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/4.jpg)
Agenda
● Introducing Liveperson and its CI
● Creating CI framework with Apache Maven
● How we’ve made 200 developers write in puppet?
● Scaling from 4 CI flows to > 100 and making our
developers’ life easier
● Using Jenkins as a deployment tool
● Roadmap: Auto provisioning and containers
![Page 5: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/5.jpg)
Introducing Liveperson and its CI
![Page 6: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/6.jpg)
Introducing Liveperson and its CI
SaaS platform for creation of meaningful
connections through real-time engagement
Monitor web visitor’s behavior
(Over 1.5 B visits each month)
Conduct behavioral ranking
Provide the engagement platform
(Over 10 M chats each month)
SaaS & Cloud only
![Page 7: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/7.jpg)
Introducing Liveperson and its CI
Numbers:
● 200 Developers
● 25 Scrum teams
● 500 Builds per day
![Page 8: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/8.jpg)
Creating CI framework with Apache Maven
![Page 9: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/9.jpg)
Day 1 Build Config Tool
● Project management tool
● Well defined lifecycle
● Dependency management
● Makes the build process easy
Lifecycle
compile
Deploy
Install
Test
![Page 10: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/10.jpg)
Day 2 Maven Repository
● Started with single node
● Upgraded to High Availability
● Planning to add more instances for global development
![Page 11: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/11.jpg)
Mysql Mysql
Master / Master
Replication
Artifactory High Availability
![Page 12: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/12.jpg)
Day 3 - TeamCity Pre-Test Commit
Get Code
Pre-Test commit
Commit code
Continuous Integration
Build Agent Pools
Verify your change
CLEAN
COMPILE
TEST
INSTALL
DEPLOY
3
4
1
2
![Page 13: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/13.jpg)
TeamCity High Availability
Mysql Mysql
Master / Master
Replication
Keep AliveActive Passive
![Page 14: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/14.jpg)
Day 4 Automation
We transformed our QA testes from manual to
automate
![Page 15: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/15.jpg)
Day Automation tools
Automation testing tools we use in our build
![Page 16: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/16.jpg)
Measuring our coverage
![Page 17: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/17.jpg)
Measuring our coverage
![Page 18: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/18.jpg)
Measuring our coverage
![Page 19: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/19.jpg)
Packaging
We use RPM as our packaging tool and repo
Production
![Page 20: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/20.jpg)
Packaging
Included in maven-rpm-plugin
<groupId>org.codehaus.mojo</groupId>
<artifactId>rpm-maven-plugin</artifactId>
Standard configuration:
● projversion
● summary
● description
● mapping
● preinstallScriptlet
● postinstallScriptlet
![Page 21: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/21.jpg)
How we’ve made 200 developers write in
puppet?
![Page 22: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/22.jpg)
Configuration management
We deliver everything to production with PUPPET
● 200 developers program in Puppet to automate
deployment
● Trained those 200 developers to write in Puppet
● We created standards in LP for deployment and
configuration
![Page 23: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/23.jpg)
Puppet configuration design
Environment
ManifestModule
Production
Building Blocks
Production
grade basic
modules
Configuration
parameters
passed per env
Product puppet
module
![Page 24: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/24.jpg)
Configuration management
Production
CD
![Page 25: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/25.jpg)
Orchestration
Mcollective
![Page 26: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/26.jpg)
Scaling from 4 CI flows to > 100 and making
our developers life easier
![Page 27: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/27.jpg)
Step #1 Maven plugins
So how did we connect all these
tools together ?
By extending Maven lifecycle with
new maven plugins.
What is a plugin ?
Plugins are the central feature of
Maven that allow for the reuse of
common build logic across multiple
projects.
![Page 28: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/28.jpg)
Practice #1 Maven plugins
Practice #1 Maven plugins
![Page 29: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/29.jpg)
Security analysis
During “nightly” builds and “release” build we
added the following code analysis:
● Security vulnerability and
regulation compliance
● Infrastructure vulnerability
● SaaS open source
management tool
![Page 30: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/30.jpg)
?
How do i create a java project with Liveperson standard ?
● Where do I manage the code ( SVN /
GIT )
● What is LP project layout standard ?
● Which parentPom version to use ?
● How do I create the Puppet
deployment Module?
● What about CI tools. How to create a
build for my project ?
![Page 31: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/31.jpg)
Step 2 Making our developers life easier
Stored in
repositoryRPM
ResourcesLP
ParentPom
Code
coverage
Server to
run on
Promotion
Tool
Security
Analysis
Java project
Puppet
module
CI Build
config
Click Here
The power of Maven
Archetype
![Page 32: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/32.jpg)
All together now
Commit
CI
Servers
Deploy
Production
TeamCitySVN
compile Unit
tests
Code
analysis
QA
Servers
Promote
GDE
Servers
E2E
Servers
Security
checks
![Page 33: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/33.jpg)
Jenkins as a deployment tool
Same deployment framework for
CI / QA / PRODUCTION
![Page 34: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/34.jpg)
● CI with Openstack on the fly
● Auto provisioning of full environments
● Moving with the same technology to Docker
● Support Global development
● GIT development
Roadmap
![Page 36: CI / CD @ Liveperson › sites › default › files › ... · 2014-07-28 · Jenkins User Conference Israel #jenkinsconf Gidi Samuels July 16, 2014 CI / CD @ Liveperson #jenkinsconf](https://reader033.fdocuments.us/reader033/viewer/2022060508/5f2381f2f00bc922b015f1f2/html5/thumbnails/36.jpg)
Jenkins User Conference Israel #jenkinsconf
Thank You To Our Sponsors
Platinum
Gold
Silver