Download - Continuous Delivery at SAP IT, Alon Aizenberg

INTERNAL

Continuous Delivery @ SAP IT

Alon Aizenberg SAP Global IT / Social Collaboration Platform team January 2013

©  2013 SAP AG. All rights reserved. 2 Public

About SAP

§  40 years old

§  Leader in enterprise applications

§  ~ 70K employees

§  ~ 183K customers

§  ~ 130 countries


Social Collaboration Platform team

We produce many platforms


Search Consolidation Project

§  One stop when searching SAP related content

§  ~ 100 machines in all environments

§  ~ 40 in productive deployments

§  ~ 2 years in continuous

delivery mode


The Problems

?


The Problems

Physical hardware,

manual setup

Manual

deployment Manual QA

phases

Big features, long

development cycles


Virtualization, InfrastructureAutomation

The Tools

Deployment

Automation Test

Automation

Continuous

Delivery


The Tools


Infrastructure

Automation


Infrastructure Automation

§  Start with manual VM operations

§  Command line wrapper allows automation

§  Evolve, full Infrastructure as code

§  Projects, environments, machine descriptors and automation scripts in source control

Infrastructure

Automation


Project Configuration Example

- name: prod

servers:

- name: search01

runlist:

- role[search-server]

- name: search02

runlist:

- role[search-server]

- name: db

template: SLES_11_64bit

cpus: 8

memory: 16384mb

runlist:

- role[hanadb-server]

- name: index01

runlist:

- role[indexing-server]

- name: index02

runlist:


- name: index03

runlist:


- name: ls5133

physical: true

dnsname: search11111

ipaddress: 11.11.11.11

runlist:

- role[trex-name-server]

Infrastructure

Automation

chefrepo: [email protected]:srch/chef-repo-int.git


Chef usage

§  Each project has a chef repository in git

§  Shared automations have their own repositories in git

§  Use librarian-chef and git to create a full chef repository

Before uploading the scripts to chef server

Infrastructure

Automation


Project A Chef repository

Chef Server

Production Project A

production environment

Shared cookbook repo /

Cookbook shelf

Chef Usage Infrastructure

Automation

Combined chef repository


Librarian-Chef Configuration Examples

site "http://cookbooks.sap.corp:8080" cookbook 'monsoon’ # loads jvm cookbook because tomcat declares it’s dependent on jvm cookbook 'tomcat’ # use git repo to load a cookbook cookbook 'saml2', :git => 'git://github.sap.corp/srch/saml2’

Infrastructure

Automation


Barkeeper

§  The glue

§  Manages projects and environments

§  Manages VMs

§  Manages chef servers

§  Handles permissions

§  UI, REST API and CLI

Infrastructure

Automation

Project configuration and chef repository

Barkeeper VMware

API


Barkeeper Infrastructure

Automation


Deployment

Automation


Deployment Automation

Dedicated cookbooks automate

application deployment and configuration

Deployment

Automation


Test

Automation


Test Automation

§  End to end functional acceptance tests

§  80% of functional tests are automated

§  Manual QA

§  Integrations with other systems

§  Not everything is automated, yet

Test

Automation


Continuous

Delivery


Continuous Delivery

§  Bamboo Continuous Integration server

§  Orchestrates the flow from code push to productive deployment

§  Build grid

§  Parallel execution

Continuous

Delivery


Bamboo jobs

Build, unit test

Package, publish

Deploy Test, functional tests

Deploy QA, sanity tests

Deploy Prod

Artifact / Binary repository

Source/Chef repositories

Commit

Test

QA

Prod

Environments

Pull deployment automation, binaries and artifacts


Pipeline Interactions – Deploy Environment

Bar K

eeper

GIT Project Repo

Chef Server

VM 1

VM 2

VM 3

VM 4

VM 5

VM 6

Create chef server

Create VMs

Continuous

Delivery


Pipeline Interactions - Install Environment

Build Agents

Bar K

eeper

Bam

boo

GIT Chef Repo

Chef Server Chef Server

VM 1

VM 2

VM 3

VM 4

VM 5

VM 6

Shared cookbooks shelf

Sync

Run chef clients

Continuous

Delivery


Pipeline Interactions - Test Application

Build Agents

Bam

boo

Chef Server Chef Server

VM 1

VM 2

VM 3

VM 4

VM 5

VM 6

REST API Tests

Functional tests

Git source repo with test automation

Continuous

Delivery


How did we get there?

§  2 years

§  Evolution not revolution

§  Constantly rethink and improve

§  Tech talks, documentation, encourage hands on

§  Make the tools easy to use for everyone


A change in the state of mind

§  Not only coding

§  Master always shippable

§  Feature toggles

§  Must be maintained

§  Be ready to handle production

§  If you build it… they will come


Lessons learned

§  Management buy-in

§  Fast pipeline

§  Application adaptations

§  Testing automation is big investment

§  Tests should be easy to write and run

§  Automate everything – build infrastructure, load balancers, load tests, monitoring, analytics etc.


? Questions


Thank you

Twitter @alonaizenberg

Email [email protected]