IBM InterConnect 2016 Greg Hodgkinson 2238 Thriving DevOps at BMI (Prolifics)
-
Upload
greg-hodgkinson -
Category
Software
-
view
31 -
download
1
Transcript of IBM InterConnect 2016 Greg Hodgkinson 2238 Thriving DevOps at BMI (Prolifics)
CONNECT WITH US:IT: Customized to Your Advantage
Thriving DevOps at Broadcast Music Inc. DOD-2238Growing from nothing to world class.
Public | Copyright © 2014 Prolifics
Greg Hodgkinson – Prolifics, Global Director of Lifecycle Tools and
Methodology
Jim Harvey - Broadcast Music Inc., Sr. Dir. of Quality Assurance and Release
Management
CONNECT WITH US:
What attendees will learn What has made DevOps thrive at Broadcast
Music Inc.? What are the conditions and driving forces
behind a thriving DevOps practice? What does this thriving practice look like? Who is involved, what tools are they using, how
have they benefitting from these tools? What are they achieving? How is DevOps making a difference to the
business?
2
CONNECT WITH US:IT: Customized to Your Advantage
Introductions
Public | Copyright © 2014 Prolifics
CONNECT WITH US:
Who Are We Making Happy?
4
Everyone! (that listens to music)
CONNECT WITH US:
Speaker Introduction
5
JIM HARVEYSenior Director of Quality Assurance and Release ManagementBroadcast Music Inc.Jim Harvey is the Senior Director of Quality Assurance and Release Management for Broadcast Music Inc. (BMI). He has over 16 years’ experience in Software Development. Jim is currently evolving the Software Development Life Cycle at BMI as it relates to Requirements Management, Software Testing and Change Control management.
GREG HODGKINSONDirector of Lifecycle Tools and MethodologyProlificsGreg has 20 years of experience in assisting customers achieve success by optimizing their teams. Optimizations include adopting agile practices, DevOps tools and practices, and SOA techniques.
He is very much a practitioner with responsibility for successful delivery at a number of leading companies. He presents on DevOps and agile SOA approaches at major IBM events, has co-authored a Redbook on SOA solutions, and contributes to DeveloperWorks.
CONNECT WITH US:
Broadcast Music Inc. at a Glance
Broadcast Music, Inc. - 1939 Performing Rights Organization
(PRO) - Pay public performance royalties
Operate on a non-profit-making basis
7 locations: Nashville, New York, Los Angeles, Atlanta, Miami, Puerto Rico, London
600 employees, 7.5 million works, over 500,000 songwriters and composers
6
CONNECT WITH US:
The Performers
7
Israel Kamakawiwoʻole
Louis Armstrong
Judy Garland
CONNECT WITH US:
The Writers
8
“What a wonderful world”George David
WeissBob Thiele
“Somewhere over the rainbow”1939
1967
CONNECT WITH US:
2016: The Digital Music Ecosystem
Over 500 digital music services worldwide
Offer consumers the opportunity to legally access up to 26 million songs
Challenge: Stay relevant and competitive?
9
CONNECT WITH US:
Prolifics at a Glance
10
5-Years CompoundAnnual Growth Rate
19%
EmployeesWorldwide1,500
Global PresenceUnited States, United Kingdom, Germany, India
20+Technology
Accelerators
550+Technical
Certifications
Over 10 Technology and Solutions Awards since 2009 including
Business Agility, Customer Integration and Digital Experience, the first ever Beacon Laureate for Business Agility
Over 160 global customersare currently Fortune
1000 companies
Best-in-class architects andspecialty experts:
BPM, Integration, Digital Experience, Security, Testing, Business Analytics and Enterprise Content Management
End-to-End Project Expertise
Rate of Repeat Engagements*
91%
Years inBusiness
35+Offices
14 Awards
Technology Expertise
Fortune 1000
*based on % revenue Source – December 2013 internal revenue metrics
CONNECT WITH US:IT: Customized to Your Advantage
Challenge and Success
2/18/16 11
CONNECT WITH US:
Why the need for DevOps at BMI?
Rapidly innovating music industry
Means BMI needs rapid innovation themselves
New music services => new integrations
New offerings = > new systems
Explosion of artists => increased need for scale
===========================Net: Need to deliver a lot of software innovation, quickly.
12
CONNECT WITH US:
The BMI Transformation
Over a period of 4-5 years BMI have fundamentally changed their ability to deliver software innovation.
13
New Approaches New Runtime Platforms
New Development Tools
Agile
BPM
SOA
DevOps
IBM BPMIBM WebSphere Portal
IBM ODMWebSphere DataPower
IBM Integration BusWebSphere ESB
IBM CastIronIBM DataStage
OracleIBM WSRR
Focal Point
DOORS NG
Rational Software Arch.
Rational Team Concert
Rational Quality Mgr.
IBM UrbanCode Deploy
CONNECT WITH US:
How Did They Change (Successfully) So Fast?
They key to this rapid transformation: Good processes and tools!
14
To be successful with these…
…required that they be successful with these.
In a nutshell: all the things that helped them be efficient! DevOps especially relevant.
CONNECT WITH US:
Adding Agile and RTC
First challenge: Herding the cats!
Scrum
Structure for teams Focus of sprints Course correction Simple – get going quickly!
RTC
Progress visible Keeps everyone on same
page Easy configuration
management Opportunity for integrated
builds
15
CONNECT WITH US:
Toe in the Water: Continuous Integration
Opportunity: Integrated builds for quality feedback loop
Continuous integration of WESB apps
Java apps easy, start tackling newly adopted WESB
Developed build machine plugin to automate builds of WESB ears
Used in CI cycles: immediate feedback for devs (learning new tech)
Little effort -> lots of value
16
CONNECT WITH US:
Obvious Next Step: Automated Deploys
Challenge: Delays to get changes into test
Continuous deployment of WESB apps
Added automated deployment of WESB ears to plugin
Either on demand, or as soon as ready
No more waiting; testers testing earlier
Reduced delays mean more testing done; quality increases
17
CONNECT WITH US:
Expand Across Enterprise Architecture
Opportunity: Success breeds success
Full-stack build and deploy
Made use of Prolifics investment
Common framework Leveraged common steps Already had plugins for IIB
and Portal Expanded to all app types Integrated stack no longer
held back by slowest individual release pipeline
18
CONNECT WITH US:
Added Database Deployments
Challenge: But what about the data? New plugin for automating deploy of schema updates
Implemented using same framework and standards as apps for simplicity
DBAs place scripts into source control
Automation picks up and upgrades schema
Works across all environments
19
CONNECT WITH US:
Added Publishing of Services
Challenge: And what about the service registry?
New plugin to publish to WSRR
Again, same framework and standards leveraged for simplicity
Uploads WSDLs, XSDs, and other file changes
Updates version and other metadata
Consistent across all environments
One less thing to remember to do
20
*Bad luck Brian
CONNECT WITH US:
Ramping up on Automated Deployment
Challenge: Push button deployments – too many buttons!
Purpose-built deployment engine: UrbanCode Deploy
Wrap existing deployment automation as UCD plugins - quickly and painless move
Orchestration Cross-tier deployments
simple Easier to elaborate deploys
of individual components too
Snapshots of complex apps simple
Conditional gates on environments
21
CONNECT WITH US:
Integrated Automated Testing
Opportunity: Include existing automated tests Plugin to wrap testing tool(s)
Tests moved into source control Builds pull out tests and execute
them automatically Can then
Trigger when you need, Schedule, Or include as part of
build/deploy/test process Lots more value from test scripts
Regression testing Smoke testing
22
CONNECT WITH US:
Dockerising Build Agents
Challenge: Build infrastructure inflexible time sink
Created Docker images of all build agent configurations
Install build agent software Install automation Install supporting software
– in some cases this is considerable
Easier to isolate build agents
Easier to spin up new agents
Easier to refactor infrastructure
23
CONNECT WITH US:
Next: Automated Provisioning of Environments
Challenge: Full set of test environments takes up a lot of tin
Create Docker images for all test environment configurations
Saves time – also allows new behaviors
Spin up new environments on demand
Stand-up/tear-down as part of test automation
Much better usage of tin – easier to share across projects and teams
24
CONNECT WITH US:
Beyond That: Into the Future
Opportunity: What else can we do with this automation?
Think big for the future!
Increasingly hands-free automation
Self scaling environments Detect shortage Provision new environment Lay on top app and config
Self healing environments Detect issue Provision different version Apply known fix
25
CONNECT WITH US:IT: Customized to Your Advantage
Our Approach
2/18/16 26
CONNECT WITH US:
Practice Agility!
The key to getting started and keeping going.
27
Some might… We did…
Start by envisioning a big program of work.
Started simple – Then build on initial success, now into 4th year of continued improvement – best multi-year projects are based on a small success.
Static delivery plan
Good principles of agility – Do something, course correct, adapt to changing needs, focus on strategic plan but still keep agility to respond to immediate need
Fix-it-all in first 6 monthsPick an area to improve - Don’t “boil the ocean”. We started by just fixing Continuous Integration.
CONNECT WITH US:
Focus on Practices and Benefits!
Focus on what is important.
28
Some might… We did…
Just focus on putting in place new tools and technologies.
Consider the practices – We thought through what it is we’re trying to do and chose the right tools and technologies to support.
Get caught up in “what is hot in DevOps”.
Focus on benefit – As we intended on keeping resource/budget small, we had to make sure that everything we did gave “best bang for buck”.
Practices & Benefits
CONNECT WITH US:
Cross Boundaries!
Important to keep eyes open to wider opportunities.
29
Some might… We did…
Stick with the areas they are comfortable with e.g. Java.
Cover it all! - Thrived on figuring out how to automate assembly and deploy of the trickier areas e.g. database, repository-based dev, service registries. Ensure that all parts of the app are included.
Just focus on developers and operations teams.
Consider all involved teams – Any role that is involved in releasing apps, including DBAs, testers.
CONNECT WITH US:
Keep Efficient!
DevOps sounds like it needs a big team.
30
Some might… We did…
Go it alone, or outsource it all.
Best of both worlds – Prolifics involvement allows BMI to focus on writing and testing apps, while BMI release engineering team has still had direct control of app releases.
Create a big team to solve a big problem.
Small as possible - Achieved almost unbelievable levels of efficiency with team of just 2.5 FTEs (and one of those is a manager). Low cost, disproportionate benefits.
Either do no training, or overcommit to training.
Pragmatic value-driven training - Train by example, support with JIT workshops and Wiki-based documentation.
CONNECT WITH US:
Leverage Expertise!
Working with a partner has allowed BMI to make much more progress.
31
Some might… We did…
“Not invented here!” – lets build our own automation.
Reuse! - Leveraged automation from the Prolifics DevOps Labs. Huge difference in how much has been achieved as well as level of quality.
Give us 6 months and we’ll have our own automation development capability.
Engage specialists - Prolifics DevOps labs efficiencies have allowed us to be responsive to BMIs needs – we’ve created the engine.
CONNECT WITH US:IT: Customized to Your Advantage
What Does Success Look Like?
Public | Copyright © 2014 Prolifics
CONNECT WITH US:
Planning and Prioritizing Work
Apply agility to planning
Start from backlog Release plans track what
we’re currently working on Thrice weekly touchpoints Focus on prioritizing:
Hot current issues Hot opportunities for
improvement Strategic improvements
Constantly maximizing value!
33
CONNECT WITH US:
Prototyping New Automation
Uncover risk, uncover opportunity Seek to understand before we
automate What are artifacts? Where do
artifacts live? What are APIs/tools?
Approach Do it manually. Then automate
manually – command-line. Then script automation
Examples! Involve experts and teams -
Demo and get feedback
34
DeveloperWorks Knowledge Center
Samples Try it all out!
CONNECT WITH US:
Implement Automation in RTC
Great automation development support Configuration management of
automation and example apps Inject automation (with changes)
into example build workspace Build engine to do test runs – attach
logs, perform activity logging Link up development work to plan
items in release plan Record of progress, issues, solutions JBE simple automation engine – use
Ant as standard
35
CONNECT WITH US:
Architect for Ease of Maintenance and Extensibility
Well factored plugin model Ant-based core Standard assembly and deploy steps
atomic steps Additional atomic steps e.g. stop app,
undeploy app, set default app, etc. Composite steps are mini-processes Reusable utilities in common Package up libs where appropriate Helper scripts where appropriate Interface with APIs and tools
appropriate to target platform Version to handle upgrades of target
platforms
36
APIs and Tools
Target Platform
Core Automation Plugin
build.xml
libs
scripts
Atomic•assembleApp•deployApp•etc.
Composite•assembleAndDeployApp
common.xml
CONNECT WITH US:
Wrap Plugins for UrbanCode Deploy
Port deploy automations to enterprise deployment platform Simple set of wrapper files
around proven automation Benefits
Keep orchestration (deploy processes) flexible
One button deploys for multi-component, multi-tier apps
Snapshot whole app and move through environments
Gates, approvals, etc.
37
UCD Plugin
Core Automation Plugin
build.xml
libs
scripts
Atomic•assembleApp•deployApp•etc.
Composite•assembleAndDeployApp
common.xml
plugin.xml
wrapper scripts.groovyAtomic• deployApp• etc.
component template.json
• component processes
CONNECT WITH US:
Continuous Integration and Provisioning of Plugins
Practice what we preach – optimized delivery of plugins Automate plugin assemble and
deploy Core plugins and UCD plugins Reduce time-to-test
Run in a continuous integration cycle to detect issues early
Combine with running of automation steps to test working
Feedback loop Snapshot plugins – promote to PROD
38
DEV/TEST PROD
DEV/TESTDeploy
PROD Deploy
DEV/TESTDeploy
PROD Deploy
JBE
UCD Serve
r
JBE
JBE
JBE
UCD Serve
r
CONNECT WITH US:
Containerize build agents Creating a flexible DevOps
infrastructure with Docker Build agent host installs take
time Docker gives simple way of
Codifying install process Shipping around “golden”
images Starting/stopping as many
instances (containers) as you wish
Keeping build machine stacks isolated
Provisioning new machines “instantly”
Prevent build machines from clogging up with installs and temporary files
39
Before:
After:
CONNECT WITH US:IT: Customized to Your Advantage
How Has This Helped?
Public | Copyright © 2014 Prolifics
CONNECT WITH US:
What has been achieved?
Automated assemble and deploy across 10 technology platforms.
Accelerated delivery pipelines for 11+ application development streams.
Oversight of deployments for all involved projects.
Core Build/Release team of 2.5 FTEs
Big Difference, Little Budget!
41
CONNECT WITH US:
What Benefits are Being Seen?
Able to reinvest upwards of 10 resources into more valuable activities.
Delivery is sooner. Testers are able to test more. Quality of code delivered is
higher. Visibility of delivery has
increased. Outages due to deployment
error close to eliminated.
42
CONNECT WITH US:
Impact on the Business, Customers, and Us?
Able to keep up with exploding number of music delivery channels.
Able to deliver more features that make it easier to do business.
Giving a better and more reliable quality of service to customers.
Collected royalties result in a healthier music industry, meaning better music for all of us!
43
CONNECT WITH US:
Thank YouYour Feedback is Important!
Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your
smartphone, laptop or conference kiosk.
CONNECT WITH US:
Contact Us
Public | Copyright © 2016 Prolifics 45
www.prolifics.com
+1 646 435 2470