Package Repositories: The Unsung Heroes of Configuration and Release Management

1

Webcast Starts at 1:00 EDT

… no you shouldn’t be hearing anything yet

#PackageReposWebinar

2

We start in less than 15 min



3




4


… no you shouldn’t be hearing anything yetGetting excited? I am.


5

Package Repositories

The unsung heroes.


6

Eric’s Bio

I’m a Lead Consultant at Urbancode where I helps customers get the most out of their build, deploy and release processes. I have 9 years of automation experience throughout the application life-cycle in roles as a developer, test automation engineer, and support engineer. I’ve been at the forefront of CI & CD for 7+ years

Eric [email protected]@EricMinick


mailto:[email protected]

7

Serious problems from top to bottom

• Dan, new developer: can’t compile• Pam, experienced programmer: dealing with a

merge conflict• Tom, tester: just had a batch of bugs rejected• Owen, operations: dealing with a production

failure• Cynthia, CIO: has just been informed the

organization failed an audit


8

Agenda

• Packages and configuration management• Traditional approaches• Elements of a successful solution• Recipe for adoption


9

Agenda


10

Configuration management

• ITIL Definition*: “The Process responsible for maintaining information about Configuration Items required to deliver an IT Service, including their Relationships.”

• Understand what we are releasing and how it relates to everything else.

*Source: ITIL v3 Service Transition. 2007.

11

Lots of stuff to version

Develop / Build Runtime

“Cod

e”In

fras

truc

ture

source

3rd Party libs

Internal Libs

Config templates

VM ImagesEnvironmentTemplates

Deployment Manifests

Deployable Builds

EnvironmentManifests

Sub-Projects

Middleware


12

Agenda



13

“How do I make this build work?”

• Developer attempting to build– new developer feels the pain

• Build machine view of binaries– magic build machine

• Dependency information comes in the form of link errors


14

“Use Google”


15

“I’ll email that to you”


16

“It’s in the lib directory”

Binary dependencies are versioned with the source code


17

“Go get that off the file share”

• Single source for the organization• Dependencies are now scriptable• Version explosion– hard to know when to remove an old version– hard to know which to use


18

“I’m waiting for the internet to download”

• Using Maven Central as a binary repository


19

Deploy & Release: almost as bad as build


20

Agenda

• Binaries and configuration management• Traditional approaches for dealing with

binaries• Elements of a successful solution• Recipe for adoption


21

What they need

• Developers– A description of dependencies– Location to get them (and easy updates)– Controlled official versions of dependencies

• QA– What’s in my environment: A deployment manifest

• Ops– The same manifest & everyone else’s house in order

• CIO / Audit– Inventory of who deployed what where


22

ITIL Definitive Media Library

• Location where the definitive and approved versions of all software configuration items are securely stored

• Includes:– archive and retention periods– environment support (e.g. test and live environments)– tamper resistance

Service Transition, ITIL v 3; Lacy & Macfarlane; 2007


23

Good binary management

• Package Repository– authoritative place to store versioned binaries– access control– checksums for tamper resistance– release meta-data– retention periods

• Dependency management– ITIL: “relationships between configuration items”


24

Build Dependencies

• “Description of dependencies” is complex

• Requires:– track compile time and runtime dependencies– automated retrieval from repository– traceable: give me a manifest


25

Deployment Manifests

• Collection of versioned packages to deploy• How (also versioned)– Process– Configuration rules

source: http://www.flickr.com/photos/expertinfantry/5449659589/

26

Package repos: hand off from Dev to Ops

AKA: DSL, DML, Artifact Repo, Binary Repo


27 #PackageReposWebinar

28

Agenda

• Binaries and configuration management• Traditional approaches for dealing with

binaries• Elements of a successful solution• Recipe for adoption


29

Our recommended recipe

1. The dependency audit2. Decide who will control dependency rules3. Decide who will control the repository4. Establish a binary artifact repository5. Link scripts to repository6. Migrate dependencies to repository7. Deny the old methods8. Develop tested stacks


30

Dependency Audit

• Begin researching dependency relationships• Need to avoid breaking what we have today• Validate files are the version they claim to be• Requires understanding relationships between

teams and their components• Where is there commonality? Conflicts?


31

Decide who will control repository

• Will we have one repo or several?

• How do new 3rd party artifacts get added?– architecture checks for duplication– security and compliance concerns

• What internal systems can register versions?

• What policies are used for removing old versions?


32

Decide who will control of dependency rules

• Developers often best understand the project’s needs

• Architects suggest components that should be used everywhere

• CM team often set policy for what’s allowed• QA knows what’s been tested• Operations knows what’s approved for

production use


33

Establish a package repository

• Select a repository– Codestation, Maven, Yum, SCM Tool….

• Implement it– installation– configuration– disaster recovery– retention policy– security rules• authentication & authorization


34

Link scripts to repository

• Build scripts• Deployment scripts• Provisioning scripts

• Create a “walking skeleton” for a smooth, incremental transition

http://alistair.cockburn.us/Walking+skeleton


35

Migrate dependencies to repository

• Should be easy

• It won’t be


36

Deny the old methods

• Turn off the file share

• Firewall off Maven Central

• Reject commits of libraries to source control


37

Develop tested stacks

• Identify groups of components commonly used together

• Test versions of those groups and create a “stack” – a version of the group

• If you build your app on this stack, it will work in our environment– With infrastructure automation this becomes PaaS


38

Checking in with the team

• Dan, new developer: quickly up to speed• Pam, experienced programmer: getting work

done• Tom, tester: not wasting time on bogus bugs• Owen, operations: working a 40 hour work

week• Cynthia, CIO: working on new initiatives


39

Key Takeaways

• The package repository as a key role in your tool chain

• Audit what you are using in this place now and consider alternatives that are better tailored

• If it’s important, version it


40

Package Repo bundled in our Products

• AnthillPro– All in one continuous delivery platform

• uBuild – Build automation and CI for the hard problems

• uDeploy– Deployment and release management

• uProvision– Spins up virtual environments. Integrated with

VMWare, Azure and EC2

41

Q&A

@UrbanCodeSoft@EricMinick

Slideshare.net/Urbancode


http://slideshare.net/Urbancode

Package Repositories: The Unsung Heroes of Configuration and Release Management

Technology

Transcript of Package Repositories: The Unsung Heroes of Configuration and Release Management