Implementing Docker in Production at Scale

1

Implementing Dockerin Production at Scale

Sean P. Kane & Karl MatthiasLead Site Reliability Engineers

2Confidential ©2008-15 New Relic, Inc. All rights reserved.

This document and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.

Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,” “expects” or words of similar import.

Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov.

New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.


Overview - New Relic’s conversion to Docker containers

- Building an MVP platform

- Lessons we learned from 1.5 years in production

- Building a v.2 platform


The Incentives

4

http://newrelic.com/insights

SOA initiative Monolithic to Micro-Services

New product launch Greener Fields Doubling our server count

Inconsistent deployment story Per team Capistrano & Rake

http://newrelic.com/insights


A Simple Platform that Scaled Well…

Where Our Platform Got Us

5

5

250+ production containers

75+ project

s

15+ teams


Simple approach…

Our First Platform

6

6

Solve Deployment First Simple Solutions Work. Avoid Big Design Up Front


Our First Platform

7

7

Remote Logging

Static Load Balancing

Use dedicated ports, everywhereOur New

PlatformOld School

Service Discovery


Our First Platform

8

8

Process Management

Supervisord in all the containers


Our First Platform

9

9

Remote Logging

Supervisord, Heka, and Syslog

Build out your logging story up front


Our First Platform

10

10

Nagios and cAdvisor

New Relic Server Monitor w/ Docker support

/status/check (200 OK)

Monitor overall service health versus containers.

Stats and Monitoring


Our First Platform

11

11

Stats & Monitoring

New Relic Server Monitor w/Docker support


Our First Platform

12

12

Application Standards

/status/checkExternalized ConfigurationStandardized base images for Docker


Our First Platform

Shipright (Internal Tool)

Resource Management

Port Registry (Internal Tool)

Centurion

Service Discovery

Docker build tool


Our First Platform


Resource Management


Centurion

Old-school Service Discovery

Noisy neighbors - cgroups

A single, large Docker cluster is better than multiple smaller team-based clusters


Our First Platform


Resource Management


Centurion

Service Discovery

Centurion

Where do services live?


Our First Platform


Resource Management


Centurion

Service Discovery

Docker Deployment Tool for Repeatable Deployments

https://github.com/newrelic/centurion Rolling, zero-down-time

deploys

Any engineer can deploy any app

Configuration & secrets are not in the application repo

http://github.com/newrelic/centurion






Our New Platform

Mesos - Resource offers

Marathon – Scheduler

Custom Executor

Dogestry – S3 CLI image registry

https://github.com/dogestry/dogestry








Service Discovery

Distributed Load Balancing

Health Check Monitoring

Intend to Open Source later in 2015

Tooling We’re Building

18

18

Sidecar


DEMO


The simple approach worked.

It provided us time to learn as an organization.

It allowed us to iterate fast and often.

It provided us the time to build something better. That we needed.

Conclusion

20

20


Presenter Name, Title and or Date

Thanks

@relistan – Karl Matthias

@spkane – Sean P. Kane

http://goo.gl/AM943Mhttp://newrelic.com/docker

https://twitter.com/relistan

https://twitter.com/spkane

http://goo.gl/AM943M

http://newrelic.com/docker




Implementing Docker in Production at Scale

Internet

Transcript of Implementing Docker in Production at Scale