Federated Graphite in Docker - Denver Docker Meetup
-
Upload
phil-zimmerman -
Category
Technology
-
view
602 -
download
1
Transcript of Federated Graphite in Docker - Denver Docker Meetup
Containing Graphite(And Herding Cats)
Trying to Make the World at TWC a Better Place
*Drinker of Good Beer
Phil Zimmerman*
@phil_zimmerman
Graphite: OverviewLegos and Herding CatsShow Me Some Cool Shit
Meet The Players
Registrator
A Self-Serve Interface to Your Cluster
Distributed init For Long-running Services
You Know The Company Behind Consul(Think Vagrant)
Key/Value StoreService Discovery (HTTP and DNS)
Failure DetectionMulti Datacenter
Queries Consul and Updates SpecifiedTemplates on the Filesystem
Registrator
https://github.com/gliderlabs/registrator
Auto Register/De-register Docker Containers
Graphite Overview
GraphiteCarbon-cache
WhisperGraphite Web
Carbon-relay
Carbon-cacheReceives and Writes Incoming Metrics
Caches and flushes data
2 Receiver ports, 1 Query Port
Whisper DatabaseTime-series Data
Graphite WebQueries carbon-cache
Reads whisper dataAND
Carbon-relayLoad Balances and Proxies Metrics
Can Write To Multiple carbon-caches
For High Volume of Metrics
Carbon Relay [:2003]
Metrics Metrics Metrics
Carbon Cache [:2003] [:2004] [:7002]
Whisper Data
Carbon Webapp [:80]
postgresql (dashboards)
[:5432]
Typical Single-node Setup
Does Anyone See Containers??
Master Webapp [:80]
pgsql[:5432]
Clustered Setup
Carbon Relay [:2003]
Metrics Metrics Metrics
Carbon Relay [:2003]
Carbon Relay [:2003]
Carbon Relay [:2003]
Carbon Cache [:2003] [:2004] [:7002]
Whisper Data
Carbon Cache [:2003] [:2004] [:7002]
Carbon Webapp [:80]
MemcachedMemcachedMemcached [:11211]
Carbon Relay [:2003]
Carbon Cache [:2003] [:2004] [:7002]
Whisper Data
Carbon Cache [:2003] [:2004] [:7002]
Carbon Webapp [:80]
And You Thought You Were Too Old
To Play With Legos
How Do We Orchestrate All Of This??
Without Service Discovery
Containers
Service AnnouncementService Discovery
Configuration
What We NeedWhen Containers Start/Stop, They Register/
De-register Their ServicesWhen Dependent Service Endpoints Change,
Config Files Auto-Update and Trigger Service Restart
Registrator
The Service ContainersCarbon-cache: Dockerfile
The Service ContainersCarbon-relay: Dockerfile
The Service ContainersCarbon-webapp: Dockerfile
The Service Containerspostgresql: Dockerfile
The Service Containersmemcached
The Data Volume ContainersWhisperCreate a carbon-cache container without
running the cmd (use entrypoint)
The Data Volume Containersgraphite-dbCreate a postgresql container without
running the cmd (use entrypoint)
The Data Volume Containersconsul-templateCreate a consul-template container
without running the cmd (use entrypoint)
Service Announcement/Registration
RegistratorRuns in a container Pass consul url as argument for docker run
Service Announcement/Registration
docker run -d -P -e SERVICE_5432_NAME=graphite-db
--volumes-from dashboard-data twcable/postgresql93:0.2
Start & Register postgresql
Service Discoverygraphite-master-webapplocal_settings.ctmpl (consul-template)
Service Discoverygraphite-master-webapptmpl.json - config for consul-template
Services Registered In
Consul
Great, Now How Do You Launch All of These
Containers??
Registrator
Mesos Master !
Mesos Marathon
Zookeeper
Mesos Master !
Mesos Marathon
Zookeeper
Mesos Master !
Mesos Marathon
Zookeeper
Mesos Slave !
Mesos Docker Consul
Mesos Slave !
Mesos Docker Consul
Mesos Cluster
Marathon JSON Deploy Definition
Post JSON To Running Marathon Instance
Show Me Something Cool
Helpful Linkshttp://mesos.apache.org/https://mesosphere.github.io/marathon/https://consul.io/https://github.com/hashicorp/consul-templatehttps://github.com/gliderlabs/registrator
https://grey-boundary.io/the-architecture-of-clustering-graphite/
http://philzim.com/2014/11/12/service-discovery-orchestration-with-mesos-and-consul/