Monitoring Docker Environments with Outlyer
-
Upload
todd-radel -
Category
Technology
-
view
147 -
download
0
Transcript of Monitoring Docker Environments with Outlyer
![Page 1: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/1.jpg)
Monitoring, done differently.
Monitoring Docker Environments with Outlyer
Todd RadelDirector, Customer Success
![Page 2: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/2.jpg)
About Me
2
● 20 years’ software development experience
● Java, .NET, Python, Perl, C++
● APM industry veteran4 years at AppDynamics
tradel
http://lnked.in/tradel
@tradel
![Page 3: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/3.jpg)
Agenda
• How Outlyer works with Docker
• Installing Outlyer agents on Docker
• How to use Host View to visualize your environment
• How plugins and packs work with containers
• Open house for Q&A
3
![Page 4: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/4.jpg)
How Outlyer works with Docker
4
![Page 5: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/5.jpg)
“Docker is hard.”
5
Colin HemmingsCTO, Outlyer
![Page 6: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/6.jpg)
Challenges in the Docker world
• Many versions of Docker, Kubernetes, Swarm, Rancher, etc.
• Breaking API changes between versions
• Evolving and releasing rapidly
• Networking model causes gaps in visibility
6
![Page 7: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/7.jpg)
Comparing Outlyer to other solutions
Basic metrics only (CPU/memory)
Over 500 metrics from base plugin (CPU, memory, network, I/O, load average, etc)
Difficult installation Installation with single command
No service monitoring Most plugins work in Docker
7
![Page 8: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/8.jpg)
A note about labels and tags
• Docker uses “labels” to attach metadata to containers
• Kubernetes/Swarm/Rancher add many of their own labels
• Outlyer agent automatically imports Docker labels as Outlyer tags
(but only label names, not values (yet))
8
Outlyer
TAGS
Docker
LABELS
![Page 9: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/9.jpg)
Installing Outlyer Agent on Docker
9
![Page 10: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/10.jpg)
The “hard” way
DATALOOP_AGENT_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
DATALOOP_NAME=docker_container_name
docker run -d --net host \
-e "DATALOOP_AGENT_KEY=${DATALOOP_AGENT_KEY}" \
-e "DATALOOP_NAME=${DATALOOP_NAME}" \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /proc:/rootfs/proc:ro \
-v /sys/fs/cgroup:/rootfs/sys/fs/cgroup:ro \
outlyer/agent:latest
10
![Page 11: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/11.jpg)
The hard way
DATALOOP_AGENT_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
DATALOOP_NAME=docker_container_name
docker run -d --net host \
-e "DATALOOP_AGENT_KEY=${DATALOOP_AGENT_KEY}" \
-e "DATALOOP_NAME=${DATALOOP_NAME}" \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /proc:/rootfs/proc:ro \
-v /sys/fs/cgroup:/rootfs/sys/fs/cgroup:ro \
outlyer/agent:latest
11
Needed to allow the agent to
gather host metrics
Automatically uses the latest
version - or check tags at
https://hub.docker.com/r/outlyer
/agent/ for specific versions
![Page 12: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/12.jpg)
Using Docker Compose
wget https://rawgit.com/outlyerapp/docker-alpine/master/agent/docker-compose.yaml
vim docker-compose.yaml
docker-compose -f docker-compose.yaml up
12
![Page 13: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/13.jpg)
Using Docker Compose
version: '2'
services:
dataloop-docker:
image: outlyer/agent:latest
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock:rw
- /sys/fs/cgroup:/rootfs/sys/fs/cgroup:ro
- /var/lib/docker/:/var/lib/docker:ro
- /proc:/rootfs/proc:ro
environment:
- DATALOOP_NAME=outlyer-agent
- DATALOOP_AGENT_KEY=<your dataloop agent key>
13
![Page 14: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/14.jpg)
Kubernetes Deployment
wget https://rawgit.com/outlyerapp/docker-alpine/master/agent/kubernetes_daemonset.yaml
kubectl apply -f kubernetes_daemonset.yaml
• Deploys one container on every host in cluster (masters and workers)
• Lights up your entire cluster with a single command
14
![Page 15: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/15.jpg)
Swarm and Rancher Deployment
• Swarm does not currently support host networking
Use manual method for now
• Deploy on Rancher through Rancher UI
Config files will be available soon
15
![Page 16: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/16.jpg)
Using Host View to visualize your cluster
16
![Page 17: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/17.jpg)
What is Host View?
• Displays health of every container
and host
• Sort, group, and filter by tags
or Docker labels
• Visualize health by
CPU/memory/presence
• Show links between containers
and hosts
• Container status at a glance
17
![Page 18: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/18.jpg)
How plugins and packs work in Docker
18
![Page 19: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/19.jpg)
The problem
• Most plugins (and Nagios scripts) want to access “localhost” for ports,
files, etc.
HOST = 'localhost'
PORT = 27017
• But the agent runs in a different container from the app you want to
monitor
19
![Page 20: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/20.jpg)
The magic solution
• Docker compatibility layer
• Transparently intercepts common API calls to localhost
• Replaced with calls to target container
20
![Page 21: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/21.jpg)
The magic solution
Most plugins and packs work the same inside and outside of Docker!
21
![Page 22: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/22.jpg)
Adapting your own plugins
• See if the client library is already covered by plugin-helper.(Requests, MongoClient, Psycopg2, Socket, Subprocess, others)
• If so: no code changes necessary!
• If not: use the plugin-helper to find the target container.
22
![Page 23: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/23.jpg)
Using the plugin helper library
try:
from outlyer.plugin_helper.container import is_container, get_container_ip
if is_container():
container_ip = get_container_ip()
except ImportError:
print(“Plugin helper library is not available!”)
print(“Please upgrade your Outlyer agent to the latest version.”)
23
![Page 24: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/24.jpg)
Coming Soon!
24
![Page 25: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/25.jpg)
New features in Fall 2017 release
• Full support for key/value labels
• Host View becomes the default view
• “Scoped” dashboards with label selectors
25
![Page 26: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/26.jpg)
Questions & Answers
26
![Page 27: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/27.jpg)
Resources
27
![Page 28: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/28.jpg)
Resources
• Monitoring Docker environments with Outlyer
(product documentation)
• Plugin Helper library
(for adapting your plugins to Docker)
• Support Slack
(sign up and ask questions)
28
![Page 29: Monitoring Docker Environments with Outlyer](https://reader031.fdocuments.us/reader031/viewer/2022022415/5a64915f7f8b9a94568b4c3b/html5/thumbnails/29.jpg)
Thank You!
29