Monitoring Big Data Systems - "The Simple Way"
-
Upload
demi-ben-ari -
Category
Engineering
-
view
773 -
download
5
Transcript of Monitoring Big Data Systems - "The Simple Way"
![Page 1: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/1.jpg)
Monitoring Big Data Systems Done "The Simple Way"
Demi Ben-Ari - CTO @ Panorays
![Page 2: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/2.jpg)
About Me
Demi Ben-Ari, Co-Founder & CTO @ Panorays● BS’c Computer Science – Academic College Tel-Aviv Yaffo● Co-Founder “Big Things” Big Data Community
In the Past:● Sr. Data Engineer - Windward● Team Leader & Sr. Java Software Engineer,
Missile defense and Alert System - “Ofek” – IAFInterested in almost every kind of technology – A True Geek
![Page 3: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/3.jpg)
Agenda
● A lot of (NOT) funny Jokes● Problem definition and Environment● Monitoring pipeline solutions
○ Metrics○ Datastore○ Dashboards○ Alerting
● Summary● (Not going to address Service discovery and monitoring)
![Page 4: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/4.jpg)
Say “Distributed”, Say “Big Data”,Say….
![Page 5: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/5.jpg)
What is Big Data (IMHO)? And What to Monitor?● Systems involving the “3 Vs”:
What are the right questions we want to ask?○ Volume - How much?
■ Amount per second / minute / hour / day….■ Gigabytes, Terabytes, Petabytes…
○ Velocity - How fast?■ Count per second / minute / hour / day….
○ Variety ■ What kind? (Difference) ■ Sensor Data, Logs, Data Streams, Financial Transactions, Geo Locations...
![Page 6: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/6.jpg)
Monolith Structure
OS CPU Memory Disk
Processes Java Application Server
Database
Web Server
Load Balancer
Users - Other Applications
Monitoring System
UI
![Page 7: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/7.jpg)
Distributed Microservices Architecture
Service A
Queue
DB
Service B
DBCache
Cache DBService C
Web Server
DB
Analytics Cluster
Master
Slave Slave Slave
Monitoring System???
![Page 8: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/8.jpg)
Some basic concepts
![Page 9: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/9.jpg)
Basic Concepts● Monitoring
○ Collecting, processing, aggregating, and displaying real-time quantitative data about a system
● White-box○ Monitoring based on metrics exposed by the internals of the system○ logs, interfaces JMX of JVM, etc
● Black-box ○ Testing externally visible behavior as a user would see it.
● Dashboard○ An application that provides a summary view of a service’s core metrics.
![Page 10: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/10.jpg)
Basic Concepts● Alert
○ A notification intended to be read by a human and that is pushed to a system such as a bug or ticket queue, an email alias or a pager.
● Root cause○ A defect in a software or human system that - if repaired, instills confidence that this
event won’t happen again in the same way.● Node and machine
○ Used interchangeably to indicate a single instance (physical server, virtual machine or container). There might be multiple services worth monitoring on a single machine.
● Push○ Any change to a service’s running software or its configuration.
● KPI - Key Performance Indicator
![Page 11: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/11.jpg)
Data flow and Environment(Our Use Case)
![Page 12: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/12.jpg)
Data Flow Diagram
External Data
Source
Analytics Layers
Data Pipeline
Parsed Raw
Entity Resolution Process
Building insightson top of the entities
Data Output Layer
Anomaly Detection
Trends
UI for End Users
![Page 13: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/13.jpg)
Environment Description
Cluster
Dev TestingLive
Staging ProductionEnv
OB1K
RESTful Java Services
![Page 14: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/14.jpg)
Situations
![Page 15: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/15.jpg)
MongoDB + Spark
Worker 1
Worker 2
….
….
…
…
Worker N
Spark Cluster
Master
Write
Read
MasterSahrded MongoDB
Replica Set
![Page 16: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/16.jpg)
Cassandra + Spark
Worker 1
Worker 2
….
….
…
…
Worker N
Cassandra Cluster
Spark Cluster
Write
Read
![Page 17: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/17.jpg)
Cassandra + Serving
Cassandra Cluster
Write
Read
UI ClientUI Client
UI ClientUI Client
Web ServiceWeb
ServiceWeb ServiceWeb
Service
![Page 18: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/18.jpg)
Problems● Multiple physical servers
● Multiple logical services
● Want Scaling => More Servers
● Even if you had all of the metrics○ You’ll have an overflow of the data
● Your monitoring becomes a “Big Data” problem itself
![Page 19: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/19.jpg)
The what really “Distributed” Means
The DevOps Guy
(It might be you)
![Page 20: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/20.jpg)
So...Let’s Start!
![Page 21: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/21.jpg)
Report to Where?● We chose: ● Graphite (InfluxDB) + Grafana● Can correlate System and
Application metrics in one place :)
![Page 22: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/22.jpg)
Report to Where?● Save DevOps efforts if you’re willing to Pay :)● Hosted Graphite
○ https://www.hostedgraphite.com/
● Throwing the “Big Data” volume monitoring problem at someone else
![Page 23: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/23.jpg)
Connections Connections...
http://www.mememaker.net/meme/connections-connections-everywhere2/
![Page 24: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/24.jpg)
Drivers to Datastores● Actions they usually do:
○ Open connection○ Apply actions
■ Select■ Insert■ Update■ Delete
○ Close connection
● Do you monitor each?○ Hint: Yes!!!! Hell Yes!!!
● Creating a wrapper in any programming language and reporting the metrics○ Count, execution times, errors…○ A bit of Infrastructure code that will give great visibility
![Page 25: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/25.jpg)
Monitoring Operation System
![Page 26: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/26.jpg)
Monitoring Operation System Metrics● What to measure:
○ CPU○ Memory○ Disk Space
● How to measure:○ CollectD or StatsD reporting to Graphite○ New Relic
■ Nice and easy UI■ Even the free account gives great tool■ Alerting of thresholds
![Page 27: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/27.jpg)
Monitoring Cassandra
![Page 28: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/28.jpg)
Monitoring Cassandra● OpsCenter - by DataStax
![Page 29: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/29.jpg)
Monitoring Cassandra● Is the enough?...
We can connect it to Graphite also (Blog: “Monitoring the hell out of
Cassandra”)
● Plug & Play the metrics to Graphite - Internal Cassandra mechanism
● Back to the basics: dstat, iostat, iotop, jstack
![Page 30: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/30.jpg)
Monitoring Cassandra
![Page 31: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/31.jpg)
Monitoring Cassandra - Alternative
![Page 32: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/32.jpg)
Monitoring Cassandra - Some more :)● Cyanite: http://cyanite.io/
Graphite with Cassandra backend as a datasource.
● Nodetool - Cassandra tool
● Back to the basics: dstat, iostat, iotop, jstack
![Page 33: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/33.jpg)
Some help from “the Cloud”
![Page 34: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/34.jpg)
Monitoring via AWS’s CloudWatch
![Page 35: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/35.jpg)
Google Stackdriver (GCP)● Can integrate both GCP and Amazon accounts
![Page 36: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/36.jpg)
Monitoring Spark
![Page 37: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/37.jpg)
What to monitor in an Apache Spark Cluster● Application execution
● Resource consumption and allocation
● Task Failures
● Environment and Amount of servers
● Physical OS metrics
● Infrastructure services
![Page 38: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/38.jpg)
Ways to Monitoring Spark● Sending Metrics: Spark → Graphite (Execution)
● http://spark.apache.org/docs/latest/monitoring.html
![Page 39: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/39.jpg)
Ways to Monitoring Spark● Sending Metrics: Spark → Graphite (JVM metrics)
● http://spark.apache.org/docs/latest/monitoring.html
![Page 40: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/40.jpg)
Ways to Monitoring Spark● Grafana-spark-dashboards
○ Blog: http://www.hammerlab.org/2015/02/27/monitoring-spark-with-graphite-and-grafana/
● Spark UI - Online on each application running● Spark History Server - Offline (After application finishes)● Spark REST API
○ Querying via inner tools to do ad-hoc monitoring
● Back to the basics: dstat, iostat, iotop, jstack● Blog post by Tzach Zohar - “Tips from the Trenches”
![Page 41: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/41.jpg)
Monitoring Your Data
https://memegenerator.net/instance/53617544
![Page 42: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/42.jpg)
Data Questions?
● Did all of the computation occur?
● Are there any data layers missing?
● How much data do we have? (Volume)
● Is all of the data in the Database?
![Page 43: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/43.jpg)
Data Answers!● KISS (Keep it simple stupid)
● Jenkins + Maven (JUnit) for the rescue
● Creating a maven “monitoring” project.○ Running scheduled tasks, each for the relevant data source
■ Database data existence
■ S3 files existence
■ Data flow that keeps on coming from sensors
■ (Any other data source that you can imagine…)
○ Scheduled task that write amount metrics to Graphite -> Dashboards
○ Report task execution to Graphite
![Page 44: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/44.jpg)
Data Answers!● The method doesn’t really matter, as long as you:
○ Can follow the results over time
○ Know what your data flow and know where things might fail
○ It’s easy for anyone to add more monitoring
(For the ones that add the new data each time…)
○ It don’t trust others to add monitoring
(It will always end up the DevOps’s “fault” -> No monitoring will be
applied)
![Page 45: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/45.jpg)
Logging?Monitoring?
https://lh4.googleusercontent.com/DFVcH-E5XKj8cbhEtI0qabmf_wwVqWWvk0pK5H5rnC_kVxY2tXClKfzV-LvAH61YRLJUEvtO9amjWfjcY4Z57VBYCuQ95_hdAVEHgLAuepJiArH0wJERWuzzmgnPysCiIA
![Page 46: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/46.jpg)
● Elastic● Architecture:
Server
Server
Server
ELK - Elasticsearch + Logstash + Kibana
Shippers
Queue
Indexer Web UIStorage
![Page 47: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/47.jpg)
● (Simpler) Architecture:○ The problem: Log42 only works with TCP :( => Log4J2 works with UDP too
Server
Server
Server
ELK - Elasticsearch + Logstash + Kibana
Indexer Web UIStorage
TCP / UDP
![Page 48: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/48.jpg)
ELK - Elasticsearch + Logstash + Kibana
http://www.digitalgov.gov/2014/05/07/analyzing-search-data-in-real-time-to-drive-decisions/
![Page 49: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/49.jpg)
ELK - Elasticsearch + Logstash + Kibana
http://blog.takipi.com/log-management-tools-face-off-splunk-vs-logstash-vs-sumo-logic/
![Page 50: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/50.jpg)
Who else Logs?
● Graylog2
● ….
● Logging As a Service :)
○ Logz.io (http://logz.io/blog/deploy-elk-production)
○ Logly
○ sematext
![Page 51: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/51.jpg)
How does it look in real life?
● http://www.digitalgov.gov/2015/01/07/elk/● http://www.ragedsyscoder.com/monitoring-slides/file/img/tvs.jpg
![Page 52: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/52.jpg)
Did someone say “Dashboard”?
http://www.funpic.hu/_files/pictures/original/86/71/27186.jpg
![Page 53: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/53.jpg)
Redash● http://redash.io/● Open Source: https://github.com/getredash/redash● Came out as one of many Open source tool by Everything.me● Created and Maintained by Arik Fraimovich (You rock!)● Written in Python● Has an on-premise and hosted solution
רןאאקמ●
![Page 54: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/54.jpg)
Redash - Data Sources
![Page 55: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/55.jpg)
Redash - Screenshots
![Page 56: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/56.jpg)
Redash - Screenshots
![Page 57: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/57.jpg)
Redash - the “Why?”● Having multiple data sources in the organization
● Wanting to see all a combination of data sources in one place
● It’s open source and ready to use
● Why implement fancy UI and spend a lot of time?!?!?
● So...just use it!
![Page 58: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/58.jpg)
Alerting
![Page 59: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/59.jpg)
Alerting● Syren - Open source● Reporting to:
○ Email, Flowdock, HipChat, HTTP,
Hubot, IRCcat, PagerDuty,
Pushover, SLF4J, Slack, SNMP, Twilio
![Page 60: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/60.jpg)
ELK - And what about alerting???● Elastalert
● http://engineeringblog.yelp.com/2015/10/elastalert-alerting-at-scale-with-elasticsearch.html● http://engineeringblog.yelp.com/2016/03/elastalert-part-two.html
![Page 61: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/61.jpg)
Some more alerting● Cloudwatch and Stackdriver has their own alerting mechanism● New Relic has it’s own alerting too● Even with our Jenkins tests we’ve created alerting via emails
○ Beware of “Spam”
● Find which solution you would like as long as:○ You can notice what is wrong => when it’s wrong ○ Be able to “Acknowledge” your errors○ Do something you won’t be able to ignore :)
![Page 62: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/62.jpg)
Summary - Monitoring Stack
Alerting
Metrics Collection
Datastore
Dashboard
Data Monitoring
Log Monitoring
![Page 63: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/63.jpg)
Conclusions● Correlating Application and System metrics!!!!● Ask the right monitoring questions and answer them with Dashboards● KISS - simple is key, what’s hard, we tend not to do at all● Alert about what you can actually react to (And to the relevant person)● Measure whatever you can - only way to know if you’re improving● Monitor your business KPIs too.
● If all of the above is not enough,
Graphs are fricking cool!
http://www.rantlifestyle.com/2013/09/23/how-happy-this-baby-is-will-shock-you/
![Page 64: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/64.jpg)
Questions?
![Page 65: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/65.jpg)
�Demi Ben-Ari● LinkedIn● Twitter: @demibenari● Blog: http://progexc.blogspot.com/● Email: [email protected]● “Big Things” Community�Meetup, YouTube, Facebook, Twitter● GDG Cloud
Thanks! my contact:
![Page 66: Monitoring Big Data Systems - "The Simple Way"](https://reader034.fdocuments.us/reader034/viewer/2022050719/5871969c1a28ab044e8b4c99/html5/thumbnails/66.jpg)
Resources● Monitoring distributed systems - A case study in how Google monitors its
complex systems