Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular...
Transcript of Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular...
![Page 1: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/1.jpg)
Grafana and MySQL Benefits and Challenges
![Page 2: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/2.jpg)
About me Philip Wernersbach Software Engineer
Ingram Content Group
https://github.com/philip-wernersbach
https://www.linkedin.com/in/pwernersbach
2
![Page 3: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/3.jpg)
• I work in Ingram Content Group’s Automated Print On Demand division
• We have an automated process in which publishers (independent or corporate) request books via a website, and we automatically print, bind, and ship those books to them
• This process involves lots of hardware devices and software components
3
![Page 4: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/4.jpg)
The Problem
4
![Page 5: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/5.jpg)
The Problem “How do we aggregate and track metrics from our hardware and software sources, and display those data points in a graph format to the end user?” à Grafana!
5
![Page 6: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/6.jpg)
Which data store should we use with Grafana? ▸ Out of the box, Grafana supports
Elasticsearch, Graphite, InfluxDB, KairosDB, OpenTSDB
6
![Page 7: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/7.jpg)
Which data store should we use with Grafana? ▸ We compared the options and tried
InfluxDB ▸ There were several sticking points
with InfluxDB, both technical and organizational, that caused us to rule it out
7
![Page 8: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/8.jpg)
Which data store should we use with Grafana? ▸ We already have a MySQL cluster
deployed, System Administrators and Operations know how to manage it
▸ Decided to go with MySQL as a data store for Grafana
8
![Page 9: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/9.jpg)
The Solution: Ingram Content’s Grafana-MySQL
Integration
9
![Page 10: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/10.jpg)
The Integration
▸ Written in Nim ▸ Emulates an InfluxDB server ▸ Connects to an existing
MySQL server ▸ Protocol compatible with
InfluxDB 0.9.3 ▸ Acts as a proxy that converts
the InfluxDB protocol to the MySQL protocol and vice-versa
10
![Page 11: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/11.jpg)
The Integration
11
Grafana
Integration Proxy (InfluxDB compatible)
MySQL
![Page 12: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/12.jpg)
The Challenges!
12
![Page 13: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/13.jpg)
InfluxDB ▸ Time Series
database ▸ “time” is a special
value, has special meaning
▸ Applies special logic to “time”
SMART!
The Challenges! Database Engine Comparison
MySQL ▸ Generic relational
database ▸ “time” is a generic
data value
▸ Doesn’t apply special logic
DUMB!
13
![Page 14: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/14.jpg)
The Challenges! Query Comparison
InfluxDB “SELECT count(bar1) FROM foo WHERE bar1 > 0 AND bar2 > 0 AND time > now() - 7d GROUP BY time(1h)”
14
![Page 15: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/15.jpg)
The Challenges! Query Comparison
MySQL “SELECT time, count(bar1) FROM foo WHERE bar1 > 0 AND bar2 > 0 AND time > NOW(6) - INTERVAL 7 DAY GROUP BY YEAR(time), MONTH(time), DAY(time), HOUR(time) ORDER BY time ASC”
15
![Page 16: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/16.jpg)
The Challenges! InfluxQL != SQL
InfluxQL != SQL ▸ InfluxQL is SQL-like, but
different enough that it can’t be passed through to MySQL
16
![Page 17: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/17.jpg)
The Challenges! InfluxQL != SQL
▸ “time” is SELECT’d automatically implicitly in InfluxDB
▹ “SELECT bar FROM foo” à “SELECT time, bar FROM FOO”
▸ GROUP’ing on “time” is smart in
InfluxDB, and dumb in MySQL ▹ See slides 16 and 17 ▸ “time” in epoch format with
millisecond precision is a float in MySQL
▹ “FROM_UNIXTIME(1444667802.145)”
17
![Page 18: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/18.jpg)
The Challenges! InfluxQL != SQL
▸ InfluxDB series are roughly equivalent to MySQL tables, but the management syntax is sometimes different
▹ “DROP SERIES FROM foo” à “DELETE FROM foo”
▸ InfluxDB allows quoted identifiers, MySQL is more restrictive
▹ “SELECT * FROM foo WHERE bar > ‘0’” à Silently returns garbage in MySQL
▹ Grafana quotes identifiers aggressively, so we had to patch it
18
![Page 19: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/19.jpg)
The Challenges! InfluxQL != SQL
There’s probably more InfluxQL-SQL incompatibilities, but these are the ones that we’ve run into with Grafana at Ingram Content Group.
19
![Page 20: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/20.jpg)
The Benefits!
20
![Page 21: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/21.jpg)
The Benefits!
▸ Allows us to leverage System Administrator’s and Operation’s existing knowledge of MySQL
▸ Allows us to process large amounts of data
▹ InfluxDB had issues when we tried to insert about a million points at a time, MySQL handles this with no problem
21
![Page 22: Grafana and MySQL Benefits and Challenges · 2019-12-18 · The Benefits! We can use full, regular SQL for our Grafana queries in addition to InfluxQL Think JOINs, etc. We’re doing](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6dbe47a0d0da0e362dbbb3/html5/thumbnails/22.jpg)
The Benefits!
▸ We can use full, regular SQL for our Grafana queries in addition to InfluxQL
▹ Think JOINs, etc.
▸ We’re doing massive batch inserts, and MySQL’s query cache makes Grafana queries significantly faster
▹ This is obviously use case specific, but shows how you can tune MySQL to your specific use case
22