Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016...

46

Transcript of Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016...

Page 1: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 2: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 3: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 4: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 5: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 6: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 7: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 8: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 9: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring
Page 10: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

Peter Zaitsev, CEO, Percona

February 24, 2016 Percona Technical Webinars

Using Grafana for MySQL Monitoring

Page 11: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 2

Visualization

A picture is worth a thousand words

Page 12: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 3

Why Grafana ?

Fantastic Visualization solution for Graphs

Open Source

Great Community and Momentum

Not tied to single Data Source

Page 13: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 4

Technology Percona Way

Focus on finding best

Solution

Do not reinvent the

bicycle

Page 14: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 5

Why Prometheus

Great out of the box support for MySQL

Smart Practically focused team behind

Multi-Dimensional Data model for modern world

Powerful and simple language for visualization

Integrates with Grafana well-ish

Page 15: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 6

Different Monitoring Needs

Alerting Capacity Planning

Troubleshooting Performance Analyses

Page 16: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 7

Way to deal with Data

Capture lots of data

Less frequently if too expensive

Build Dashboards for most important things

Custom Visualization for specific problem analyses

Page 17: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 8

Not Just Metrics

We use Grafana together with other tools

“Query Analytics” for MySQL Queries

Pt-stalk to capture snapshots

Page 18: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 9

USE Method by Brendan Gregg

Utilization Saturation

Errors

Page 19: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 10

Database Environment Matters

Capture OS Level

information

Correlate OS and MySQL

Data

Page 20: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 11

Examples Lets Look at some graphs

Page 21: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 12

Use Interval Templating

•  Custom Intervals to match Prometheus Resolution

Page 22: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 13

Match Step and Rate

•  Resolution 1/1 makes most sense

Page 23: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 14

IO Latency

•  Exponential graph format •  Use dots for volatile data

Page 24: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 15

IOPS

•  Great indication of Work Storage handles

Page 25: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 16

Correlate with 2-Axis Graph

•  Increase iops but not latency – great!

Page 26: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 17

Disk Bandwidth

•  Staircase line is a lot less missleading

Page 27: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 18

Disk IO Utilization

•  Often Misunderstood figure

Page 28: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 19

Disk IO Load

•  Similar to LoadAvg but focused on the specific disk only

Page 29: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 20

Active Users

•  Which users are using system the most •  Can also see by CPU usage,

Connections, Rows changed etc

Page 30: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 21

Active Tables

•  Which tables get most reads or writes ? •  Rapid Changes often signal problems

Page 31: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 22

Auto Increment Usage

•  Do not run out of Auto Increment Numbers!

Page 32: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 23

Check Query Response Time

•  Great Performance Metric for Stable Workloads

•  Percona Server Feature

Page 33: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 24

Connected vs Running

•  More of Connected threads are “Running” for saturated systems

Page 34: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 25

Top Counters

•  Use topk() function in Prometheus to show top series from the group

Page 35: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 26

File IO

•  Where does File IO “Load” Comes from ?

Page 36: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 27

IO Bandwidth utilization

•  Can use Performance Schema to get more details on file IO

Page 37: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 28

Or from SHOW STATUS

•  If you do not have Performance Schema enabled

Page 38: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 29

Insight into cause of “waits”

•  Note – not everything is Performance Schema instrumented

Page 39: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 30

Time Offset

•  Can use Time offset to compare to yesterday, 1 week ago etc

Page 40: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 31

Transaction History

•  Most common “slow killer” of system performance

Page 41: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 32

Purge Progress

•  For Innodb History Length Growth Diagnostics

Page 42: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 33

Purge Delay Management

•  One way to battle Transaction History Growth

Page 43: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 34

Wish List

Prometheus Plugin is rather New

Heatmap Charts

Pie Charts

Built in Annotations

MySQL, MongoDB Plugins

Page 44: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 35

Want to try it out ?

Working on simple package for MySQL Monitoring

Checkout out preview at https://hub.docker.com/r/percona/pmm/

Page 45: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 36 www.percona.com

Percona Live Data Performance Conference

•  April 18-21 in Santa Clara, CA at the Santa Clara Convention Center

•  Register with code “WebinarPL” to receive 15% off at registration

•  MySQL, NoSQL, Data in the Cloud

www.perconalive.com

Page 46: Percona – The Database Performance Experts · Peter Zaitsev, CEO, Percona February 24, 2016 Percona Technical Webinars Using Grafana for MySQL Monitoring

www.percona.com 37 www.percona.com

Peter Zaitsev [email protected]

@PeterZaitsev bit.ly/PerconaJobs

Thank You!