Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf ·...
Transcript of Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf ·...
![Page 1: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/1.jpg)
Dimitri Vanoverbeke
Practical MySQL Performance Optimization
![Page 2: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/2.jpg)
www.percona.com 2
Contents of the presentation
How do we approach performance optimisations
Discuss practical optimisation tips.
What tools exist?
![Page 3: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/3.jpg)
www.percona.com 3
MySQL performance
MySQL Performance
does not Matter
![Page 4: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/4.jpg)
www.percona.com 4
What matters?
My Website/App is
important
![Page 5: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/5.jpg)
www.percona.com 5
Things to Note
Performance issues can be unrelated to MySQL
Maybe the solution is not MySQL
![Page 6: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/6.jpg)
www.percona.com 6
Many other tools
Cassandra MongoDB Hadoop ElasticSearch Redis
![Page 7: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/7.jpg)
www.percona.com 7
Putting it in Practice
Keep an unbiased view
Focus on the Stack
![Page 8: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/8.jpg)
www.percona.com 8
Performance “sandbags”
Security
Manageability
Compatibility
Compliance
Ease of use by developers
![Page 9: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/9.jpg)
www.percona.com 9
“Good Enough”
KNOW WHEN TO
STOP
![Page 10: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/10.jpg)
www.percona.com 10
So what is important in MySQL?
Queries!!!!
• Inserts • Selects • Updates • Deletes
![Page 11: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/11.jpg)
www.percona.com 11
What are we aiming for?
Quickly, Efficiently, Reliably…
![Page 12: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/12.jpg)
www.percona.com 12
Quickly
Good Response Time
For all Users and Use Cases
![Page 13: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/13.jpg)
www.percona.com 13
Efficiency
Resources
CPU, Memory, IO, Disk Space
Dollars
Human Effort
![Page 14: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/14.jpg)
www.percona.com 14
Reliably
High Availability!
High Availability and Performance Come together
![Page 15: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/15.jpg)
www.percona.com 15
Consider things for queries
• But avoid focus on extreme outliers
Look beyond averages
• Daily and Weekly cycle • Growth of load and Data Size
Make sure to look at trends
over time
• Data size changes? Think about the Future
![Page 16: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/16.jpg)
www.percona.com 16
Schema
Look at the Schema and queries together
• Minor Schema changes • Data Architecture
![Page 17: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/17.jpg)
www.percona.com 17
Schema Tips
Learn indexing • http://bit.ly/1rAtamE
Design Schema for Data access • Rethink your schema
Technical voodoo • Partitioning and Sharding • Normalization and denormalization
![Page 18: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/18.jpg)
www.percona.com 18
Other Performance influences
Infrastructure • Storage systems, virtualization, etc.
Operating system and configuration • Disk scheduler, FS, etc…
MySQL Version • Are you using Percona Server, MariaDB, MySQL?
MySQL configuration • The default configuration of Hell…
![Page 19: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/19.jpg)
www.percona.com 19
Optimization Process
Low Hanging Fruit
• MySQL Configuration
• OS settings • Indexes • Caching
Medium Level
• InfraStructure • MySQL
Version
Hard Changes
• Major schema changes
• Application architecture changes
![Page 20: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/20.jpg)
www.percona.com 20
QUICK TIPS
![Page 21: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/21.jpg)
www.percona.com 21
Hardware
CPU
Memory
Disk
Network
![Page 22: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/22.jpg)
www.percona.com 22
OS choices are simple
Linux
Server Grade
Recent
Well supporting MySQL
![Page 23: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/23.jpg)
www.percona.com 23
OS tuning
Defaults are good for Common workload
Filesystem: EXT4 or XFS
More details: http://bit.ly/MySQLonLinux
![Page 24: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/24.jpg)
www.percona.com 24
MySQL Version
New versions typically improve performance
Scalability, improved optimizer
Expect some regression
Try maybe a different flavor? Percona Server 5.6
![Page 25: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/25.jpg)
www.percona.com 25
MySQL config optimisations
Do NOT run with defaults
Do not ever obsess with tuning
More details: http://bit.ly/1vth5Cu
![Page 26: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/26.jpg)
www.percona.com 26
MySQL tuning in 1 slide
• max_connections • log_bin • innodb_buffer_pool_size • innodb_log_file_size • innodb_flush_log_at_trx_commit • innodb_flush_method=O_DIRECT
![Page 27: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/27.jpg)
www.percona.com 27
TOOLS
![Page 28: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/28.jpg)
www.percona.com 28
Tools to Consider
• pt-query-digest (Percona Toolkit) • MySQL Enterprise Monitor • Monyog • VividCortex • Percona Cloud Tools (Free Beta)
• http://cloud.percona.com
![Page 29: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/29.jpg)
www.percona.com 29
pt-query-digest
Command Line tool from Percona Toolkit
Process Logs Manualy
Powerful Filtering
Storing History
![Page 30: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/30.jpg)
www.percona.com 30
pt-query-digest example
![Page 31: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/31.jpg)
www.percona.com 31
Top Queries
![Page 32: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/32.jpg)
www.percona.com 32
Query Details
![Page 33: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/33.jpg)
www.percona.com 33
More Query Details
![Page 34: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/34.jpg)
www.percona.com 34
Percona Cloud Tools
![Page 35: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/35.jpg)
www.percona.com 35
Query Profile
![Page 36: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/36.jpg)
www.percona.com 36
Query Details
![Page 37: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/37.jpg)
www.percona.com 37
Graphs!
![Page 38: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/38.jpg)
www.percona.com 38
Different Stats and Metrics
![Page 39: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/39.jpg)
www.percona.com 39
Process Support
Discover New Queries in the Application
Mark them as “Needing Attention” or “Reviewed”
Tag queries with information about application or user facing function
Discuss it with the team and record changes using Comments
![Page 40: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/40.jpg)
www.percona.com 40
Choose Queries
![Page 41: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/41.jpg)
www.percona.com 41
Review and Comment
![Page 42: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/42.jpg)
www.percona.com 42
Where to Analyze ?
• Development to catch bad queries early
• Production to catch good queries starting to go bad
Both !
![Page 43: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/43.jpg)
www.percona.com 43
Query Performance can show modifying parameters
Workload Changes
Data Size
Changing Query Plans
Upgrading MySQL Version
Changing MySQL Settings
Hardware Changes
![Page 44: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/44.jpg)
www.percona.com 44
But Why Slow ?
• MySQL Level • OS Level • Hardware/Environment Level
Need to know more than query
level information
![Page 45: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/45.jpg)
www.percona.com 45
“Metrics”
System and MySQL Information
Hundreds of metrics captured
1 second resolution capture
Real-Time Graphs Update
![Page 46: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/46.jpg)
www.percona.com 46
Design
Graphs != Data
Can plot metrics you have available when you need it
Graphite + Ease of Use
![Page 47: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/47.jpg)
www.percona.com 47
MySQL Level
![Page 48: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/48.jpg)
www.percona.com 48
Beyond Average
![Page 49: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/49.jpg)
www.percona.com 49
System Information
![Page 50: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/50.jpg)
www.percona.com 50
Zoom In as Needed
![Page 51: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/51.jpg)
www.percona.com 51
More To Come
Active Development in Progress
Try it out, its free http://cloud.percona.com
Let us know what you think!
![Page 52: Practical MySQL Performance Optimizationfiles.meetup.com/8981312/meetuppracticalperfopt.pdf · Compatibility Compliance Ease of use by developers . 9 “Good Enough” KNOW ... •](https://reader030.fdocuments.us/reader030/viewer/2022041017/5ec9fdd2a927057f6c5245c0/html5/thumbnails/52.jpg)
www.percona.com 52
FINAL WORDS