OOW13: Accelerate your Exadata deployment with the DBA skills you already have
The 5 Minute DBA-DBA Skills for Non-DBA
-
Upload
percona2013 -
Category
Software
-
view
345 -
download
6
description
Transcript of The 5 Minute DBA-DBA Skills for Non-DBA
THE 5 MINUTE DBA DBA Skills for Non-DBA’s
Matt Yonkovit ,VP (Consulting) – PerconaPercona University, Washington DC
September 12, 2013
Non-DBA’s in charge of DB’s
Company Confidential December 2010
-3-
• System Administrators
• Web Administrators
• Developers
WHAT CAN YOU DO IN 5 MINUTES?The top tips to ensure your database is running smoothly
Company Confidential December 2010
-5-
BIG DISCLAIMER!
Company Confidential December 2010
-6-
• Its important to distinguish this up front:– I am giving you the easy answers, which are not always
the correct answers..
– If you only have 5 minutes to spend on one of these
database topics, you lose a lot of flexibility
– I could spend lots of time talking about everything here,
instead I am focusing on maximizing your benefit.
• This presentation assumes you are running on Linux/Unix
• Probably the most common problem to get called on is some sort of slowness
• Keys to success– Identify what is slow
– When its slow
– Can you repeat it
• Dig in
The Slow Down Of Doom!
Company Confidential December 2010
-10-
No sense in targeting area’s that do not matter
Target your pain first
Company Confidential December 2010
-11-
• Google is your friend to research problems• But don’t tune via Google• Trust but verify
Know where to look
Company Confidential December 2010
-12-
• Know the tools out in the community• Have them ready to jump in and fight fires at a
moments notice• Tools like:
– Percona Toolkit
– Cacti
– Home grown scripts
– etc
Know thy tools
Company Confidential December 2010
-13-
• Define what is slow? If its a single page, the process is drastically different then if its general slowness.
• Assume General Slowness– #1 What has changed? A new deployment? More users?
Etc? Sometimes a roll-back of a buggy code release is
the best fix.
• DO NOT ASSUME ITS THE DATABASE!• Start with the obvious and the quick and move to
more complex
Tuning walk through
Company Confidential December 2010
-14-
• 5.1 -vs- 5.0 provides improvements
• 5.5 -vs- 5.1 provides improvements
• 5.6 –vs- 5.5 provides improvements
HUGE Differences in Versions!
Company Confidential December 2010
-15-
• Often stock OS repos are out of date!• Watch out for old versions installed as
dependencies of off the shelf software• Just because a hosting provider installed it does not
mean its the latest or best
Where did you get it?
Company Confidential December 2010
-16-
• Innodb• Innodb Plugin• XtraDB• TokuDB• MyISAM• Aria• Many Many more....
Storage Engines
Company Confidential December 2010
-17-
• Old Kernel?• Multi-core servers, scalability getting better (~32 cores + )• Fit your hot data into memory!
– Typically more memory is better
• IO is a Major bottleneck– RAID For your databases is a must
– Battery Backed Controller
– RAID 10 typically better then RAID 5
– SSD's for optimal performance
• Network should not be forgotten• 32bit binary or OS on 64 bit machine
Hardware & OS
Company Confidential December 2010
-18-
• Next I start by looking at the server (Sar, vmstat, top, iostat -x)... I typically start low and work my way into the database– Too much io, maybe not enough memory, missing
indexes, or bad config– Cpu maxed out could be lots of users, lots of concurrent
queries, or something else.– Is the problem even MySQL? Often times we see issues
with non-database software– top process is something else ( i.e. apache, php, cron
job, etc )– Network is often overlooked
Review OS Metrics
Company Confidential December 2010
-19-
• Sar is your friend, every unix/linux server should setup sar to collect stats in the background– Sar Can give you historical reports on : CPU, Disk,
Memory, Network, and a lot more.
– Scheduled to collect data via cron
• Iostat is valuable in gauging disk performance– avwait+svctim = response time (iostat -x)
Systat
Company Confidential December 2010
-21-
VMSTAT
Company Confidential December 2010
-26-
• Can give you a realtime view of changes on the system to CPU/Memory
Percona Toolkit
Company Confidential December 2010
-28-
• Collection of scripts used for monitoring system and DB performance
• pt-stalk• pt-diskstats• pt-pmp• Pt-Ioprofiler• pt-mext
Cacti
Company Confidential December 2010
-29-
• Great Visual Presentation of Historical Stats• MySQL, Apache, Sphinx, Memcached, (Entire
Stack)• Percona Maintains the MySQL Templates• Can add a lot of value...
Example:
Mysterious Slowdown
Percona Monitoring Plugins- Cacti!
Company Confidential December 2010
-33-
Grab them here: http://www.percona.com/software/percona-monitoring-plugins/
Configuration Options – Top 10
Company Confidential December 2010
-36-
General Configuration Options• query_cache_size (Disable in many versions)• log_slow_queries• long_query_time• max_connections• expire_logs_days
MyISAM Specific• key_buffer_size
Configuration Options
Company Confidential December 2010
-37-
Innodb Specific Options• innodb_buffer_pool_size• innodb_flush_log_at_trx_commit• innodb_log_file_size
Plugin/Xtradb/Mysql 5.5 Innodb Options• innodb_io_capacity• innodb_read_ahead• innodb_write_io_threads• innodb_read_io_threads
Online Tools to help you set your my.cnf!
Company Confidential December 2010
-38-
• https://tools.percona.com/wizard
Memory and Swap
Company Confidential December 2010
-39-
• Disk IO is bad, Avoiding swapping things that should be in memory to disk ( do not disable swap however )
• SSD’s/Fast Disks• Hot Data in Memory!• read_buffer, read_rnd_buffer, sort_buffer, join_buffer
are all allocated per thread so be careful!• You want temp tables to be built in memory not on disk
Check Current Activity
• Mysql
– “show full processlist”
– “show global status”
– “Show innodb status”
• Mysqladmin
• Slow Query Log
• pt-query-digest
• Mysqldumpslow
• Innotop
• Mytop
• pt-mext
Company Confidential December 2010
-40-
Show Processlist
Company Confidential December 2010
-41-
mysql> show processlist\G
*************************** 1. row ***************************
Id: 52
User: bench
Host: domU-12-31-39-0C-41-81.compute-1.internal:42315
db: simple_benchmark
Command: Sleep
Time: 0
State:
Info: NULL
*************************** 14. row ***************************
Id: 65
User: bench
Host: domU-12-31-39-0C-41-81.compute-1.internal:42328
db: simple_benchmark
Command: Query
Time: 0
State: init
Info: insert into simple_benchmark.simple_comments (lookup_id, comment, mydate) values ( '8519163', 'Comme
*************************** 15. row ***************************
Id: 66
User: bench
Host: domU-12-31-39-0C-41-81.compute-1.internal:42329
db: simple_benchmark
Command: Sleep
Time: 0
State:
Info: NULL
Mysqladmin
Company Confidential December 2010
-42-
• Mysqladmin extended -r -i1 will give you mysql stats at 1 second intervals, change the -i to increase decrease...
• A few stats of note:
pt-mext
Company Confidential December 2010
-43-
• Mext can take the output from mysqladmin and put the status variables side by side, this makes it very convenient to look for differences
Slow Query Log
Company Confidential December 2010
-44-
• The Slow Query log is your friend! As mentioned earlier
queries that take over a certain amount of time are
logged here. (Percona Patches for complete picture )
• Summarize the slow query log with percona toolkit, pt-
query-digest or mysqldumpslow if the percona toolkit is
not available
• Look for not only the longest running query, but also the
one with lots of executions and cumulated time
Memcached/Redis/Other Caching
Company Confidential December 2010
-47-
• Review: More memory the better
• Caching frequently used data is an excellent way to
boost performance
• This typically requires changes to your app
• Performance improvement can be dramatic
Query Tuning
Company Confidential December 2010
-48-
• Review the slow query log, parse the log with pt-
query-digest.
– If I see something here, I will analyze it
• If your still at a loss, try using tcpdump to dump
SQL statements into pt-query-digest and look for
repeated queries. ( Easier then the general query
log if you need sub second slow queries).
Query Tuning
Company Confidential December 2010
-49-
• Use Explain Plan
• Add Indexes where needed, do not over index
– pt-duplicate-key-checker
• Lots of small queries can sometimes be more
impactful than 1 big one.