Webinar slides: MySQL Query Tuning Trilogy: Indexing and EXPLAIN - deep dive

Copyright 2016 Severalnines AB

1

Your host & some logistics

I'm Jean-Jérôme from the Severalnines Team and I'm your host for today's webinar!

Feel free to ask any questions in the Questions section of this application or via the Chat box.

You can also contact me directly via the chat box or via email: [email protected] during or after the webinar.


2

About Severalnines and ClusterControl


3

What we do

Manage Scale

Monitor Deploy


4

ClusterControl Automation & Management

! Provisioning ! Deploy a cluster in minutes ! On-premises or in the cloud (AWS)

! Monitoring ! Systems view ! 1sec resolution ! DB / OS stats & performance advisors ! Configurable dashboards ! Query Analyzer ! Real-time / historical

! Management ! Multi cluster/data-center ! Automate repair/recovery ! Database upgrades ! Backups ! Configuration management ! Cloning ! One-click scaling


5

Supported Databases


6

Customers


MySQL Query Tuning - Indexing & EXPLAIN

September 27, 2016

Krzysztof Książek

Severalnines

[email protected]

7

mailto:[email protected]


8

Agenda

! Indexes

! How B-Tree actually works?

! MyISAM vs. InnoDB

! Different types of indexes

! EXPLAIN

! Overview

! EXPLAIN PARTITIONS

! EXPLAIN EXTENDED

! EXPLAIN JSON


9

Indexes - theory


10

Indexes - theory

! Let’s start with some theory

! MySQL uses multiple index types

! B-Tree and B+Tree indexes (MyISAM, InnoDB and Memory)

! Fulltext index (MyISAM and InnoDB)

! Spatial index (MyISAM and InnoDB)

! Hash index (Memory)


11

Indexes - B-Tree index

! Let’s check how B-Tree index is designed:


12


! We have a root node and four leaf pages

! Each entry in a leaf page has a pointer to a row related to this index entry

! Each leaf page has a pointer to the beginning of another leaf page

! This allows for a quick range scan of the index

! Index lookup is also pretty efficient


13


! Let’s insert a row with indexed value of 20

! It is added to the third leaf page as there’s still some free space there


14


! Let’s insert a row with indexed value of ’50’ - it won’t fit in the leaf page - page has to be split:


15


! Before and after the split


16


! Splitting a leaf node requires several writes to happen

! Splitting a root node requires even more writes as another level of leaf nodes has to be created

! As you can see, adding data to the index is more expensive than just inserting a row - management of the index adds significant overhead

! You have to keep in mind that indexes are, on the one hand, great as they speed up reads

! On the other hand, they slow down write operations

! More indexes not always better


17



18



19

Query tuning - indexing - MyISAM vs. InnoDB

! Two ‘types’ of index

! Primary Key - always unique

! Secondary index - may or may not be unique

! In MyISAM primary and secondary indexes are structured in the same way

! They contain pointers to data

! InnoDB is different

! Primary key - it’s a clustered index

! Instead of pointer to the row, it contains full row

! Speeds up PK lookups significantly

! Secondary index contain PK values

! PK values are used to retrieve data via PK lookup

! Long PK may significantly affect performance

Indexes - MyISAM vs. InnoDB


20

Indexes - diferent types of indexes


21

Indexes - B-Tree! B-Tree indexes can speed up queries which

match a full value

! Index can cover multiple columns - a composite index (col1, col2, col3)

! It can be used by queries with matching leftmost prefix

! SELECT * FROM table WHERE col1 = ‘1234’;

! Indexes can also work with leftmost part of the column:

! SELECT * FROM table WHERE col1 LIKE ‘1%’;

! B-Tree indexes can be used for range queries

! SELECT * FROM table WHERE col1 > ‘1%’ AND col1 < ‘4%’;

! Leaf nodes in a B-Tree index contain pointers to the rows (or PK values) but they also contain the data for the indexed column itself

! It can be used to create covering indexes

! KEY covering (col1, col2)

! SELECT col1, col2 FROM table1 WHERE col1 = 123


22

! This is a type of index that can be used for text lookups

! They do not work by comparing values, it’s more like keyword searching

! Available in both MyISAM and InnoDB (since 5.6)

! Can’t be used for WHERE conditions - you should be using the MATCH AGAINST operator

! You may use both FULLTEXT and B-Tree index on the same column

Indexes - Full text indexes


23

Indexes - Hash indexes! Hash indexes (user controlled) are used only in

MEMORY engine and can be used for exact lookups

! InnoDB uses HASH indexes as a part of Adaptive Hash Index - it’s not manageable by user

! For each row a hash is calculated on a column and then it’s used for lookups

! Let’s say hash value for a column is ‘123456’

! Data will be stored in index as a hash value -> pointer to the row

! Hash index most likely will be smaller than B-Tree, making it faster

! As long as there won’t be too many collisions

! Index can be used only for direct lookups

! It can’t be used for range queries

! Index on (col1, col2) can’t be used to locate data based only on ‘col1’ value


24

Indexes - gotchas

! Make sure you do not use functions on the left side of the WHERE clause:

! SELECT column1 FROM table WHERE LOWER(column2) = 'some value’;

! This won’t use index on (column2)

! SELECT column1 FROM table WHERE column2 = UPPER(‘some value’);

! This will use index on (column2)


25

Indexes - gotchas

! You should not mix data types

! SELECT * FROM table WHERE varcharcolumn = 12345;

! It won’t use index on (varcharcolumn)

! You can explicitly convert integer to a string

! SELECT * FROM table WHERE varcharcolumn = ‘12345’;

! This one will use index on (varcharcolumn)


26

EXPLAIN


27

EXPLAIN

! EXPLAIN gives you an option to check the query execution plan


28

EXPLAIN

! ‘Type’ column - type of a join

! eq_ref - rows will be accessed using an index, one row is read from the table for each combination of rows from the previous tables.

! ref - multiple rows can be accessed for a given value so we are using standard, non-unique index to retrieve them

! index_merge - rows are accessed through the index merge algorithm. Multiple indexes are used to locate matching rows.


29

EXPLAIN

! ‘Type’ column - type of a join

! range - only rows from a given range are being accessed, index is used to select them

! index - full index scan is performed. It can be either a result of the use of a covering index or an index can be used to retrieve rows in a sorted order

! ALL - full table scan is performed


30

EXPLAIN

! ‘possible_keys’ and ‘key’ tells us about indexes which could be used for the particular query and the index chosen by the optimizer as the most efficient one

! ‘key_len’ tells about the length of the index (or a prefix of an index) that was chosen to be used in the query

! ‘ref’ column tells us which columns or constants are compared to the index picked by the optimizer. In our case this is ‘const’ as we are using a constant in the WHERE clause. This can be another column, for example in the case of a join. You can also see ‘func’ when we are comparing a result of some function


31

EXPLAIN

! ‘rows’ gives us an estimate of how many rows the query will scan, this estimate is based on the index statistics from the storage engine therefore it may not be precise

! ‘Extra’, prints additional information relevant to how the query is going to be executed. You can see here information about different optimizations that will be applied to the query

! Using Index - only index will be used to retrieve rows (covering index)

! Using filesort - additional pass is required to retrieve rows in a sorted order


32

EXPLAIN

! JOINs - EXPLAIN output looks more complex

! Three tables are involved in the query

! Start with the ‘actor’ tableusing ‘idx_actor_last_name’ index

! Then ‘film_actor’ table was joined using PK

! Finally, ‘film’ table was joined using PK

! Plan looks optimal

! 1x1x1 rows will be scanned


33

EXPLAIN PARTITIONS

! One new column (‘partitions’) was added to EXPLAIN output

! It contains values ‘p10,p11,p12’

! Optimizer decided, based on WHERE clause, that only those partitions have to be accessed

! type: index - a PK scan will be used and then WHERE clause condition will be checked


34

EXPLAIN EXTENDED

! EXPLAIN EXTENDED followed by SHOW WARNINGS shows exact execution plan

! Please note <in_optimizer> in the SHOW WARNINGS output

! Subquery is not dependent therefore it can be materialized

! ( <materialize> (/* select#2 */ select 'M' from èmployees`.èmployees` è1` where 1 )


35

EXPLAIN JSON

! EXPLAIN can print the output in JSON format

! It not only changes the format but also a new content is added

! Information about used parts of the composite index

! Additional information about subquery

! Is it dependent?

! Is it materialized?


36

EXPLAIN JSON


37

Thank You!! Blog posts covering query tuning process:

! http://severalnines.com/blog/become-mysql-dba-blog-series-database-indexing

! http://severalnines.com/blog/become-mysql-dba-blog-series-using-explain-improve-sql-queries

! Register for the next part of our trilogy:

! http://severalnines.com/upcoming-webinars

! Install ClusterControl:

! http://severalnines.com/getting-started

! Contact: [email protected]

http://severalnines.com/blog/become-mysql-dba-blog-series-database-indexing

http://severalnines.com/blog/become-mysql-dba-blog-series-using-explain-improve-sql-queries

http://severalnines.com/upcoming-webinars

http://severalnines.com/getting-started

mailto:[email protected]

Webinar slides: MySQL Query Tuning Trilogy: Indexing and EXPLAIN - deep dive

Internet

Transcript of Webinar slides: MySQL Query Tuning Trilogy: Indexing and EXPLAIN - deep dive