Oracle OpenWorld 2013 - HOL9737 MySQL Replication Best Practices
-
Upload
sven-sandberg -
Category
Technology
-
view
524 -
download
1
description
Transcript of Oracle OpenWorld 2013 - HOL9737 MySQL Replication Best Practices
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 121
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 122
MySQL ReplicationBest Practices
Luís Soares Sr. Software Engineer
Sven Sandberg Sr. Software Engineer
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.
The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
Agenda
Replication Basics
Crash-Safe Slaves: Replication Metadata in System Tables.
On-line Data Verification: Replication Event Checksums
Tuning and Optimizing Row-based Replication
Improving the Slave Scalability: Multi-Threaded Slave Applier
Automated Fail-over and Slave Positioning: Global Transaction Ids
Summary
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
Replication Basics
MySQL Master Server– Changes data
– Logs changes (Events) into a file (Binary Log)
MySQL Slave Server
– Retrieves events from the master
– Replays the events on the slave's databases
MySQL Replication Components.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
Replication Basics
The Binary Log– File based log that records the changes on the master.
– Statement or Row based format (may be intermixed).
– Split into transactional groups.
BEGIN ...Ev1 Ev2 COMMIT
MySQL Replication Components: Binary Log.
Binary Log FileBinary Log File
BEGIN ...Ev1 Ev2 COMMIT
Event Layout ona Binary Log File
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
Replication BasicsMySQL Replication Components: Binary Log.
Binary Log
Binarylogfiles
Index
Under the Hood
Binary log files: mysql-bin.000001, mysql-bin.000002, …
- The actual data.
Index: mysql-bin.index
- An index over the binary log files.
Log coordinate:
- binlog file name + event offset in the file (3.23.15+)
- Global Transaction Identifiers (5.6+)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
Replication BasicsMySQL Replication Architecture.
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
ClientDumpthread
IOthread
SQLthread
Network
Masterinfo
RelayLogInfo
ReplicationMetadata
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Replication Basics
Asynchronous Replication (MySQL 3.23.15+)– Transactions are committed and externalized without interaction with
the replication layer.
– Events are sent to slave after the commit operation is acknowledged.
– Faster but vulnerable to lost updates on server crashes and inconsistency.
– Built into the server.
Semi-synchronous Replication (MySQL 5.5+)
– Master commits transaction but waits until one slave acknowledges having received and stored the event before replying to the client.
Changes Propagation.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
Replication Basics
Hands-On
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11
Replication BasicsGetting MySQL.
Head to
– http://dev.mysql.com
... and get the latest MySQL 5.6 generic tar.gz package from there
… or alternatively, you can find MySQL 5.6 linux generic package at your local directory /usr/local/mysql56 .
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12
Replication BasicsSetting Up Working Directories.
$ tar xfvz /usr/local/mysql....tar.gz directory=$HOME/hol9737
$ mkdir $HOME/hol9737/$ cd $HOME/hol9737/
Create a directoryfor storing the binaries (e.g., under your home).
Unpack the package (e.g., using tar).
$ mv $HOME/hol9737/mysql... $HOME/hol9737/mysql56
Rename the directory for simplicity
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
Replication BasicsSetting Up MySQL.
$ mysql56/scripts/mysql_install_db basedir=$HOME/hol9737/mysql56 datadir=$HOME/hol9737/master user=`whoami`
CreateTwo Data
Directories
$ mysql56/scripts/mysql_install_db basedir=$HOME/hol9737/mysql56 datadir=$HOME/hol9737/slave user=`whoami`
Createtwo Data
Directories
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
Replication BasicsSetting Up MySQL Master Server.
[mysqld]serverid=1logbin=masterbinlogerror=master.errport=12000
Create a defaults file called $HOME/hol9737/master.cnf
Set the unique server id.
Turn ON the binary log.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
Replication Basics
$ mysql56/bin/mysqld defaultsfile=$HOME/hol9737/master.cnf lcmessagesdir=$HOME/hol9737/mysql56/share datadir=$HOME/hol9737/master/
Start themaster server.
$ mysql5.6/bin/mysql u root port 12000 protocol=tcp prompt='master> '
Log in to themaster server.
Setting Up MySQL Master Server.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Replication BasicsInspecting the Master Status.
master> SHOW BINARY LOGS;... master> SHOW BINLOG EVENTS;... master> SHOW MASTER STATUS;
What binlog files are in use?
What's in the first binary log?
What binary log is in use what is its position?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17
master> SHOW BINARY LOGS;
master> SHOW BINLOG EVENTS;
master> SHOW MASTER STATUS;
Replication BasicsInspecting the Master Status.
++++++| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |++++++| masterbin.000001 | 120 | | | |++++++1 row in set (0.00 sec)
+++++++| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |+++++++| masterbin.000001 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 |+++++++1 row in set (0.00 sec)
+++| Log_name | File_size |+++| masterbin.000001 | 120 |+++1 row in set (0.00 sec)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18
Replication BasicsThe Binary Log Layout.
Log Events
Format description
Rotate
Log Events
Format description
Rotate
Log Events
Format description ● Multiple Files.
● Files begin with Format Description event.
● Each log file ends with a Rotate event.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19
Replication BasicsSetting Up Replication User (that the slave will connect with).
master> CREATE USER `rpl_user`@`localhost`;master> GRANT REPLICATION SLAVE ON *.* TO `rpl_user`@`localhost` IDENTIFIED BY 'secret';master> FLUSH PRIVILEGES;
We could useany user.
Needs replicationgrants to read any change
on the master
Better use a dedicated
user for connectingthe slave.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Replication BasicsSetting Up MySQL Slave Server.
[mysqld]serverid=2relaylog=slaverelaybinlogerror=slave.errport=12001
Set the unique server id.
Set the name for the relay log.
Create a defaults file called $HOME/hol9737/slave.cnf
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21
Replication BasicsStarting Up The MySQL Slave Server.
$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/slave.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/slave/
Start theslave server.
$ mysql5.6/bin/mysql u root port 12001 –protocol=tcp prompt='slave> '
Log in to theslave server.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22
Replication BasicsStarting the Slave Threads.
slave> CHANGE MASTER TO MASTER_HOST = 'localhost', MASTER_PORT = 12000, MASTER_USER = 'rpl_user', MASTER_PASSWORD = 'secret';
Point the slave serverto the master
server.
Use the credentialswe had granted
before.
slave> START SLAVE;Start the slave
threads.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
Replication BasicsInspecting the Slave Status.
slave> SHOW SLAVE STATUS;
Inspect the slave status.
(...) Slave_IO_State: Waiting for master to send event Master_Host: localhost Master_User: rpl_user Master_Port: 12000 Connect_Retry: 60 Master_Log_File: masterbin.000002 Read_Master_Log_Pos: 120 Relay_Log_File: slaverelaybin.000003 Relay_Log_Pos: 284 Relay_Master_Log_File: masterbin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes(...)
Both slave threads are up and running!
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24
Replication BasicsReplicating From Master to Slave.
master> USE test;master> CREATE TABLE t1 (a INT); slave> USE test; slave> SHOW TABLES;master> INSERT INTO t1 VALUES(1); slave> SELECT * FROM t1;
Use thetest db.
Create a table.
Slave has replicated the table.
Slave has replicated the data.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25
Replication BasicsWhat's in the Binary Log?
master> SHOW BINLOG EVENTS [IN 'masterbin.0000002'];
+++++++| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |+++++++| masterbin.000002 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 || masterbin.000002 | 120 | Query | 1 | 217 | use `test`; CREATE TABLE t1 (a INT) || masterbin.000002 | 217 | Query | 1 | 296 | BEGIN || masterbin.000002 | 296 | Query | 1 | 395 | use `test`; INSERT INTO t1 VALUES (1) || masterbin.000002 | 395 | Xid | 1 | 426 | COMMIT /* xid=21 */ |+++++++5 rows in set (0.00 sec)
The “CREATE TABLE” statement.
The “INSERT” statement.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
Replication BasicsReplicating From Master to Slave – binary log formats.
• Three formats:– STATEMENT – every change logged as a statement.
● Re-executed on the slave.
– ROW – every change logged as data.
● Data changes are applied on the slave.
– MIXED – every change logged either as data or statements.
● Automatically switches from statement to row on non-deterministic statements.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
Replication BasicsReplicating From Master to Slave – binary log formats.
master> set binlog_format=ROW;master> INSERT INTO t1 VALUES(2);master> SHOW BINLOG EVENTS;
+++++++| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |+++++++| masterbin.000002 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 || masterbin.000002 | 120 | Query | 1 | 217 | use `test`; CREATE TABLE t1 (a INT) || masterbin.000002 | 217 | Query | 1 | 296 | BEGIN || masterbin.000002 | 296 | Query | 1 | 395 | use `test`; INSERT INTO t1 VALUES (1) || masterbin.000002 | 395 | Xid | 1 | 426 | COMMIT /* xid=21 */ || masterbin.000002 | 426 | Query | 1 | 498 | BEGIN || masterbin.000002 | 498 | Table_map | 1 | 543 | table_id: 70 (test.t1) || masterbin.000002 | 543 | Write_rows | 1 | 583 | table_id: 70 flags: STMT_END_F || masterbin.000002 | 583 | Xid | 1 | 614 | COMMIT /* xid=27 */ |+++++++9 rows in set (0.00 sec)
Let's changethe format.
The second “INSERT” statement.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
Replication BasicsMaster replication files.
$ ls la master/
(...)rwrw 1 XXXXXX XXXXXX 143 Sep 26 11:10 masterbin.000001rwrw 1 XXXXXX XXXXXX 614 Sep 26 11:40 masterbin.000002rwrw 1 XXXXXX XXXXXX 40 Sep 26 11:10 masterbin.index(...)
The binary log files.
Index file over the existing binary log files.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29
Replication BasicsSlave replication files.
$ ls la slave/
(...) rwrw 1 XXXXXXX XXXXXXX 128 Sep 26 11:40 master.info(...) rwrw 1 XXXXXXX XXXXXXX 57 Sep 26 11:40 relaylog.info(...)rwrw 1 XXXXXXX XXXXXXX 337 Sep 26 11:10 slaverelaybin.000002rwrw 1 XXXXXXX XXXXXXX 778 Sep 26 11:40 slaverelaybin.000003rwrw 1 XXXXXXX XXXXXXX 50 Sep 26 11:10 slaverelaybin.index(...)
The relay log files.
Index file over the existing binary log files.
Persists IO thread repliaction metadata (master host, username, … and positioning on the master's binlog).
Persists SQL thread replication metadata. Basically, the positioning in the relay log.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30
Crash-Safe Positioning:
Storing Replication Metadata in Tables.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31
Storing Replication Metadata in Tables.
Store replication metadata in tables, within the context of regular transactions.
– Commit both data and replication positions together.
– Rollback both data and replication positions together.
– Robust, highly available setups.
– No metadata in files (master.info and relay-log.info).
– Slave knows a valid position to resume replication after a crash.
Often referred to as the “Crash-Safe Slave” or “Transactional Replication” feature.
Non-transactional engines will not work with this for obvious reasons.
Self-healing slaves.MySQL 5.6
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32
Storing Replication Metadata in Tables.Configuring the slave to use tables as metadata repository.
$ mysql5.6/bin/mysqladmin shutdown user=root port=12001 protocol=tcp
[mysqld](...) relayloginforepository=TABLEmasterinforepository=TABLE
$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/slave.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/slave/
Start theslave server.
Shutdownthe slaveserver.
Configure slaveto use tables,
instead of files.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33
Storing Replication Metadata in Tables.Inspecting the filesystem and the tables.
$ ls la slave/
master.infoand
relay-log.infoare gone!!
slave> SELECT * FROM mysql.slave_relay_log_info;
+++++++...| Master_id | Number_of_lines | Relay_log_name | Relay_log_pos | Master_log_name | Master_log_pos |...+++++++...| 2 | 6 | ./slaverelaybin.000005 | 284 | masterbin.000002 | 614 |...+++++++...1 row in set (0.00 sec)
relay-log.infowas automatically
migrated!
Let's look atthe relay infoslave table.
(if you see errors likeERROR 2013 (HY000): Lost connection to MySQL server during query- don't worry and just try again)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34
Storing Replication Metadata in Tables.Inspecting the filesystem and the tables.
slave> SELECT * FROM mysql.slave_master_info\G
mysql> select * from mysql.slave_master_info\G*************************** 1. row *************************** Master_id: 2 Number_of_lines: 23 Master_log_name: masterbin.000002 Master_log_pos: 614 Host: localhost User_name: rpl_user(...)
master.infowas automatically
migrated!
Let's look atthe slave table.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
Storing Replication Metadata in Tables.Inspecting the filesystem and the tables.
slave> SHOW CREATE TABLE mysql.slave_relay_log_info\G
mysql> SHOW CREATE TABLE mysql.slave_relay_log_info\G*************************** 1. row *************************** Table: slave_relay_log_infoCreate Table: CREATE TABLE `slave_relay_log_info` ( `Master_id` int(10) unsigned NOT NULL, `Number_of_lines` int(10) unsigned NOT NULL (...) `Relay_log_name` text CHARACTER SET utf8 (...) `Relay_log_pos` bigint(20) unsigned NOT NULL (...) `Master_log_name` text CHARACTER SET utf8 COLLATE utf8_bin NOT NULL (...) `Master_log_pos` bigint(20) unsigned NOT NULL (...) `Sql_delay` int(11) NOT NULL (...) `Number_of_workers` int(10) unsigned NOT NULL, PRIMARY KEY (`Master_id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Relay Log Information'1 row in set (0.00 sec)
InnoDB engine by default.
What's the table schema?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36
On-line Data Verification:
Replication Event Checksums
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37
Replication Event Checksums
MySQL 5.6 Replication adds replication events checksums. Each event is appended a 32 bit cyclic redundancy check (CRC32). Master generates CRC32 when writing events to the binary log. Multiple verifications points (on the master and/or on the slave).
– Can be turned on/off independently.
Cyclic Redundancy Checks Per Events.
BEGIN ...Ev1 Ev2 COMMITCRC
Binary Log FileBinary Log File
CRC
CRC
CRC
BEGIN ...Ev1 Ev2 COMMITCRC
CRC
CRC
CRC
One CRC per Event.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38
Replication Event Checksums
--binlog-checksum = CRC32 | NONE– Turns on/off generation of CRCs on the master.
– SET @@global.binlog_checksum
● --master-verify-checksum=0 | 1
– Dump thread and user sessions verify checksums.
– SET @@global.master_verify_checksum
● --slave-sql-verify-checksum=0 | 1
– SQL thread verifies checksums.
– SET @@global.slave_sql_verify_checksums
User Interface for Controlling CRC.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.39
Replication Event ChecksumsConfiguring the Master to Generate Checksums.
$ mysql5.6/bin/mysqladmin shutdown user=root port=12000 protocol=tcp
[mysqld](...) binlogchecksum=CRC32
$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/master.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/master/
Start themaster server.
Shutdownthe master
server.
Configure masterto generate
event checksums.
This is actually the default currently.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.40
Replication Event ChecksumsInspecting the Replication Event Checksums.
master> INSERT INTO t1 VALUES(3);
Let's insertanother row into
test.t1.
$ mysql5.6/bin/mysqlbinlog master/masterbin.000002
# at 120#120926 12:26:24 server id 1 end_log_pos 199 CRC32 0x9a914962 Query thread_id=1 exec_time=0 error_code=0(...)BEGIN/*!*/;# at 199#120926 12:26:24 server id 1 end_log_pos 298 CRC32 0x4de04ef5 Query thread_id=1 exec_time=0 error_code=0use test/*!*/;SET TIMESTAMP=1348658784/*!*/;INSERT INTO t1 VALUES (3)/*!*/;# at 298#120926 12:26:24 server id 1 end_log_pos 329 CRC32 0xe4f8efc1 Xid = 20COMMIT/*!*/;
Valid CRCs.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41
Tuning and Optimizing Row-based Replication
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42
Row-based Replication Enhancements
RBR works with partial rows in addition to full rows.– Before Image – only fields required to find the row.
– After Image – only fields that actually changed.
May be also configured to exclude BLOBS only (when not needed).
Reduces memory footprint
Reduces network bandwidth usage
Reduces binary log size.
SET binlog_row_image={MINIMAL|NOBLOB|FULL} – default is FULL
Optimized Row-based Replication Logging.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43
Row-based Replication EnhancementsOptimized Row-based Replication Logging.
Before Image After Image
Full Rows
Rows without Blobs
Minimal Rows
Blob
PrimaryKey
Changed Columns
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44
Row-based Replication EnhancementsUsing Full Rows.
master> FLUSH LOGS;master> SET binlog_format=ROW;master> CREATE TABLE t2 (int1 INT, int2 int, float float, txt text, PRIMARY KEY (a));master> INSERT INTO t2 VALUES (1, 1, 1.0, 'ola');master> UPDATE t2 SET int2=2;
Create a new binlog file.
NeedRow
Format.
Updatethe table.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45
Row-based Replication EnhancementsUsing Full Rows.
$ mysql5.6/bin/mysqlbinlog v master/masterbin.000003
# at 242#120926 12:55:17 server id 1 end_log_pos 664 CRC32 (...)
BINLOG 'Je1iUBMBAAAAMgAAAFACAAAAAEgAAAAAAAEABHRlc3QAAn (...)(...)'/*!*/;### UPDATE test.t2### WHERE### @1=1### @2=1### @3=1 ### @4='ola'### SET### @1=1### @2=2### @3=1 ### @4='ola'# at 314
The full rows are written to the binary log.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46
Row-based Replication EnhancementsUsing Rows Without Blobs.
master> FLUSH LOGS;master> SET binlog_row_image=NOBLOB;master> UPDATE t2 SET int2=3;
Create a new binlog file.
Change toNoBlob images.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47
Row-based Replication EnhancementsUsing Rows Without Blobs.
$ mysql5.6/bin/mysqlbinlog v master/masterbin.000004
# at 242#120926 13:04:09 server id 1 end_log_pos 304 (...)
BINLOG 'Oe9iUBMBAAAAMgAAAPIAAAAAAEgAAAAAAAEABHRlc3QAAnQyAAQ (...)(...)### UPDATE test.t2### WHERE### @1=1### @2=2### @3=1 ### SET### @1=1### @2=3### @3=1 # at 304
The blob field was not included in the before and after images since it was not needed for the update operation.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48
Row-based Replication EnhancementsUsing Minimal Rows.
master> FLUSH LOGS;master> SET binlog_row_image=MINIMAL;master> UPDATE t2 SET int2=4;
Create a new binlog file.
Change toMinimal images.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.49
Row-based Replication EnhancementsUsing Minimal Rows.
$ mysql5.6/bin/mysqlbinlog v master/masterbin.000005
# at 242#120926 13:08:09 server id 1 end_log_pos 288 (...)
BINLOG 'KfBiUBMBAAAAMgAAAPIAAAAAAEgAAAAAAAEABHRlc3QAAn (...)(...)### UPDATE test.t2### WHERE### @1=1### SET### @2=4# at 288
Only the PK was stored in the binary log.
Only the changed field was stored in the binary log.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.50
Row-based Replication EnhancementsOptimized Row-based Replication Logging.
The different types of images take:– FULL: 314 – 242 = 72 bytes
– NOBLOB: 304 – 242 = 62 bytes
– MINIMAL: 288 – 242 = 46 bytes
With only one row and a very simple update statement, very simple table, we have saved (FULL vs MINIMAL):
– 72 – 46 = 26 bytes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.51
Row-based Replication Enhancements
Log the original query along with the RBR events:
– Enhances auditing and debugging.
– Extra event preceding the Rows events.
– Replicated everywhere in the topology, together with the Rows events themselves.
– User can turn the behavior on and off.
Informational Log Events for Row-based Replication.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.52
Row-based Replication EnhancementsEnabling the logging of the Rows_query_log_event.
master> SET binlog_format=ROW;master> SET binlog_rows_query_log_events=ON;master> use test;master> INSERT INTO t1 VALUES (6);master> UPDATE t1 SET a=7;master> SHOW BINLOG EVENTS IN 'masterbin.000005';
Turn onrows query log events.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.53
Row-based Replication EnhancementsTesting the logging of the Rows_query_log_event.
+++++++| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |+++++++| masterbin.000009 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 || masterbin.000009 | 120 | Query | 1 | 192 | BEGIN || masterbin.000009 | 192 | Rows_query | 1 | 241 | # INSERT INTO t1 VALUES (6) || masterbin.000009 | 241 | Table_map | 1 | 286 | table_id: 70 (test.t1) || masterbin.000009 | 286 | Write_rows | 1 | 326 | table_id: 70 flags: STMT_END_F || masterbin.000009 | 326 | Xid | 1 | 357 | COMMIT /* xid=81 */ || masterbin.000009 | 357 | Query | 1 | 429 | BEGIN || masterbin.000009 | 429 | Rows_query | 1 | 470 | # UPDATE t1 SET a=7 || masterbin.000009 | 470 | Table_map | 1 | 515 | table_id: 70 (test.t1) || masterbin.000009 | 515 | Update_rows | 1 | 591 | table_id: 70 flags: STMT_END_F || masterbin.000009 | 591 | Xid | 1 | 622 | COMMIT /* xid=82 */ |+++++++11 rows in set (0.01 sec)
Queries are logged along with the changes.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54
Row-based Replication EnhancementsTesting the logging of the Rows_query_log_event.
(...)# at 192#120926 14:34:14 server id 1 end_log_pos 241 CRC32 0xddde516c Rows_query# INSERT INTO t1 VALUES (6)# at 241#120926 14:34:14 server id 1 end_log_pos 286 CRC32 0xb10b6695 Table_map: `test`.`t1` mapped to number 70(...)# at 429#120926 14:34:21 server id 1 end_log_pos 470 CRC32 0xd3a90e9b Rows_query# UPDATE t1 SET a=7# at 470#120926 14:34:21 server id 1 end_log_pos 515 CRC32 0xdd3c50a9 Table_map: `test`.`t1` mapped to number 70(...)
$ mysql5.6/bin/mysqlbinlog vv master/masterbin.000005
Queries are logged along with the changes.
Extra verbose so that mysqbinlog outputs the Rows_query_log_event.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.55
Improving the Slave Scalability: Multi-Threaded Slave Applier
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.56
Multi-Threaded Slave
Multiple execution threads to apply replication events to the slave(s). Splits processing between worker threads based on schema.
– Multi-tenant systems friendly.
Concurrent transactions commit independently.
– Intra-schema consistency is always guaranteed.
– Inter-schema eventual consistency.
Properties of the Multi-Threaded Slave.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.57
Multi-Threaded SlaveMulti-threaded Slaves.
RELAYLOG
Coordinator Coordinator ThreadThread
T1T2
DB2
DBn
DB1
IO ThreadIO Thread Worker Worker ThreadsThreads
Transaction apply path at the slave.Transaction apply path at the slave.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.58
Multi-Threaded SlaveSetting up a load that will use two workers.
slave> STOP SLAVE;slave> SET GLOBAL slave_parallel_workers=2;master> SET binlog_format=STATEMENT;master> CREATE DATABASE db1;master> CREATE DATABASE db2;master> CREATE TABLE db1.t1 (a INT);master> CREATE TABLE db2.t1 (a INT);master> INSERT INTO db1.t1 SELECT sleep(5);master> INSERT INTO db2.t1 SELECT sleep(5);
Not a requirement for MTS, but needed
for this example.
Set 2worker threads.
Use sleep to makethe slave threads apply
concurrently.
Create twodatabases.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.59
Multi-Threaded SlaveSetting up a load that will use two workers.
slave> SELECT * FROM mysql.slave_worker_info\G
mysql> select * from mysql.slave_worker_info\GEmpty set (0.00 sec)
slave> START SLAVE; wait 56 secondsslave> SELECT * FROM mysql.slave_worker_info\G
Positioning table for the slave threads.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.60
Multi-Threaded Slave
mysql> select * from mysql.slave_worker_info\G*************************** 1. row *************************** Master_id: 2 Worker_id: 0 Relay_log_name: ./slaverelaybin.000017 Relay_log_pos: 1098 Master_log_name: masterbin.000008 Master_log_pos: 1159(...) Checkpoint_seqno: 5 Checkpoint_group_size: 64 Checkpoint_group_bitmap: ( *************************** 2. row *************************** Master_id: 2 Worker_id: 1 Relay_log_name: ./slaverelaybin.000017 Relay_log_pos: 882 Master_log_name: masterbin.000008 Master_log_pos: 943(...) Checkpoint_group_size: 64 Checkpoint_group_bitmap: 2 rows in set (0.00 sec)
Even though these tables are not for monitoring, we can see two entries filled, meaning that two slave workers were engaged.
Since they commit independently, there is a bitmap saved to pinpoint which positions have been applied by each worker.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.61
Automatic Fail-over and Replication Positioning:Global Transaction Identifiers
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.62
Motivation: Seamless Fail-over.
Servers sometimes crash (hardware error, bug, meteor hit...) Promote a slave to replace crashed master Method to represent positions in replication stream is CRUCIAL!
– Traditional: FILENAME+OFFSET
● local, absolute, manual
– We introduce TRANSACTION IDENTIFIERS
● global, logical, automatic
Global Transaction Identifiers
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.63
A
(master)
B
(slave)
C
(slave2)
Global Transaction IdentifiersMotivation: Seamless Fail-over.
Enable fail-over if master crashes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.64
A
(master)
B
(slave)
C
(slave2)
Crash!
Global Transaction IdentifiersMotivation: Seamless Fail-over.
Enable fail-over if master crashes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.65
A
(master,crashed)
B
(slave)
C
(slave2)
Global Transaction IdentifiersMotivation: Seamless Fail-over.
Enable fail-over if master crashes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.66
A
(master,crashed)
B
(slave)
C
(slave2)
Global Transaction IdentifiersMotivation: Seamless Fail-over.
Enable fail-over if master crashes
fail-over
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.67
A
(master,crashed)
B
(slave)
C
(slave2)
Global Transaction IdentifiersMotivation: Seamless Fail-over.
Enable fail-over if master crashes
fail-over
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.68
Global Transaction IdentifiersGlobal Transaction Identifiers.
Generate a global transaction identifier on commit:– server_uuid:number
a61678ba488942799e5845ba840af334:1– server_uuid identifies the server; globally unique
– number is incremented by 1 for each transaction on this server
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.69
Global Transaction IdentifiersGlobal Transaction Identifiers.
Generate a global transaction identifier on commit:– server_uuid:number
a61678ba488942799e5845ba840af334:1– server_uuid identifies the server; globally unique
– number is incremented by 1 for each transaction on this server
Write GTID to binary log
GTID BEGIN ...Ev1 Ev2 COMMIT
Transaction 1
GTID BEGIN ...Ev1 Ev2 COMMIT
Transaction 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.70
Global Transaction IdentifiersGlobal Transaction Identifiers.
Generate a global transaction identifier on commit:– server_uuid:number
a61678ba488942799e5845ba840af334:1– server_uuid identifies the server; globally unique
– number is incremented by 1 for each transaction on this server
Write GTID to binary log
Preserve GTID when slave re-executes transaction
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.71
Global Transaction IdentifiersNew protocol.
New replication protocol:– Slave sends to master:
range of identifiers of executed transactions to master
– Master sends all other transactions to slave
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.72
Global Transaction IdentifiersNew protocol.
New replication protocol:– Slave sends to master:
range of identifiers of executed transactions to master
– Master sends all other transactions to slave
(slave)
id1,trx1,id2,trx2
binlog
(master)
Aid1,trx1,id2,trx2,id3,trx3
binlog
B
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.73
Global Transaction IdentifiersNew protocol.
New replication protocol:– Slave sends to master:
range of identifiers of executed transactions to master
– Master sends all other transactions to slave
(slave)
id1,trx1,id2,trx2
(master)
id1,trx1,id2,trx2,id3,trx3
binlog
A 2. id3, trx3, …
1. id1…id2 binlog
B
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.74
Global Transaction IdentifiersNew protocol used in failover.
(slave)
id1,trx1,id2,trx2,id3,trx3
B
binlog
A
(crashed)(master)
Aid1,trx1,id2,trx2,id3,trx3
binlog
(slave)
Cid1,trx1,id2,trx2
binlog
Failover using new protocol
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.75
Global Transaction IdentifiersNew protocol used in failover.
A
(crashed)(master)
Aid1,trx1,id2,trx2,id3,trx3
binlog
(slave)
Cid1,trx1,id2,trx2
binlogCrash! (slave)
id1,trx1,id2,trx2,id3,trx3
B
binlog
Failover using new protocol
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.76
Global Transaction IdentifiersNew protocol used in failover.
Failover using new protocol
A
(crashed)(crashed)
Aid1,trx1,id2,trx2,id3,trx3
binlog
Cid1,trx1,id2,trx2
binlog
id1,trx1,id2,trx2,id3,trx3
B
binlog
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.77
Global Transaction IdentifiersNew protocol used in failover.
(new master)
id1,trx1,id2,trx2,id3,trx3
B
binlog
A
(crashed)(crashed)
Aid1,trx1,id2,trx2,id3,trx3
binlog
(slave)
id1,id2
id3, trx3,...C
id1,trx1,id2,trx2
binlog
Failover using new protocol
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.78
Global Transaction IdentifiersConfigure the master and slave to use Global Transaction Identifiers.
$ mysql5.6/bin/mysqladmin shutdown user=root port=12000 protocol=tcp
Shutdownthe master
server.
$ mysql5.6/bin/mysqladmin shutdown user=root port=12001 protocol=tcp
Shutdownthe slaveserver.
master> RESET MASTER;
slave> STOP SLAVE;slave> RESET SLAVE;
Clear non-GTID statefrom binary logs
Clear non-GTID statefrom relay logs
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.79
Global Transaction IdentifiersConfigure the master and slave to use Global Transaction Identifiers.
[mysqld](...) gtidmode=onenforcegtidconstistencylogslaveupdates
configure master to use GTIDs
[mysqld](...) gtidmode=onenforcegtidconstistencylogslaveupdateslogbin=slavebin
configure slave to use GTIDs
GTIDs ON only works if binary logging is ON.
Master and slave must eitherboth have GTIDs ON,
or both have GTIDs OFF
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.80
Global Transaction IdentifiersConfigure the master and slave to use Global Transaction Identifiers.
$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/slave.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/slave/
$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/master.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/master/
Startthe master
server.
Startthe slaveserver.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.81
Global Transaction IdentifiersConfigure the master and slave to use Global Transaction Identifiers.
slave> CHANGE MASTER TO MASTER_AUTO_POSITION = 1, MASTER_HOST = 'localhost', MASTER_PORT = 12000, MASTER_USER = 'rpl_user', MASTER_PASSWORD = 'secret';slave> START SLAVE;
Start the slave threads.
Tell the slave threads to use GTIDs for positioning.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.82
Global Transaction IdentifiersTry it out.
master> CREATE TABLE t3 (a INT);master> INSERT INTO t3 VALUES (1);master> SELECT @@GLOBAL.GTID_EXECUTED;
mysql> SELECT @@GLOBAL.GTID_EXECUTED;
++| @@GLOBAL.GTID_EXECUTED |++| 8D4B6B8509AE11E2BCA8002318251294:12 |
++
Create and populatea table.
Create and populatea table.
See what GTIDswere assigned
to the transactions.
slave> SELECT @@GLOBAL.GTID_EXECUTED;mysql> SELECT @@GLOBAL.GTID_EXECUTED;
++| @@GLOBAL.GTID_EXECUTED |++| 8D4B6B8509AE11E2BCA8002318251294:12 |
++
The same GTIDshave been applied
on the slave.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.83
Global Transaction IdentifiersTry it out – GTIDs in master's binary log.
master> SHOW BINLOG EVENTS IN 'masterbin.000002';
+++++++
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+++++++
| masterbin.000002 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 |
| masterbin.000002 | 120 | Previous_gtids | 1 | 151 | |
| masterbin.000002 | 151 | Gtid | 1 | 199 | SET @@SESSION.GTID_NEXT= '[uuid]:1' |
| masterbin.000002 | 199 | Query | 1 | 296 | use `test`; CREATE TABLE t3 (a INT) |
| masterbin.000002 | 296 | Gtid | 1 | 344 | SET @@SESSION.GTID_NEXT= '[uuid]:2' |
| masterbin.000002 | 344 | Query | 1 | 423 | BEGIN |
| masterbin.000002 | 423 | Query | 1 | 522 | use `test`; INSERT INTO t3 VALUES (1) |
| masterbin.000002 | 522 | Xid | 1 | 553 | COMMIT /* xid=17 */ |
+++++++
8 rows in set (0,00 sec)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.84
Global Transaction IdentifiersTry it out – GTIDs in slave's binary log.
slave> SHOW BINLOG EVENTS IN 'slavebin.000001';
+++++++
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+++++++
| slavebin.000001 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 |
| slavebin.000001 | 120 | Previous_gtids | 1 | 151 | |
| slavebin.000001 | 151 | Gtid | 1 | 199 | SET @@SESSION.GTID_NEXT= '[uuid]:1' |
| slavebin.000001 | 199 | Query | 1 | 296 | use `test`; CREATE TABLE t3 (a INT) |
| slavebin.000001 | 296 | Gtid | 1 | 344 | SET @@SESSION.GTID_NEXT= '[uuid]:2' |
| slavebin.000001 | 344 | Query | 1 | 423 | BEGIN |
| slavebin.000001 | 423 | Query | 1 | 522 | use `test`; INSERT INTO t3 VALUES (1) |
| slavebin.000001 | 522 | Xid | 1 | 553 | COMMIT /* xid=17 */ |
+++++++
8 rows in set (0,00 sec)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.85
Global Transaction IdentifiersSet Up A Second Slave.
$ mysql5.6/scripts/mysql_install_db basedir=$HOME/hol9737/mysql5.6 datadir=$HOME/hol9737/slave2 user=`whoami`
Createnew DataDirectory
[mysqld]serverid=3relaylog=slave2relaybinlogerror=slave2.errport=12002gtidmode=onenforcegtidconsistencylogslaveupdateslogbin=slavebin
Create a defaults file called $HOME/hol9737/slave2.cnf
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.86
Global Transaction IdentifiersSet Up a Second Slave.
$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/slave2.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/slave2/
$ mysql5.6/bin/mysql u root port 12002 –protocol=tcp prompt='slave2> '
Start theslave2 server.
Log in to theslave2 server.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.87
Global Transaction IdentifiersSet Up a Second Slave.
slave2> CHANGE MASTER TO MASTER_AUTO_POSITION = 1, MASTER_HOST = 'localhost', MASTER_PORT = 12000, MASTER_USER = 'rpl_user', MASTER_PASSWORD = 'secret';
Point slave2to the master
server.
Use the credentialswe had granted
before.
slave2> START SLAVE;
Start the slavethreads.
Setup slave2to use GTIDs for
positioning
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.88
Global Transaction IdentifiersTry out the second slave.
slave2> SHOW BINLOG EVENTS IN 'slavebin.000001';
+++++++
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+++++++
| slave2bin.000001 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 |
| slave2bin.000001 | 120 | Previous_gtids | 1 | 151 | |
| slave2bin.000001 | 151 | Gtid | 1 | 199 | SET @@SESSION.GTID_NEXT= '[uuid]:1' |
| slave2bin.000001 | 199 | Query | 1 | 296 | use `test`; CREATE TABLE t3 (a INT) |
| slave2bin.000001 | 296 | Gtid | 1 | 344 | SET @@SESSION.GTID_NEXT= '[uuid]:2' |
| slave2bin.000001 | 344 | Query | 1 | 423 | BEGIN |
| slave2bin.000001 | 423 | Query | 1 | 522 | use `test`; INSERT INTO t3 VALUES (1) |
| slave2bin.000001 | 522 | Xid | 1 | 553 | COMMIT /* xid=17 */ |
+++++++
8 rows in set (0,00 sec) The new slavehas replicated from
the master
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.89
Global Transaction IdentifiersRecap: we want to simulate master crash and fail-over when one slave is behind.
slave
id1,trx1,id2,trx2,id3,trx3
binlog
A
(crashed)master
id1,trx1,id2,trx2,id3,trx3
binlog
slave2
id1,trx1 id2,trx2
binlog
Crash!
The situation we want to handle
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.90
Global Transaction IdentifiersRecap: we want to simulate master crash and fail-over when one slave is behind.
slave
id1,trx1,id2,trx2,id3,trx3
binlog
A
(crashed)master
id1,trx1,id2,trx2,id3,trx3
binlog
slave2
id1,trx1 id2,trx2
binlog
The situation we want to handle
failover
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.91
Global Transaction IdentifiersRecap: we want to simulate master crash and fail-over when one slave is behind.
slave
id1,trx1,id2,trx2,id3,trx3
binlog
A
(crashed)master
id1,trx1,id2,trx2,id3,trx3
binlog
slave2
id1,trx1 id2,trx2
binlog
The situation we want to handle
fail-over
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.92
Global Transaction IdentifiersMake slave2 fall behind.
slave2> STOP SLAVE;
Stop the slave threads to ensurethat slave2 does not receive updates
master> INSERT INTO t3 VALUES (99);master> SELECT @@GLOBAL.GTID_EXECUTED; slave> SELECT @@GLOBAL.GTID_EXECUTED;slave2> SELECT @@GLOBAL.GTID_EXECUTED;
++| @@GLOBAL.GTID_EXECUTED |++| [uuid]:13 |
++
++| @@GLOBAL.GTID_EXECUTED |++| [uuid]:13 |
++
++| @@GLOBAL.GTID_EXECUTED |++| [uuid]:12 |
++
Execute transaction on masterto make slave2 fall behind.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.93
Global Transaction IdentifiersKill the master.
$ pkill 9 f mysqld.*master
Kill themaster server.
master> SHOW TABLES;
The masteris dead
ERROR 2013 (HY000): Lost connection to MySQL server during query
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.94
Global Transaction IdentifiersFailover: make 'slave' a master of 'slave2'.
slave2> CHANGE MASTER TO MASTER_HOST = 'localhost', MASTER_PORT = 12001, MASTER_USER = 'rpl_user', MASTER_PASSWORD = 'secret';
Fail-over, i.e.,switch to use 'slave'
as the master.
slave2> STOP SLAVE;Stop slave threads.
slave2> START SLAVE; Start slave threads.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.95
Global Transaction IdentifiersFail-over worked by just pointing the slave to another server!
slave2> SELECT @@GLOBAL.GTID_EXECUTED;mysql> SELECT @@GLOBAL.GTID_EXECUTED;
++| @@GLOBAL.GTID_EXECUTED |++| 8D4B6B8509AE11E2BCA8002318251294:13 |
++'slave' is up to date now.
The last transactionwas copied from 'slave'slave2> SELECT * FROM test.t3;
slave2> SELECT * FROM test.t3;
++
| a |
++
| 1 |
| 99 |
++
'slave2' is up to date now.The last transaction
was copied from 'slave'
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.96
Summary
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.97
Summary
This HOL showcased some MySQL Replication best practices Several features from MySQL 5.6 that have direct user visibility were highlighted:
– Multi-threaded slave
– Global Transaction Identifiers
– Optimized Row-based Replication
– Transactional Replication Positions
– Replication Event Checksums
In the end the attendee was able to get a glimpse of some of the new MySQL 5.6 features.
– But there is much more to explore.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1298