MySQL 5.6 Replication - Universidad Autónoma de Chiapas · MySQL 5.6 Replication: ... material,...

MySQL 5.6 Replication: Supporting High Scale Web & Cloud Services

Mat Keep

MySQL Product Management

[email protected]

Andrew Morgan

MySQL Product Management

[email protected]

<Insert Picture Here>

Session Agenda

• Development Priorities

• MySQL 5.6 Replication Enhancements

• Resources to Get Started

Copyright 2012 Oracle Corporation 2

The presentation is intended to outline our general

product direction. It is intended for information

purposes only, and may not be incorporated into any

contract. It is not a commitment to deliver any

material, code, or functionality, and should not be

relied upon in making purchasing decisions.

The development, release, and timing of any

features or functionality described for Oracle’s

products remains at the sole discretion of Oracle.


Cloud

Web & Enterprise OEM & ISVs

Industry Leaders Rely on MySQL Replication is Key to Scale & HA


http://images.google.com/imgres?imgurl=http://www.infinitytechnology.com/uploadedImages/Home/TechFocus_News/AlcatelLucent_Hor_2col_lrg.gif&imgrefurl=http://www.infinitytechnology.com/phonesystems.aspx&h=251&w=720&sz=11&hl=en&start=4&sig2=urYvX47BOQRqvADY_UMA0w&um=1&tbnid=pA91nzVKrUFImM:&tbnh=49&tbnw=140&ei=94SgSOaRCaqkpATZu-jgBw&prev=/images?q=alcatel+lucent&um=1&hl=en&suggon=0&rls=com.microsoft:en-us&sa=N

• MySQL 5.6 builds on MySQL 5.5 by improving:

• Performance and Scalability

• Optimizer for better query execution times, diagnostics

• Performance Schema for better instrumentation

• InnoDB for better transactional throughput

• NoSQL API for better flexibility

• Replication largest set of enhancements ever released!

• Available now under GPL

• Development Milestone Release model enabling early

evaluation

MySQL Database 5.6: A Better MySQL.

HIGH PERFORMANCE AUTOMATED FAILOVER & RECOVERY

DATA INTEGRITY DEV/OPS AGILITY

MySQL Replication: Development Priorities


Translating to User Needs

• Improves read consistency from slaves

• Reduces risk of data loss if master fails

PERFORMANCE

• Minimize service interruption

• Self-Healing

FAILOVER & RECOVERY

• Correct, consistent & accessible

DATA INTEGRITY

• Reducing TCO

• Quickly responding to business demand

DEV/OPS AGILITY


MySQL 5.6 Replication

• Multi-Threaded Slaves

• Optimized Row-Based Replication

PERFORMANCE

• Global Transaction Identifiers

• Replication Failover & Admin Utilities

• Crash Safe Slaves & Binlog

FAILOVER & RECOVERY

• Replication Event Checksums

DATA INTEGRITY

• Replication Utilities

• Time-Delayed Replication

• Informational Log Events

• Remote Binlog Backup

• Server UUIDs

DEV/OPS AGILITY


PERFORMANCE

5x Higher Performance with MySQL 5.6

0 5 10

QPS 58.11 144.4 282.53

0

50

100

150

200

250

300

Qu

eri

es p

er

Seco

nd

Worker Threads

Multi-Threaded Slave Performance

Multi-Threaded Slaves

• Increases slave throughput,

reducing lag

• All transactions received into

slave’s relay log

• Implements multiple SQL

threads, based on database

• Applies events to different

databases in parallel

• Great for systems which

isolate application data using

databases – e.g. multi-tenant


SQ

L-A

I/O

Relay

Log

T1 T2 T3 T4

SQ

L-B

SQ

L-C

DB-A DB-B DB-C

T1

T4

T2 T3

Coordinator


• Throughput of slave increased by allowing multiple

slave threads:

• 0 – functionality disabled

• Up to 1024

•Exec_Master_Log_Posn in SHOW SLAVE STATUS

now shows the low-water-mark

• Configure using:

slave-parallel-workers=4

• Only ensures sequencing within a database

• Suitable if ordering of updates between databases is not

required



Master INSERT INTO dba.tab1...

INSERT INTO dba.tab2...

INSERT INTO dbb.tab1...

INSERT INTO dbc.tab2...


INSERT INTO dbb.tab3...


INSERT INTO dbc.tab2...


Slave thread1

dba.tab1

dba.tab2

dba.tab1

dba.tab3

dba.tab4

dbb.tab1

dbc.tab2

dbb.tab3

dbc.tab2

Slave thread2


• Transactions applied in parallel on

the slave

• Sequencing of transactions within a

given database will be the same

Optimized Row Based Replication

• Increases replication throughput for master and slave

• Reduces Binlog size, memory requirements & network

bandwidth

• Only replicates those elements of the Row image that

have changed


Primary Key Changed Columns

Optimized Row Based Replication

• Default is to include full before & after image for every

changed row

• New option: binlog-row-image=minimal

• Reduces space:

• Inserts: only after image

• Deletes: only before image

• Updates: after image (modified columns) + primary key or

indexed columns of before image


AUTOMATED FAILOVER & RECOVERY

Global Transaction Identifiers

• Foundation for reliable, automatic failover & recovery • Unique identifier for each replication event written to the Binlog

• Simple to track & compare replication across the cluster

• Automatically identify the most up-to-date slave for failover

• Deploy complex replication topologies

• Eliminates Dev/Ops overhead

Master

GTID=123456

GTID=123456

GTID=123456 GTID=123456


Before Global Transaction IDs


Master

{my-bin.000132,

782}

{my-bin.000101,

873}

{my-bin.000099,

123}

{my-bin.00088,

1027}

Before Global Transaction IDs Failover & Recovery


Master

{my-bin.000132,

782}

{my-bin.000101,

873}

{my-bin.000099,

123}

{my-bin.00088,

1027}

CHANGE MASTER TO??

Must convert {file,pos} from

old master to one

from the new master

With Global Transaction IDs


Master

GTID=123456

GTID=123456

GTID=123456 GTID=123456

With Global Transaction IDs Failover & Recovery


Master

GTID=123456

GTID=123456

GTID=123456 GTID=123456

CHANGE MASTER TO??

Slave(s) can start

from same GTID

Global Transaction ID Utilities

• Self-Healing, Low Administration replication cluster

• Automated failover & recovery

• mysqlfailover Utility

• Switchover & administration

• mysqlrpladmin Utility

• Delivers HA within the core MySQL distribution

• Eliminates the need to integrate 3rd party HA frameworks

• Allows extensibility to support variety of HA mechanisms

Replication Failover Utility

Failover

Utility

Monitoring

• Automatic failover & slave promotion

• Default is to promote most up-to-date

slave, based on GTID

• Slave promotion policies are fully

configurable; select slave(s) that are

candidates for promotion

• Implemented as a command-line utility

• Option to bind in your own pre/post

failover scripts

• For example, initiate VIP failover

• Never lose replicated events

• Nominated slave automatically retrieves any

missing updates from other slaves


Fail-

Over

Failed

Master

Slaves

Auto-Failover &

Slave Promotion

Promoted

Master

Use of mysqlfailover $ mysqlfailover --master=root@rod:3306

--discover-slaves-login=root

MySQL Replication Monitor and Failover Utility

Failover Mode = auto Next Interval = Mon Mar 19 15:56:03 2012

Master Information

------------------

Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB

mysql-bin.000001 571

Replication Health Status

+------------+-------+---------+--------+------------+---------+

| host | port | role | state | gtid_mode | health |

+------------+-------+---------+--------+------------+---------+

| rod | 3306 | MASTER | UP | ON | OK |

| jane | 3306 | SLAVE | UP | ON | OK |


| freddy | 3306 | SLAVE | UP | ON | OK |

+------------+-------+---------+--------+------------+---------+

Q-quit R-refresh H-health G-GTID Lists U-UUIDs L-log entries


Fail-

Over

mysqlfailover – master failed

Failover starting...

# Candidate slave jane:3306 will become the new master.

# Preparing candidate for failover.

# Creating replication user if it does not exist.

# Stopping slaves.

# Performing STOP on all slaves.

# Switching slaves to new master.

# Starting slaves.

# Performing START on all slaves.

# Checking slaves for errors.

# Failover complete.

# Discovering slaves for master at jane:3306

Failover console will restart in 5 seconds.


Fail-

Over

mysqlfailover – monitoring

resumes MySQL Replication Monitor and Failover Utility

Failover Mode = auto Next Interval = Mon Mar 19 16:05:12 2012

Master Information

------------------

Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB

mysql-bin.000001 1117

UUIDs

+------------+-------+---------+--------+------------+---------+

| host | port | role | state | gtid_mode | health |

+------------+-------+---------+--------+------------+---------+

| jane | 3306 | MASTER | UP | ON | OK |


| freddy | 3306 | SLAVE | UP | ON | OK |

+------------+-------+---------+--------+------------+---------+


Fail-

Over

Replication Administration Utility

• Perform switchover to eliminate

downtime during planned

maintenance

• Start and stop slaves

• Slave discovery & monitoring

• Slave status, thread status

• Replication processing, including any lag

• Configure slave promotion policies

• Non GTID-functionality can be used

with pre-MySQL 5.6 releases

Master

Slaves

Administration

Utility

Status &

Switchover


Admin

Planned switchover to new master with mysqlrpladmin

$ mysqlrpladmin --master=root@rod:3306

--slaves=root@jane:3306,root@jane:3307,root@freddy:3306

--new-master=root@jane:3306 --demote-master switchover

# Performing switchover from master at rod:3306 to slave at jane:3306.

# Checking candidate slave prerequisites.

# Waiting for slaves to catch up to old master.

# Stopping slaves.

# Performing STOP on all slaves.

# Demoting old master to be a slave to the new master.

# Switching slaves to new master.

# Starting all slaves.

# Performing START on all slaves.

# Checking slaves for errors.

# Switchover complete.

# ...done.


Switch-

Over

Crash-Safe Slaves & Binlog

Before:

– Transaction Data: in tables

– Replication Info: in files


MySQL 5.6

– Transaction Data: in tables

– Replication Info: in tables

Data

Position Info

CRASH!

Time

Data

Position Info

Time

• Automatic recovery of a slave and

Binlog after a failure

• Binlog and table data are transactionally

consistent

• Resumes replication without Dev/Op

intervention

• Automatically rolling back replication to

last committed event

• Eliminates risk of data loss or

corruption

Atomic

Atomic

Crash-Safe Slaves

• Writes and reads back only complete events or

transactions from the binary log

•master.info & relay-log.info files can be

replaced with tables

• ACID transaction prevent them getting out of sync with InnoDB

tables

--master-info-repository=TABLE

--relay-log-info-repository=TABLE

mysql.slave_master_info

mysql.slave_relay_log_info


DATA INTEGRITY

Replication Event Checksums

• Ensures replicated data is

correct, consistent and

accessible

• Detects corrupt replication events

before they’re applied

• Returns an error

• Protects entire replication path

• Memory

• Disk

• Network

• Bugs

Master

#

Slave

#


Replication Event Checksums

• Implemented in the binary and relay logs

• New mysqld options:

• binlog-checksum=CRC32

Checksums written to the binary log

• master-verify-checksum=1

Master validates checksum read from the binary log

• slave-sql-verify-checksum=1

Slave validates checksum when reading from the relay log


DEV/OPS AGILITY

MySQL Utilities

• Automate common Dev/Ops tasks

• Replication: provisioning, testing, monitoring and failover

• Database comparisons: consistency checking

• Database administration: users, connections, tables, etc

• New utilities in development, ie log analysis

• Implemented as Python scripts, plug-in for MySQL

Workbench

• Also available from LaunchPad

• Extensible to include custom scripting

• Resources: Documentation & Community Forum

• On-Demand Webinar: http://t.co/7bFAV083

• http://dev.mysql.com/doc/workbench/en/mysql-utils-man.html

• http://forums.mysql.com/list.php?155


http://dev.mysql.com/doc/workbench/en/mysql-utils-man.html










http://forums.mysql.com/list.php?155



Utility Workflow for Replication

• Check: Verifies pre-requisites for Replication

• Repl: Initiates Replication to the new slave

• Show: Display Replication topology

• Fail-Over & Admin: Detects and failovers (or switches)

from master to slave. Status monitoring

Check Repl Show Fail-Over

& Admin

Time Delayed Replication


• Configure time period before

replication events applied to

slave

• Per-slave, via execution of SQL

Thread

• Second-level granularity, up to 68

years!

• Protects against operational error

• Dropping a table, etc.

• Allows database to be inspected

without loading a back-up

Master :3306

Binary Logs

Relay Log

Relay Log

Slave 1 :3307

Slave2 :3308

10 Minute Delay

Time Delayed Replication - configuring

slave2> CHANGE MASTER TO

-> MASTER_HOST = 'localhost‘,

-> MASTER_PORT = 3306,

-> MASTER_USER = 'repl_user‘,

-> MASTER_PASSWORD = 'pw‘,

-> MASTER_DELAY = 600;

slave2> START SLAVE;


Time Delayed Replication – Doh!

slave2> STOP SLAVE;

Breath slowly!

master> SHOW BINLOG EVENTS\G *************************** 12. row ***************************

Log_name: ws2-bin.000001 Pos: 984 Event_type: Query Server_id: 1

End_log_pos: 1096 Info: use `clusterdb`; DROP TABLE `towns` /*

generated by server */

slave2> START SLAVE UNTIL

-> MASTER_LOG_FILE='ws2-bin.000001',

-> MASTER_LOG_POS=984;


Informational Log Events

• Simplifies debugging and auditing when using Row

Based Replication

• Original query written to the Binlog

• Replicated with row event to the slave

• Activate with

binlog-rows-query-log-events=TRUE

master> INSERT INTO simples VALUES

(20),(21),(22);



BEFORE: mysqlbinlog (-v):

### INSERT INTO clusterdb.simples

### SET

### @1=20 /* INT meta=0 nullable=0 is_null=0 */


### SET



### SET




AFTER: mysqlbinlog (-vv):

# at 443

#111128 16:04:24 server id 1 end_log_pos 504 Rows_query

# INSERT INTO simples VALUES (20),(21),(22)

# at 504


Other Useful Enhancements

• Remote BinLog Backup

• Creates real-time backup of binary log on remote server

• Eliminates requirement for:

• MySQL instance to parse binary log into SQL statements

• User needing privileges to each host for binary log backup

• Universally Unique Server IDs

• Enables auto-discovery and analysis of replication topology

• Simplifies management of large and highly dynamic MySQL

replication environments

How to Evaluate MySQL 5.6

http://dev.mysql.com/downloads/mysql/#downloads

Select Development Release tab


http://dev.mysql.com/downloads/mysql/

http://dev.mysql.com/downloads/mysql/

Next Steps

• Evaluate the new features

• DevZone Article:

http://dev.mysql.com/tech-resources/articles/mysql-5.6-replication.html

• Questions & Feedback: forums.mysql.com/list.php?26

• Bugs: bugs.mysql.com/

• Replication documentation

• dev.mysql.com/doc/refman/5.6/en/replication.html



http://bugs.mysql.com/

http://dev.mysql.com/doc/refman/5.6/en/replication.html



MySQL 5.6 Replication: Supporting High Scale Web & Cloud Services

Remote Binary Log Back-up


MySQL

Master

MySQL

Slave

Relay Log

Backup

Binary Log Backup

• Enhances operational efficiency

• Creates real-time backup of binary

log on remote server

• Adds a “raw” flag to the

mysqlbinlog command

• Sent via the replication channel

• No need for…..

• MySQL instance to parse binary log

into SQL statements

• User needing privileges to hosts for

binary log backup

Remote Binary Log Backup

• Writes to a local file with the same name as the original

$> mysqlbinlog --read-from-remote-

server --raw –h secret_server -P 3306

-u root mysql-bin.000001


Universally Unique Server IDs

• Enables auto-discovery and analysis of replication

topology

• UUID automatically generated for each new server that

becomes part of the replication cluster

• Can be viewed with systems monitoring tools, i.e.

MySQL Enterprise Monitor

• Simplifies management of large and highly dynamic

MySQL replication environments


Universally Unique Server IDs

• A UUID generated when the MySQL server first starts

• Stored in auto.cnf file

• Available as:

•server_uuid system variable for local host

•SHOW SLAVE HOSTS for UUIDs of slaves

•SHOW SLAVE STATUS for UUIDs of master

slave> SHOW SLAVE STATUS\G *************************** 1. row ***************************

Slave_IO_State: Waiting for master to send event

....

Master_UUID: e43ac311-19cc-11e1-a06e-002318c418fd

....


How to get Utilities

• Available on Launchpad

•https://launchpad.net/mysql-utilities

•bzr branch lp:mysql-utilities

•Requires Connector/Python

•https://launchpad.net/myconnpy

•bzr branch lp:myconnpy

• Available as a plugin in MySQL Workbench

•http://www.mysql.com/downloads/workbench/

• Documentation is here:

•http://dev.mysql.com/doc/workbench/en/mysql-

utilities.html

https://launchpad.net/mysql-utilities





https://launchpad.net/myconnpy

https://launchpad.net/myconnpy

http://www.mysql.com/downloads/workbench/

http://www.mysql.com/downloads/workbench/

http://dev.mysql.com/doc/workbench/en/mysql-utilities.html





MySQL 5.6 Replication - Universidad Autónoma de Chiapas · MySQL 5.6 Replication: ... material,...

Documents

Transcript of MySQL 5.6 Replication - Universidad Autónoma de Chiapas · MySQL 5.6 Replication: ... material,...