Percona XtraDB Cluster€¦ · WSREP and Galera • WSREP API is a project to develop generic...

Percona XtraDB Cluster

Kenny Gryp@gryp 07 03 2017 1

Agenda

• Default asynchronous MySQL replication• Percona XtraDB Cluster:• Introduction / Features / Load Balancing

• Use Cases:• High Availability / WAN Replication / Read Scaling

• Limitations

Percona

• Percona is the oldest and largest independent MySQL Support, Consulting, Remote DBA, Training, and Software Development company with a global, 24x7 staff of over 100 serving more than 2,000 customers in 50+ countries since 2006

• Our contributions to the MySQL community include:• Percona Server for MySQL, Percona XtraDB Cluster• Percona Server for MongoDB• Percona XtraBackup: online backup• Percona Toolkit• Percona Monitoring & Management…

Agenda

• Default asynchronous MySQL Replication

• Percona XtraDB Cluster:• Introduction / Features / Load Balancing

• Limitations

MySQL Replication

If your HA is based on MySQL Replication -You may be playing a dangerous game !

Traditional Replication Approach

Server 1 Server 2replication stream

“master” “slave”

MySQL Replication 7

• Common Topologies:• Master-Master (Only 1 active master)• 1 or more layers of replication

• Slaves can be used for reads:• asynchronous, stale data is the rule• data loss possible (*semi-sync)

MySQL Replication 8

• non-trivial:• external monitoring• scripts for failover• add node == restore backup• much better in

MySQL 5.6: GTIDs

MySQL Replication

Agenda

• Default asynchronous MySQL Replication• Percona XtraDB Cluster:• Introduction / Features / Load Balancing

• Limitations

11Data Centric

Server 1 Server 2 Server 3 Server N ...

12Percona XtraDB Cluster

13Percona XtraDB Cluster

• All nodes have a full copy of the data• Every node is equal• No central management, no SPOF

14Understanding Galera

38The cluster can be seen as a meeting !

PXC (Galera cluster) is a meeting

bfb912e5-f560-11e2-0800-1eefab05e57d

PXC (Galera cluster) is a meeting 22

4fd8824d-ad5b-11e2-0800-73d6929be5cf

New meeting !

What is Percona XtraDB Cluster ?

• Percona Server• + WSREP patches• + Galera library• + Utilities (init, SST and cluster check

scripts)• + Percona Changes to Galera

• This is a free open source solution, Percona Server is a MySQL alternative which offers breakthrough performance, scalability, features, and instrumentation. Self-tuning algorithms and support for extremely high-performance hardware make it the clear choice for organisations that demand excellent performance and reliability from their MySQL database server.

Percona Server

WSREP and Galera

• WSREP API is a project to develop generic replication plugin interface for databases (WriteSet Replication)

• Galera is a wsrep provider that implements multi-master, synchronous replication

• Other 'Similar' Implementations: • MariaDB Galera Cluster• Galera Cluster For MySQL

Fullcompatibility with existing

systems

Minimal effortsto migrate

Minimal effortsto return back

to MySQL

31Features

• 'Synchronous' Replication• Multi Master• Parallel Applying• Quorum Based• Certification/Optimistic Locking• Automatic Node Provisioning• PXC Specific Features

32Features

33(Virtual) Synchronous Replication

• Writesets (transactions) are replicated to all available nodes on commit (and queued on each)

• Writesets are individually “certified” on every node, deterministically.Either it is committed on all nodes or no node at all (NO 2PC)

• Queued writesets are applied on those nodes independently and asynchronously

• Flow Control avoids too much ‘lag’

• Reads can read old data• Flow Control (by default 16 trx) avoids lag• wsrep_sync_wait=1 can be enabled to

ensure full synchronous reads• Latency: writes are fast, only at COMMIT,

communication with other nodes happen

36Stale Reads

37Latency

38Features

Multi-Master Replication

• You can write to any node in your cluster*• Writes are ordered inside the cluster

writes

writeswrites

40Features

Parallel Replication

• Standard MySQL

Application writes N threads Apply 1 thread

(MySQL 5.6:max 1 thread per schema, 5.7: Interval Based)

Parallel Replication

• PXC / Galera

Application writes N threads Apply M threads

(wsrep_slave_threads)

43Features

Quorum Based

• If a node does not see more than 50% of the total amount of nodes: reads/writes are not accepted.

• Split brain is prevented• This requires at least 3 nodes to be effective• a node can be an arbitrator (garbd), joining the

communication, but not having any MySQL running

• Can be disabled (but be warned!)

• Loss of connectivity

Quorum Based 45

Network Problem

Does not accept Reads & Writes

• 4 Nodes • 2 DC

Quorum Based 46

• Default quorum configuration:4 Nodes, 0 Nodes have quorum

Quorum Based 47

Network Problem

• With garbd Arbitrator:

Quorum Based Arbitrator 48

Network Problem

garbd(no data)

49Features

50Certification

51Optimistic Locking

• Communication to the other nodes of the cluster only happens during COMMIT, this affects locking behavior.

• Optimistic Locking is done:• InnoDB Locking happens local to the

node• During COMMIT/Certification, the other

nodes bring deadlocks

• Some Characteristics:• also COMMIT and SELECT’s can fail on

deadlock• Might require application changes:

Not all applications handle this properly

53Traditional InnoDB Locking

system 1Transaction 1

Transaction 2BEGIN

Transaction1

BEGINUPDATE t WHERE id=14

UPDATE t WHERE id=14...

COMMIT

Waits on COMMIT in trx 1

54InnoDB Locking With PXC

Transaction 2BEGIN

Transaction1

COMMITDML,COMMIT or SELECT

ERROR due row conflict

system 2

55InnoDB Locking With PXC

Transaction 2BEGIN

Transaction1

COMMITCOMMIT

ERROR due row conflict

system 2

...ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction

57Features

Automatic Node Provisioning

• When a node joins the cluster:• the data is automatically copied • when finished: the new node is automatically

ready and accepting connections• 2 different types of joining:

– SST (state snapshot transfer): full copy of the data

– IST (incremental state transfer): send only the missing writesets (if available)

StateTransfer Summary

Full dataSST

IncrementalIST

New node

Node longtime

disconnected

Nodedisconnected

short time

Snapshot State Transfer

mysqldump

Smalldatabases

Donordisconnectedfor copy time

Faster

XtraBackup

Donoravailable

Slower

Incremental State Transfer

Node wasin the cluster

Disconnectedfor maintenance

Nodecrashed

writes

writeswrites

new node joining

data is copied via SST or IST

writes

writeswrites

new node joiningwhen ready

writes

64Features

65PXC Specific Features

• PXC Strict mode• Extended PFS support• ProxySQL integration• SST/XtraBackup Changes• Bug-Fixes

66PXC Strict Mode

Prevent experimental/unsupported features:• Only Allow InnoDB Operations• Prevent Changing binlog_format!=ROW• Require Primary Key on tables• Disable Unsupported Features:• GET_LOCK, LOCK TABLES, CTAS

• FLUSH TABLES <tables> WITH READ LOCK

• tx_isolation=SERIALIZABLE

67PERFORMANCE_SCHEMA Support

• Improve troubleshooting/debugging using Performance Schema

• Threads• Files• Stages• Mutexes

• https://www.percona.com/live/plam16/sessions/exploring-percona-xtradb-cluster-using-performance-schema

• https://www.percona.com/doc/percona-xtradb-cluster/5.7/manual/performance_schema_instrumentation.html

68ProxySQL

• Layer 7 MySQL Load Balancer• Open Source • R/W Splitting• Connection management• Transparent Reconnect• Query Caching (TTL)• 'Sharding' (User, Schema or Regex)

69ProxySQL & PXC Integration

• PXC includes ProxySQL as load balancer:• proxysql-admin• ProxySQL schedulers:

• Health Checks• Reconfigures Nodes

• PXC Maintenance Mode• Tell Load Balancer to rebalance load

ProxySQL

• Different possible setups:• be a dedicated layer• integrated at application layer• integrated at database layer

Dedicated shared ProxySQL application server 1 application server 2 application server 3

PXC node 1 PXC node 2 PXC node 3

ProxySQL

Dedicated shared ProxySQL application server 1 application server 2 application server 3

ProxySQL

ProxySQL on application sideapplication server 1 application server 2 application server 3

ProxySQL ProxySQL ProxySQL

ProxySQL 'Endless' Possibilitiesapplication server 1 application server 2 application server 3

ProxySQL ProxySQL ProxySQL

ProxySQL ProxySQL

75Load Balancers Alternatives

• ProxySQL (L7)• HAProxy (L4)• ScaleArc (L7, $$)• MySQL Router (L7, quite new)• MaxScale (L7, $ BS-License)

Agenda

• Use Cases:• High Availability / WAN Replication / Read

Scaling• Limitations

77Use Cases

• High Availability• WAN Replication• Read Scaling

High Availability 78

• Each node is the same (no master-slave)• Consistency ensured, no data loss• Quorum avoids split-brain• Cluster issues are immediately handled on• no ‘failover’ necessary• no external scripts, no SPOF

79WAN replication

• No impact on reads• No impact within a trx• Communication only happens during

COMMIT (or if autocommit=1)• Use higher timeouts and

send windows

• Beware of increased latency• Within EUROPE EC2

• COMMIT: 0.005100 sec• EUROPE <-> JAPAN EC2

• COMMIT: 0.275642 sec

80WAN replication - latency

81WAN replication with MySQL asynchronous replication

• You can mix both types of replication• Good option on slow WAN link• Requires more nodes

MySQL MySQL

Cluster Segmentation - WAN 82

83Read Scaling

• No Read/Write Splitting necessary in App.• Write to all nodes(*)

Example Architecture

Application Layer

ProxySQL

Agenda

• Limitations

86Limitations

• Supports only InnoDB tables• MyISAM support will most likely stay in

alpha.• The weakest node limits write

performance• All tables must have a Primary Key!

87Limitations

• Large Transactions are not recommended if you write on all nodes simultaneously

• Long Running Transactions• If the workload has a hotspot then

(frequently writing to the same rows across multiple nodes)

• Solution: Write to only 1 node

88Limitations

• By Default DDL is done in TOI• Blocks ALL writes during schema change• For certain operations:

wsrep_osu_method=RSU

• pt-online-schema-change

Summary

• Limitations

90Resources

• Percona XtraDB Cluster website: http://www.percona.com/software/percona-xtradb-cluster/

• PXC articles on Percona Blog: https://www.percona.com/blog/category/percona-xtradb-cluster/

• Slideshare: https://www.slideshare.net/Grypyrg/pxc-introduction-34774802

Questions?Kenny Gryp - @gryp

Percona XtraDB Cluster€¦ · WSREP and Galera • WSREP API is a project to develop generic...

Documents

Transcript of Percona XtraDB Cluster€¦ · WSREP and Galera • WSREP API is a project to develop generic...

Galera Documentation - Galera Cluster for MySQLgaleracluster.com/documentation-webpages/galera-documentation.pdf · Galera Documentation, Release Galera Cluster is a synchronous multi-master

Percona XtraDB 集群

Galera replication

Galera Multi Master Synchronous My S Q L Replication Clusters

MariaDB 10.5 Galera Features · through 10.4 Streaming Replication . Galera Galera PluginGalera Galera XA Transaction Support XA begin XA trx. ... NBO Replication node-1 node-2 NBO

MySQL Servers Working as a Team - Replication or Galera ...€¦ · MySQL Teamwork: Replication or Galera Cluster, joerg.bruehe@fromdual.com, 2016 April, CC-BY-SA 1 / 42 MySQL Servers

OpenStack High Availability Technologies€¦ · MySQL Galera cluster • Synchronous multi-master cluster for MySQL/InnoDB database • Database replication is not simply replication

Become a MySQL DBA - Webinars - Schema Changes for MySQL Replication & Galera Cluster (slides)

FOSDEM 2012: Synchronoues multi-master replication with Galera

Synchronous Replication For MySQL with Percona XtraDB Cluster

Percona XtraDB Cluster Documentation › hubfs › Manuals › Percona_XtraDB... · 2017-10-10 · Percona XtraDB Cluster Documentation, Release 5.7.17-29.20 Percona XtraDB Clusteris

Galera with Docker: How Synchronous Replication and Linux Containers mesh together

Scaling State Machine Replication · MySQL Group Replication Galera Cluster, … 2. State machine replication is intuitive & simple •Replication transparency

Galera #50

Galera: Multi-Master Synchronous MySQL Replication Clusters

Migration Workshop PXC 5.5 to MariaDB 10 - FromDual · 2012 – Percona XtraDB Cluster 5.5/Galera v2 2013 – MySQL 5.6/Galera v3 2013 – MariaDB Galera Cluster 5.5/Galera v2 2013

Mastering Galera

Percona XtraDB Cluster Documentation

Red Hat OpenStack Platform 10 Understanding Red Hat ... · Galera: Red Hat OpenStack Platform uses the MariaDB Galera Cluster to manage database replication. Highly available services

Galera Replication PLL 2011