Percona XtraDB Cluster SF Meetup

93
Percona XtraDB Cluster powered by Galera Vadim Tkachenko Percona Inc, co-founder, CTO www.percona.com www.mysqlperformanceblog.com

Transcript of Percona XtraDB Cluster SF Meetup

Page 1: Percona XtraDB Cluster SF Meetup

Percona XtraDB Clusterpowered by Galera

Vadim TkachenkoPercona Inc, co-founder, CTO

www.percona.comwww.mysqlperformanceblog.com

Page 2: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

This talk online• PowerPoint• http://bit.ly/PXC-meetup

• PDF• http://bit.ly/PXC-meetup-pdf

• Google Docs• http://bit.ly/PXC-meetup-gdocs

Page 3: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

This talk

High Availability

Replication

Cluster

Page 4: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

What is HA

Availability

Avail ~ Ability

Ability to Avail

Page 5: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Availability by redundancy

Page 6: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Duplicate resources

Page 7: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Failover

Page 8: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Probability of failure

Single server: P

Two servers:

P/2

X servers: P/X

Page 9: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Probability of failure

1 2 3 4 5 6 7 8 9 10

N of servers

Pro

bab

ilit

y o

f fa

ilu

re

Page 10: Percona XtraDB Cluster SF Meetup

Easy ?Not if we deal with databases

Percona XtraDB Cluster

Page 11: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Database

Page 12: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Redundancy ?

Page 13: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Database availability is hard

Service availabili

ty

Data availabili

ty

Page 14: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Replication

Page 15: Percona XtraDB Cluster SF Meetup

MySQL Replication

Percona XtraDB Cluster

Page 16: Percona XtraDB Cluster SF Meetup

If your HA is based on MySQL Replication –You are doing it wrong

Page 17: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

What is wrong with MySQL replication ?

“a”

Page 18: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

What is wrong with MySQL replication ?

“a” in async

Page 19: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

What is wrong with MySQL replication ?

“async”

vs

“sync”

Page 20: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Async

Page 21: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Async

Page 22: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

sync

Page 23: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Didn’t we just reinvent DRBD ?

Page 24: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

DRBD

Page 25: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Clustering

Page 26: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Percona XtraDB ClusterFree and Open Source

Page 27: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Percona XtraDB Cluster

Page 28: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Virtually synchronoushttp://en.wikipedia.org/wiki/Virtual_synchrony

Page 29: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Virtually synchronous

Page 30: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

synchronous

replication

multi-master

replication

parallel applying on slaves

data consistenc

y

automatic node

provisioning

Page 31: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

synchronous

replication

multi-master

replication

parallel applying on slaves

data consistenc

y

automatic node

provisioning

Page 32: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Multi-master: MySQL

Page 33: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Multi-master: XtraDB Cluster

Page 34: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

synchronous

replication

multi-master

replication

parallel applying on slaves

data consistenc

y

automatic node

provisioning

Page 35: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Parallel apply: MySQL

Page 36: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Parallel apply: XtraDB Cluster

Page 37: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

synchronous

replication

multi-master

replication

parallel applying on slaves

data consistenc

y

automatic node

provisioning

Page 38: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

XtraDB Cluster data consistency

Page 39: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

synchronous

replication

multi-master

replication

parallel applying on slaves

data consistenc

y

automatic node

provisioning

Page 40: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Node provisioning

Page 41: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

CAP theoremhttp://en.wikipedia.org/wiki/CAP_theorem

Page 42: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Pick only TWO

Consistency

Node availabilit

y

Partition Tolerance

Page 43: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Network failure

Page 44: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

MySQL Replication

Access to all systems - YES

Data consistency - NO

Page 45: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

XtraDB Cluster

Access to all systems - NO

Data consistency - YES

Page 46: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

3 nodes is the minimal recommended configuration

Page 47: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Split brain

Which system to make available ?

Page 48: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Split brain

But you deal with consequences

You still can have this setup

Page 49: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Choice

MySQL Replication:Access to all systems

Data consistency XtraDB Cluster:

Page 50: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

MySQL replication

based

MMM

MHA

Flipper

PRM

Page 51: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Percona XtraDB Cluster details

Page 52: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

PerconaXtraDB Cluster

Percona Server

WSREP patches

Galera library

Page 53: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Full compatibil

ity with existing systems

Page 54: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Minimal efforts to migrate

Page 55: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Minimal efforts to

return back to MySQL

Page 56: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

So, is this a perfect solution?

Page 57: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Limitationssome will be solved later

Page 58: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Only InnoDB tables are supported

Page 59: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

OPTIMISTIC locking for transactions on different servershttp://en.wikipedia.org/wiki/Optimistic_concurrency_control

Page 60: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Traditional locking

Page 61: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Optimistic locking

Page 62: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

The write performance is limited by weakest node

Page 63: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Write performance

Page 64: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

For write intensive applications there could be datasize limit per nodeNot physical but logical

Page 65: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Join process. Step 1

Page 66: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Join process. Step 2

Page 67: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Join process: step 3

Page 68: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

This is software + hardware solution

Page 69: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

InnoDB write performance

Page 70: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

InnoDB performance + ACID

Page 71: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Cluster performance

•10 GigE •Infiniband

Network

•SSD•PCI-e Flash

Storage

Page 72: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Join process

Page 73: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

State Transfer

Full data SST

New node

Node long time

disconnected

Incremental IST

Node disconnect

ed short time

Page 74: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Snapshot State Transfer

Mysqldump

Small databases

Rsync

Donor disconnected for copy

time

faster

XtraBackup

Donor disconnect

ed for short time

slower

Page 75: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Incremental State TransferNode was in cluster

Disconnected for

maintenance

Node Crashed (work in

progress)

Page 76: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Scaleability

Page 77: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Scaleability

Scale ~ Ability

Ability to Scale

Page 78: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Scaleability is similar to availability

Page 79: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

XtraDB Cluster: Reads scalability is easy

Page 80: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Write scalability is complicated

Page 81: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

N servers scale to :

•N factor100% reads

•……

•N/2 factor50/50

•……

•1 or const100% writes

Page 82: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

10 servers scale to :

• 1 server: 100 q/s• 10 servers: 1000

q/s

100% reads

• 1 server: 100 q/s• 10 servers: 500

q/s50/50

• 1 server: 100 q/s• 10 servers: 100

q/s (can be more)

100% writes

Page 83: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

FAQQuestions I am asked

Page 84: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

It looks so easy. Why did not you implement it earlier?It is not easy.Computer science of group communication and distributed transactions.Credits to Codership Oy

Page 85: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

How fast is it?Reasonably fast.

Page 86: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Can I replicate XtraDB Cluster to MySQL Replication?Yes

Page 87: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Async MySQL Replication

Page 88: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Would I install it on a production system?Yes. I am going to upgrade MySQLPerformanceBlog.com to use XtraDB Cluster

Page 89: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

How it is compared to MySQL Cluster?It is different

Page 90: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

XtraDB Cluster

MySQL Cluster

Easy to migrate Easy to use Cloud / EC2 Changes in an application

Write scaling 99.999%

Page 92: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Credits• WSREP patches and Galera library is developed by Codership Oy

Page 93: Percona XtraDB Cluster SF Meetup

Percona XtraDB Cluster

Thank you!

Questions ?

You can try Percona XtraDB Cluster today!