Architecture and Design of MySQL Powered Applications

Peter Zaitsev CEO, Percona Highload++ 2014 Moscow, Russia 31 Oct 2014

About Percona

Open Source Software for MySQL

Ecosystem

• Percona Server

• Percona XtraDB Cluster (PXC)

• Percona Xtrabackup

• Percona Toolkit

Solutions to succeed with MySQL

• Consulting

• Support

• Managed Services

2

About Presentation

What's in your Toolbox ?

What Questions you should ask ?

Common MySQL Architecture

Patterns

3

LiveJournal Architecture 10 Years Ago

4

10 Years ago we see

Load Balancers

FrontEnd vs BackEnd Web Server Separation

MemcacheD (Caching)

Global Database for Shared Data

Read Write Splitting

Sharding “Database Clusters”

5

History Lesson

MySQL is quite Mature

Battle tested Architecture Patterns

Most Problems have proven solutions

6

YOUR TOOLBOX NOW

Well… certain things have changed

7

Very Powerful Hardware

• 16GB to 512GB Lots of

Memory

• 1000 IOPS to 100.000 IOPS

• 10ms latency to 0.1ms IO Latency

Very Fast Storage

• 4 Cores to 32 Cores Many CPU

Cores

8

Reasonable Software

•MySQL

•Linux Kernel

•File Systems

•Etc..

Can use this

Hardware Pretty Well

9

Cloud

Dynamic Scalability

Throw Away Server Mentality

Agility

Less involved Operations

10

Automation

Development Testing Deployment Operations

11

New Languages and Frameworks

More Agile

Less “system” code

More Client Side Development

Less Database Dependence

ORM

12

Multiple “Clients”

Desktop Web Browsers

Mobile Web Browsers

“Apps”

API

13

Not Only “MySQL”

• Memcache, Redis

• RabbitMQ

• Cassandra, MongoDB

• Hadoop

• Column Store Engines, Vertica

Right Tools

for Right Job!

14

More HA Options

Mature MySQL Replication

Percona XtraDB Cluster, Galera

MySQL Cluster

Proprietary cloud HA

15

Ready solutions for Replication Management

MHA

PRM

Continuent Tungsten

16

Making Sharding Better

• Clustrix

• MemSQL

• ScaleDB

Not Quite MySQL

• ScaleArc, ScaleBase, Tesora

• MySQL Proxy, MaxScale, ProxySQL

Proxy Solutions

• Vitess

• JetPants

• MySQL Fabric

OpenSource Frameworks

17

ARCHITECTURE QUESTIONS

18

Basics

Make some Right decisions Early

Your first Architecture will not be perfect

Some poor choices are very expensive to fix

19

Consultants are like Lawyers

You can waste a lot of money with them

Getting right advice on the right topics will save you a lot of trouble in the end

20

Dimensions to consider

Application Scale

High Availability

Team Experience

21

Approach to Scaling

Avoid Over-Engineering

Give yourself some

runway

Do Some Capacity Planning

22

High Availability

Real High Availability is Really Expensive

Medium Availability is getting more affordable

People are often leading cause of downtime

23

Team Experience

Chose Solutions which your Team can Support

Often Developers know little about Databases

Right Development, Operational Process Is a Key

Tools to Support your Process

• Percona Toolkit, Percona Cloud Tools

• Systematically find and review new queries

24

Simplicity

• Less complexity lower costs Use fewer

components when possible

• Monitor

• Troubleshoot

Each components needs experience

to

• “Follow the Crowd”

• Unless you have a reason not to There is safety in

numbers

25

World of “Small” Applications

• Really

Majority of Applications are

Small

• Especially with support of caching

May well run on single database

instance

• Active intranet site for company with 200K employees

• Ecommerce site supporting $10M+/month in Sales

Such “Small” Applications can

be significant

26

Lets do Some Math

Modern MySQL

Instance Handle 100K

QPS

20 Queries per User

Interaction

5K User Interactions Per Second

144M User Interactions a

Day (Considering

3x peak to average)

10M Daily Active users

(with Medium

Engagement)

27

PRACTICAL CHOICES

Basic Architecture Patterns

28

Baseline

Single MySQL Instance

No Caching

No HA

No Supplemental Technologies

29

Simple High Availability

HA with DBaaS (Amazon RDS)

HA with MySQL Replication

HA with Percona XtraDB Cluster

30

What do you need to Scale ?

Reads

Writes Data Size

31

Load Management Tricks

Move the load

In Time (Doing Batch Processing

at Night)

In Space (Use Slave for Reports)

32

Queueing

Used by Most high volume architectures

Deal with Load Spikes

Increase Reliability

Scale parts of the system which need to

Many solutions

• RabbitMQ

• Redis

• Gearman

33

Scaling Reads

Replication (and Read Write Splitting)

Caching (Memcache, Redis etc)

Percona XtraDB Cluster (Read Anywhere)

34

Scaling Writes

• Using Different Servers for Different Parts of Application

Functional Partitioning

• Horizontal Partitioning over many servers Sharding

35

Scaling Data Size

•Depends on the Application and Needs

•10TB good rule of thumb

How Much We Can

Practically Store in Single

MySQL Instance

these days

36

How Is it Possible

“Online” Schema Change

MySQL 5.6

Pt-online-schema-change (Percona

Toolkit)

Fast Backup

Percona Xtrabackup

MySQL Enterprise Backup

Modern Filesystem/Volume

Managers

Fast Network

10Gbps affordable

High Performance

Storage

Flash/SSD

Compression

Innodb

TokuDB

37

Summary

MySQL is Mature. Solutions Available for Many Problems

Make few choices right in the beginning

Hardware and Software advances allow to go a long way with simple architectures

38

39

Thank You!

Peter Zaitsev pz@percona.com