Архитектура приложений с использованием MySQL, Петр...

39
Architecture and Design of MySQL Powered Applications Peter Zaitsev CEO, Percona Highload++ 2014 Moscow, Russia 31 Oct 2014

description

Доклад Петра Зайцева на HighLoad++ 2014.

Transcript of Архитектура приложений с использованием MySQL, Петр...

Page 1: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

Architecture and Design of MySQL Powered Applications

Peter ZaitsevCEO, PerconaHighload++ 2014Moscow, Russia31 Oct 2014

Page 2: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

2

About Percona

Open Source Software for MySQL

Ecosystem•Percona Server•Percona XtraDB Cluster (PXC)•Percona Xtrabackup•Percona Toolkit

Solutions to succeed with MySQL

•Consulting•Support •Managed Services

Page 3: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

3

About Presentation

What's in your Toolbox ?

What Questi

ons you

should

ask ?

Common

MySQL Architecture Patter

ns

Page 4: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

4

LiveJournal Architecture 10 Years Ago

Page 5: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

5

10 Years ago we see

Load Balancers

FrontEnd vs BackEnd Web Server Separation

MemcacheD (Caching)

Global Database for Shared Data

Read Write Splitting

Sharding “Database Clusters”

Page 6: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

6

History Lesson

MySQL is quite Mature

Battle tested Architecture Patterns

Most Problems have proven solutions

Page 7: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

7

YOUR TOOLBOX NOWWell… certain things have changed

Page 8: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

8

Very Powerful Hardware

•16GB to 512GBLots of Memory

•1000 IOPS to 100.000 IOPS•10ms latency to 0.1ms IO Latency

Very Fast Storage

•4 Cores to 32 CoresMany CPU Cores

Page 9: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

9

Reasonable Software

•MySQL •Linux Kernel•File Systems•Etc..

Can use this

Hardware Pretty

Well

Page 10: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

10

Cloud

Dynamic Scalability

Throw Away Server Mentality

Agility

Less involved Operations

Page 11: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

11

Automation

Development

Testing

Deployment

Operations

Page 12: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

12

New Languages and Frameworks

More Agile

Less “system” code

More Client Side Development

Less Database Dependence

ORM

Page 13: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

13

Multiple “Clients”

Desktop Web Browsers

Mobile Web Browsers

“Apps”

API

Page 14: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

14

Not Only “MySQL”

•Memcache, Redis•RabbitMQ•Cassandra, MongoDB•Hadoop•Column Store Engines, Vertica

Right Tools for

Right Job!

Page 15: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

15

More HA Options

Mature MySQL Replication

Percona XtraDB Cluster, Galera

MySQL Cluster

Proprietary cloud HA

Page 16: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

16

Ready solutions for Replication Management

MHA

PRM

Continuent Tungsten

Page 17: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

17

Making Sharding Better

•Clustrix•MemSQL•ScaleDB

Not Quite MySQL

•ScaleArc, ScaleBase, Tesora•MySQL Proxy, MaxScale, ProxySQL

Proxy Solutions

•Vitess•JetPants•MySQL Fabric

OpenSource Frameworks

Page 18: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

18

ARCHITECTURE QUESTIONS

Page 19: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

19

Basics

Make some Right decisions Early

Your first Architecture will not be perfect

Some poor choices are very expensive to fix

Page 20: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

20

Consultants are like Lawyers

You can waste a lot of money with them

Getting right advice on the right topics will save you a lot of trouble in the end

Page 21: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

21

Dimensions to consider

Application Scale

High Availability

Team Experience

Page 22: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

22

Approach to Scaling

Avoid Over-Engineering

Give yourself some runway

Do Some Capacity Planning

Page 23: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

23

High Availability

Real High Availability is Really Expensive

Medium Availability is getting more affordable

People are often leading cause of downtime

Page 24: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

24

Team Experience

Chose Solutions which your Team can Support

• Percona Toolkit, Percona Cloud Tools• Systematically find and review new queries

Often Developers know little about Databases

Right Development, Operational Process Is a Key

Tools to Support your Process

Page 25: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

25

Simplicity

•Less complexity lower costsUse fewer

components when possible

•Monitor•Troubleshoot

Each components needs experience

to

•“Follow the Crowd”•Unless you have a reason not to

There is safety in numbers

Page 26: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

26

World of “Small” Applications

•ReallyMajority of

Applications are Small

•Especially with support of caching

May well run on single database

instance

•Active intranet site for company with 200K employees•Ecommerce site supporting $10M+ in Sales

Such “Small” Applications can be

significant

Page 27: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

27

Lets do Some Math

Modern MySQL Instance Handle 100

K QPS

20 Queries per Use

r Interaction

5K Use

r Interactions Per Second

144M User

Interaction

s a Day

(Considering 3x peak

to avera

ge)

10M Daily Activ

e users (with Medium

Engagement)

Page 28: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

28

PRACTICAL CHOICESBasic Architecture Patterns

Page 29: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

29

Baseline

Single MySQL Instance

No Caching

No HA

No Supplemental Technologies

Page 30: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

30

Simple High Availability

HA with DBaaS (Amazon RDS)

HA with MySQL Replication

HA with Percona XtraDB Cluster

Page 31: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

31

What do you need to Scale ?

Reads

Writes Data Size

Page 32: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

32

Load Management Tricks

Move the load

In Time (Doing Batch Processing at Night) In Space (Use Slave for Reports)

Page 33: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

33

Queueing

Used by Most high volume architectures

• RabbitMQ• Redis• Gearman

Deal with Load Spikes

Increase Reliability

Scale parts of the system which need to

Many solutions

Page 34: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

34

Scaling Reads

Replication (and Read Write Splitting)

Caching (Memcache, Redis etc)

Percona XtraDB Cluster (Read Anywhere)

Page 35: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

35

Scaling Writes

•Using Different Servers for Different Parts of Application

Functional Partitioning

•Horizontal Partitioning over many servers

Sharding

Page 36: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

36

Scaling Data Size

•Depends on the Application and Needs•10TB good rule of thumb

How Much We Can

Practically Store in

Single MySQL Instance

these days

Page 37: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

37

How Is it Possible

“Online” Schema Change

MySQL 5.6Pt-online-schema-change

(Percona Toolkit)

Page 38: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

38

Summary

MySQL is Mature. Solutions Available for Many Problems

Make few choices right in the beginning

Hardware and Software advances allow to go a long way with simple architectures

Page 39: Архитектура приложений с использованием MySQL, Петр Зайцев (Percona)

39

Thank You!Peter Zaitsev

[email protected]