Skalierbarkeit mit MariaDB und MaxScale

MariaDB Roadshow 2014Hamburg - Berlin - Frankfurt

Ralf Gebhardt, Sales Engineerralf.gebhardt@skysql.com

© SkySQL – The MariaDB Company

MaxScale Objectives

• Highly scalable • Transparent to the application• Highly available• Extendible• Flexible• Modules specify what MaxScale is :

router / firewall / data extract tool /…• Open source, GPL2 software since 8th

of Jan. 2014

© SkySQL – The MariaDB Company

How? In a nut shell

Core of MaxScale is multi-threaded, event-driven network IO processor

10/06/2014

Client authenticates once - with MaxScale

Alter, block or log client requests and results

Content-aware or unaware routers

Configuration and state aware monitors

Allows multiple protocols for clients and servers

© SkySQL – The MariaDB Company

MaxScale 1.0 beta

● Announced 7th of July● First Beta Release

● Connection Routing● Master, Slave, Synced

● Read/Write Splitter● Statement based routing● Balance Load on Slaves

● Monitor for Replication and Galera

● Filter● Query Logging● RegEx Match/Replace● Tee – Second Service● Top – Per Connection Long

Query logging● Protocols

● MySQL Client, MySQL Backend● Telnet● HTTPD

© SkySQL – The MariaDB Company

Basic Setup

● Clients access MaxScale● MaxScale in the app server,

UNIX sockets● Database servers accessed

via TCP

● Clients● Application Server

Database Cluster

© SkySQL – The MariaDB Company

Typical Scenarios

● MySQL Replication with● Read connection load

balancing● Read/Write statement

splitting● Galera Cluster

● Connection load balancing● Read/Write statement

splitting● Write conflict avoidance

(planned)

© SkySQL – The MariaDB Company

Inside MaxScaleMySQL Local Authentication

Backends Clients

ok

“read user, host” connect/auth

Users are loaded at start

© SkySQL – The MariaDB Company

Inside MaxScaleRead/write splitting

“call my_proc(1)”

Master

Slave

Query Classifier

© SkySQL – The MariaDB Company

Inside MaxScale Routing

© SkySQL – The MariaDB Company

Inside MaxScale Routing

© SkySQL – The MariaDB Company

Read Connection Routing - Replication

● For applications that are “Replication aware”, i.e. they can use 2 connections, one for the master and one for the slave servers

● Each application server uses 2 or more connections:1 R/W, 1+ R/O

● Connection load balancing is fast, lightweight and it provides high availability

● MaxScale connects the R/W client connection to the master and it load balances the R/O client connection to one of the slaves

● MHA may provide the automatic failover in case of fault of the master

© SkySQL – The MariaDB Company

Read Scalability with MySQL ReplicationUsing MaxScale’s Connection Load Balancing

ClientClient Client For applications that need read scalability and load balancing and can use separate connections for reads and writes.

Each client establishes one connection for writes and one for reads

Master Slave Slave

[write service]type=servicerouter=readconnrouterouter_options=masterservers=s1,s2,s3,..,sn

[read service]type=servicerouter=readconnrouterouter_options=slaveservers=s1,s2,s3,..,sn

Slave Slave...

© SkySQL – The MariaDB Company

Connection Routing - Galera

● For applications that can use the“all-master” capabilities of Galera

● Each application server uses only 1 connection

● MaxScale monitors the state of each Galera Cluster and it applies load balancing only on synced nodes

● MaxScale load balances the client connections to the Galera nodes

© SkySQL – The MariaDB Company

Read/Write Splitting - Replication

● For applications that have been designed to work with a single server and require read scalability

● Each application server uses only 1 connection

● MaxScale monitors the state of each node and it applies load balancing only on the available slaves

● MaxScale creates 2 or more connections, one for R/W on the master node and one R/O load balanced on the slave nodes

● Slave automatic failover, server weighting, replication consistency can be applied

© SkySQL – The MariaDB Company

Read/Write Splitting - Galera

● For applications designed to work with a single database server, they now require high availability and read scalability without application changes

● Read/Write requests are sent to a designated master in order to maintain full compatibility also at locking level

● Read/Only requests are sent to other masters, when no locking is needed (depending on the isolation level)

● MaxScale monitors the availability of the Galera nodes

● No need to set Replication consistency checks on MaxScale

© SkySQL – The MariaDB Company

Query Monitor

● For developers and devops who need to monitor/troubleshoot their applications and the traffic between the application server and the database

● Incoming queries and outcoming results are tracked by MaxScale and info are stored in log files or they are sent to external data handlers

● Developers can have a precise logging of their queries or they can analyse the top queries that are sent to the database

● The query monitor uses filters to operate. It can coexist with other services

© SkySQL – The MariaDB Company

Query Transformation

● For applications that cannot be modified, but queries must be evaluated and in some cases transformed, for example to apply different reads or writes

● Incoming queries are evaluated by filters using regular expressions

● The queries can be passed as is, they can be transformed or even split/trasnformed in multiple queries

● Developers can apply modifications to legacy applications without changes to the code

© SkySQL – The MariaDB Company

Query Rewrite and FilteringUsing MaxScale’s Filters

ClientClient ClientFor applications with multiple different backend databases. For database firewalling, etc.

CREATE TABLE SUPPLIER(ID INTEGER, NAME VARCHAR) TYPE=INNODB

[regex]type=filtermodule=regexfiltermatch=TYPEreplace=ENGINE

CREATE TABLE SUPPLIER(ID INTEGER, NAME VARCHAR) ENGINE=INNODB

© SkySQL – The MariaDB Company

Query rewrite and filtering (cont.)Using MaxScale’s Filters

ClientClient ClientFor sloppy administrators :-)

Select count(*) f0rm mysql.user

[regex]type=filtermodule=regexfiltermatch=[Ff][Oo0][Rr][Mm]replace=FROM

Select count(*) FROM mysql.user

© SkySQL – The MariaDB Company

What’s coming

● Filters● Routing hints● Multiple Slave Support● Weight based routing● Backend failure tolerance● Enforce Node Replication consistency for

MariaDB 10 & MySQL 5.6

© SkySQL – The MariaDB Company

Extending with filters

● May be connected to form arbitrary chains ● Allows inspection, modification and rejection of requests and results

● blacklist● data mining / log

© SkySQL – The MariaDB Company

Get involved

● Check on GitHub● https://github.com/skysql/MaxScale

● MariaDB Source● https://downloads.mariadb.org/

● Bugs report● http://bugs.skysql.com/buglist.cgi?product=Maxscale

● Google groups● https://groups.google.com/forum/#!forum/maxscale

● Binary Tarball● http://downloads.skysql.com/files/SkySQL/MaxScale

● SkySQL website● http://www.skysql.com

Fragen?

www.mariadb.comwww.facebook.com/mariadb.dbms

www.twitter.com/mariadb

Vielen Dank!

Skalierbarkeit mit MariaDB und MaxScale

© Copyright 2014 SkySQL Corporation Ab.SkySQL and MariaDB are trademarks or registered trademarks of SkySQL Corporation Ab in the European Union and United States of America and/or other

countries. MySQL is a trademark of Oracle Corporation Inc. Other names may be trademarks of their respective owners.