MariaDB MaxScale – Der neue intelligente Datenbank-Proxy

© 2015, MariaDB Corp. © 2015, MariaDB Corp.

MariaDB Roadshow 2015

MariaDB MaxScaleDer neue intelligente Datenbank-Proxy

Ralf Gebhardt

© 2015, MariaDB Corp.

Needs of Web Scale organization

●  Ability to support growing user base ○  Scalability

●  Deliver quality customer experience ○  High Availability ○  Continuity ○  Performance ○  Data Security

●  Control cost ○  Development Friendliness ○  Improve DBA Productivity

●  Competitive Advantage ○  Innovation Rollout ○  New Service Creation


Challenges of Scalability withMariaDB & MySQL

Application Business Logic

Application Database Connection Management

MySQL/MariaDB connector

●  Complex application ●  Reduced Development Agility

and DBA Productivity ●  Longer time to market

●  For security requirements additional database firewall needed

Database Firewall Database Firewall

●  Load Balance queries ●  Send queries to node per load balancing ●  Detect node failure and failover ●  Detect node addition or removal

●  Determine which shard to send the query ●  If sharding schema changes, or a new shard

added, ○  update the application code

MariaDB/MySQL Replication

MariaDB Galera Cluster

1:100 101:200 201:300 301:...

Database Firewall

Data Sharding


Scalability with MaxScale for MariaDB/MySQL replication

Application Business Logic

Application Database Connection Management

MySQL/MariaDB connector

MaxScale

Connection and Statement based load balancing

●  Simplifies applications ○  Improved Developer productivity

●  Cluster configuration transparent to applications ○  Improved DBA productivity

●  Monitors backend database high availability ○  Improved Business Continuity

●  Load Balance queries ●  Route queries to node per load balancing algorithm ●  Detect node failure and failover ●  Detect node addition or removal

●  Send queries over a single server connection to MaxScale

MariaDB Galera Cluster MariaDB/MySQL Replication


MariaDB MaxScale: Lightweight, Modular, Intelligent, Transparent

Mon

itor Filter &

Logging

Routing

MaxScaleCore

Routing

●  MaxScale core is a high performance message switch.

●  Module APIs allow for multiple modules of each type.

●  Modules can beclient-facing, back-endfacing, internal, or connectto other DBs.

●  Eases customer-driven development.

Flexible architecture, no change to clients.


Client and Back-End Protocols

● Client protocols let applicationsconnect to MaxScale ○  Initial support for MySQL connectors. ○  Future support planned for JSON/BSON, others.

● Back-end protocols allow MaxScale to connect to different database and cluster technologies. ○  Initial support for MySQL Replication, MariaDB

Replication, Galera Cluster, and MySQL Server with NDB storage engine.

Protocol


Routing and Load Balancing

● Routing modules implementfundamental proxy algorithms. ○ Connection based routing: establishes route upon

connection, doesn’t examine requests. ○ Statement based routing: examines requests to

dynamically route requests to the appropriate back-end.

● Modular design allows for load balancing, HA failover, sharding, and other routing policies.

Routing


Filtering, Logging, and Auditing

● Can log requests that meet filtercriteria such as time-out, accessspecific row, etc.

● Configurable firewall blocks or allows requests based on policy.

● External filtering and transformation pipeline processes requests and results as they pass through MaxScale. ○ Allows for creation of parallel streams. ○  Transforms can alter both requests and returned data.

Filter/Log


Authentication and Monitoring

● Authentication modules letMaxScale proxy user credentials.

● Monitoring modules provide the

detailed back-end stateinformation MaxScale needs to make the right routing decisions.

Authentication

Monitor


MariaDB MaxScale: Implementing Advanced Database Services

●  MaxScale services interface to clients and servers through protocols.

●  Router modules implement policy to determine which servers are best able to handle requests.

●  Filters and logging modules implement a pipeline that can block, split, or transform requests. Simple clients, complex

architectures.

Routing Fi

lter/

Log

Client Protocol

Server Protocol

Message Core&

State Machine .log


MariaDB MaxScale 1.0 Feature List

●  Protocol ●  MySQL and MariaDB client and server

●  Monitoring ●  MariaDB Galera Cluster ●  MySQL replication cluster

●  Load Balancing ●  Connection based ●  Statement based

●  Logging ●  Query performance ●  Top N Query

●  Query transformation based on regular expressions and time ●  Query duplication to another database, storage engine or application ●  Extensible plugin architecture


Read Scalability: MySQL Replication + Connection Load Balancing

●  Applications are “cluster aware”, split reads and writes on separate connections.

●  Using connection load balancing, MaxScale routes the read/write connection to the master, and uses round-robin load balancing to route read-only connections to the slaves.

●  MaxScale monitors the cluster, adjusting resource pools should a slave fail, or should MHA failover promote a slave to master.

MaxScale


RW Scalability, HA Failover: Galera Cluster + Connection Load Balancing

●  Applications can be simple, use one connection for reads and updates.

●  Using connection load balancing, MaxScale load balances connections to each node in the Galera Cluster, round-robin.

●  MaxScale monitors the cluster, routing requests only to fully synchronized nodes.

MaxScale


RW Splitting, Read Scalability: MySQL Replication + Statement Load Balancing

●  Applications can be simple, use one connection for reads and updates.

●  Using statement load balancing, MaxScale dynamically routes writes to the master, and round-robin load balances read requests to the slaves.

●  MaxScale monitors the cluster, routing requests only to fully operational nodes.

MaxScale

R/WSplitting


Query Transformation with MaxScale for Legacy Application Compatibility

Modify queries from legacy applications on the fly - for example a MySQL 5.1 app:

1.  MaxScale accepts a query from a MySQL 5.1 compatible client,

2.  If the query matches the regular expression “/CREATE TABLE/” then MaxScale substitutes “ENGINE” for “TYPE” in that statement, else it passes the statement through the filter unchanged.

3.  Forwards the transformed statement to MariaDB or MySQL 5.6.

4.  Receives the result from the back-end. 5.  Forwards the result to the client.

MaxScale

1 5

3 4

/CREATE TABLE/

s/TYPE/ENGINE/

2

●  Application and Database can be upgraded asynchronously ○  Improves business continuity ○  Improve organizational productivity

●  Faster rollout of new database versions

MySQL 5.1

MySQL 5.6 or MariaDB 10


Query Logging with MaxScale For Performance Diagnostics

1.  MaxScale accepts a query from a client application

2.  Forwards the query to the back-end ○  Logs it into the “All Queries” ○  Logs it into the “Long Running Queries”

log files 3.  Receives the result from the back-end, 4.  Forwards the result to the client

○  Logs the result into the “All Queries” log ○  If it is one of the N longest-running

queries, logs the result into the “Long Running Queries” log file MaxScale

.log

Long Queries (Top N queries)

.log

All Queries

1 4

2

4

2

4 2 3

●  Centralized cluster wide query performance log at MaxScale ○  No need to manually correlate query

logs across cluster nodes ○  Simplified performance tuning ○  Improved DBA productivity

●  Filtering can be also used to prevent SQL injection attacks or unauthorized queries ○  Improved Security


Query Duplication with MaxScale 1.  MaxScale accepts a query from a client

application 2.  Forwards the query to the back-end

○  Sends query to Database A ○  Send query to Database B via Tee-filter

3.  Receives the result from Database A 4.  Receives the result from Database B 5.  Forwards the result to the client

MaxScale

1 5

4

2 2 3

●  Cross-DB solutions: Duplicate queries to MariaDB Replication Cluster and Galera Cluster

●  ETL: Use Tee-filter to insert into transactional database and insert into analytic database via queuing to ETL process

●  Faster and smooth rollout of new services ○  Using different DB technology and

different workloads

Database A: MariaDB InnoDB Replication Cluster

Database B: MariaDB Galera Cluster or MariaDB MyISAM or CONNECT Engine for Cassandra or Spider Engine or InfiniDB Engine or RabbitMQ for another application to process


Scalability Binlog relay: improved HA for master/slave replication

Schema based sharding

MySQL Cluster (NDB) connection load balancing

Named Server Routing

Ease of Use Integration with MDBE notification service

Nagios Plugin for monitoring

Hint based statement load balancing

Canonical Query Logging integrated with RabbitMQ

Security Database Firewall filter: Block SQL injection

What is new in MariaDB MaxScale 1.1?


Key Use Case: Bin Log Server

●  Transparent MySQL replication relay ○  Horizontally Scale Slaves without Master Overload ○  Crash Safe Disaster Recovery ○  Better Parallel Replication

●  User facing application reads from slaves are most up to date

MySQL 5.6 compatible


Key Use Case: Schema Sharding

●  Route to sharded database ●  Use Case:

A Business is hosting database for multiple customers on sharded servers. As business grows add new customers while continuing existing services ○  Each customer with its own

database schema. ○  Each Server hosts up to 600

customers. ○  Client Applications are aware of

database schema - but not the hosting server

Shard 1

MaxScale

Sharding Router

Shard 2 Shard 3 Shard 4 Shard 5

●  MaxScale Schema Shard Router Solution: ○  All applications connect to single

MaxScale ○  MaxScale routes the shard server based

on query from client ○  No impact on existing client application

when ■  New client or shard server is added ■  Database schema are moved around

shard server


Key Use Case: Database Security

●  Database Firewall Filter ○  Block query based on date/time, where clause, wildcard, regular expression match, column

match, query type(insert, select, delete, update) ○  Prevent sensitive data from leaking to the wrong parties

●  Use Case: Prevent user login data breach

●  MaxScale Database Firewall Filter Solution: ○  Configure a filter that will block any SELECT queries on “logins” table with an “OR”

operator in WHERE clause

SQL injection into this query: SELECT idFROM loginsWHERE username = 'Joe'AND password = 'anything' OR 'x'='x' Result: All the rows of the logins table

Client application generated dynamic SQL: SELECT idFROM loginsWHERE username = '$username'AND password = '$password' Result: A Single row matching the user

Secure End User Data, Prevent Identity Theft, while allowing developer agility


Key Use Case: Nagios Plugin

check_maxscale_resources

status and performance data

maxscaled and cli protocol configured in maxscale.cnf

Nagios plugin scripts and command configuration files that comes with MaxScale binaries are copied in appropriate directory

Nagios core

check_maxscale_resources

check_maxscale_threads

check_maxscale_monitor

Nagios: Leading Open Source Tool ●  Monitoring, alerting and reporting ●  More than 1 million worldwide users ●  Ease to use and integrates with enterprise monitoring ecosystem

MaxScale scripts for Nagios ●  Monitor MaxScale service, threads and resources ●  Alert upon failure or anomalies ●  Report performance data


MariaDB MaxScale Advantages

●  Developer and DBA friendly solution for ○  Scalability ○  High availability ○  Cluster wide performance tuning

●  Continuity during database upgrades ●  Data and SQL security ●  New Services with extendable plugins


MariaDB MaxScale Customer Benefits

●  DBA ○  Handle large amount of data with scalable and highly available MariaDB clusters ○  Tune database performance with query logging ○  Continuous availability to applications during cluster administration and database

upgrades ●  Developer

○  Simplified development as cluster complexity hidden from applications ○  Focus on business logic instead of cluster monitoring and tracking

●  Data Architect ○  Tee-filter and Flexible Plugin architecture opens up cross-workload and cross-DB

solutions ○  Simplified and faster rollout of database versions for new services

■  Applications continue to work during database upgrades ●  CIO

○  Scale the infrastructure with growth of business ○  Scale without sacrificing developer and DBA productivity ○  Improved customer satisfaction with application continuity and performance ○  Expand business horizon with newer solutions


MaxScale Resources

●  Product Page: https://mariadb.com/products/mariadb-maxscale ●  Product FAQ: https://mariadb.com/products/product-faqs ●  Data Sheet: MariaDB MaxScale Datasheet ●  Blogs: https://mariadb.com/blog-tags/MaxScale ●  Discussion via the Google Group: [email protected] ●  MariaDB MaxScale Webinar - Scalability at its best


Questions?

mariadb.com

[email protected]

"MySQL is a registered trademark of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. MariaDB is not affiliated with MySQL."


Extra Slides


MariaDB MaxScale 1.0 Supported OS and Databases

●  Supported OS ○  RedHat 5, 6, 7 ○  CentOS 5, 6, 7 ○  Ubuntu 12.04 LTS, 13.10, 14.04 ○  Debian 6, 7 ○  OpenSUSE 13.1 ○  SLES 11, 12 ○  Fedora 19 64 bit ○  Fedora 20 64 bit

●  Supported Backend Database ○  MariaDB Enterprise Server and Enterprise all versions ○  MariaDB Enterprise Cluster all versions ○  MySQL 5.1 and higher


Feature Benefits: MaxScale Base DBA Developer Architect Organization CIO

Overall Benefits

Database Scalability, HA Simplicity Flexibility Organizational Scalability, Productivity, Customer Satisfaction, New revenue stream opportunities

Feature Component Feature Benefits

MySQL and MariaDB Client and Server Protocol

Core Simplified Application Development that can focus on business logic

Increased developer productivity

MariaDB Galera and MySQL replication cluster monitoring

Monitor Plugin Scalability and Reliability to handle large amount of data

Free from cluster configuration complexity

Ability to support growth of business Increased developer productivity

Monitor Plugin Administer Clusters(node addition, removal, replication setup) without impacting application

Continuous availability. Zero-downtime and hidden failover at application level,

Increase DBA productivity End user satisfaction with always available web applications

Connection and Statement based load balancing

Router Plugin Optimal and homogeneous cluster nodes utilization giving high performance and high throughput.

Improved application performance with high performance database queries

End user satisfaction with high performance web applications Optimized Capex resources

Query Performance Logging Top N Query Logging

Filter Plugin Database performance tuning

End user satisfaction

Query Statement Modification based on regular expressions

Filter Plugin Client Applications do not have to upgrade at the same time of database

Applications continue to work when upgrading from older DB version to newer version

End user satisfaction with continuous availability Improved Developer productivity

Query Duplication to another storage engine or application

Tee-Filter Plugin Cross Workload and Cross Technology Applications

New customer offerings with cross workload solutions

Flexible Plugin Architecture Core Architect New Applications not possible before

New Revenue Stream


Feature Benefits: MaxScale Base DBA Developer Architect Organization CIO

Feature Component Feature Benefits

Query Duplication to another database

Monitoring Plugin for another database type

Database migration from another db to MariaDB

Continuous availability during database migration

Replication Relay Big Log Router Horizontally scale slaves without master overload

High Performance read scalability

Scalability and Performance to meet the end user expectation

Routing Hints Hint Filter Performance tuning via hints dynamically control statement-based routers

Improved Application performance and stability

Canonical Query logging Canonical Query Filter Application Validation Improved Developer productivity and application quality

Database Firewall Firewall Filter Prevent SQL Injection attacks

Data access security without application awareness

Prevent sensitive data from leaking to the wrong parties

Data access security

Schema based sharding Table Key based sharding

Sharding Router Scalability and Performance with Sharding

Application do not need to be aware of sharding

Scalability and Performance

MariaDB MaxScale – Der neue intelligente Datenbank-Proxy

Software

Transcript of MariaDB MaxScale – Der neue intelligente Datenbank-Proxy