MaxScale – An intelligent proxy Das intelligente OS ... · MariaDB or MySQL 5.6. 4. Receives the...
Transcript of MaxScale – An intelligent proxy Das intelligente OS ... · MariaDB or MySQL 5.6. 4. Receives the...
© MariaDB Corporation Ab.
MaxScale – An intelligent proxyDas intelligente OS Gateway für MariaDB und
MySQL DatenbankenJens Bollmann
What Is MaxScale?
Router
ProtocolAuth
enticatio
n
Monit
orFilters
MaxScaleCore
What is MaxScale?
• Simple Answer• A Classical Proxy
• Sits between the database clients and servers forwarding requests and responses
• More Complex Answer• A proxy platform for building highly tailored and configurable proxy
implementations• Database aware• Built with a pluggable architecture
3
© MariaDB Corporation Ab
MaxScale is also
•Open Source (github)https://github.com/mariadb-corporation/MaxScale
•GA•Developed with input from customers and users•Setup in many production environments
• Enterprise ready•Training available
© MariaDB Corporation Ab.
An intermediate layer
MaxScale
Clients
Proxy
Databases
Database Router
MaxScale Benefits
• Transparently decouple applications from database deployment environment
• Scale out through middle layer• Improve availability without adding application
complexity• Add flexibility without burdening every application• Improve filtering and logging capabilities
– e.g. „One slow query log for all db instances“ • Allow multiple applications talking to listeners
6
Applications
Databases
© MariaDB Corporation Ab
MaxScale Architecture• Generic core
• Module APIs allow for multiple modules of each type
• Modules can be • client-facing• back-end facing• Internal• connect to other DBs
• Flexible architecture, no change to clients
Router
ProtocolAuth
enticatio
n
Monit
orFilte
rs
MaxScaleCore
MaxScale Core
• Provides core services for• configuration• networking• scheduling• query classification• logging• buffer management• plugin loading• request flow
8
• Using Linux epoll• Events are network request
• Provides protocol implements for• Client -> MaxScale• MaxScale -> Database
•Initial support for MySQL/MariaDB protocol•Potential future support for others (JSON/BSON)
• Also used for administration protocols
Protocols
9
Protocol
Authentication
10
Authentic
ati
on
• Authentication modules provide for authentication mechanism• Allows MaxScale to proxy user credentials
• Currently only MySQL/MariaDB user accounts• User accounts are loaded on startup • User accounts re-verified for each failure
Filter/Logging
• A filter may block, modify or log a request as it passes through MaxScale
• Filters may be built up into chains
• Filters may duplicate requests
11
Filter
Available Filters
12
Filter
1. Statement counting Filter• Counts the number of statements
2. Query Logging Filter• Writes all statements for a session into a log file
3. Query Rewrite Filter • Allows a regexp to be defined, along with replacement text
4. Top Filter• Capture the longest running queries within a client session
5. Tee Filter• Duplicate all queries coming through MaxScale
Monitors
• Monitors the database environment• Provides availability and status information
• Allows the router modules to make intelligent decisions where to send queries
• Tailored to particular database configurations• Implemented modules for monitoring:
• Standard MySQL/MariaDB monitoring• Galera Cluster node monitoring
13
Mon
itor
Routers
• Routes requests to backend database servers using a combination of• Data from monitoring• Routing algorithm• Hints from filters• Request characteristics
• Two classes of routers• Connection based routing• Statement based routing
• R/W split router available
14
Router
Putting it all together
15
Protocol
Protocol
FilterFilter Router
Monitor
Router Protocol
Client Application
Use cases
Router
ProtocolAuth
enticatio
n
Monit
orFilters
MaxScaleCore
© MariaDB Corporation Ab.
Load Balancing
MySQL Replication + Connection Load Balancing
Each application server uses 2 connections: 1 R/W, 1 R
MaxScale connects the R/W client connections to the master and the R
connections are load-balanced to all slaves
© MariaDB Corporation Ab.
Read/Write Splitting
MySQL Replication + R/W Split routing
Each application server uses only 1 connection
MaxScale creates 2 connections, one for R/W on the master node and one for R/O
load balanced on the slave nodes
R/WSplitting
MaxScale monitors the state of each node and only applies
operations on available slaves
© MariaDB Corporation Ab.
HA with Galera
Galera Cluster + Connection based routing
Each application server uses only 1 connection
Only fully synchronised nodes are included
MaxScale monitors the state of the nodes and load balances all
operations
© MariaDB Corporation Ab.
Galera with a Write Master
Galera Cluster + R/W split routing
Each application server uses only 1 connection
If the “master” node fails a new one can be elected immediately
MaxScale selects one node as ”master” and the other nodes as
”slaves”
© 2015, MariaDB Corp.
Query TransformationModify queries from legacy applications on the fly - for example a MySQL 5.1 app:
1. MaxScale accepts a query from a MySQL 5.1 compatible client,
2. If the query matches the regular expression “/CREATE TABLE/” then MaxScale substitutes “ENGINE” for “TYPE” in that statement, else it passes the statement through the filter unchanged.
3. Forwards the transformed statement to MariaDB or MySQL 5.6.
4. Receives the result from the back-end.5. Forwards the result to the client.
Filte
r &
Lo
ggi
ng
1 5
3 4
/CREATE TABLE/
s/TYPE/ENGINE/
2
● Application and Database can be upgraded asynchronously○ Improves business continuity○ Improve organizational productivity
● Faster rollout of new database versions
MySQL 5.1
MySQL 5.6or MariaDB 10
© 2015, MariaDB Corp.
Query Logging 1. MaxScale accepts a query from a client
application2. Forwards the query to the back-end
○ Logs it into the “All Queries”○ Logs it into the “Long Running Queries”
log files3. Receives the result from the back-end,4. Forwards the result to the client
○ Logs the result into the “All Queries” log
○ If it is one of the N longest-running queries, logs the result into the “Long Running Queries” log file
.log
Long Queries(Top N
queries)
.log
All Queries
1 4
2
4
2
42 3
● Centralized cluster wide query performance log at MaxScale○ No need to manually correlate query logs across cluster
nodes○ Simplified performance tuning○ Improved DBA productivity
● Filtering can be also used to prevent SQL injection attacks or unauthorized queries○ Improved Security
© 2015, MariaDB Corp.
Query Duplication1. MaxScale accepts a query from a client
application2. Forwards the query to the back-end
○ Sends query to Database A○ Send query to Database B via Tee-filter
3. Receives the result from Database A4. Receives the result from Database B5. Forwards the result to the client
Filte
r &
Lo
ggi
ng
1 5
4
22 3
● Cross-DB solutions: Duplicate queries to MariaDB Replication Cluster and Galera Cluster
● ETL: Use Tee-filter to insert into transactional database and insert into analytic database via queuing to ETL process
● Faster and smooth rollout of new services ○ Using different DB technology and different
workloads
Database A: MariaDB InnoDB Replication Cluster
Binlog Router
24
MySQLClient
MySQLBackend
MyS
QLM
on
BinlogRouter
binlog cache
binlog cache
MySQLClient
MySQLBackend
MyS
QLM
on
BinlogRouter
MaxScale Roadmap
Router
ProtocolAuth
enticatio
n
Monit
orFilters
MaxScaleCore
New Feature Development
• Write Partitioning•Multi-master environments•Direct writes to appropriate servers
•Simple sharding•Schema based sharding•Key-based sharding
•Binlog router already in production•MaxScale contains copy of master binlog
26
© 2015, MariaDB Corp.
MaxScale Roadmap
MaxScale (future versions):○ Integration with MariaDB Enterprise○ Hinting added to Statement Load Balancing○ Binlog relay: improved HA for master/slave replication○ Optimized Galera routing for deadlock avoidance○ MySQL Cluster (NDB) connection load balancing○ Logging integrated with RabbitMQ○ Firewall filter: Block SQL injection○ Schema based Sharding
More Information
• Contact Us: https://mariadb.com/about/contact• Binary packages on https://mariadb.com/ • Source in GitHub: https://github.com/mariadb-corporation/MaxScale • Discussion via the Google Group: [email protected]• Bugs database: http://bugs.mariadb.com• MaxScale blogs: https://mariadb.com/blog-tags/MaxScale
28
© MariaDB Corporation Ab.
Questions ?Jens Bollmann
Hall 3, D 36