Webinar slides: MySQL & MariaDB load balancing with ProxySQL & ClusterControl: introduction webinar

Click here to load reader

  • date post

    12-Apr-2017
  • Category

    Internet

  • view

    162
  • download

    2

Embed Size (px)

Transcript of Webinar slides: MySQL & MariaDB load balancing with ProxySQL & ClusterControl: introduction webinar

  • Copyright 2017 Severalnines AB

    1

    Your host & some logistics

    I'm Jean-Jrme from the Severalnines Team and I'm your host for today's webinar!

    Feel free to ask any questions in the Questions section of this application or via the Chat box.

    You can also contact me directly via the chat box or via email: [email protected] during or after the webinar.

  • Copyright 2017 Severalnines AB

    2

    About Severalnines and ClusterControl

  • Copyright 2017 Severalnines AB

    3

    What we do

    Manage Scale

    Monitor Deploy

  • Copyright 2017 Severalnines AB

    4

    ClusterControl Automation & Management

    ! Provisioning !Deploy a cluster in minutes !On-premises or in the cloud (AWS)

    !Monitoring ! Systems view ! 1sec resolution !DB / OS stats & performance advisors !Configurable dashboards !Query Analyzer ! Real-time / historical

    !Management !Multi cluster/data-center ! Automate repair/recovery ! Database upgrades ! Backups !Configuration management !Cloning !One-click scaling

  • Copyright 2017 Severalnines AB

    5

    Supported Databases

  • Copyright 2017 Severalnines AB

    6

    Customers

  • Copyright 2017 Severalnines AB

    MySQL & MariaDB load balancing with ProxySQL & ClusterControl: introduction webinar

    February 28, 2017

    Krzysztof Ksiek

    Severalnines

    [email protected]

    Ren Canna

    ProxySQL

    [email protected]

    7

    mailto:[email protected]:[email protected]

  • Copyright 2017 Severalnines AB

    8

    Agenda

  • Copyright 2017 Severalnines AB

    9

    Agenda

    ! ProxySQL - introduction

    ! ProxySQL Internals

    ! Failover with ProxySQL

    ! Demo of ProxySQL support in ClusterControl

  • Copyright 2017 Severalnines AB

    10

    ProxySQL - introduction

  • Copyright 2017 Severalnines AB

    11

    Main motivations

    ! empower the DBAs

    ! improve operation

    ! understand and improve performance

    ! create a proxy layer to shield the database

    ! High performance and High Availability

  • Copyright 2017 Severalnines AB

    12

    ! on-the-fly rewrite of queries

    ! caching reads outside the database server

    ! connection pooling and multiplexing

    ! complex query routing and read/write split

    ! load balancing

    ! real time statistics

    !monitoring

    ProxySQL Features

    ! High Availability and Scalability

    ! seamless failover

    ! firewall

    ! query throttling

    ! query timeout

    ! query mirroring

    ! runtime reconfiguration

  • Copyright 2017 Severalnines AB

    13

    Hostgroups and Query Routing

    ! All backends are grouped into hostgroups

    ! Hostgroups have logical functionalities

  • Copyright 2017 Severalnines AB

    14

    Basic design

  • Copyright 2017 Severalnines AB

    15

    ! HostGroup0 (HG0): Write masters

    ! HostGroup1( HG1): Read slaves

    ! Read/Write split

    Example of hostgroup setups! HG0: main write masters

    ! HG1: main read slaves

    ! HG2: reporting slaves

    ! HG3: ad-hoc queries slaves

    ! HG4: data warehouse write masters

    ! HG5: data warehouse read slaves

    ! HG6: remote site servers

    ! HG7: test servers

    ! HG8 : mirror for traffic on HG0

    ! HG9 : mirror for traffic on HG1

  • Copyright 2017 Severalnines AB

    16

    Basic design

  • Copyright 2017 Severalnines AB

    17

    Basic design

  • Copyright 2017 Severalnines AB

    18

    Clustered ProxySQL Architecture

  • Copyright 2017 Severalnines AB

    19

    Clustered ProxySQL at scale

    ! Tested with:

    ! 8 app servers with 3k clients connections each (24k total)

    ! 4 middle layer proxysqls processing 4k connections each from local proxysqls (16k total)

    ! 256 backends/shard (meaning 256 routing rules) processing 600 connections each (150k total)

    ! Single ProxySQL was tested with up to 150k connections

    ! As today, ProxySQL is able to process up to 750k QPS

  • Copyright 2017 Severalnines AB

    20

    ProxySQL Internals

  • Copyright 2017 Severalnines AB

    21

    ProxySQL Modules

  • Copyright 2017 Severalnines AB

    22

    Queries Processor

    ! Based on Queries Rules

    ! Defines what to cache

    ! Defines the hostgroup target

    ! Timeout/delay

    ! Firewall

    ! Mirroring

    ! Rewrite queries

  • Copyright 2017 Severalnines AB

    23

    !Complex rules to match incoming traffic:

    ! regex on query

    ! regex on digest text

    ! username

    ! schemaname

    Queries rules

    !Complex rules to match incoming traffic:

    ! Source IP address

    ! Bind IP address/port

    !digest

    ! Rules can be chained

  • Copyright 2017 Severalnines AB

    24

    !Caching on the wire

    ! Internal key/value storage

    ! In memory only

    ! Pattern based

    ! Expired by timeout

    Queries Cache and Rewrite

    ! Rewrite on the wire

    ! Regex match/replace on query on digest text

    !Optionally cached or mirrored

  • Copyright 2017 Severalnines AB

    25

    User Authentication

    ! Credentials stored in the proxy

    ! User login always possible (even without backends)

    ! Max connections

    ! Login credentials are encrypted

  • Copyright 2017 Severalnines AB

    26

    Hostgroups Manager

    ! Management of servers

    ! Track servers status

    ! Tightly integrated with the connections pool

  • Copyright 2017 Severalnines AB

    27

    Connection Pool

    ! Reduced the overhead of creating new connections, and are recycled when not in use

    ! One to many connections

    ! Multiplexing & maximum connections

    ! Auto-reconnect and automatic re-execution of queries

    ! Failover management

  • Copyright 2017 Severalnines AB

    28

    Auto-reconnect and re-execution

    ! Automatic detection of failures

    ! Graceful handling

    ! Auto-reconnect when possible

    ! Pause until a backend becomes available

    ! Re-execution of queries

  • Copyright 2017 Severalnines AB

    29

    Multiplexing

    ! Reduce the number of connections against mysqld (configurable)

    ! Many clients connections (tens of thousands) can use few backend connections (few hundreds)

    ! Tracks connection status (transactions, user variables, temporary tables, etc)

    ! Order by waiting time

  • Copyright 2017 Severalnines AB

    30

    Monitoring Module

    ! It monitors backends and collects metrics

    ! Monitors replication lag and shun hosts

    ! Monitors read_only variables (replication hostgroups)

    ! Ping and terminates unresponsive nodes

  • Copyright 2017 Severalnines AB

    31

    Failover with ProxySQL

  • Copyright 2017 Severalnines AB

    32

    Failover! 2 phases process:

    ! remove host

    ! add host

    ! Seamless switchover:

    ! https://severalnines.com/blog/how-proxysql-adds-failover-and-query-control-your-mysql-replication-setup

    ! http://proxysql.blogspot.com/2015/09/proxysql-tutorial-seamless-replication.html

    ! Managed by external process

    ! Switchover in less than 1 second

  • Copyright 2017 Severalnines AB

    33

    Distributed failover

    ! Multiple ProxySQL are available in a network

    ! Failover managed by an external process that:

    ! remove host from each ProxySQL instance

    ! add new host into each ProxySQL instance

    ! Manager is not part of ProxySQL.

    ! ClusterControl can be used to detect master failure and perform a failover

  • Copyright 2017 Severalnines AB

    34

    Replication hostgroups

    ! Constantly monitor read_only variable

    ! Defines replication topology as writer(s)/readers(s)

    ! Automatically re-assign servers to the right hostgroup

    ! Possible to define unlimited number of replication clusters

  • Copyright 2017 Severalnines AB

    35

    Master switch with ClusterControl and ProxySQL

  • Copyright 2017 Severalnines AB

    36

    Failover highlight

    ! improve failover time as perceived by the application

    ! prevent errors sent to the application

    ! perform transparent database failovers: gracefully redirecting traffic without the application knowing

    ! existing applications do not have to be rewritten to autoreconnect since connections are not lost from failovers

  • Copyright 2017 Severalnines AB

    37

    Demo

  • Copyright 2017 Severalnines AB

    38

    Thank You!! Severalnines resources on ProxySQL

    ! https://severalnines.com/blog/using-clustercontrol-deploy-and-configure-proxysql-top-mysql-replication

    ! https://severalnines.com/blog/sharding-mysql-mysql-fabric-and-proxysql

    ! https://severalnines.com/blog/how-proxysql-adds-failover-and-query-control-your-mysql-replication-setup

    ! https://severalnines.com/blog/how-set-read-write-split-galera-cluster-using-proxysql

    ! Install ClusterControl:

    ! https://severalnines.com/download-clustercontrol-database-management-system

    ! Contact: [email protected]

    https://severalnines.com/blog/using-clustercontrol-deploy-and-configure-proxysql-top-mysql-replicationhttps://severalnines.com/blog/sharding-mysql-mysql-fabric-and-proxysqlhttps://severalnines.com/blog/how-proxysql-adds-failover-and-query-control-your-mysql-replication-setuphttps://severalnines.com/blog/how-set-read-write-split-galera-cluster-using-proxysqlhttps://severalnines.com/download-clustercontrol-database-management-systemmailto:[email protected]