V1212.01

SkySQL Data SuiteA New Open Source Approach to MySQL Distributed Systems

Serge Frezefond

2

Agenda

• SkySQL Cloud Data Suite Architecture

• SkySQL Cloud Data Suite on Amazon EC2

• Components for automated provisioning in AWS

• Components for high availability in EC2

• The Administration Console

• Automated Backup/Restore with S3

4

Intro to SkySQL Data Suite

6

SkySQL Cloud Data Suite

SkySQL Data SuiteSkySQL Data SuiteSkySQL Data SuiteSkySQL Data Suite

High Availability

High Availability

Full-TextFull-Text

ReplicationReplication

GaleraGalera

DRBDDRBD

AnalyticsAnalyticsAnalyticsAnalytics

Carrier GradeCarrier GradeCarrier GradeCarrier Grade

Online Apps.Online Apps.Online Apps.Online Apps.

SharedStorage

SharedStorage

7

Cloud Data Suite vs Enterprise Data Suit

Cloud Data SuiteCloud Data Suite

•Deployed in Amazon AWSDeployed in Amazon AWS•Automatic configuration and provisioningAutomatic configuration and provisioning•MariaDB 5.5MariaDB 5.5•MySQL Replication with Pacemaker/CorosyncMySQL Replication with Pacemaker/Corosync

•Administration Console with Backup/RestoreAdministration Console with Backup/Restore•SkySQL Enterprise Monitoring aka MONyog - TrialSkySQL Enterprise Monitoring aka MONyog - Trial•PhpMyAdminPhpMyAdmin•Community SupportCommunity Support•Development ReleaseDevelopment Release•Fully OSS (with MONyog Trial)Fully OSS (with MONyog Trial)

Cloud Data SuiteCloud Data Suite

•Deployed in Amazon AWSDeployed in Amazon AWS•Automatic configuration and provisioningAutomatic configuration and provisioning•MariaDB 5.5MariaDB 5.5•MySQL Replication with Pacemaker/CorosyncMySQL Replication with Pacemaker/Corosync

•Administration Console with Backup/RestoreAdministration Console with Backup/Restore•SkySQL Enterprise Monitoring aka MONyog - TrialSkySQL Enterprise Monitoring aka MONyog - Trial•PhpMyAdminPhpMyAdmin•Community SupportCommunity Support•Development ReleaseDevelopment Release•Fully OSS (with MONyog Trial)Fully OSS (with MONyog Trial)

Enteprise Data SuiteEnteprise Data Suite

•Deployed in Clouds, virtualised env. and bare OSDeployed in Clouds, virtualised env. and bare OS•Standard download packagesStandard download packages•MariaDB, MySQL, Percona ServersMariaDB, MySQL, Percona Servers•MySQL Replication with MHA, DRBD, Galera or MySQL Replication with MHA, DRBD, Galera or shared storage and Pacemaker/Corosyncshared storage and Pacemaker/Corosync•ZManda Recovery ManagerZManda Recovery Manager•SkySQL Enterprise Monitory aka MONyogSkySQL Enterprise Monitory aka MONyog•SQLyogSQLyog•Enterprise Technical SupportEnterprise Technical Support•Production ReleaseProduction Release•Mix of OSS and CommercialMix of OSS and Commercial

Enteprise Data SuiteEnteprise Data Suite

•Deployed in Clouds, virtualised env. and bare OSDeployed in Clouds, virtualised env. and bare OS•Standard download packagesStandard download packages•MariaDB, MySQL, Percona ServersMariaDB, MySQL, Percona Servers•MySQL Replication with MHA, DRBD, Galera or MySQL Replication with MHA, DRBD, Galera or shared storage and Pacemaker/Corosyncshared storage and Pacemaker/Corosync•ZManda Recovery ManagerZManda Recovery Manager•SkySQL Enterprise Monitory aka MONyogSkySQL Enterprise Monitory aka MONyog•SQLyogSQLyog•Enterprise Technical SupportEnterprise Technical Support•Production ReleaseProduction Release•Mix of OSS and CommercialMix of OSS and Commercial

8

SkySQL Cloud Data Suite Sep 2012

1111

2222

3333

1.Configure your DB system on config.skysql.com or cloud.skysql.com

2.Deploy the DB system:• Online• Configuration pull from AWS• Manually

1.Connect the DB system toyour application

9

SkySQL Cloud Data Suite Dec 2012

• Rel. 1.1 with bug fixing

• New Administration Console with:

• rpm package and installation on Linux OS

• Administration and monitoring of replicated and standalone servers

• Customisable graphs

• Customisable commands

10

SkySQL Cloud Data Suite Architecture

11

SkySQL Cloud Data Suite Configurator

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

DeploymentEngine

ProvisioningQueue

DeploymentQueue

AMI

userdata.sh

User Account

Deployment triggered by the Deployment triggered by the first boot of each instancefirst boot of each instance

Deployment triggered by the Deployment triggered by the first boot of each instancefirst boot of each instance

12

SkySQL Cloud Data Suite

Terminal

Admin Console

PhpMyAdmin

MONyog

13

SkySQL Data Suite Building Block

Replicator

Data StoreAdminConsole

14

SkySQL Cloud Data Suite Sep 2012Technologies and Software

Replicator

Data StoreAdminConsole

FRONT ENDFRONT ENDApacheApacheTomcatTomcat

FRONT ENDFRONT ENDApacheApacheTomcatTomcat

BACK ENDBACK ENDSQLiteSQLite

BACK ENDBACK ENDSQLiteSQLite

MariaDBMariaDBMariaDBMariaDB

Linux 3.2 Linux 3.2 (Amazon (Amazon Distro)Distro)

Linux 3.2 Linux 3.2 (Amazon (Amazon Distro)Distro)

MONyogMONyogTrialTrial

MONyogMONyogTrialTrial

PHPMyAdminPHPMyAdminPHPMyAdminPHPMyAdmin

MySQL MySQL ReplicationReplication

MySQL MySQL ReplicationReplicationGlusterFSGlusterFSGlusterFSGlusterFSPacemakerPacemaker

CorosyncCorosyncPacemakerPacemakerCorosyncCorosync XtrabackupXtrabackupXtrabackupXtrabackup

15

SkySQL Cloud Data Suite Sep 2012Data Store

Data Store

PublicPublicSkySQL AMISkySQL AMI

PublicPublicSkySQL AMISkySQL AMI

MariaDB 5.5MariaDB 5.5MyISAM, InnoDB, XtraDB, MyISAM, InnoDB, XtraDB,

Enhanced Optimizer, Virtual Enhanced Optimizer, Virtual ColumnsColumns

MariaDB 5.5MariaDB 5.5MyISAM, InnoDB, XtraDB, MyISAM, InnoDB, XtraDB,

Enhanced Optimizer, Virtual Enhanced Optimizer, Virtual ColumnsColumns

From standalone toFrom standalone to1 Master + 3 Slaves1 Master + 3 SlavesFrom standalone toFrom standalone to1 Master + 3 Slaves1 Master + 3 Slaves

System and DataSystem and Datafile systemsfile systems

System and DataSystem and Datafile systemsfile systems

Pacemaker controls the Pacemaker controls the automatic failoverautomatic failover

Pacemaker controls the Pacemaker controls the automatic failoverautomatic failover

16

SkySQL Cloud Data Suite Sep 2012Administration Console

AdminConsole

GUI to monitor the GUI to monitor the status of the status of the

ClusterCluster

GUI to monitor the GUI to monitor the status of the status of the

ClusterCluster

Extended Extended Monitoring from Monitoring from

MONyog as a trialMONyog as a trial

Extended Extended Monitoring from Monitoring from

MONyog as a trialMONyog as a trial

PhPMyAdmin as PhPMyAdmin as graphical user toolgraphical user toolPhPMyAdmin as PhPMyAdmin as

graphical user toolgraphical user tool

Backup/Restore Backup/Restore to/from S3 using to/from S3 using

XtrabackupXtrabackup

Backup/Restore Backup/Restore to/from S3 using to/from S3 using

XtrabackupXtrabackupStart/Stop, Isolate, Start/Stop, Isolate,

Promote nodesPromote nodesStart/Stop, Isolate, Start/Stop, Isolate,

Promote nodesPromote nodesmysql command mysql command

line interfaceline interfacemysql command mysql command

line interfaceline interface

Automatic failover for the Automatic failover for the admin and monitoring admin and monitoring

serversservers

Automatic failover for the Automatic failover for the admin and monitoring admin and monitoring

serversservers

17

SkySQL Cloud Data Suite Sep 2012Replicator

Replicator

Standard MySQL Standard MySQL Replication in asynchronous Replication in asynchronous

and semi-synchronous and semi-synchronous modemode

Standard MySQL Standard MySQL Replication in asynchronous Replication in asynchronous

and semi-synchronous and semi-synchronous modemode

GlusterFS on ext4 to replicate GlusterFS on ext4 to replicate the data collection and the data collection and

monitoringmonitoring

GlusterFS on ext4 to replicate GlusterFS on ext4 to replicate the data collection and the data collection and

monitoringmonitoring

Automatic failover of database Automatic failover of database servers and console servers and console

componentscomponents

Automatic failover of database Automatic failover of database servers and console servers and console

componentscomponents

18

SkySQL Data Suite Provisioning

AMI

userdata.sh

User Account

Variable definition - they Variable definition - they will be used later when will be used later when other scripts are calledother scripts are called

Variable definition - they Variable definition - they will be used later when will be used later when other scripts are calledother scripts are called

On-the-fly configuration On-the-fly configuration files, such as my.cnffiles, such as my.cnf

On-the-fly configuration On-the-fly configuration files, such as my.cnffiles, such as my.cnf

Execute predefinedExecute predefinedscripts already in the scripts already in the

AMIAMI

Execute predefinedExecute predefinedscripts already in the scripts already in the

AMIAMI

Adaptive, distributed Adaptive, distributed setupsetup

Adaptive, distributed Adaptive, distributed setupsetup

Elastic IP onElastic IP onthe Master Serverthe Master Server

Elastic IP onElastic IP onthe Master Serverthe Master Server

Currently, the Administration Currently, the Administration Console is on the Master serverConsole is on the Master server

Currently, the Administration Currently, the Administration Console is on the Master serverConsole is on the Master server

19

SkySQL Data Suite Deployment

20

SkySQL Data Suite Deployment

21

Automated Provisioning in AWS

22

Starting Here!

You can try it for You can try it for FREE! Even without FREE! Even without

ananAWS accountAWS account

You can try it for You can try it for FREE! Even without FREE! Even without

ananAWS accountAWS account

23

Simplified Interface

Immediate HelpImmediate HelpImmediate HelpImmediate Help

Real Time CheckReal Time CheckReal Time CheckReal Time Check

24

Advanced Configuration

25

Cost Estimation

26

Delivery Mechanism

27

Online Deployment

The keys will be The keys will be discarded immediatelydiscarded immediatelyafter the provisioningafter the provisioning

The keys will be The keys will be discarded immediatelydiscarded immediatelyafter the provisioningafter the provisioning

28

Extra AWS & Contact Info

29

Welcome Email

mysql command linemysql command linemysql command linemysql command line

Administration ConsoleAdministration ConsoleAdministration ConsoleAdministration Console

ssh connectionssh connectionssh connectionssh connection

How to remove SCDSHow to remove SCDSHow to remove SCDSHow to remove SCDS

An inventory of the An inventory of the AWS componentsAWS components

An inventory of the An inventory of the AWS componentsAWS components

IMPORTANT!IMPORTANT!Your pem keyYour pem keyIMPORTANT!IMPORTANT!Your pem keyYour pem key

30

Job Done!

31

SkySQL Cloud Data Suite in Amazon EC2

32

SecurityIdentity Access Manager Roles

• EIP AddressAssociate/Disassociate, Describe

• InstancesStart, Stop, DescribeAttributes, Status, Tags, Net IFCreate, Describe

• S3 BucketsList, Create, Delete

• ObjectsGet, Put, List, Delete

33

SecuritySecurity Groups

• ssh

• http

• MySQL

• MONyog

34

Storage

35

Components for High Availability in EC2

36

Automatic Failover with PacemakerCRM Configuration

Replicator

Data StoreAdminConsole

37

Automatic Failover with PacemakerCRM Configuration

Replicator

Data StoreAdminConsole

[root@node2 init.d]# [root@node2 init.d]# crm statuscrm status============Last updated: Sat Dec ============Last updated: Sat Dec 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via crm_attribute on node2Stack: openaisCurrent DC: node2 - partition crm_attribute on node2Stack: openaisCurrent DC: node2 - partition with quorumVersion: 1.1.7-6.el6-with quorumVersion: 1.1.7-6.el6-148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 expected votes14 Resources configured.============Online: [ node2 expected votes14 Resources configured.============Online: [ node2 node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL] node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL] Masters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIPMasters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIP(lsb:amazon_master_eip):(lsb:amazon_master_eip): Started node2 Clone Set: apache-php-Started node2 Clone Set: apache-php-myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 ] Resource Group: monyog monyog-fs] Resource Group: monyog monyog-fs (ocf::custom:Filesystem):(ocf::custom:Filesystem):

Started node2 monyog-serviceStarted node2 monyog-service (lsb:MONyogd):(lsb:MONyogd): Started Started node2 Resource Group: admin-console console-fsnode2 Resource Group: admin-console console-fs(ocf::custom:Filesystem):(ocf::custom:Filesystem): Started node2 SkySQL-monitorStarted node2 SkySQL-monitor(lsb:skysql_monitor):(lsb:skysql_monitor): Started node2 Tomcat7Started node2 Tomcat7(ocf::heartbeat:tomcat):(ocf::heartbeat:tomcat): Started node2Started node2

[root@node2 init.d]# [root@node2 init.d]# crm statuscrm status============Last updated: Sat Dec ============Last updated: Sat Dec 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via crm_attribute on node2Stack: openaisCurrent DC: node2 - partition crm_attribute on node2Stack: openaisCurrent DC: node2 - partition with quorumVersion: 1.1.7-6.el6-with quorumVersion: 1.1.7-6.el6-148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 expected votes14 Resources configured.============Online: [ node2 expected votes14 Resources configured.============Online: [ node2 node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL] node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL] Masters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIPMasters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIP(lsb:amazon_master_eip):(lsb:amazon_master_eip): Started node2 Clone Set: apache-php-Started node2 Clone Set: apache-php-myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 ] Resource Group: monyog monyog-fs] Resource Group: monyog monyog-fs (ocf::custom:Filesystem):(ocf::custom:Filesystem):

Started node2 monyog-serviceStarted node2 monyog-service (lsb:MONyogd):(lsb:MONyogd): Started Started node2 Resource Group: admin-console console-fsnode2 Resource Group: admin-console console-fs(ocf::custom:Filesystem):(ocf::custom:Filesystem): Started node2 SkySQL-monitorStarted node2 SkySQL-monitor(lsb:skysql_monitor):(lsb:skysql_monitor): Started node2 Tomcat7Started node2 Tomcat7(ocf::heartbeat:tomcat):(ocf::heartbeat:tomcat): Started node2Started node2

Node 2Node 2Node 2Node 2

Elastic IPElastic IPElastic IPElastic IP

Public IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPs

admin-console and admin-console and MONyog groupMONyog group

admin-console and admin-console and MONyog groupMONyog group

38

The Administration Console

39

Admin Console Dec 2012

40

Admin Console Dec 2012

Access to Access to MONyogMONyog

Access to Access to MONyogMONyog

MasterMasterServerServerMasterMasterServerServer

ContextContextSensitiveSensitiveContextContext

SensitiveSensitiveContextContext

SensitiveSensitiveContextContext

SensitiveSensitive

Customisable Customisable GraphsGraphs

Customisable Customisable GraphsGraphs

41

Admin Console Dec 2012

Access to Access to PhpMyAdminPhpMyAdmin

Access to Access to PhpMyAdminPhpMyAdmin

Customisable Customisable CommandsCommands

Customisable Customisable CommandsCommands

Backup/RestoreBackup/RestoreBackup/RestoreBackup/Restore

42

Administration Console Dec 2012Customizable Graphs

Name SQL Icon Delta Avg Chart

Monitors

43

Administration Console Dec 2012Customizable Commands

Desc Icon Order

Commands

Desc Icon Script

Steps

CommandStep

44

Automated Backup/Restore with S3

45

Automated BackupUsing Xtrabackup from the Admin Console

46

Automated BackupUsing Xtrabackup from the Admin Console

47

Automated BackupUsing Xtrabackup from the Admin Console

48

Why is the SkySQL Cloud Data Suite important for you?

• You can test MariaDB/MySQL on AWS for FREE

• You can use and fully control a ready made highly available solution AWS

• You can compare MySQL on EC2 and RDS

• You can study/review security and availability aspects in AWS

• You can use some of the SCDS components in your environment

49

SkySQL Cloud Data Suite & OpenStack ?

• Nova, glance, swift, keystone, horizon, quantum ceph

• Position for public/private/hybrid cloud

• Rackspace,HPcloud,VMware ...CloudWatt, Numergy

• Our solution will be there

•

50

Is not Amazon RDS enough for my needs ?It is fully managed !

• Provisioning

• Patching

• Backup/ PointInTIme recovery

• Replicas

• HA / Failover

• monitoring

51

Or maybe Google Cloud SQL ?

• New offer

• Similar to AWS RDS

• Rely on storage feature for HA

• Fully managed

52

Private/Hybrid cloudEucalyptus / VMware Vcloud

• They own a huge market share

• Eucalyptus is tied to Amazon / open core

• VMware is tied to VMware and fully closed

• Excellent solutions but ...

53

For more information...

• Try the Configurator and How-To Video: http://cloud.skysql.com

• The Admin Console: http://www.skysql.com/downloads/community/skysql-console

• Forums: https://www.skysql.com/forums/skysql-cloud-data-suite

• Web: https://www.skysql.com/cloud

• Bugs: http://bugs.skysql.com

• Try it! http://config.skysql.com

• Free SkySQL account available for testing

• Test on your account