MariaDB Cassandra Interoperability Cassandra Storage Engine in MariaDB
Cloud MariaDB Data suite
Transcript of Cloud MariaDB Data suite
V1212.01
SkySQL Data SuiteA New Open Source Approach to MySQL Distributed Systems
Serge Frezefond
2
Agenda
• SkySQL Cloud Data Suite Architecture
• SkySQL Cloud Data Suite on Amazon EC2
• Components for automated provisioning in AWS
• Components for high availability in EC2
• The Administration Console
• Automated Backup/Restore with S3
4
Intro to SkySQL Data Suite
6
SkySQL Cloud Data Suite
SkySQL Data SuiteSkySQL Data SuiteSkySQL Data SuiteSkySQL Data Suite
High Availability
High Availability
Full-TextFull-Text
ReplicationReplication
GaleraGalera
DRBDDRBD
AnalyticsAnalyticsAnalyticsAnalytics
Carrier GradeCarrier GradeCarrier GradeCarrier Grade
Online Apps.Online Apps.Online Apps.Online Apps.
SharedStorage
SharedStorage
7
Cloud Data Suite vs Enterprise Data Suit
Cloud Data SuiteCloud Data Suite
•Deployed in Amazon AWSDeployed in Amazon AWS•Automatic configuration and provisioningAutomatic configuration and provisioning•MariaDB 5.5MariaDB 5.5•MySQL Replication with Pacemaker/CorosyncMySQL Replication with Pacemaker/Corosync
•Administration Console with Backup/RestoreAdministration Console with Backup/Restore•SkySQL Enterprise Monitoring aka MONyog - TrialSkySQL Enterprise Monitoring aka MONyog - Trial•PhpMyAdminPhpMyAdmin•Community SupportCommunity Support•Development ReleaseDevelopment Release•Fully OSS (with MONyog Trial)Fully OSS (with MONyog Trial)
Cloud Data SuiteCloud Data Suite
•Deployed in Amazon AWSDeployed in Amazon AWS•Automatic configuration and provisioningAutomatic configuration and provisioning•MariaDB 5.5MariaDB 5.5•MySQL Replication with Pacemaker/CorosyncMySQL Replication with Pacemaker/Corosync
•Administration Console with Backup/RestoreAdministration Console with Backup/Restore•SkySQL Enterprise Monitoring aka MONyog - TrialSkySQL Enterprise Monitoring aka MONyog - Trial•PhpMyAdminPhpMyAdmin•Community SupportCommunity Support•Development ReleaseDevelopment Release•Fully OSS (with MONyog Trial)Fully OSS (with MONyog Trial)
Enteprise Data SuiteEnteprise Data Suite
•Deployed in Clouds, virtualised env. and bare OSDeployed in Clouds, virtualised env. and bare OS•Standard download packagesStandard download packages•MariaDB, MySQL, Percona ServersMariaDB, MySQL, Percona Servers•MySQL Replication with MHA, DRBD, Galera or MySQL Replication with MHA, DRBD, Galera or shared storage and Pacemaker/Corosyncshared storage and Pacemaker/Corosync•ZManda Recovery ManagerZManda Recovery Manager•SkySQL Enterprise Monitory aka MONyogSkySQL Enterprise Monitory aka MONyog•SQLyogSQLyog•Enterprise Technical SupportEnterprise Technical Support•Production ReleaseProduction Release•Mix of OSS and CommercialMix of OSS and Commercial
Enteprise Data SuiteEnteprise Data Suite
•Deployed in Clouds, virtualised env. and bare OSDeployed in Clouds, virtualised env. and bare OS•Standard download packagesStandard download packages•MariaDB, MySQL, Percona ServersMariaDB, MySQL, Percona Servers•MySQL Replication with MHA, DRBD, Galera or MySQL Replication with MHA, DRBD, Galera or shared storage and Pacemaker/Corosyncshared storage and Pacemaker/Corosync•ZManda Recovery ManagerZManda Recovery Manager•SkySQL Enterprise Monitory aka MONyogSkySQL Enterprise Monitory aka MONyog•SQLyogSQLyog•Enterprise Technical SupportEnterprise Technical Support•Production ReleaseProduction Release•Mix of OSS and CommercialMix of OSS and Commercial
8
SkySQL Cloud Data Suite Sep 2012
1111
2222
3333
1.Configure your DB system on config.skysql.com or cloud.skysql.com
2.Deploy the DB system:• Online• Configuration pull from AWS• Manually
1.Connect the DB system toyour application
9
SkySQL Cloud Data Suite Dec 2012
• Rel. 1.1 with bug fixing
• New Administration Console with:
• rpm package and installation on Linux OS
• Administration and monitoring of replicated and standalone servers
• Customisable graphs
• Customisable commands
10
SkySQL Cloud Data Suite Architecture
11
SkySQL Cloud Data Suite Configurator
ConfigurationEngine
RulesRepository
ProvisioningEngine
SoftwareRepository
DeploymentEngine
ProvisioningQueue
DeploymentQueue
AMI
userdata.sh
User Account
Deployment triggered by the Deployment triggered by the first boot of each instancefirst boot of each instance
Deployment triggered by the Deployment triggered by the first boot of each instancefirst boot of each instance
12
SkySQL Cloud Data Suite
Terminal
Admin Console
PhpMyAdmin
MONyog
13
SkySQL Data Suite Building Block
Replicator
Data StoreAdminConsole
14
SkySQL Cloud Data Suite Sep 2012Technologies and Software
Replicator
Data StoreAdminConsole
FRONT ENDFRONT ENDApacheApacheTomcatTomcat
FRONT ENDFRONT ENDApacheApacheTomcatTomcat
BACK ENDBACK ENDSQLiteSQLite
BACK ENDBACK ENDSQLiteSQLite
MariaDBMariaDBMariaDBMariaDB
Linux 3.2 Linux 3.2 (Amazon (Amazon Distro)Distro)
Linux 3.2 Linux 3.2 (Amazon (Amazon Distro)Distro)
MONyogMONyogTrialTrial
MONyogMONyogTrialTrial
PHPMyAdminPHPMyAdminPHPMyAdminPHPMyAdmin
MySQL MySQL ReplicationReplication
MySQL MySQL ReplicationReplicationGlusterFSGlusterFSGlusterFSGlusterFSPacemakerPacemaker
CorosyncCorosyncPacemakerPacemakerCorosyncCorosync XtrabackupXtrabackupXtrabackupXtrabackup
15
SkySQL Cloud Data Suite Sep 2012Data Store
Data Store
PublicPublicSkySQL AMISkySQL AMI
PublicPublicSkySQL AMISkySQL AMI
MariaDB 5.5MariaDB 5.5MyISAM, InnoDB, XtraDB, MyISAM, InnoDB, XtraDB,
Enhanced Optimizer, Virtual Enhanced Optimizer, Virtual ColumnsColumns
MariaDB 5.5MariaDB 5.5MyISAM, InnoDB, XtraDB, MyISAM, InnoDB, XtraDB,
Enhanced Optimizer, Virtual Enhanced Optimizer, Virtual ColumnsColumns
From standalone toFrom standalone to1 Master + 3 Slaves1 Master + 3 SlavesFrom standalone toFrom standalone to1 Master + 3 Slaves1 Master + 3 Slaves
System and DataSystem and Datafile systemsfile systems
System and DataSystem and Datafile systemsfile systems
Pacemaker controls the Pacemaker controls the automatic failoverautomatic failover
Pacemaker controls the Pacemaker controls the automatic failoverautomatic failover
16
SkySQL Cloud Data Suite Sep 2012Administration Console
AdminConsole
GUI to monitor the GUI to monitor the status of the status of the
ClusterCluster
GUI to monitor the GUI to monitor the status of the status of the
ClusterCluster
Extended Extended Monitoring from Monitoring from
MONyog as a trialMONyog as a trial
Extended Extended Monitoring from Monitoring from
MONyog as a trialMONyog as a trial
PhPMyAdmin as PhPMyAdmin as graphical user toolgraphical user toolPhPMyAdmin as PhPMyAdmin as
graphical user toolgraphical user tool
Backup/Restore Backup/Restore to/from S3 using to/from S3 using
XtrabackupXtrabackup
Backup/Restore Backup/Restore to/from S3 using to/from S3 using
XtrabackupXtrabackupStart/Stop, Isolate, Start/Stop, Isolate,
Promote nodesPromote nodesStart/Stop, Isolate, Start/Stop, Isolate,
Promote nodesPromote nodesmysql command mysql command
line interfaceline interfacemysql command mysql command
line interfaceline interface
Automatic failover for the Automatic failover for the admin and monitoring admin and monitoring
serversservers
Automatic failover for the Automatic failover for the admin and monitoring admin and monitoring
serversservers
17
SkySQL Cloud Data Suite Sep 2012Replicator
Replicator
Standard MySQL Standard MySQL Replication in asynchronous Replication in asynchronous
and semi-synchronous and semi-synchronous modemode
Standard MySQL Standard MySQL Replication in asynchronous Replication in asynchronous
and semi-synchronous and semi-synchronous modemode
GlusterFS on ext4 to replicate GlusterFS on ext4 to replicate the data collection and the data collection and
monitoringmonitoring
GlusterFS on ext4 to replicate GlusterFS on ext4 to replicate the data collection and the data collection and
monitoringmonitoring
Automatic failover of database Automatic failover of database servers and console servers and console
componentscomponents
Automatic failover of database Automatic failover of database servers and console servers and console
componentscomponents
18
SkySQL Data Suite Provisioning
AMI
userdata.sh
User Account
Variable definition - they Variable definition - they will be used later when will be used later when other scripts are calledother scripts are called
Variable definition - they Variable definition - they will be used later when will be used later when other scripts are calledother scripts are called
On-the-fly configuration On-the-fly configuration files, such as my.cnffiles, such as my.cnf
On-the-fly configuration On-the-fly configuration files, such as my.cnffiles, such as my.cnf
Execute predefinedExecute predefinedscripts already in the scripts already in the
AMIAMI
Execute predefinedExecute predefinedscripts already in the scripts already in the
AMIAMI
Adaptive, distributed Adaptive, distributed setupsetup
Adaptive, distributed Adaptive, distributed setupsetup
Elastic IP onElastic IP onthe Master Serverthe Master Server
Elastic IP onElastic IP onthe Master Serverthe Master Server
Currently, the Administration Currently, the Administration Console is on the Master serverConsole is on the Master server
Currently, the Administration Currently, the Administration Console is on the Master serverConsole is on the Master server
19
SkySQL Data Suite Deployment
20
SkySQL Data Suite Deployment
21
Automated Provisioning in AWS
22
Starting Here!
You can try it for You can try it for FREE! Even without FREE! Even without
ananAWS accountAWS account
You can try it for You can try it for FREE! Even without FREE! Even without
ananAWS accountAWS account
23
Simplified Interface
Immediate HelpImmediate HelpImmediate HelpImmediate Help
Real Time CheckReal Time CheckReal Time CheckReal Time Check
24
Advanced Configuration
25
Cost Estimation
26
Delivery Mechanism
27
Online Deployment
The keys will be The keys will be discarded immediatelydiscarded immediatelyafter the provisioningafter the provisioning
The keys will be The keys will be discarded immediatelydiscarded immediatelyafter the provisioningafter the provisioning
28
Extra AWS & Contact Info
29
Welcome Email
mysql command linemysql command linemysql command linemysql command line
Administration ConsoleAdministration ConsoleAdministration ConsoleAdministration Console
ssh connectionssh connectionssh connectionssh connection
How to remove SCDSHow to remove SCDSHow to remove SCDSHow to remove SCDS
An inventory of the An inventory of the AWS componentsAWS components
An inventory of the An inventory of the AWS componentsAWS components
IMPORTANT!IMPORTANT!Your pem keyYour pem keyIMPORTANT!IMPORTANT!Your pem keyYour pem key
30
Job Done!
31
SkySQL Cloud Data Suite in Amazon EC2
32
SecurityIdentity Access Manager Roles
• EIP AddressAssociate/Disassociate, Describe
• InstancesStart, Stop, DescribeAttributes, Status, Tags, Net IFCreate, Describe
• S3 BucketsList, Create, Delete
• ObjectsGet, Put, List, Delete
33
SecuritySecurity Groups
• ssh
• http
• MySQL
• MONyog
34
Storage
35
Components for High Availability in EC2
36
Automatic Failover with PacemakerCRM Configuration
Replicator
Data StoreAdminConsole
37
Automatic Failover with PacemakerCRM Configuration
Replicator
Data StoreAdminConsole
[root@node2 init.d]# [root@node2 init.d]# crm statuscrm status============Last updated: Sat Dec ============Last updated: Sat Dec 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via crm_attribute on node2Stack: openaisCurrent DC: node2 - partition crm_attribute on node2Stack: openaisCurrent DC: node2 - partition with quorumVersion: 1.1.7-6.el6-with quorumVersion: 1.1.7-6.el6-148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 expected votes14 Resources configured.============Online: [ node2 expected votes14 Resources configured.============Online: [ node2 node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL] node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL] Masters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIPMasters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIP(lsb:amazon_master_eip):(lsb:amazon_master_eip): Started node2 Clone Set: apache-php-Started node2 Clone Set: apache-php-myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 ] Resource Group: monyog monyog-fs] Resource Group: monyog monyog-fs (ocf::custom:Filesystem):(ocf::custom:Filesystem):
Started node2 monyog-serviceStarted node2 monyog-service (lsb:MONyogd):(lsb:MONyogd): Started Started node2 Resource Group: admin-console console-fsnode2 Resource Group: admin-console console-fs(ocf::custom:Filesystem):(ocf::custom:Filesystem): Started node2 SkySQL-monitorStarted node2 SkySQL-monitor(lsb:skysql_monitor):(lsb:skysql_monitor): Started node2 Tomcat7Started node2 Tomcat7(ocf::heartbeat:tomcat):(ocf::heartbeat:tomcat): Started node2Started node2
[root@node2 init.d]# [root@node2 init.d]# crm statuscrm status============Last updated: Sat Dec ============Last updated: Sat Dec 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via crm_attribute on node2Stack: openaisCurrent DC: node2 - partition crm_attribute on node2Stack: openaisCurrent DC: node2 - partition with quorumVersion: 1.1.7-6.el6-with quorumVersion: 1.1.7-6.el6-148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 expected votes14 Resources configured.============Online: [ node2 expected votes14 Resources configured.============Online: [ node2 node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL] node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL] Masters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIPMasters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIP(lsb:amazon_master_eip):(lsb:amazon_master_eip): Started node2 Clone Set: apache-php-Started node2 Clone Set: apache-php-myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 ] Resource Group: monyog monyog-fs] Resource Group: monyog monyog-fs (ocf::custom:Filesystem):(ocf::custom:Filesystem):
Started node2 monyog-serviceStarted node2 monyog-service (lsb:MONyogd):(lsb:MONyogd): Started Started node2 Resource Group: admin-console console-fsnode2 Resource Group: admin-console console-fs(ocf::custom:Filesystem):(ocf::custom:Filesystem): Started node2 SkySQL-monitorStarted node2 SkySQL-monitor(lsb:skysql_monitor):(lsb:skysql_monitor): Started node2 Tomcat7Started node2 Tomcat7(ocf::heartbeat:tomcat):(ocf::heartbeat:tomcat): Started node2Started node2
Node 2Node 2Node 2Node 2
Elastic IPElastic IPElastic IPElastic IP
Public IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPs
admin-console and admin-console and MONyog groupMONyog group
admin-console and admin-console and MONyog groupMONyog group
38
The Administration Console
39
Admin Console Dec 2012
40
Admin Console Dec 2012
Access to Access to MONyogMONyog
Access to Access to MONyogMONyog
MasterMasterServerServerMasterMasterServerServer
ContextContextSensitiveSensitiveContextContext
SensitiveSensitiveContextContext
SensitiveSensitiveContextContext
SensitiveSensitive
Customisable Customisable GraphsGraphs
Customisable Customisable GraphsGraphs
41
Admin Console Dec 2012
Access to Access to PhpMyAdminPhpMyAdmin
Access to Access to PhpMyAdminPhpMyAdmin
Customisable Customisable CommandsCommands
Customisable Customisable CommandsCommands
Backup/RestoreBackup/RestoreBackup/RestoreBackup/Restore
42
Administration Console Dec 2012Customizable Graphs
Name SQL Icon Delta Avg Chart
Monitors
43
Administration Console Dec 2012Customizable Commands
Desc Icon Order
Commands
Desc Icon Script
Steps
CommandStep
44
Automated Backup/Restore with S3
45
Automated BackupUsing Xtrabackup from the Admin Console
46
Automated BackupUsing Xtrabackup from the Admin Console
47
Automated BackupUsing Xtrabackup from the Admin Console
48
Why is the SkySQL Cloud Data Suite important for you?
• You can test MariaDB/MySQL on AWS for FREE
• You can use and fully control a ready made highly available solution AWS
• You can compare MySQL on EC2 and RDS
• You can study/review security and availability aspects in AWS
• You can use some of the SCDS components in your environment
49
SkySQL Cloud Data Suite & OpenStack ?
• Nova, glance, swift, keystone, horizon, quantum ceph
• Position for public/private/hybrid cloud
• Rackspace,HPcloud,VMware ...CloudWatt, Numergy
• Our solution will be there
•
50
Is not Amazon RDS enough for my needs ?It is fully managed !
• Provisioning
• Patching
• Backup/ PointInTIme recovery
• Replicas
• HA / Failover
• monitoring
51
Or maybe Google Cloud SQL ?
• New offer
• Similar to AWS RDS
• Rely on storage feature for HA
• Fully managed
52
Private/Hybrid cloudEucalyptus / VMware Vcloud
• They own a huge market share
• Eucalyptus is tied to Amazon / open core
• VMware is tied to VMware and fully closed
• Excellent solutions but ...
53
For more information...
• Try the Configurator and How-To Video: http://cloud.skysql.com
• The Admin Console: http://www.skysql.com/downloads/community/skysql-console
• Forums: https://www.skysql.com/forums/skysql-cloud-data-suite
• Web: https://www.skysql.com/cloud
• Bugs: http://bugs.skysql.com
• Try it! http://config.skysql.com
• Free SkySQL account available for testing
• Test on your account