Intro To The SkySQL Cloud Data Suite

V1212.01

Ivan Zoratti

SkySQL Data SuiteA New Open Source Approach to MySQL Distributed Systems

Wednesday, 5 December 12

Who is Ivan

?Wednesday, 5 December 12

SkySQL•#1 provider of MySQL-based

solutions, for software and services

•Backed by Monty Program AB and 3rd parties active in the MySQL ecosystem

•Committed to provide the best solutions around MySQL for the Community and for the Enterprise


PAGE

Agenda

• SkySQL Cloud Data Suite Architecture

• SkySQL Cloud Data Suite on Amazon EC2

• Components for automated provisioning in AWS

• Components for high availability in EC2

• The Administration Console

• Automated Backup/Restore with S3

4Wednesday, 5 December 12

PAGE

Agenda

• SkySQL Cloud Data Suite Architecture

• SkySQL Cloud Data Suite on Amazon EC2

• Components for automated provisioning in AWS

• Components for high availability in EC2

• The Administration Console

• Automated Backup/Restore with S3

4

WARNING!We have lots of slides


Intro to SkySQL Data SuiteWednesday, 5 December 12

PAGE

SkySQL Data Suite

6

SkySQL Data Suite

High Availability

Full-Text

Replication& MHA

Galera

DRBD

Analytics

Carrier Grade

Online Apps.

SharedStorage


PAGE

SkySQL Cloud Data Suite

7

SkySQL Data Suite

High Availability

Full-Text

Replication

Galera

DRBD

Analytics

Carrier Grade

Online Apps.

SharedStorage


PAGE

Cloud Data Suitevs

Enterprise Data Suite

8

Cloud Data Suite

•Deployed in Amazon AWS•Automatic configuration and provisioning•MariaDB 5.5•MySQL Replication with Pacemaker/Corosync

•Administration Console with Backup/Restore•SkySQL Enterprise Monitoring aka MONyog - Trial•PhpMyAdmin•Community Support•Development Release•Fully OSS (with MONyog Trial)

Enteprise Data Suite

•Deployed in Clouds, virtualised env. and bare OS•Standard download packages•MariaDB, MySQL, Percona Servers•MySQL Replication with MHA, DRBD, Galera or shared storage and Pacemaker/Corosync

•ZManda Recovery Manager•SkySQL Enterprise Monitory aka MONyog•SQLyog•Enterprise Technical Support•Production Release•Mix of OSS and Commercial


PAGE

SkySQL Cloud Data Suite Sep 2012

1. Configure your DB system on config.skysql.com or cloud.skysql.com

2. Deploy the DB system:• Online• Configuration pull from AWS• Manually

3. Connect the DB system toyour application

1

2

3


PAGE

SkySQL Data Suite Dec 2012

• Rel. 1.1 with bug fixing

• New Administration Console with: • rpm package and installation on Linux OS

• Administration and monitoring of replicated and standalone servers

• Customisable graphs

• Customisable commands


SkySQL Data Suite ArchitectureWednesday, 5 December 12

PAGE

SkySQL Data Suite Configurator

12

AMI

User Account


PAGE


12

ConfigurationEngine

RulesRepository

AMI

User Account


PAGE


12

ConfigurationEngine

RulesRepository

ProvisioningQueue

AMI

User Account


PAGE


12

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

ProvisioningQueue

AMI

User Account


PAGE


12

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

ProvisioningQueue

DeploymentQueue

AMI

User Account


PAGE


12

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

DeploymentEngine

ProvisioningQueue

DeploymentQueue

AMI

User Account


PAGE


12

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

DeploymentEngine

ProvisioningQueue

DeploymentQueue

AMI

DatabaseDatabase

Database Database Database

userdata.sh

User Account

Deployment triggered by the first boot of each instance


PAGE

SkySQL Cloud Data Suite

13

DatabaseDatabase


$ ssh -i SDS193500.pem [email protected][ec2-user@node1 ~]$ ps -efUID PID PPID C STIME TTY TIME CMDroot 1157 1 0 Nov29 ? 00:00:02 /usr/sbin/glusterd --pid-file=/var/run/glusterd.pidroot 1840 1 2 Nov29 ? 00:11:34 /usr/sbin/glusterfsd -s localhost --volfile-id glust...root 1889 1 0 Nov29 ? 00:00:50 /usr/sbin/glusterfsd -s localhost --volfile-id glust...root 1895 1 0 Nov29 ? 00:00:01 /usr/sbin/glusterfs -s localhost --volfile-id gluste...root 1901 1 0 Nov29 ? 00:00:01 /usr/sbin/glusterfs -s localhost --volfile-id gluste...root 4484 1 8 Nov29 ? 00:36:13 /usr/sbin/glusterfs --volfile-id=/gluster-monyog --......root 1311 1 0 Nov29 ? 00:00:34 corosync220 1317 1311 0 Nov29 ? 00:00:12 /usr/libexec/pacemaker/cibroot 1318 1311 0 Nov29 ? 00:00:05 /usr/libexec/pacemaker/stonithdroot 1319 1311 0 Nov29 ? 00:00:12 /usr/lib64/heartbeat/lrmd220 1320 1311 0 Nov29 ? 00:00:03 /usr/libexec/pacemaker/attrd220 1321 1311 0 Nov29 ? 00:00:01 /usr/libexec/pacemaker/pengine220 1322 1311 0 Nov29 ? 00:00:05 /usr/libexec/pacemaker/crmd...root 1534 1 0 Nov29 ? 00:00:00 sendmail: accepting connectionssmmsp 1542 1 0 Nov29 ? 00:00:00 sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue...root 2293 1 0 Nov29 ? 00:00:00 /bin/sh /usr/bin/mysqld_safe --defaults-file=/etc/my.cnf --p...mysql 2791 2293 0 Nov29 ? 00:00:28 /usr/sbin/mysqld --defaults-file=/etc/my.cnf --basedir=/usr ......root 3467 1 0 Nov29 ? 00:00:01 /usr/sbin/httpd -DSTATUS -f /etc/httpd/conf/httpd.confapache 25416 3467 0 Nov29 ? 00:00:00 /usr/sbin/httpd -DSTATUS -f /etc/httpd/conf/httpd.conf...root 4578 1 1 Nov29 ? 00:07:56 /usr/local/MONyog/bin/MONyog-bin -s

Terminal

Admin Console

PhpMyAdmin

MONyog


PAGE

SkySQL Data Suite Building Block

14

Replicator

DatabaseDatabase


Data StoreAdminConsole


SkySQL Cloud Data Suite Sep 2012Technologies and Software

Replicator

DatabaseDatabase



FRONT ENDApacheTomcat

BACK ENDSQLite

MariaDB

Linux 3.2 (Amazon Distro)

MONyogTrial

PHPMyAdmin

MySQL ReplicationGlusterFSPacemaker

Corosync Xtrabackup


SkySQL Cloud Data Suite Sep 2012Data Store

DatabaseDatabase


Data Store

PublicSkySQL AMI

MariaDB 5.5MyISAM, InnoDB, XtraDB, Enhanced

Optimizer, Virtual Columns

From standalone to1 Master + 3 Slaves

System and Datafile systems

Pacemaker controls the automatic failover


SkySQL Cloud Data Suite Sep 2012Administration Console

DatabaseDatabase


AdminConsole

GUI to monitor the status of the

Cluster

Extended Monitoring from MONyog as a trial

PhPMyAdmin as graphical user tool

Backup/Restore to/from S3 using

XtrabackupStart/Stop, Isolate,

Promote nodesmysql command

line interface

Automatic failover for the admin and

monitoring servers


SkySQL Cloud Data Suite Sep 2012Replicator

Replicator

DatabaseDatabase




Replicator

DatabaseDatabase


Standard MySQL Replication in asynchronous and semi-

synchronous mode



Replicator

DatabaseDatabase



synchronous modeGlusterFS on ext4 to replicate the

data collection and monitoring



Replicator

DatabaseDatabase



synchronous modeGlusterFS on ext4 to replicate the

data collection and monitoring

Automatic failover of database servers and console components


PAGE

SkySQL Data Suite Provisioning

19

AMI

DatabaseDatabase


userdata.sh

User Account


PAGE


19

AMI

DatabaseDatabase


userdata.sh

User AccountVariable definition - they will

be used later when other scripts are called


PAGE


19

AMI

DatabaseDatabase


userdata.sh



On-the-fly configuration files, such as my.cnf


PAGE


19

AMI

DatabaseDatabase


userdata.sh




Execute predefinedscripts already in the AMI


PAGE


19

AMI

DatabaseDatabase


userdata.sh





Adaptive, distributed setup


PAGE


19

AMI

DatabaseDatabase


userdata.sh






Elastic IP onthe Master Server


PAGE


19

AMI

DatabaseDatabase


userdata.sh






Elastic IP onthe Master Server

Currently, the Administration Console is on the Master server


PAGE

SkySQL Data Suite Deployment


PAGE


20

Database


PAGE


20

Database

Database

Database

Database

Database


PAGE


20

Database

Database

Database

Database

Database

Database

Database

Database

Database


PAGE


20

Database

Database

Database

Database

Database

Database

Database

Database

Database

Database

Database

Database

Database

Database


PAGE



Automated Provisioning in AWS


PAGE

Starting Here!

23

You can try it for FREE! Even without an

AWS account


PAGE

Simplified Interface

24

Immediate Help

Real Time Check


PAGE

Advanced Configuration


PAGE

Cost Estimation


PAGE

Delivery Mechanism


PAGE

Online Deployment

28

The keys will be discarded immediatelyafter the provisioning


PAGE

Extra AWS & Contact Info


PAGE

Welcome Email

30

mysql command line

Administration Console

ssh connection

How to remove SCDS

An inventory of the AWS components

IMPORTANT!Your pem key


PAGE

Job Done!


SkySQL Cloud Data Suite in Amazon EC2


PAGE

SecurityIdentity Access Manager Roles

33

• EIP Address• Associate/Disassociate, Describe

• Instances• Start, Stop, Describe• Attributes, Status, Tags, Net IF

• Create, Describe

• S3 Buckets• List, Create, Delete• Objects

• Get, Put, List, Delete


SecuritySecurity Groups

• ssh

• http

• MySQL

• MONyog


PAGE

Storage


Components for High Availability in EC2


PAGE

Automatic Failover with PacemakerCRM Configuration

37

Replicator

DatabaseDatabase

Database Database



PAGE

Automatic Failover with PacemakerCRM Configuration

38

Replicator

DatabaseDatabase

Database Database


[root@node2 init.d]# crm status============Last updated: Sat Dec 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via crm_attribute on node2Stack: openaisCurrent DC: node2 - partition with quorumVersion: 1.1.7-6.el6-148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 expected votes14 Resources configured.============

Online: [ node2 node3 node1 node4 ]

Master/Slave Set: ms_resMySQL [resMySQL] Masters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIP (lsb:amazon_master_eip): Started node2 Clone Set: apache-php-myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 ] Resource Group: monyog monyog-fs (ocf::custom:Filesystem): Started node2 monyog-service (lsb:MONyogd): Started node2 Resource Group: admin-console console-fs (ocf::custom:Filesystem): Started node2 SkySQL-monitor (lsb:skysql_monitor): Started node2 Tomcat7 (ocf::heartbeat:tomcat): Started node2

Node 2

Elastic IP

Public IPsPublic IPsPublic IPs

admin-console and MONyog group


The Administration ConsoleWednesday, 5 December 12

PAGE

Admin Console Dec 2012


PAGE


41

Access to MONyog

MasterServer

ContextSensitiveContext

Sensitive

Customisable Graphs


PAGE


42

Access to PhpMyAdmin

Customisable Commands

Backup/Restore


Administration Console Dec 2012Customizable Graphs

Name SQL Icon Delta Avg Chart

Monitors


Administration Console Dec 2012Customizable Commands

Desc Icon Order

Commands

Desc Icon Script

Steps

CommandStep


Automated Backup/Restore with S3Wednesday, 5 December 12

PAGE

Automated BackupUsing Xtrabackup from the Admin Console


PAGE



PAGE

Why is the SkySQL Cloud Data Suite important for you?

49

• You can test MariaDB/MySQL on AWS for FREE

• You can use and fully control a ready made highly available solution AWS

• You can compare MySQL on EC2 and RDS

• You can study/review security and availability aspects in AWS

• You can use some of the SCDS components in your environment


PAGE

For more information...

50

• Try the Configurator and How-To Video: http://cloud.skysql.com

• The Admin Console: http://www.skysql.com/downloads/community/skysql-console

• Forums: https://www.skysql.com/forums/skysql-cloud-data-suite

• Web: https://www.skysql.com/cloud

• Bugs: http://bugs.skysql.com

• Try it! http://config.skysql.com• Free SkySQL account available for testing• Test on your account


http://cloud.skysql.com








https://www.skysql.com/forums/skysql-cloud-data-suite




https://www.skysql.com/cloud

https://www.skysql.com/cloud

http://bugs.skysql.com

http://bugs.skysql.com

http://config.skysql.com

http://config.skysql.com

Thank You!

[email protected]

www.slideshare.net/izorattiwww.skysql.com


mailto:[email protected]








Intro To The SkySQL Cloud Data Suite

Technology

Transcript of Intro To The SkySQL Cloud Data Suite