MySQL DBaaS with OpenStack Trove

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

MySQL DBaaSWith OpenStack Trove

Matt LordMySQL Product Manager@mattalord

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 3

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


• “Since 2000, 52% of the companies in the Fortune 500 have either gone bankrupt, been acquired, ceased to exist, or dropped off the list• Digitalization of business is a key factor in this accelerated pace of change.

Information flows faster• Cloud is the foundation for digital transformation– Ubiquity and ease of adoption– Unlimited and dynamic capacity– Helps you innovate faster”

Ray Wang, Constellation Research

Cloud: ”Single Most Disruptive Technology”

Source: http://www.forbes.com/sites/oracle/2014/12/19/ray-wang-cloud-is-the-foundation-for-digital-transformation/

4


MySQL: Most Popular Open Source Database in the Cloud

5


Program Agenda

An Introduction to OpenStack

An Introduction to Trove

Why MySQL Enterprise Edition?

Integrating MySQL Enterprise with Trove

What’s Coming Next?

1

2

3

4

5

7Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

An Introduction to OpenStack


The OpenStack Mission: to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.


What is OpenStack?• OpenStack is an open source cloud operating system designed to control

large scale cloud deployments• OpenStack controls compute, storage, network, identity management,

orchestration, and much more


OpenStack Components

• Compute Service : Nova• Object Storage Service : Swift• Block Storage Service : Cinder• Image Service : Glance• Identity Service : Keystone• Network Service : Neutron• Telemetry Service : Ceilometer

• Orchestration Service : Heat• Database Service : Trove• Dashboard : Horizon• Data Processing : Sahara• Bare-Metal Provisioning : Ironic

10


OpenStack Overview


• Virtually all of the OpenStack components require access to back-end database services to store state and configuration information.

• MySQL is generally considered to be the de facto database for OpenStack

• MySQL Cluster can also be used for higher availability

Basic OpenStack Architecture

12


OpenStack is About Abstraction and Integration

13

Block Storage service

Cinder

Network service

Neutron

Compute service

NovaApplication Programming Interfaces

Plugins thenImplement theAPI (partial list)

Common Interface – OpenStack API


Goals: OpenStack With Oracle Linux and Oracle VM

Interoperability

• Help make OpenStack a first class solution for enterprise customers

• Work with the OpenStack community to support enterprise applications in an OpenStack cloud

• Provide choice for managing hypervisors with Oracle Linux and Oracle VM

• Ensure Oracle Linux and Oracle VM can work with any OpenStack solution customers choose

• Work with third party vendors to offer joint solutions

14


OpenStack Across Oracle’s Portfolio

HorizonCloud Management

Oracle Linux , Oracle Solaris, Oracle VM,

NovaCompute Virtualization

Oracle Linux , Oracle Solaris, Oracle Virtual

Networking

NeutronCloud Networking

Oracle Linux, Oracle Solaris, Oracle ZS3, Oracle Axiom, Oracle StorageTek

Tape, Oracle FS1

Cinder/SwiftCloud Storage

Oracle Linux, Oracle Solaris, Oracle VM

Templates

GlanceImage Deployment

15


End to End Full Stack Support

Control nodeCompute node nCompute node 1

VMVM

VMVM

VMVM

Clusterware


Oracle OpenStack for Oracle Linux Value Proposition • Fully Integrated Enterprise Cloud Solutions with Single Support– Integrated support for all components in the stack including applications, database,

middleware, guest OS, host OS, Virtualization and OpenStack– Large eco-system of OL/OVM HCL of 3rd party servers and storage– Large partner network for testing and supporting plug-ins and ISV modules

• No additional support cost for OpenStack– For customers with Premier Support for Oracle Linux or Oracle VM or Oracle System

• Enterprise Class (and also less demanding non-enterprise workloads) –OpenStack will not succeed without broad adoption in the enterprise– Focus on deployment, upgrade, stability and supportability for OpenStack using

Oracle’s experience in implementing the world’s toughest enterprise workloads


OpenStack Momentum• Launched in 2010, OpenStack has grown to become one of the largest open source

project ever created. Some numbers from http://www.stackalytics.com/ as of May 2015:– Over 400 companies contributed code to OpenStack (Kilo)– Almost 2700 individuals contributed code to the project (Kilo)

Austin Bexar Cactus Diablo Essex Folsom Grizzly Havana Icehouse Juno0

20

40

60

80

100

120

140

Companies Contributing Up to Juno

Austin Bexar Cactus Diablo Essex Folsom Grizzly HavanaIcehouse Juno0

200

400

600

800

1000

1200

1400

1600

Individuals Contributing Up to Juno


OpenStack Momentum

Supporting Companies: http://www.openstack.org/foundation/companies/

19

http://www.openstack.org/foundation/companies/


“@WalmartLabs is now running in excess of 100,000 cores of OpenStack on its compute layer. And that’s growing by the day. It’s also the technology that ran parent company Walmart’s prodigious Cyber Monday and holiday season sales operations. If that’s not production, I’m not sure what is.”Barb Darrow, Gigaom Research

OpenStack Comes up Huge for Walmart

20


Fujitsu Migrates its Internal IT to OpenStack• Fujitsu will migrate its internal IT to OpenStack to save 35 billion yen• Will involve about 640 large-scale and complex systems across 13,000

servers globally. Include CRM/sales-support systems, SCM/ECM systems, corporate systems, such as for HR and accounting, and a global communications platform for 170,000 employees across some 540 group companies.

“The new cloud services platform will be used to quickly build systems that, by employing the latest technologies, such as mobile technologies and big data, can respond quickly to a changing business environment.”

21


Top 5 Business Drivers for OpenStack

Source: http://superuser.openstack.org/articles/openstack-user-survey-insights-november-2014

22


OpenStack Deployments


23


Workloads & Databases


24


ApplicationPhotobucket hosts more than 10 billion images and videos from 100 million registered members. Users upload more than 4 million images and videos per day from the Web and mobile devices.

Key Business BenefitPhotobucket chose Openstack to improve operational efficiencies. Server utilization was only in the 15%-20% range for 800 bare-metal servers. Now, Photobucket operates MySQL databases serving content from 350-400 physical servers. New servers are deployed in 2-3 min as opposed to 2-3 hours.

Why OpenStack“Everyone's walking around with sort of a smile on their face, because they're not fighting to keep the infrastructure running.”

Jay KistlerDirector of Operations, Photobucket

25


ApplicationCERN, the European Organization for Nuclear Research, is one of the world’s largest and most respected centers for scientific research.

Key Business BenefitAs CERN’s data processing needs continued increasing while staffing remained fixed, the organization decided that cloud computing could provide a complementary approach to deliver services at scale to its physicists.

CERN uses Openstack for its cloud, and built a Database as-a-Service with MySQL and Oracle VM.

Why OpenStack“Cloud technology has allowed us to be much more responsive to our user community allowing them to explore the frontiers of science without waiting for hardware to be delivered and configured.”

Tim Bell, Manager of Infrastructure Services, CERN

CERN

26


Getting Started With OpenStack• http://www.openstack.org/software/start/

• Oracle OpenStack for Oracle Linux:http://www.oracle.com/technetwork/server-storage/openstack/linux/downloads/index.html?ssSourceSiteId=ocomen 27

http://www.openstack.org/software/start/

http://www.openstack.org/software/start/

http://www.oracle.com/technetwork/server-storage/openstack/linux/downloads/index.html?ssSourceSiteId=ocomen




An Introduction to Trove


Current State of Database Deployments

• Ever Growing Database Population• Multiple versions and patch levels • Compliance challenges• Poor resource utilization• High cost of deployment and operation

Siloed, Dispersed, Varied and Complex

29


DBaaS Key Capabilities

Self Service Metering Elasticity

Service Catalog High Availability Security

End-users can provision, monitor & manage for greater agility

Allows chargeback for cost allocation

Support dynamic workloads with minimal excess capacity

Standardized database services for rapid, error-free deployment

Avoiding costly unplanned downtime

Protecting against external attacks, misuse of information & ensuring regulatory compliance

30


Database as a Service (DBaaS)

• Reduces Database Sprawl• Supports Rapid Provisioning• Enhances Security• Improves Performance & Availability• Automates Management Improves agility, security & reduces TCO Foundation for digital transformation, helps you innovate faster

With MySQL Enterprise Edition

MarketsandMarkets forecasts the Cloud Database and DBaaS Market to grow from $1.07 billion in 2014 to $14.05 billion by 2019, at a Compound Annual Growth Rate (CAGR) of 67.30% during the forecast period of 2014-2019.

31


DBaaS with OpenStack Trove• Trove provides scalable and reliable cloud database as a service

provisioning functionality • Cloud users and DBAs can provision and manage multiple MySQL Enterprise

Edition instances• Automates complex administrative tasks including deployment,

configuration, patching, backups & restores and monitoring• Became an official part of OpenStack with the Icehouse release in April

2014. – Improvements in each subsequent & upcoming release

32


Trove Overview

• Built on OpenStack– Backups to Swift– Images in Glance– Nova management– Data in Cinder

• Major components– API – Taskmanager – Conductor– Guest Agent


Provisioning Models

PDB

Pre-provisioned hardware

Oracle 11g


Mongo DB


MySQL

OpenStack Cloud

Oracle 12c Container Database

PDB PDB

Trove DBaaS

TroveInstance

TroveInstance

Oracle

TroveInstance

MySQ

L

Cluster

NovaInstance

Oracle

NovaInstance

Oracle

Trove Instance

MySQL Cluster

Single Instance Provisioning

Cluster Provisioning

Multi-tenant DBMS

Provisioning

Provision from Inventory

PDB

PDB

* Oracle Database 11g and 12c are currently only supported in Tesora DBaaS Enterprise

*

*


Trove Stack

• Built on OpenStack– Backups to Swift– Images in Glance– Nova management– Data in Cinder

• Major components– API – Taskmanager – Conductor– Guest Agent


Commercial Alternative : Tesora DBaaS Editions

OpenStack Trove Community Edition

Enterprise DBaaS Features

Early Access to Upstream

24/7 Support

Guest Images Proprietary DBs

Simplified Install/Config

Community Code

Enterprise Edition

Simplified Install/Config

Extensive Testing/Bug Fixes

Guest Images O/S DBs

Community CodeCommunity Code

Extensive Testing/Bug Fixes

Guest Images O/S DBs

Enhanced GUI Management

More Trove Features, Earlier

36


Why MySQL Enterprise?


Why MySQL Enterprise Edition?In Addition to all the MySQL Features you Love

Insure Your Deployments

Get the Best Results

Delight Customers

Improve Performance & Scalability

Enhance Agility & Productivity

Reduce TCO

Mitigate Risks

Get Immediate

Help if/when Needed

Increase Customer

Satisfaction

38


Management ToolsAdvanced Features Support

• Scalability• High Availability• Security• Audit• Encryption

•Monitoring• Backup• Development• Administration•Migration

• Technical Support• Consultative Support•Oracle Certifications

MySQL Enterprise Edition

39


MySQL Monitoring on OpenStack

• Easy integration with automation tools• Real-time MySQL performance and availability monitoring• Visually find & fix problem queries• Expert advice on MySQL setup and configuration• Advanced host/OS monitoring• Stay on top of backups and security• Ensure all instances are optimal

MySQL Enterprise Monitor & Query Analyzer

MySQL Enterprise Monitor 3.0Saves you time. Fixes problems

you cannot find yourself.

40


• Online, non-locking backup and recovery – Complete MySQL instance backup (data and config)– Partial backup and restore

• Direct Cloud storage backups via Swift API• Incremental backups & Point-in-time recovery• Advanced compressed and encryption• Backup to tape (SBT)• Backup validation• Optimistic backups• Cross-Platform (Windows, Linux, Unix)

MySQL Enterprise BackupData Protection on OpenStack

41


• PAM (Pluggable Authentication Modules)– Access external authentication methods– Standard interface (Unix, LDAP, Kerberos, others)– Proxied and non-proxied users

• Windows– Access native Windows services– Authenticate users already logged into Windows (Windows Active Directory)

• Pluggable Authentication API

MySQL Enterprise AuthenticationData Protection on OpenStack

42


• MySQL encryption libraries – Symmetric encryption AES256– Public-key / asymmetric cryptography

• Key management– Generate public and private keys– Key exchange methods: RSA, DSA, DH

• Sign and verify data– Cryptographic hashing for digital signing, verification, & validation

MySQL Enterprise EncryptionData Protection on OpenStack

43


MySQL Enterprise FirewallData Protection on OpenStack

• Real Time Protection– Queries analyzed and matched against White List– Automated creation of approved list of SQL

command patterns on a per user basis

• Blocks SQL Injection Attacks– Positive Security Model

• Block Suspicious Traffic– Out of Policy Transactions detected & blocked

• Transparent– No changes to application required

44


Data Protection on OpenStackMySQL Enterprise Audit

• Out-of-the-box logging of connections, logins, and query

• User defined policies for filtering, and log rotation

• Dynamically enabled, disabled: no server restart

• XML-based audit stream per Oracle Audit Vault specification

• Easily implemented via MySQL 5.6 Audit API

Adds regulatory compliance to MySQL applications (HIPAA, Sarbanes-Oxley, PCI, etc.)

45


• Provides 20x better scalability

• Plugin improves sustained performance as user connections grow

MySQL Scalability in OpenStackMySQL Enterprise Scalability: Thread Pool

Reduces the number of provisioned machine instances needed to meet

performance requirements

46


MySQL HA in OpenStack

• High Availability– Server monitoring with auto-promotion and

transparent application failover

• Optionally scale-out through sharding– Application provides shard key– Range or Hash– Tools for resharding– Global updates & tables

• Fabric-aware connectors rather than proxy: Python, Java, PHP, .NET, C (labs)– Lower latency, bottleneck-free

• Server provisioning using OpenStack – Supports the Nova and Neutron APIs

MySQL Enterprise High Availability + Sharding-Based Scale-out

MySQL Fabric

Connector

Application

Read-slaves

mappings

SQL

HA group

Read-slaves

HA group

Connector

Application

47


MySQL Enterprise Support• Largest MySQL engineering and support organization• Backed by the MySQL developers• World-class support, in 29 languages• Hot fixes & maintenance releases• 24x7x365• Unlimited incidents• Consultative support• Global scale and reach Get immediate help for any MySQL

issue, plus expert advice

48


MySQL Consultative SupportMake the Most of your Deployments

• Remote troubleshooting• Replication review• Partitioning review• Schema review• Query review• Performance tuning• ...and more

49


Scaling, Security, and Data ProtectionMySQL Enterprise ScalabilityMySQL Enterprise MonitorMySQL Enterprise BackupMySQL Enterprise SecurityMySQL Enterprise EncryptionMySQL Enterprise AuditMySQL Enterprise AuthenticationMySQL Enterprise High AvailabilityOracle Enterprise Manager for MySQL

50


Integrating MySQL Enterprise & Trove


MySQL Enterprise & Trove• Create a MySQL Enterprise machine image for use with Trove– Pull a standard image from Glance and add/install:• MySQL Enterprise Server• Default MySQL config (/etc/mysql/my.cnf) that enables all Enterprise Plugins– Can later use a Trove configuration group to sync and manage options across all instances

• MySQL Enteprise Backup• MySQL Enteprise Monitor Agent– Configured to know how to reach out to the MySQL Enterprise Monitor Server

• Trove Guest Agent– Configured to know how to reach the Trove Server (API, Taskmanager, Conductor)– Customized backup strategy definitions for MySQL Enterprise Backup• trove/guestagent/strategies/backup/mysql_impl.py• See: http://dev.mysql.com/doc/mysql-enterprise-backup/3.12/en/mysqlbackup.usage.html

https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/mysql_impl.py



http://dev.mysql.com/doc/mysql-enterprise-backup/3.12/en/mysqlbackup.usage.html


• Store customized MySQL Enterprise image in Glance–Or skip all of the image related steps with official certified Oracle

Linux + MySQL Enterprise Trove images

• Create necessary entries in Keystone• User, role, tenant, service, endpoint

• Create MySQL datastore within Trove– Use trove-manage client • Sync Trove information to OpenStack’s MySQL data store (trove-manage

db_sync)• Create MySQL datastore using new Trove machine image in Glance

MySQL Enterprise & Trove


• Configure MySQL Enterprise Backups– Specify MySQL Enterprise Backup as tool for backups

(/etc/trove/trove.conf): [mysql]

backup_strategy=mysqlbackup

backup_incremental_strategy={'mysqlbackup': 'incremental'}

• Instances can then be created with trove client or via Horizon web UI




• Launch MySQL EE instance– Flavor from Nova– Cinder volume for the datadir

• Initialize the instance– Specify default accounts– Specify default schemas

• Optionally restore from a backup


What’s Next?


Plans for MySQL Enterprise• MySQL 5.7– Improved scalability, performance, security, and ease-of-use– Server Team Blog : http://mysqlserverteam.com

• MySQL Group Replication– Easy HA with active-active clusters– HA Team Blog : http://mysqlhighavailability.com

• MySQL Development Labs : http://labs.mysql.com• MySQL Fabric – new versions & features• Stay tuned!

http://mysqlserverteam.com/



http://mysqlhighavailability.com/

http://labs.mysql.com/


Plans for OpenStack• Kilo -- Released April 2015– Bare-Metal Provisioning (Ironic)• Also supports provisioning with Oracle VM VirtualBox

• Liberty -- October 2015–Queue Service (Zaqar)– Shared file system (Manila)– DNS Service (Designate)– Key Management (Barbican)– Additional details: http://status.openstack.org/release/

• Unknown – Container Service (Magnum)

https://wiki.openstack.org/wiki/Ironic

https://wiki.openstack.org/wiki/Zaqar

https://wiki.openstack.org/wiki/Manila

https://wiki.openstack.org/wiki/Designate

https://wiki.openstack.org/wiki/Barbican

http://status.openstack.org/release/

http://status.openstack.org/release/

https://wiki.openstack.org/wiki/Magnum


Plans for Trove• High Availability – Auto-scaling up and down based on demand– Failure detection and auto-failover

• Disaster Recovery– Cross region replication and failover

• Support for additional datastores • Better integration with other OpenStack modules• Improved security, monitoring, metering• API for accessing datastore log files– For example: Enterprise Audit log, Slow Query log, Error log, etc.


Plans for Oracle OpenStack Platform• Support of Additional OpenStack Services– Heat, MySQL A/A Clusters, Trove, Ceilometer, Ironic, and Murano– Ease of upgrade, installation, configuration and reconfiguration, supportability

• Templates for OpenStack: Linux, Heat, Enterprise Applications...• Support Oracle Applications in Murano Application Catalog • Nova support for Docker and LXC• Docker Images for Controller Node (Including HA)• Trove support for both the Oracle Database and MySQL• Oracle Virtual Networking OpenStack support to the network layer • CEPH back-end support for both Cinder and Swift

61


Plans for Oracle OpenStack Platform Cont.• Backup and restore of OpenStack repositories and metadata– Easier Disaster Recovery

• Discovery of physical server and component states like NICs, disks etc. to allow for easier remote configuration– Ease of adding new compute nodes to existing deployment

• Using BTRFS to enhance performance and availability of Swift• Applying insights from Oracle’s internal enterprise testing and deployments• Work with partners for joint support– Certification of third party vendor plug-ins

62


Appendix


OpenStack Resources• Docs– http://docs.openstack.org/

• Wiki– https://wiki.openstack.org/wiki/Main_Page

• Users Stories and Case Studies– http://www.openstack.org/user-stories/

• Community help– https://wiki.openstack.org/wiki/IRC– https://wiki.openstack.org/wiki/Mailing_Lists

http://docs.openstack.org/

https://wiki.openstack.org/wiki/Main_Page

http://www.openstack.org/user-stories/

https://wiki.openstack.org/wiki/IRC

https://wiki.openstack.org/wiki/Mailing_Lists


Trove Resources• Docs– http://docs.openstack.org/developer/trove/

• Wiki– https://wiki.openstack.org/wiki/Trove

• Launchpad– https://launchpad.net/trove

• Community help– #openstack-trove on Freenode

• Events– http://www.tesora.com/troveday/

http://docs.openstack.org/developer/trove/

https://wiki.openstack.org/wiki/Trove

https://launchpad.net/trove

https://wiki.openstack.org/wiki/IRC

http://www.tesora.com/troveday/


Oracle OpenStack Resources• Product web page (data sheet, white paper, videos, FAQ)– http://www.oracle.com/us/technologies/linux/openstack/overview/index.html

• Documentation: Installation and User Guide, HOL– http://www.oracle.com/technetwork/server-storage/openstack/linux/documentation/in

dex.html

• Downloads– http://www.oracle.com/technetwork/server-storage/openstack/linux/downloads/index.

html

http://www.oracle.com/us/technologies/linux/openstack/overview/index.html

http://www.oracle.com/technetwork/server-storage/openstack/linux/documentation/index.html

http://www.oracle.com/technetwork/server-storage/openstack/linux/documentation/index.html

http://www.oracle.com/technetwork/server-storage/openstack/linux/downloads/index.html

http://www.oracle.com/technetwork/server-storage/openstack/linux/downloads/index.html


MySQL Enterprise Resources• Manual– https://dev.mysql.com/doc/refman/5.7/en/

• Additional info– https://www.mysql.com/products/enterprise/

• Community forums– http://forums.mysql.com/

• Support– https://support.oracle.com/

• Server Team Blog– http://mysqlserverteam.com

https://dev.mysql.com/doc/refman/5.7/en/

https://www.mysql.com/products/enterprise/

http://forums.mysql.com/

https://support.oracle.com/



MySQL Central @ OpenWorld

• Keynotes• Conferences Sessions• Birds-of-a-feather sessions• Tutorials• Hands-on Labs• Demos• Receptions & Customer Appreciation Event• Access to Oracle OpenWorld Extensive Content

October 25-29, 2015 | San Francisco

68

Register Now & Save!

MySQL DBaaS with OpenStack Trove

Software

Transcript of MySQL DBaaS with OpenStack Trove