Everything you wanted to know about Trove but didn't know whom to ask!
description
Transcript of Everything you wanted to know about Trove but didn't know whom to ask!
Everything you wanted to know about Trove but didn't know
whom to ask!Nikhil Manchanda [HP]
PTL, TroveEmail: [email protected]
IRC: SlickNikTwitter: @SlickNik@hphelioncloud
Amrith Kumar [Tesora]Core Reviewer, TroveEmail: [email protected]
IRC: amrithTwitter: @amrithkumar
@tesoracorp
Tweet about this presentation #openstack #trove
What is Trove• Trove Mission
“To provide scalable and reliable Cloud Database as a Service provisioningfunctionality for both relational and non-relational database engines, and to continue to improve its fully-featured and extensible open source framework.”
https://wiki.openstack.org/wiki/Trove
• Provisioning and de-provisioning.• Administration.• Configuration.• Backup and Restore.• Clustering & Replication.
• Create and manipulate tables, collections, namespaces.• Select, Insert, Update, Delete Data.
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20142
An Overview of Trove
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20143
Trove Architecture [1]
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20144
Trove Architecture [2]
Keysto
ne
Cinder VolumeCinder VolumeCinder Volume
No
va-Netw
orkin
g/N
eutro
nTrove API
Message Bus
Trove Conductor
Trove Task Manager
Compute Instance
Guest Agent SQL/NoSQL
Backup
Data
BackupBackups
DB
BackupBackup
Guest Images
DataData
Nova
Cinder
Swift
Glance
Trove
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20145
Trove Architecture [3]
Trove Guest Instance
Operating System
Database Management System
Trove Guest Agent
Trove Controller
Trove API
Message Bus
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20146
Trove Taskmanager
Trove Conductor
Where Can I Get Trove?• Use the source, Luke!
• OpenStack Trove Project• https://git.openstack.org/cgit/openstack/trove• Mirrored at - http://github.com/openstack/trove.git
• Trove python client binding and command line client• https://git.openstack.org/cgit/openstack/python-troveclient• Mirrored at – http://github.com/openstack/python-troveclient.git
• Trove design specifications for blueprints• https://git.openstack.org/cgit/openstack/trove-specs• Mirrored at – http://github.com/openstack/trove-specs
• Trove scripts for installation and testing, and elements for building guest images• https://git.openstack.org/cgit/openstack/trove-integration• Mirrored at – http://github.com/openstack/trove-integration.git
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20147
trove/guestagent/strategies
backup/
[…]
replication/
restore/
storage/
trove/guestagent/
[…]
backup/
common/
datastore/
[…]
strategies/
[…]
trove/guestagent/strategies/backup
base.py
base.pyc
couchbase_impl.py
couchbase_impl.pyc
__init__.py
__init__.pyc
mysql_impl.py
mysql_impl.pyc
postgresql_impl.py
grep class guestagent/strategies/backup/mysql_impl.py
class MySQLDump(base.BackupRunner):
class InnoBackupEx(base.BackupRunner):
class InnoBackupExIncremental(InnoBackupEx):
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20148
Trove Modularity
Getting Started with Trove
• As a Trove user• OpenStack distribution that includes Trove (such as HP Helion Dev Platform)
• http://www8.hp.com/us/en/cloud/hphelion-openstack.html• Tesora DBaaS platform, a Trove packaging tailored for the enterprise
• http://tesora.com/download-tesora-dbaas-platform-enterprise-edition
• As a Trove developer$ git clone http://github.com/openstack/trove-integration$ cd trove-integration/scripts$ ./redstack install$ ./redstack kick-start mysql
• On top of DevStack• Add to localrc:
• ENABLED_SERVICES+=,trove,tr-api,tr-tmgr,tr-cond• Swift should also be enabled for Backup and Restore.
• ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account• SWIFT_HASH=<swift-hash-here>
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 20149
Provisioning a Trove Instance
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201410
$ trove create
<instance-name> <flavor-id>
--size <volume-size>
• Support for flavors
• Support for volumes using Cinder
• Optional parameters to create• Image per ‘datastore type’ and ‘version’
• Support AZs using --availability_zone
• Support for Neutron using --nic
{"instance": {
"status": "BUILD","updated": "2014-06-05T19:33:46","name": "test","links": [
{"href": "https://region-a.geo-
1.database.hpcloudsvc.com/v1.0/39745227274204/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",
"rel": "self"},{
"href": "https://region-a.geo-1.database.hpcloudsvc.com/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",
"rel": "bookmark"}
],"created": "2014-06-05T19:33:46","id": "ce629494-a64d-41ed-b73c-04cb28bb33bb","flavor": {
"id": "1002","links": [
{"href": "https://region-a.geo-
1.database.hpcloudsvc.com/v1.0/39745227274204/flavors/1002","rel": "self"
},{
"href": "https://region-a.geo-1.database.hpcloudsvc.com/flavors/1002",
"rel": "bookmark"}
]},"datastore": {
"version": "5.5","type": "mysql"
}}
}
Managing the Trove Database
• Resize flavor
• Resize volume
• Datastore specific extensions:• Create Database / Schema
• Create Users
• Grant Users Permissions on Databases
• Enable a Root User
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201411
Trove is Tuned
• Trove sets up sane defaults• InnoDB only• Disable load data infile and select into outfile
• Ability to automatically tune settings by flavor – e.g. for my.cnf• Buffer Pool Size, • Log file size• max_connections, etc.
• API to programmatically target configuration groups for settings
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201412
Trove is Secure
• Removes anonymous user
• Removes non-localhost users
• Removes local file access
• Mangles root user password
• Sets up Security Groups to manage access (via API)
• User SSH access to datastore not required
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201413
Backup and Restore
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201414
$ trove backup-create <backup-name> <instance-id>
• Optional params:• Description through --description
• Incremental backups using –parent
• To Restore backup use create:
$ trove create
<instance-name> <flavor-id>
--size <volume-size>
--backup <backup-id>
{
"backup": {
"instance_id": "9a31e420-aee6-4329-92f7-4b6b3790bcd4",
"status": "NEW",
"updated": "2014-06-05T19:43:28",
"locationRef": null,
"name": "DEMO_instance_test_backup",
"created": "2014-06-05T19:43:28",
"size": null,
"id": "f811a9cf-f397-4e07-a21f-7134094976f8",
"description": null
}
}
Backups in Trove
• Fully Managed
• Triggered and Tracked via API
• Streamed to Swift (OpenStack Object Storage)
• Incremental as well as Full backups
• Multiple formats per datastore supported via strategies:
• XtraBackup (Percona)
• mysqldump
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201415
Replica Instances
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201416
$ trove create
<instance-name> <flavor-id>
--size <volume-size>
--replica_of <instance-id>
• Support for async mysql replication (mysql slave instances)
• Manual detach using
$ trove update <instance-id>
--detach-replica-source
{"instance": {
"status": "BUILD","updated": "2014-06-05T19:33:46","name": "test","links": [
{"href": "https://region-a.geo-
1.database.hpcloudsvc.com/v1.0/39745227274204/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",
"rel": "self"},{
"href": "https://region-a.geo-1.database.hpcloudsvc.com/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",
"rel": "bookmark"}
],"created": "2014-06-05T19:33:46","id": "ce629494-a64d-41ed-b73c-04cb28bb33bb","flavor": {
"id": "1002","links": [
{"href": "https://region-a.geo-
1.database.hpcloudsvc.com/v1.0/39745227274204/flavors/1002","rel": "self"
},{
"href": "https://region-a.geo-1.database.hpcloudsvc.com/flavors/1002",
"rel": "bookmark"}
]},"datastore": {
"version": "5.5","type": "mysql"
}}
}
Clusters
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201417
$ trove cluster-create <datastore> <ds-version>
• Optional parameters to cluster-create• --instance <flavor_id=flavor_id,volume=volume>
• Specify multiple times to create multiple instances for your cluster
• Initial support in Juno added for MongoDB Clusters• Sets up mongo config server, and mongo query routers
• Transparent and driven by configuration options
• Support for adding shards to existing cluster for horizontal scale out.
Juno and Kilo
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201418
Completed in Juno
• Async MySQL replication (master-slave)
• Clusters for MongoDB
• Neutron Support
• Support for PostgreSQL
• Config-groups enhancements• Configuration groups per datastore / version
• Config-groups for MongoDB
• Backups for Cassandra and Couchbase
• Additional Tempest Tests
Planned for Kilo
• Building out clusters -- including Semi Synchronous (Galera) mysql clusters
• Async Replication v2 – failover support
• Associate flavors with datastores
• Access datastore logs via API
• Removing deprecated oslo-incubator code
• Upgrade testing through grenade
• YOUR IDEA HERE!
Your Idea Here
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201419
• Growing community of contributors• 128 contributors from 28 companies• 1723 commits, and 231829 lines of code
• Open to new ideas, and code
• Lots of room for improvement
• Find us at #openstack-trove on FreeNode.
Thank You!
Got Questions?
11/6/2014All you ever wanted to know about Trove!
OpenStack Summit (Paris), 201420
Nikhil Manchanda [HP]PTL, Trove
Email: [email protected]: SlickNik
Twitter: @SlickNik@hphelioncloud
Amrith Kumar [Tesora]Core Reviewer, TroveEmail: [email protected]
IRC: amrithTwitter: @amrithkumar
@tesoracorp
Tweet about this presentation #openstack #trove