High performance MySQL in an OpenStack cloud
23
Matt Griffin George Lorch Tom Diederich January 15, 2015 Cloud Online Meetup: High performance MySQL in an OpenStack cloud
-
Upload
openstackonline -
Category
Technology
-
view
560 -
download
2
Transcript of High performance MySQL in an OpenStack cloud
Matt GriffinGeorge Lorch
Tom DiederichJanuary 15, 2015
Cloud Online Meetup:High performance MySQL in an
OpenStack cloud
www.percona.com
Agenda● MySQL guests in the cloud● Benchmarking the MySQL core of OpenStack● Supporting the OpenStack community
Questions: In ReadyTalk
Slides: bit.ly/percona-jan2015-cloud-online-meetup
www.percona.com
About us
Percona delivers high performance MySQL to our clients and the community.
Open Source Software● Percona Server● Percona XtraDB Cluster
(Galera)● Percona XtraBackup● Percona Toolkit
100% MySQL-compatible
MySQL Services● Support● Consulting● Managed Services
Community● Events: Percona Live,
OpenStack Live, Percona University
● MySQL Performance Blog● Related communities
www.percona.com
Percona & OpenStackMySQL is the top database for OpenStack services
MySQL is one of the top use cases for guests in the cloud
We have performance expertise to share with the developer community and production users OpenStack User Survey results, November 2014
http://superuser.openstack.org/articles/openstack-user-survey-insights-november-2014
www.percona.com
MySQL Guests in the Cloud(Trove DBaaS)
www.percona.com
Utility userNeeded a MySQL user to perform specific management functions
Potential risk in an actual user making changes to this user
Created a special type of MySQL user in Percona Server designed to● Do specific tasks● Be protected/hidden
Currently only used by HP Helion
www.percona.com
Options modifiersOperators can set limits on MySQL configuration changes to reduce their risk.e.g., maximum-query_cache_size=4M
Percona extended MySQL’s five included options with:● minimum● hidden● readonly
Enhanced oversight for cloud operators over guest DBs
Available now when using Trove with Percona Server
www.percona.com
Enforcing a storage engineInnoDB is most likely the best choice… but many other options exist
An operator can now lock down
End result is enhanced consistency and reliability in guest environments
Available now when using Trove with Percona Server
www.percona.com
Security and CapacityCan disable LOAD DATA INFILE and SELECT INTO OUTFILE● Both potentially high risk resources to interact with the
file system● Percona extended –secure-file-priv option
Added Binary and Slow Log size and rotation● Set tighter control in Percona Server over how database
logs behave and the disk capacity they consume per instance
Both available now when using Trove with Percona Server
www.percona.com
Percona Server + XtraBackupPercona Server 5.6 with Percona XtraBackup 2.2.x enables:● Changed page tracking● Reduced locking during backups
Enabled by default when deploying a Trove image containing Percona Server and Percona XtraBackup
www.percona.com
Percona XtraBackupAdditional benefits of Percona XtraBackup● Reduced disk usage with enhanced compression● Higher security with AES 64-256 encryption● Simpler backups administration
Coming soon● Asymmetric/public key encryption● Streaming of backups to Swift (currently in Alpha)
www.percona.com
Additional Resources
OpenStack Summit ParisPercona Server features for OpenStack and Trove OpsGeorge Lorch, PerconaVipul Sabhaya, HP
www.percona.com
Benchmarking the MySQL Core of OpenStack
www.percona.com
ObjectivesExamine the database communication and load in an OpenStack environment
Identify database-layer bottlenecks at various levels of concurrency
Understand the query load placed against the database server during typical requests to Nova and Neutron
www.percona.com
Benchmarking SetupInitial setup: 7 AWS instances● 1 controller nodes● 1 compute worker node● 1 network node● 3 PXC nodes● 1 Rally benchmark node
c3.8xlarge instance type except compute● 24 cores / 60GB RAM / SSD
ephemeral drives
Updated setup: 16 AWS instances● 1 controller node (we want more
in the future)● 10 compute worker nodes● 1 network + queue node● 3 PXC nodes● 1 Rally benchmark node
c3.8xlarge instance type except compute● 24 cores / 60GB RAM / SSD
ephemeral drives
c3.large● 2 cores / 3.75GB RAM / SSD
ephemeral drives
www.percona.com
Load TestsBoot 10,000 servers● Some increase in response
time● Some errors (Galera
certification timeouts)● Database wasn’t the
bottleneck
Then boot/delete 5,000 more servers● Writing to several nodes
significantly improves performance
● Some services retry on certification timeouts (e.g., @_retry_on_deadlock in Nova) but some don’t
● Most operators will probably not see timeouts in production
www.percona.com
Lessons Learned I● SELECT … FOR UPDATE can be an issue for Galera
clusters powering OpenStack● Hidden behind ‘deadlock’ error
● Better performance when combining Network and Queue node
● Use pt-query-digest (in Percona Toolkit) to analyze workloads. “Options” very helpful to filter output. Look for:● Slowest transactions (default output)● Large transactions
www.percona.com
Lessons Learned II● Database is not the bottleneck
● Controller node’s API functions are saturated earlier with maxed out CPU but little stress on the database
● Likely many opportunities for database optimizations for production clouds given identified query-level bottlenecks● Targeted indexes will help
● Continue examining OpenStack’s MySQL workload across services and with future releases● Future patches coming
www.percona.com
Additional ResourcesOpenStack Summit ParisMySQL and OpenStack Deep DivePeter Boros, PerconaJay Pipes, Mirantis
Ansible playbooks: Set up your own OpenStack benchmarking cluster on AWS
“OpenStack users shed light on Percona XtraDB Cluster deadlock issues” by Peter Boros
“Understanding reservations, concurrency, and locking in Nova” by Jay Pipes
www.percona.com
Supporting the OpenStack Community
www.percona.com
Contributions in Progress● Trove clustering● HA Guide update● MySQL Performance Blog: OpenStack advice based on
our MySQL expertise● Peter and Jay’s Ansible playbooks are evolving
www.percona.com
Hands-on tutorials covering technologies including:● Trove (DBaaS)● Barbican (security)● Ceph (storage)● Neutron (networking)
Sessions on topics such as:● Rally benchmarking● Trove in the real world● Stories from enterprise users
like Time Warner Cable● Cloud Foundry PaaS
Early Bird registration ends soonhttp://www.percona.com/live/openstack-live-2015/
www.percona.com
QuestionsMatt Griffin [email protected] George Lorch [email protected] Tom Diederich [email protected]
Slides: bit.ly/percona-jan2015-cloud-online-meetup
