ACCELERATING MYSQL WITH FLASHBLADE A FRAMEWORK FOR FASTER BACKUP & RECOVERY USING FLASHBLADE
TECHNICAL WHITE PAPER
2
TABLE OF CONTENTS
INTRODUCTION ........................................................................................................................................ 3
THE PROBLEM .......................................................................................................................................... 3
THE SOLUTION ......................................................................................................................................... 3
OVERVIEW ................................................................................................................................................. 4
AUDIENCE ................................................................................................................................................. 4
MYSQL OVERVIEW ................................................................................................................................... 4
SOLUTION DESIGN .................................................................................................................................. 4
Solution Overview .............................................................................................................................. 4
Environment Overview ..................................................................................................................... 5
Server Configuration ......................................................................................................................... 5
Virtual Machine Configuration ....................................................................................................... 6
MySQL Configuration ......................................................................................................................... 7
FlashBlade Configuration ................................................................................................................ 8
BACKUP AND RESTORE TESTS AND RESULTS ................................................................................ 10
Logical Backup Test .......................................................................................................................... 10
Restore Logical Backups Test ....................................................................................................... 13
Physical Backup Test ........................................................................................................................ 15
Restore Physical Backups Test ..................................................................................................... 18
BEST PRACTICES FOR MYSQL ON FLASHBLADE ............................................................................ 21
CONCLUSION .......................................................................................................................................... 23
REFERENCES .......................................................................................................................................... 24
ABOUT THE AUTHOR ............................................................................................................................ 25
3
THE PROBLEM
Enterprise usage of MySQL may range anywhere from a handful to hundreds of databases. With the advent of
virtualization, usage has been steadily increasing. In addition to the growing number of MySQL databases, the
growth of data that feeds these databases has been staggering.
This poses a key challenge to database administrators, not only in managing databases but in backing up databases.
DBAs have to perform backups periodically and restores as needed within agreed service levels, and those SLAs
change frequently to meet business requirements.
THE SOLUTION
FlashBlade™ is a ground-breaking scale-out flash storage system from Pure Storage®. Engineered with a massively
parallel architecture from software to hardware, it has emerged as the industry-leading solution for use cases like
artificial intelligence and modern analytics that require the highest performance. Many customers have tapped this
performance to modernize their backup and recovery infrastructure.
The current FlashBlade system can support a read rate of 16 GBps and a write rate of 4.5 GBps, in a single 4U chassis.
The sustained write rate of 4.5 GBps is equivalent to a backup rate of 15 TB/hour, which is well above the normal
required backup rate for many customers. The restore rates from FlashBlade can be higher than 45 TB/hour with
concurrent restore operations.
MySQL customers can now direct their MySQL backups to FlashBlade to accelerate backup and restore significantly.
Customers can realize these backup/restore rates without requiring any exotic and costly software on their hosts for
block differencing purposes.
While the performance of FlashBlade is very impressive, even more attractive is its non-disruptive scalability.
The capacity and performance of FlashBlade can be scaled in a non-disruptive manner, simplifying the operational
procedures of a customer’s backup/recovery process. The system can be scaled in seconds, without running special
commands or extra cables. Additionally, FlashBlade's in-line compression feature reduces capacity requirements by
almost 66% for most typical databases – driving costs down significantly while not sacrificing performance. Allowing
the storage system to handle compression also saves dramatically on host CPU cycles.
INTRODUCTION
MySQL is the preferred database platform for the majority of websites worldwide,
as well as thousands of corporate web-based applications. This open-source
database is ubiquitous: most SaaS providers and top social media companies
have deployed MySQL as the database of choice for their key services. As MySQL
deployments and data volumes grow, the importance of efficient backup and
recovery infrastructure becomes critical.
OVERVIEW
The purpose of this technical white paper is to provide techniques and guidelines to rapidly backup and restore
MySQL databases on FlashBlade. The goal of this document is to help MySQL administrators to meet their backup/
recovery service level objectives.
AUDIENCE
The target audience for this document includes but is not limited to MySQL database administrators, storage
administrators, IT managers, system architects, sales engineers, field consultants, professional services, and partners
who are looking to design and deploy MySQL backups on FlashBlade. A working knowledge of MySQL, Linux®, server,
storage, and networks is assumed but is not a prerequisite to read this document.
MYSQL OVERVIEW
MySQL is very simple to use and at the same time provides the flexibility and scalability to support data growth.
One of the key reasons enterprises choose MySQL is for its total cost of ownership (TCO). Even with the commercial
Enterprise version, there are no up-front licensing costs, which means lower capital spending.
MySQL databases can be backed up in two ways:
• Physical backups of directories and files that store database contents. These backups are faster, as they
involve file copying without conversion. They are portable only to other servers that have similar OS and
MySQL database characteristics. Physical backup tools include mysqlbackup of MySQL Enterprise Backup
from Oracle, Percona XtraBackup (a free, open source, online backup solution for MySQL), MariaDB, and
Percona Server for MySQL.
• Logical backups of database structure and content. These are slower than physical backups, as the server
must access database information and convert it to a logical format. The advantage of logical backups is
that they are machine-independent and highly portable. mysqldump is the preferred and most prevalent tool
used by DBAs to perform logical backup and restore of MySQL databases.
SOLUTION DESIGN
Solution Overview
The solution to accelerate MySQL data protection, be it physical or logical, is to enable parallelism at the application
level and to use the Pure Storage FlashBlade storage system as the target and source, respectively, for backup and
restore activity.
FlashBlade is a scale-out NAS system that is engineered with massively parallel architecture. On top of the
performance gained from flash technology, enabling parallel connections to the FlashBlade system from the
host across different IP addresses and ports can better utilize the compute and network resources of the
blades in the FlashBlade system.
4
5
In this paper, we tested both the physical and logical backup and restore of a MySQL database on FlashBlade.
For physical backup and restore, we chose Percona Xtrabackup, while for logical backup and restore we chose
mysqldump. For the sake of clarity, we performed only full backups and restores in our tests.
Environment Overview
As part of this solution, we wanted to simulate a real-world scenario in which numerous MySQL databases are
distributed across multiple storage systems and backed up to the same target at the same time.
The setup includes 20 virtual machines running MySQL on Red Hat Linux 7.4 across four ESX servers. VMFS
datastores were setup across three Pure Storage FlashArrays (//M20, //M50, and //X70) that were connected to these
ESX servers. The 20 virtual machines were distributed across these three storage systems (4 on //M20, 8 on //M50,
and 8 on //X70).
Each virtual machine ran MySQL with a source database size of 70 GB. As the backups were to be performed on
to FlashBlade, volumes/filesystems from FlashBlade were mounted over NFS protocol on these virtual machines.
As the backups were to be targeted to a single destination, the same NFS filesystems were mounted across all
virtual machines.
FIGURE 1. MySQL databases distributed across multiple storage systems and backed up to the same target
Server Configuration
Four Intel® CPU based Cisco UCS B-series B200 M4 blade servers were deployed to host the ESX server. Each server
had a Cisco UCS VIC 1340 card, and they were connected by four ports from each Cisco Fabric extender of the Cisco
UCS chassis to the Cisco Fabric Interconnect, which was in turn connected to the Cisco MDS 9148S for upstream
connectivity to the Pure Storage FlashArrays. The server configuration is described in Table 1.
COMPONENT DESCRIPTION
PROCESSOR 4 X INTEL XEON E5-2609 V4 1.7 GHZ (2 CPUS WITH 8 CORES EACH)
MEMORY 64 GB @ 2.4GHZ (2 X 32GB)
HBA4 X 10G PORTS ON CISCO UCS VIC 1340 (UCSB-MLOM-40G-03) 40GBPS
NIC6 INTERFACES ON CISCO UCS VIC 1340
5 CONFIGURED - 1 FOR PUBLIC, 4 FOR BACKUP
UCS FIRMWARE (ACROSS ALL COMPONENTS) 3.1 (2B)
TABLE 1. UCS Blade configuration
Four data VIPs (virtual IPs) were created on FlashBlade, each one of them configured to the VLAN (201 to 204).
The ESX servers were configured with five virtual switches, one for public traffic and the remaining four for FlashBlade
traffic. The four virtual switches for backup traffic in ESX were tagged with corresponding VLANs (201 to 204).
Virtual Machine Configuration
Each VM was configured with 4 vCPUs and 4 GB of memory.
FIGURE 2. Typical VM configuration
The VM was configured with two disks. Red Hat O.S was installed on the first disk and the MySQL database data
on the second disk. The second disk was carved out from one of the three Pure FlashArrays (//M20, //M50, //X70).
The VM was configured with five network adapters; one was used for the public network and another four for the
6
backup network. The NFS filesystem from FlashBlade, where the MySQL backup was to be stored, was mounted
across these four interfaces. Four private IP addresses were configured for the four interfaces.
MySQL Configuration
The open source MariaDB variant of MySQL was installed on the Red Hat Linux 7.4 virtual machines.
+----------------------------+----------------------+
| Variable_name | Value |
+----------------------------+----------------------+
| innodb_version | 5.5.52-MariaDB-38.3 |
| protocol_version | 10 |
| slave_type_conversions | |
| version | 5.5.56-MariaDB |
| version_comment | MariaDB Server |
| version_compile_machine | x86_64 |
| version_compile_os | Linux |
+----------------------------+----------------------+
For the physical backups, Percona Xtrabackup 2.2.13 was installed on top of the MySQL software. In this paper, we
used the utility innobackupex, which is a perl wrapper for the xtrabackup tool, as innobackupex allows additional
restore features like copy-back or move-back that would copy or move the backup files to the datadir location.
As of Percona Xtrabackup 2.3+, the features of innobackupex are incorporated into xtrabackup and
innobackupex may be deprecated in the next major release.
The following are some of the key parameters that were configured in the my.cnf file.
datadir = /m01/data
innodb_file_per_table=1
The datadir parameter was configured to point to the Pure FlashArray volume where the MySQL database was
to be stored.
7
The innodb_file_per_table was enabled to store the tables and indexes in a separate .ibd datafile, as opposed
to a single, big, monolithic datafile. There are various advantages to file-per-table tablespaces – like better file
management, reduced fragmentation, faster truncate, etc. – but the key one in this context is acceleration of the
backup process, and specifically with physical backups through XtraBackup. XtraBackup allows multiple files
to be backed up concurrently using the parallel option.
FlashBlade Configuration
COMPONENT DESCRIPTION
FLASHBLADE 14 X 17TB BLADES
CAPACITY224 TB RAW
151.63 TB USABLE (WITH NO DATA REDUCTION)
CONNECTIVITY 4 X 40 GB/S ETHERNET (DATA)
1 GB/S ETHERNET (MANAGEMENT PORT)
PHYSICAL 4U
SOFTWARE PURITY//FB 2.1.1
TABLE 2. FlashBlade configuration
The FlashBlade network settings were configured with four subnets across four VLANs.
FIGURE 3. FlashBlade network settings
8
For the MySQL backup target, we created two NFS filesystems, each with a size of 10TB. Logical backups were sent
to the NFS filesystem named mysql-backup-01, and physical backups were sent to the NFS filesystem named mysql-
backup-02. We created two NFS filesystems to identify the data reduction rate (if any) on FlashBlade specific to the
type of backups.
FIGURE 4. Creating NFS filesystems
Even though the target for the logical backup is a single filesystem, mounting it on a single mount point across all
VMs on the same interface would make that underlying adapter on the ESX a bottleneck and limit bandwidth to
1 GB/s (10 Gbps). Thus, to maximize bandwidth, we mounted the same filesystem across four interfaces on all 20
virtual machines across the four subnets.
Similarly, the target filesystem for physical backups was mounted on four interfaces on all 20 VMs.
Note: Be sure to size the NFS filesystem(s) to hold the MySQL backup. Ideally, the filesystems should be
sized based on the retention and frequency of backups. Of course, you will benefit from the data reduction
offered by FlashBlade; the data reduction rate will vary based on the type of data. As the space is always thin-
provisioned, there is no penalty for creating a larger-sized filesystem.
9
BACKUP AND RESTORE TESTS AND RESULTS
Our backup and restore tests included the following four tests across logical and physical backup mechanisms.
They were tested singly and on 20 virtual machines to showcase the scalability aspect of the solution.
1. Logical backup using mysqldump
2. Restore logical backup
3. Physical backup using Percona Xtrabackup
4. Restore physical backup using Percona Xtrabackup
Logical Backup Test
Each VM was configured with backup scripts like the one below, directing the mysqldump output on to FlashBlade
across different mount points. The mysqldump commands were run in parallel to overcome the single threaded
limitation of mysqldump, especially if run at database level. Since the virtual machine was configured with four vCPUs,
we limited the parallel threads to four.
As the test environment had only one database (dss), we had to run the logical backups at table level. If you have
multiple databases, you can run mysqldump in parallel at the database level.
The backups are organized under the same NFS filesystem by the servername, which is passed as a parameter
whether it was invoked from a single virtual machine or across all 20 of them.
mysqldump -u root -q dss table1 > /b01/backup/$server/table1.sql &
mysqldump -u root -q dss table2 > /b02/backup/$server/table2.sql &
mysqldump -u root -q dss table3 > /b03/backup/$server/table3.sql &
mysqldump -u root -q dss table4 > /b04/backup/$server/table4.sql &
wait
mysqldump -u root -q dss table5 > /b01/backup/$server/table5.sql &
mysqldump -u root -q dss table6 > /b02/backup/$server/table6.sql &
mysqldump -u root -q dss table7 > /b03/backup/$server/table7.sql &
mysqldump -u root -q dss table8 > /b04/backup/$server/table8.sql &
Even though the same filesystem (mysql-backup-01) was mounted across four mount points, the four mount points
were configured across different subnets, enabling network aggregations that can yield higher bandwidth.
LOGICAL BACKUP ON A SINGLE VIRTUAL MACHINE
The backup script was run on a single virtual machine to get a baseline. It took 9 minutes 11 seconds to backup 56.4 GB
(actual data, not including white spaces) of the MySQL database, yielding a throughput of 104 MB/s. One option to
increase throughput is to increase the number of parallel mysqldump sessions, provided you have available compute
resources (as in our case). We were limited by the 4 vCPUs at the virtual machine level.
10
The FlashBlade GUI shows bandwidth usage of around 110 MB/s for most of the backup activity.
FIGURE 5. Bandwidth usage on FlashBlade
LOGICAL BACKUP ON 20 VIRTUAL MACHINES
The backup script listed in the earlier section was run across all 20 virtual machines. The 20 virtual machines
were driving a total of 1.1 TB of backup data into FlashBlade across the four interfaces.
mysqldump -u root -q dss table1 > /b01/backup/$server/table1.sql &
mysqldump -u root -q dss table2 > /b02/backup/$server/table2.sql &
mysqldump -u root -q dss table3 > /b03/backup/$server/table3.sql &
mysqldump -u root -q dss table4 > /b04/backup/$server/table4.sql &
wait
mysqldump -u root -q dss table5 > /b01/backup/$server/table5.sql &
mysqldump -u root -q dss table6 > /b02/backup/$server/table6.sql &
mysqldump -u root -q dss table7 > /b03/backup/$server/table7.sql &
mysqldump -u root -q dss table8 > /b04/backup/$server/table8.sql &
The 1.1 TB of backup across 20 virtual machines took 14 minutes to complete, yielding a throughput of 1.34 GB/s.
The following screenshot shows bandwidth metrics on FlashBlade. The 20 virtual machines were generating
around 1.51 GB/s of write bandwidth for most of the activity.
FIGURE 6. Bandwidth usage on FlashBlade
11
The following screenshots show read bandwidth metrics during the same time across the three FlashArrays on
which MySQL databases were hosted.
//M20
//M50
//X70
FIGURE 7. Bandwidth usage on three FlashArrays hosting MySQL databases
Meanwhile, FlashBlade has reduced the backup data by 2.1:1. The actual physical storage space consumed is
544.05 Gb for the 1.1 TB of the full backup.
FIGURE 8. Physical storage space consumed on FlashBlade
12
Restore Logical Backups Test
Similar to the backup, the restore was performed by running mysql commands in parallel. As the logical backups were
performed at the table level, the restore was included in the database (dss) where the tables are restored.
RESTORE OF A LOGICAL BACKUP ON A SINGLE VIRTUAL MACHINE
The following restore script was run on the single virtual machine to restore the dss database at the table level.
As part of the restore, MySQL creates a table and inserts the records into the table.
Note: If you are restoring at the database level on a remote server using sql script, be sure to create the
database before the restore, unless the sql dump was created with the --database option.
mysql -u root -q dss < /b01/backup/$server/table1.sql &
mysql -u root -q dss < /b02/backup/$server/table2.sql &
mysql -u root -q dss < /b03/backup/$server/table3.sql &
mysql -u root -q dss < /b04/backup/$server/table4.sql &
wait
mysql -u root -q dss < /b01/backup/$server/table5.sql &
mysql -u root -q dss < /b02/backup/$server/table6.sql &
mysql -u root -q dss < /b03/backup/$server/table7.sql &
mysql -u root -q dss < /b04/backup/$server/table8.sql &
To restore 70 GB worth of database on a single virtual machine, it took 37 minutes 21 seconds at a read throughput
of 32 MB/s on FlashBlade, while averaging around 100 MB/s on the FlashArray the database is restored to. This
sub-optimal throughput issue was not due to the infrastructure, but caused by MySQL. The restore process performs
inserts of records one at a time from the sql file and, as such, is not benefitting from any bulk operations.
FIGURE 9. Bandwidth usage on FlashBlade
FIGURE 10. Bandwidth usage on FlashArray
13
RESTORE OF A LOGICAL BACKUP ON 20 VIRTUAL MACHINES
As with the single virtual machine, the following restore script was run across 20 virtual machines, and the
server name was passed as the parameter to restore the database from the right set of backups.
mysql -u root -q dss < /b01/backup/$server/table1.sql &
mysql -u root -q dss < /b02/backup/$server/table2.sql &
mysql -u root -q dss < /b03/backup/$server/table3.sql &
mysql -u root -q dss < /b04/backup/$server/table4.sql &
wait
mysql -u root -q dss < /b01/backup/$server/table5.sql &
mysql -u root -q dss < /b02/backup/$server/table6.sql &
mysql -u root -q dss < /b03/backup/$server/table7.sql &
mysql -u root -q dss < /b04/backup/$server/table8.sql &
The restore of MySQL databases across 20 virtual machines took 56 minutes 19 seconds, which was more than
four times the backup time. Overall 1.1 TB was read from FlashBlade at a throughput of 341 MB/s and restored on to
three FlashArrays.
The following screenshot shows read bandwidth from FlashBlade when the restore was performed.
FIGURE 11. Bandwidth usage on FlashBlade
The following screenshots show write bandwidth metrics across the three FlashArrays to which the MySQL
database was restored.
//M20
14
//M50
//X70
FIGURE 12. Bandwidth usage on three FlashArrays to which the MySQL database is restored
Overall, FlashBlade is significantly underutilized, even while a logical backup and restore of 20 virtual machines was
performed, leaving room for more consolidation of high bandwidth workloads.
Physical Backup Test
For the physical backup test, Percona Xtrabackup was configured on the virtual machine. Xtrabackup provides an
option to run the backups in parallel, but it requires the innodb_file_per_table to be enabled, which stores the table in
a separate .ibd file.
The following is the command that was run to perform a full backup of the database. To differentiate the backups
and ascertain if there are any differences in data reduction rates, the backups were sent to a different NFS filesystem
(mysql-backup-02). If you do not pass the --no-timestamp option, the backups will be created in a timestamped
directory (YYYY-MM-DD_hh-mm-ss).
innobackupex --defaults-file=/etc/my.cnf --user=root --parallel=4 /x01/xtrabackup/$server
Unlike the logical backup, where we were able to run parallel commands and direct the output to four different mount
points to take advantage of the separate interfaces, Xtrabackup doesn’t allow multiple destinations. Thus, we rotated
the destination for every virtual machine between /x01, /x02, /x03, and /x04. As they are mounted on separate
subnets, we were able to use available network interfaces to spread out backup traffic.
15
Alternatively, we could have mounted the same filesystem on the same mount point /x01, but mounted it across
different subnets in every virtual machine to accomplish the same behavior.
By using all four interfaces across the 20 virtual machines, we were able to use the available bandwidth and not
overwhelm a single interface, which would have limited our bandwidth to 10 Gbps (1 GB). Meanwhile, physical backup
on a single virtual machine would be limited to 10 Gbps or 1 GB. In our setup, the virtual machine’s compute would
become the bottleneck before the network bandwidth is saturated at 10 Gbps.
PHYSICAL BACKUP ON A SINGLE VIRTUAL MACHINE
The full backup of a MySQL database with a size of 70 GB took 152 seconds at a throughput of 471 MB/s, which is over
4 times the performance of the logical backup.
FIGURE 13. Bandwidth usage on FlashBlade
The improvement in performance over logical backup is due to the parallel option and to the implementation of
Xtrabackup’s backup feature.
PHYSICAL BACKUP ON TWENTY VIRTUAL MACHINES
The backup command was run across all 20 virtual machines at the same time. The only difference in the backup
across all the virtual machines is the destination directory. We rotated the destinations from /x01 to /x04 across the 20
virtual machines.
innobackupex --defaults-file=/etc/my.cnf --user=root --parallel=4 /x01/xtrabackup/$server
The backup of MySQL databases across 20 virtual machines containing 1.35 TB took 359 seconds at a throughput
of 3.85 GB/s.
16
FIGURE 14. Bandwidth usage on FlashBlade
As seen from the above FlashBlade bandwidth metrics, concurrent backups from 20 virtual machines pushed
the write bandwidth to 4 GB/s for most of the activity.
The following screenshots show the bandwidth metrics across the three FlashArrays (//X70, //M50, and //M20).
As //X70 and //M50 were hosting 8 virtual machines each, the read bandwidth was hovering over 1.6 GB/s for most
of the activity while //M20 was hovering around 750MB/s.
//M70
//M50
//X20
FIGURE 15. Bandwidth usage on three FlashArrays to which the MySQL database is restored
17
Meanwhile, the data reduction on FlashBlade for the physical backups was also 2.1:1 – like the logical backups.
FIGURE 16. Bandwidth usage on FlashBlade
Restore Physical Backups Test
Percona Xtrabackup has an extra step prior to restoring a backup: it prepares the backup for restore. The prepare step
makes the files consistent at a single instant in time by undoing all uncommitted transactions and playing back the
committed transactions from the transaction logs. The prepare operation can be performed on any machine, and not
necessarily on the originating server. It can be done right after the backup or just before the restore.
To prepare the backup with innobackupex, the --apply-log option should be used along with the backup directory.
If the backup was taken without the --no-timestamp option, be sure to provide the timestamped directory name.
innobackupex --apply-log /x01/xtrabackup/$server/$1
Once the backup is prepared, it can be restored. The backup can be restored back to the same datadir location or it
can be copied to another location.
innobackupex --defaults-file=/etc/my.cnf --copy-back /x01/xtrabackup/$server/$1
The --copy-back option copies the contents from the backup directory to the datadir directory. The datadir must
be empty for the --copy-back option, as it will not over write the existing directory unless the ---force-non-empty-
directories option is specified.
Once the restore is complete, the files ownership should be updated to mysql before starting the MySQL instance.
# chown –R mysql:mysql /m01/data
Restore timing and performance is generally limited by the target FlashArray as backup data is read out of FlashBlade
and written into the FlashArray. As we have multiple FlashArrays, FlashBlade can deliver higher performance, since a
15-blade system can deliver read bandwidth in the range of 16GB/s.
RESTORE OF A PHYSICAL BACKUP ON A SINGLE VIRTUAL MACHINE
As part of the Xtrabackup restore, the files are copied back to the datadir location. There are not many options to fine-
tune restore performance at the Xtrabackup level.
18
19
The prepare step of the restore took 2 seconds, and the actual restore of 70 GB of backup data from a single virtual
machine took 219 seconds at a throughput of 327 MB/s.
In the screenshots below, the FlashBlade GUI shows average read bandwidth while the FlashArray GUI displays write
bandwidth on the target FlashArray to which the data is written. The read performance from FlashBlade is directly
dependent on the write performance on the target FlashArray.
FIGURE 17. Bandwidth usage on FlashBlade
FIGURE 18. Bandwidth usage on FlashArray
RESTORE OF A PHYSICAL BACKUP ON TWENTY VIRTUAL MACHINES
The prepare and restore processes were performed at the same time on 20 virtual machines. Similar to the backup
process, the restores were performed from the same NFS filesystem, but from different network interfaces to spread
the bandwidth across the four interfaces.
To restore 1.35 TB of data across 20 virtual machines, it took 8 minutes 27 seconds, yielding a throughput of 2.73 GB/s.
As seen from the FlashBlade bandwidth metrics, the read bandwidth hovered close to 3 GB/s for most of the restore
and started coming down based on activity at the individual host level.
FIGURE 19. Bandwidth usage on FlashBlade
20
The following screenshots show write bandwidth on the three target FlashArrays.
//M20
//M50
//X70
FIGURE 20. Bandwidth usage on three FlashArrays to which the MySQL database is restored
The target FlashArrays are not anywhere close to their maximum potential write performance: their write performance
for this test is purely dependent on the performance of XtraBackup and its implementation. However, XtraBackup
restore performance is over six times that of the logical restores through the sql dump file. Meanwhile, FlashBlade has
a lot more room left to serve other demanding workloads that can stress both reads and writes.
21
BEST PRACTICES FOR MYSQL ON FLASHBLADE
ENABLE PARALLELISM
To increase read and write bandwidth on FlashBlade, use client level parallelization techniques as applicable.
This increases the number of connections to FlashBlade.
USE MULTIPLE NETWORK INTERFACES
To enhance network bandwidth, be sure to have multiple network interfaces on the client. These multiple interfaces
can be configured on a single subnet or on multiple subnets.
SINGLE SUBNET
IO performance in a MySQL environment with multiple interfaces and a single subnet is limited to the speed of the first
interface that is picked by the OS. This is because the OS returns the first path when multiple paths are available on
the subnet, and thus traffic is always routed through the first path.
For example, in the setup below, FlashBlade has a single IP address on which the NFS filesystem will be mounted from
the client, and the client has two interfaces.
Client interface NFS server
ens7f0 10.21.108.193 10.21.108.10
ens7f1 10.21.108.194 10.21.108.10
[root@mysql-m50-01 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.21.108.1 0.0.0.0 UG 0 0 0 ens7f0
10.21.108.0 0.0.0.0 255.255.255.0 U 0 0 0 ens7f0 <-- first route
10.21.108.0 0.0.0.0 255.255.255.0 U 0 0 0 ens7f1 <-- route ignored
As per the routing table, traffic can go through the first interface (ens7f0) as the destination, and the mask fits for both
routes. The OS will invariably choose the first route.
In this case, to enhance bandwidth, it is recommended to use NIC bonding at the client level.
MULTIPLE SUBNETS
Use a separate subnet for each interface. With multiple subnets, there is no need to bond the network interfaces to
aggregate bandwidth across the available interfaces. The routing will be automatic in the case of multiple subnets.
Client interface NFS server
ens7f0 10.21.108.193 10.21.108.10
ens7f1 10.21.107.194 10.21.107.10
22
[root@mysql-m50-01 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.21.108.1 0.0.0.0 UG 0 0 0 ens7f0
10.21.108.0 * 255.255.255.0 U 0 0 0 ens7f0
10.21.107.0 * 255.255.255.0 U 0 0 0 ens7f1
In this case, these are two dynamic routes and, based on the traffic, the route is selected automatically.
As such, if you decide to use multiple subnets, they should be configured on both the client and the FlashBlade side.
Multiple subnets can be configured in the FlashBlade GUI under Network Settings.
LINUX MOUNT OPTIONS
To mount the NFS filesystem on Linux, use the following mount options. Do not specify the rsize or wsize options,
as the system can get the default offered by FlashBlade, which is 524288.
rw, bg, nointr, hard, tcp, vers=3, nolock, noac, actimeo=0
MYSQL CONFIGURATIONS
For InnoDB, enable the innodb_file_per_table that stores each table in its own .ibd file, which helps in space
management, speeds up truncate table operations, reduces fragmentation, and improves backup and restore time
when used with MySQL Enterprise Backup or Percona XtraBackup.
23
CONCLUSION
Businesses need a scalable and performant storage system to meet their recovery time objectives in the context
of a shrinking backup time window. Pure Storage FlashBlade provides a scalable, highly available, data reducible,
and performant solution to meet enterprise MySQL backup and restore requirements. The logical backup tests of
MySQL across 20 virtual machines illustrated FlashBlade write rates of over 1.5GB/s (limited only by MySQL’s backup
tool implementation), while the physical backup tests across 20 virtual machines yielded a rate of 3.85 GB/s by
taking advantage of the available bandwidth in FlashBlade. Similarly, the restore rates of 20 virtual machines using
physical backups yielded 2.73GB/s of read bandwidth from FlashBlade – which had a lot more room left to support
additional workloads.
Having FlashBlade as the backup target alone won’t provide stellar results unless the source system is capable of
reading and writing data out and in at the required rates. Without adequate speed, the source system becomes the
bottleneck. However, complementing FlashBlade, Pure’s FlashArray system provides the necessary performance,
with scalable read bandwidth for backup activity and comparable write bandwidth for restore activity.
24
REFERENCES
The following documents and links were referred to in preparing this document.
1. Pure Storage Community pages
https://support.purestorage.com
2. MySQL documentation
https://dev.mysql.com/doc/
3. Percona Xtrabackup
https://www.percona.com/software/mysql-database/percona-xtrabackup
25
ABOUT THE AUTHOR
Somu Rajarathinam is the Pure Storage Oracle Solutions Architect
responsible for defining database solutions based on the company’s
products, performing benchmarks, and developing reference
architectures for Oracle databases on Pure.
Somu has over 20 years of Oracle database experience, including
as a member of Oracle Corporation’s Systems Performance and
Oracle Applications Performance Groups. His career has also
included assignments with Logitech, Inspirage, and Autodesk, ranging
from providing database and performance solutions to managing
infrastructure, to delivering database and application support, both
in-house and in the cloud.
Web: www.somu.us
Twitter: @purelydb
[email protected] | 800-379-PURE | @PURESTORAGE
© 2017 Pure Storage, Inc. All rights reserved.
Pure Storage, the “P” Logo, FlashBlade, and Pure1 are trademarks or registered trademarks of Pure Storage, Inc. in the U.S. and other countries. Cisco UCS, and Cisco Nexus are registered trademarks of Cisco Systems, Inc. in the U.S. and other countries. Red Hat Enterprise Linux is the registered trademark of Red Hat, Inc. in the U.S. and other countries. MySQL is the trademark of Oracle, MariaDB Corporation AB and others.
The Pure Storage product described in this documentation is distributed under a license agreement and may be used only in accordance with the terms of the agreement. The license agreement restricts its use, copying, distribution, decompilation, and reverse engineering. No part of this documentation may be reproduced in any form by any means without prior written authorization from Pure Storage, Inc. and its licensors, if any.
THE DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. PURE STORAGE SHALL NOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING, PERFORMANCE, OR USE OF THIS DOCUMENTATION. THE INFORMATION CONTAINED IN THIS DOCUMENTATION IS SUBJECT TO CHANGE WITHOUT NOTICE.
ps_wp25p_mysql-fast-backup-and-restore-with-flashblade_ltr_01
Top Related