Joffrey MICHAÏE, CEO & Founder @ OceanDBABackup strategies for MySQL

MySQL User Group of Barcelona - 19-09-2016 @ Samsara

Agenda› Introduction› Backup Concepts› Different Backup Types

› Cold / Hot Backups› Logical Backups› Binary Backups

› HA as Backup› Backup Best Practices

Speaker Introduction› Made in France, Based in Barcelona, Mauritius Soon!› Unix Sysadmin (and Lead) from 2002 to 2009› MySQL Consultant 2009 to 2010

› Sun Microsystems (MySQL Ab)› Oracle

› Principal MySQL Consultant 2010 to 2015› SkySQL Ab / MariaDB Corporation

› Founder of OceanDBA› 2016 - ...

OceanDBA› Startup atmosphere ... Since April 2016› Database and Sysadmin health-care:

› Remote DBA› Consulting› Support› Sysadmin for web platforms

› Graphing and Monitoring

Backup Concepts› What is a Backup ?› What can we use backups for ?

› Data Recovery› Archiving / Accounting› Increase High-Availability› Setup Replication / Clustering› Deploy production data in dev / pre-prod / staging› Safety copy before new application update / schema change

Backup Concepts› Locking

› Do you need to block the database ? For how long ?› Storage engine or solution specifics

› Consistency› Do you need consistency ?› What happen do relational data changes performed during backup ?

› Backup Types› Full backup› Incremental› Differential

Backup Concepts› Backup is good, what about restore ?

› When was your last backup ?› Did you test it ?

› How long does it take to recover:› One line of data ?› One table ?› One database ?› One instance ?

Different Backup Types

Cold Backups› You need to stop service / lock the entire database› File copy: rsync / cp / tar / robotcopy› Backup is consistent› Require downtime, during all time of backup› (maybe) Good for your personal blog› Not good for amazon.com› You can use a slave/replica to reduce downtime

Different Backup TypesHot Backups

› Save your data while database is running!› File copy: rsync / cp / robotcopy

› Only for MyISAM * !› Can now be used for InnoDB (MariaDB 5.5/MySQL 5.7)

using InnoDB table-space import and export feature.› Logical Backups

› mysqldump / mydumper› Binary Backup

› SAN-VMware-ZFS-LVM Snapshot / Xtrabackup / MySQL Enterprise› Maybe consistent ...› May generate locking ...

Different Backup TypesLogical Backups 1 / 3

› Built-in feature: mysqldump / SELECT INTO OUTFILE› Open-source software (multi-threaded export and import): mydumper› Logical (Human readable) copy of your data:

› CREATE/INSERT statements› CSV like file per table

› One file per instance / schema / table› Useful to restore some parts of your data› Prevents from file-system / data corruption› Fast to export› Usually slow to import, can be multi-threaded if you have multiple files or use mydumper

Disable consistency / Binary log for faster restore, or use "libeatmydata":http://thelinuxjedi.blogspot.com/2011/05/libeatmydata-feed-me-seymour.html

Different Backup TypesLogical Backups 2 / 3 - mysqldump› By default, locks (only) the table being backed-up (remove if you don't want!)

› --lock-all-tables for 100% consistent backup (MyISAM or Mix)› --skip-lock-tables to disable locking (non consistent!)› --single-transaction for non-blocking and consistent InnoDB backup› --master-data=2 to store binary log position within backup› --tab to generate CSV like files. Output is one file per table

› You can pipe the output to a file / stream or directly to another mysql client!› Slow for larger datasets ( > 50GB)› Hard to split a dump into smaller part

› String manipulation / Sed experts : Have fun !

Different Backup TypesLogical Backups 3 / 3

› mysqlhotcopy (built-in tool)› Only good for MyISAM› Uses posix lock on the file› One file at a time

› mydumper (open-source, currently maintained by Percona)– Powerful parallel export and import– Very little locking for InnoDB– Useful to restore large data sets (>50GB)– No incremental / piping support (you can store data on filer)– Need a separate copy of schema (views / triggers / routines)

Different Backup Types

Binary Backups 1 / 3› Requires File-system or Disk subsystem access› Usually the fastest method to backup› Also the fastest to restore!› Doesn't prevent from binary corruption (bad memory /

controller / file-system / human mistake ...)

Different Backup TypesBinary Backups 2 / 3

› File system / proprietary snapshot› Requires a fast consistency lock (flush tables with read lock)

› Beware of long running transactions!› Lock the DB, start the snapshot, Unlock the DB› Not locking the DB may introduce data loss (check your settings!)› mylvmbackup is widely used› Good candidate for mixed databases (InnoDB/MyISAM/...)› Beware of performance hit while snapshot is running!

› Online binary copy› MySQL Enterprise Backup / Xtrabackup (open-source)› Usually The best solution for InnoDB databases

Different Backup TypesBinary Backups 3 / 3

› Xtrabackup (open-source version of MEB)› Live copy of a running DB› Non locking for transactional tables (InnoDB)› Requires locking when copying non-transactionnal data› Use --no-lock to remove locking (Can break consistency)

› Features:› Streaming / Compress / Encrypt› Incremental / Differential (InnoDB only)› Defragmentationhttps://www.percona.com/software/mysql-database/percona-xtrabackup

HA as Backup› HA can be used as backup solutions!

› MySQL Replication› Low overhead› Live stream of your production database› Onsite / Offsite streaming

› Galera Cluster› San / San Replication

› Beware: Human errors are replicated ( DROP production_table ...)› Can be combined with time-delayed replication!

› Pt-slave-delay› MySQL 5.6 / MariaDB 10.2 delayed slave› When a problem is detected, fast-forward replication thread to before problem

Binary log and PITR› Point in time recovery story ...

› Nightly full backup at 00:00› My database crashes at 23:50 (Murphy's law) ...› Is my boss OK if I restore last night backup ? Can we loose 23h50 of work ?

› Binary logs to the rescue!› Regular rsync› MariaDB Maxscale binlog server› Mysqldump binlog streaming

1) Restore last full backup2) Apply point-in-time recovery with mysqlbinlog3) You have restored you database up to 23:49 !

Backup Best Practices› What is the best backup solution (if you can afford) ?

› Regular binary backups (full + incrementals)› Regular logical backups (per table)› Regular binary log copy

› Don't forget to backup:› Configuration File(s)› Other objects (stored-procedures / triggers / events / UDFs)

› Keep the backup near production, but also keep an off-site copy› And finally, regularly test and monitor your backups!

› Restore often (Facebook test every backup!)› Set-up replication› Run consistency check (pt-table-checksum)› Sleep well !

Questions ? Thank you !