Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 +...

222

Transcript of Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 +...

Page 2: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Do I/you need a backup?

Percona XtraBackup: install, usage, tricks

Page 3: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 4: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Backups do not give you

More users More money

Increase productivity

Percona XtraBackup: install, usage, tricks

Page 5: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Backups are boring

Percona XtraBackup: install, usage, tricks

Page 6: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Not having backup

Percona XtraBackup: install, usage, tricks

Data loss

Time loss

Users loss

Money loss

Page 7: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Recovery time matters

Percona XtraBackup: install, usage, tricks

Page 8: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Regularly test your recovery procedure You may get surprised

Percona XtraBackup: install, usage, tricks

Page 9: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Backup – copy of data

Percona XtraBackup: install, usage, tricks

Page 10: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Why can’t we use just “cp”or “rsync”?

Percona XtraBackup: install, usage, tricks

Page 11: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 12: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 13: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 14: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 15: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 16: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 17: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

It works till first DROP TABLE or bad UPDATE Shhhh…. It happens

Percona XtraBackup: install, usage, tricks

Page 18: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Snapshots

Percona XtraBackup: install, usage, tricks

Page 19: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 20: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 21: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

LVM snapshot performance

Percona XtraBackup: install, usage, tricks

740

240

0

100

200

300

400

500

600

700

800

IO r

eq/s

ec

no snapshot

snapshot

Page 22: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

ZFS snapshots Reportedly works fine

Percona XtraBackup: install, usage, tricks

Page 23: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

ZFS available only

Solaris FreeBSD

Percona XtraBackup: install, usage, tricks

Page 24: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

SAN snapshots

Percona XtraBackup: install, usage, tricks

Page 25: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

NetApp

$$$

Percona XtraBackup: install, usage, tricks

Page 26: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Logical backup

Percona XtraBackup: install, usage, tricks

Page 27: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 28: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

“Backup that takes 30 hours can’t be a daily backup”

Percona XtraBackup: install, usage, tricks

Page 29: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Mysqdump benefits

Easy to use Per-table

Per-database

Percona XtraBackup: install, usage, tricks

Page 30: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Mysqdump problems

Intrusive Slow

Single thread

Percona XtraBackup: install, usage, tricks

Page 31: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Slave lag under mysqldump

Percona XtraBackup: install, usage, tricks

0

10

20

30

40

50

60

70

80

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76

Sla

ve

be

hin

d m

as

ter,

tim

e u

nit

----- Date-time ---

slave lag

Page 32: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

mydumper Parallel mysqldump

Percona XtraBackup: install, usage, tricks

Page 33: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

MySQL Enterprise Backup

Percona XtraBackup: install, usage, tricks

(previously InnoDB Hot Backup)

Page 34: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

MEB benefits

Similar to XtraBackup

Percona XtraBackup: install, usage, tricks

Page 35: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

MEB drawbacks

Closed source

Annual subscription

Percona XtraBackup: install, usage, tricks

Page 36: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Not true XtraBackup is “reverse engineering” of MEB

Percona XtraBackup: install, usage, tricks

Page 37: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

What is Percona XtraBackup

Open-source hot backup tool for MySQL-based servers

Percona XtraBackup: install, usage, tricks

Page 38: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 39: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 40: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 41: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 42: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 43: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Backup logic is identical to InnoDB recovery logic

Percona XtraBackup: install, usage, tricks

Page 44: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Backup logic is a part of

InnoDB source code No need for “reverse engineering”

Percona XtraBackup: install, usage, tricks

Page 45: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

XtraBackup source code structure

Patches to InnoDB

C code

Innobackupex script

xbstream utility

Percona XtraBackup: install, usage, tricks

Page 46: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

bzr branch lp:percona-xtrabackup https://launchpad.net/percona-xtrabackup

Percona XtraBackup: install, usage, tricks

Page 47: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Build tools

● build/ directory

● build.sh

● build.sh xtradb55

● build-rpm.sh

● xtrabackup.spec

● build-dpkg.sh

● build-binary.sh

● .tar.gz

Percona XtraBackup: install, usage, tricks

Page 48: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Binary packages

Percona XtraBackup: install, usage, tricks

Page 49: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Yum repository

• rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-2.0.3.x86_64.rpm

• yum install percona-xtrabackup

Percona XtraBackup: install, usage, tricks

Page 50: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Apt repository

• deb http://repo.percona.com/apt squeeze main • Replace “squeeze” by your favorite name

• deb-src http://repo.percona.com/apt squeeze main • apt-get update

Percona XtraBackup: install, usage, tricks

Page 51: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Binary packages

• http://www.percona.com/downloads/XtraBackup/XtraBackup-2.0.3/ • Binary tar.gz

• Centos based, may not work on SUSE

• deb

• RPM

• RedHat based

• source

Percona XtraBackup: install, usage, tricks

Page 52: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 53: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Supported engines • InnoDB/XtraDB

• Hot backup

• MyISAM • With read-lock

• Archive, CSV • With read-lock

• Your favorite exotic engine • May work if supports FLUSH TABLES WITH READ LOCK

Percona XtraBackup: install, usage, tricks

Page 54: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Supported platforms

• Linux • RedHat 5; RedHat 6

• CentOS, Oracle Linux

• Debian 6

• Ubuntu LTS

• Any other via compiling source code

Percona XtraBackup: install, usage, tricks

Page 55: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Windows

• Experimental releases

• Active Perl

Percona XtraBackup: install, usage, tricks

Page 56: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Solaris, OSX, FreeBSD

• “should work”

Percona XtraBackup: install, usage, tricks

• Build from source packages

Page 57: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Binaries structure

• innobackupex • Perl script

• xtrabackup • Percona Server 5.1; MySQL 5.1 + InnoDB-plugin

• xtrabackup_51 • MySQL 5.0; Percona Server 5.0; MySQL 5.1+builtin InnoDB

• xtrabackup_55 • MySQL 5.5; Percona Server 5.5

• xbstream • xbstream format support

Percona XtraBackup: install, usage, tricks

Page 58: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 59: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 60: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 61: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 62: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 63: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Basic command

• innobackupex /data/backup/mysql-data

Percona XtraBackup: install, usage, tricks

Page 64: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Innobackupex connects to mysqld

• --defaults-file • Datadir (default assumption /var/lib/mysql) !

• --user • --password • --host

• If you need to connected via specific IP

• --socket

Percona XtraBackup: install, usage, tricks

Page 65: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Innobackupex basic output

Percona XtraBackup: install, usage, tricks

$ innobackupex --user=DBUSER --password=DBUSERPASS /mnt/backup/...innobackupex: Backup created in directory '/mnt/backup/2012-09-30_00-00-09'innobackupex: MySQL binlog position: filename 'mysql-bin.000003', position 1946120930 00:00:53 innobackupex: completed OK!

Page 66: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 67: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

apply command

• innobackupex --apply-log /mnt/backup/2012-09-30_00-00-09

Percona XtraBackup: install, usage, tricks

Page 68: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 69: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Use memory

• innobackupex --apply-log --use-memory=10G /mnt/backup/2012-09-30_00-00-09

• use-memory == innodb_buffer_pool_size for mysqld

Percona XtraBackup: install, usage, tricks

Page 70: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Why xtrabackup –prepare 2 times?

• Second time to create innodb log files

• Just for convenience

Percona XtraBackup: install, usage, tricks

Page 71: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

/mnt/backup/2012-09-30_00-00-09

is ready for usage

Percona XtraBackup: install, usage, tricks

Page 72: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 73: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

FTWRL

• set the global read lock - after this step, insert/update/delete/replace/alter statements cannot run

• close open tables - this step will block until all statements started previously have stopped

• set a flag to block commits

Percona XtraBackup: install, usage, tricks

Page 74: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Why? Consider:

• Copy table1.frm • Copy table2.frm • Copy table3.frm • Copy table4.frm

• ...... Meantine ALTER TABLE table1 started …………..

• Copy table5.frm

• Copy table6.frm • Copy table7.frm

Percona XtraBackup: install, usage, tricks

Page 75: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

With FTWRL

• FLUSH TABLES WITH READ LOCK

• Copy table1.frm • Copy table2.frm • Copy table3.frm • Copy table4.frm

• ...... ALTER TABLE table1 --- LOCKED …………..

• Copy table5.frm

• Copy table6.frm • Copy table7.frm • UNLOCK TABLES

Percona XtraBackup: install, usage, tricks

Page 76: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

The same with MyISAM Not transactional engine. No redo logs

Percona XtraBackup: install, usage, tricks

Page 77: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

FTWRL problems

Percona XtraBackup: install, usage, tricks

Page 78: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Problem 1

• close open tables - this step will block until all statements started previously have stopped

• Long running SELECT will block FTWRL, which will block other statements – Queries piles up and lock down

server

Percona XtraBackup: install, usage, tricks

Page 79: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Pileup

• SELECT 1mln rows FROM BIG_TABLE • Blocks: FLUSH TABLES WITH READ LOCK

• Blocks all following queries:

• SELECT …

• UPDATE …

• SELECT …

Percona XtraBackup: install, usage, tricks

Page 80: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Solution: shoot long queries

Percona XtraBackup: install, usage, tricks

Page 81: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Problem 2

• Big MyISAM tables

• Block write queries on time copying MyISAM

• Example: you created 100GB MyISAM table and forgot about

Percona XtraBackup: install, usage, tricks

Page 82: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

--no-lock

Percona XtraBackup: install, usage, tricks

Page 83: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

--no-lock Do not execute FLUSH TABLES WITH READ LOCK

Percona XtraBackup: install, usage, tricks

Page 84: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 85: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

You take responsibility

• There is NO ALTER TABLE statements

• There is NO INSERT/UPDATE/DELETE to MyISAM tables

• What about Binary log position? • We will cover it

Percona XtraBackup: install, usage, tricks

Page 86: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Directory after backup

• MySQL files +

• xtrabackup_binlog_info • xtrabackup_binlog_pos_innodb

• Only after –apply-log

• xtrabackup_slave_info • When –slave-info is used

• xtrabackup_checkpoints • xtrabackup_logfile • xtrabackup_binary • backup-my.cnf

Percona XtraBackup: install, usage, tricks

Page 87: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup_binlog_info

• Master binary log position

• Result of SHOW MASTER STATUS

Percona XtraBackup: install, usage, tricks

binlog.000001 68212201

Page 88: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup_checkpoints

• Information about InnoDB checkpoints LSN, useful for incremental backup

Percona XtraBackup: install, usage, tricks

Page 89: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup_logfile

• Can be very big

• The bigger this file – the longer –apply-log process

Percona XtraBackup: install, usage, tricks

Page 90: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup_binary

• Exact xtrabackup binary used for backup

• This is used for –apply-log

Percona XtraBackup: install, usage, tricks

Page 91: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

backup-my.cnf

• NOT BACKUP of my.cnf

• Information to start mini-InnoDB instance during --apply-log

• Example:

Percona XtraBackup: install, usage, tricks

[mysqld] datadir=/data/bench/back/test/2012-04-06_13-09-42 innodb_data_home_dir=/data/bench/back/test/2012-04-06_13-09-42 innodb_log_files_in_group=2 innodb_log_file_size=1992294400 innodb_fast_checksum=0 innodb_page_size=16384 innodb_log_block_size=4096

Page 92: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Summary

Backups are important

Recovery time is important

Backup methods for MySQL

Percona XtraBackup overview

XtraBackup basic usage

Percona XtraBackup: install, usage, tricks

Page 93: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

End of part 1

Percona XtraBackup: install, usage, tricks

Questions ?

Page 94: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Part 2:● Minimizing footprint

– I/O throttling

– FS cache optimization

● Parallel file copying● Restoring individual tables● Partial backups

– individual partitions backup

Page 95: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Minimizing footprint: I/O throttling--throttle=N

Limit the number of I/O operations per second in 1 MB units

xtrabackup --throttle=1 ...

readread writewrite readread writewrite

Second 1

readread writewrite readread writewrite

Second 2

readread writewrite readread writewrite

Second 3

readread writewrite waitwait

Second 1

readread writewrite waitwait

Second 2

readread writewrite waitwait

Second 3

Page 96: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Minimizing footprint: I/O throttlingLimitation:

--throttle only has effect for InnoDB tables, other files are copied by the innobackupex script with cp or rsync.

Use streaming backups and the pv utility as a workaround:

$ innobackupex --stream=tar /tmp | pv -q -L1m | tar -xf - -C /data/backup

● pv utility● monitor the progress of data through a pipe● available in most Linux distributions● -L to limit the transfer to a specified rate

Page 97: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 98: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Minimizing footprint:FS cache optimizations

● XtraBackup on Linux:– posix_fadvise(POSIX_FADV_DONTNEED)

– hints the kernel the application will not need the specified bytes again

– works automatically, no option to enable● Didn't really work in XtraBackup 1.6, fixed in 2.0

Page 99: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Parallel file copying

● creates N threads, each thread copying one file at a time● utilizes disk hardware by copying multiple files in parallel

– best for SSDs– less seeks on HDDs due to more merged reqs by I/O scheduler– YMMV, benchmarking before using is recommended

Data directoryData directory

ibdata1ibdata1

actor.ibdactor.ibd

customer.ibdcustomer.ibd

film.ibdfilm.ibd

Backup locationBackup location--parallel=4

ibdata1

film.ibd

customer.ibd

actor.ibd

Page 100: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Parallel file copying

$ innobackupex --parallel=4 --no-timestamp /data/backup...[01] Copying ./ibdata1 to /data/backup/ibdata1[02] Copying ./sakila/actor.ibd to /data/backup/./sakila/actor.ibd[03] Copying ./sakila/customer.ibd to /data/backup/./sakila/customer.ibd[04] Copying ./sakila/film.ibd to /data/backup/./sakila/film.ibd

● works only with multiple InnoDB tablespaces(innodb_file_per_table=1)

● in XtraBackup 1.6 could not be used with streaming backups

● works with any backup types in XtraBackup 2.0

Page 101: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Restoring individual tables:export

● problem: restore individual InnoDB table(s) from a full backup to another server● use --export to prepare

● use improved table import feature in Percona Server to restore● innodb_file_per_table=1

Full backupFull backup

ibdataibdata

actor.ibdactor.ibd

customer.ibdcustomer.ibd

film.ibdfilm.ibd

export

Server B

ibdataibdata

actor.ibdactor.ibd

customer.ibdcustomer.ibd

film.ibdfilm.ibd

Server A

ibdataibdata

actor.ibdactor.ibd

customer.ibdcustomer.ibd

film.ibdfilm.ibd

backup

Page 102: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Restoring individual tables:export

Why not just copy the .ibd file?

● metadata:

– InnoDB data dictionary (space ID, index IDs, pointers to root index pages)

– .ibd page fields (space ID, LSNs, transaction IDs, index ID, etc.)

● xtrabackup --export dumps index metadata to .exp files on prepare

● Percona Server uses .exp files to update both data dictionary and .ibd on import

Page 103: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Restoring individual tables: export

$ xtrabackup --prepare --export --innodb-file-per-table=1 --target-dir=/data/backup

...

xtrabackup: export metadata of table 'sakila/customer' to file `./sakila/customer.exp` (4 indexes)

xtrabackup: name=PRIMARY, id.low=23, page=3

xtrabackup: name=idx_fk_store_id, id.low=24, page=4

xtrabackup: name=idx_fk_address_id, id.low=25, page=5

xtrabackup: name=idx_last_name, id.low=26, page=6

...

Page 104: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Restoring individual tables:import

● improved import only available in Percona Server● can be either the same or a different server instance:

– (on different server to create .frm) CREATE TABLE customer(...);

– SET FOREIGN_KEY_CHECKS=0;

– ALTER TABLE customer DISCARD TABLESPACE;

– <copy customer.ibd to the database directory>

– SET GLOBAL innodb_import_table_from_xtrabackup=1; (Percona Server 5.5)orSET GLOBAL innodb_expand_import=1; (Percona Server 5.1)

– ALTER TABLE customer IMPORT TABLESPACE;

– SET FOREIGN_KEY_CHECKS=1;

Page 105: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Restoring individual tables:import

● Improved table import is only available in Percona Server

● tables can only be imported to the same server with MySQL (with limitations):– there must be no DROP/CREATE/TRUNCATE/ALTER

between taking backup and importing the table

mysql> ALTER TABLE customer DISCARD TABLESPACE;

<copy customer.ibd to the database directory>

mysql> ALTER TABLE customer IMPORT TABLESPACE;

Page 106: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups● backup individual tables/schemas rather than the

entire dataset● InnoDB tables:

– require innodb_file_per_table=1

– restored in the same way as individual tables from a full backup

– same limitations with the standard MySQL server (same server, no DDL)

– no limitations with Percona Server when innodb_import_table_from_xtrabackup is enabled

Page 107: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:selecting what to backup

innobackupex:● streaming backups:

● --databases=”database1[.table1] ...”,e.g.: --databases=”employees sales.orders”

● local backups:● --tables-file=filename, file contains database.table, one per line

● --include=regexp, e.g.: --include='^database(1|2)\.reports.*'

xtrabackup:● --tables-file=filename (same syntax as with innobackupex)

● --tables=regexp (equivalent to --include in innobackupex)

Page 108: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:selecting what to backup

innobackupex:● streaming backups:

● --databases=”database1[.table1] ...”,e.g.: --databases=”employees sales.orders”

● local backups:● --tables-file=filename, file contains database.table, one per line

● --include=regexp, e.g.: --include='^database(1|2)\.reports.*'

xtrabackup:● --tables-file=filename (same syntax as with innobackupex)

● --tables=regexp (equivalent to --include in innobackupex)

Page 109: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:selecting what to backup

innobackupex:● streaming backups:

● --databases=”database1[.table1] ...”,e.g.: --databases=”employees sales.orders”

● local backups:● --tables-file=filename, file contains database.table, one per line

● --include=regexp, e.g.: --include='^database(1|2)\.reports.*'

xtrabackup:● --tables-file=filename (same syntax as with innobackupex)

● --tables=regexp (equivalent to --include in innobackupex)

Page 110: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:selecting what to backup

innobackupex:● streaming backups:

● --databases=”database1[.table1] ...”,e.g.: --databases=”employees sales.orders”

● local backups:● --tables-file=filename, file contains database.table, one per line

● --include=regexp, e.g.: --include='^database(1|2)\.reports.*'

xtrabackup:● --tables-file=filename (same syntax as with innobackupex)

● --tables=regexp (equivalent to --include in innobackupex)

Page 111: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:preparing

$ xtrabackup --prepare --export --target-dir=./

...120407 18:04:57 InnoDB: Error: table 'sakila/store'InnoDB: in InnoDB data dictionary has tablespace id 24,InnoDB: but tablespace with that id or name does not exist. It will be removed from data dictionary....xtrabackup: export option is specified.xtrabackup: export metadata of table 'sakila/customer' to file `./sakila/customer.exp` (4 indexes)xtrabackup: name=PRIMARY, id.low=62, page=3xtrabackup: name=idx_fk_store_id, id.low=63, page=4xtrabackup: name=idx_fk_address_id, id.low=64, page=5xtrabackup: name=idx_last_name, id.low=65, page=6...

Page 112: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:restoring

● Non-InnoDB tables– just copy files to the database directory

● InnoDB (MySQL):– ALTER TABLE ... DISCARD/IMPORT TABLESPACE

– same limitations on import (must be same server, no ALTER/DROP/TRUNCATE after backup)

● XtraDB (Percona Server):

– xtrabackup --export on prepare

– innodb_import_table_from_xtrabackup=1;

– ALTER TABLE ... DISCARD/IMPORT TABLESPACE

– no limitations

Page 113: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

reports

Partial backups:individual partitions

CREATE TABLE reports (d DATE, t TEXT)PARTITION BY RANGE (YEAR(d))( PARTITION p_2010 VALUES LESS THAN (2010), PARTITION p_2011 VALUES LESS THAN (2011), PARTITION p_2012 VALUES LESS THAN (2012), PARTITION p_cur VALUES LESS THAN (MAXVALUE));

Problem:● archive p_2010 first● purge p_2010

p_2011

p_2010

p_cur

p_2012

Page 114: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:individual partitions

Can I backup individual partitions with XtraBackup?

Yes!

MyISAM or InnoDB (with multiple tablespaces) partitions are like normal tables with specially formatted names:

$ ls -1 /usr/local/mysql/data/test/reports*

/usr/local/mysql/data/test/reports#P#p_2010.ibd

/usr/local/mysql/data/test/reports#P#p_2011.ibd

/usr/local/mysql/data/test/reports#P#p_2012.ibd

/usr/local/mysql/data/test/reports#P#p_cur.ibd

/usr/local/mysql/data/test/reports.frm

/usr/local/mysql/data/test/reports.par

Page 115: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:individual partitions

$ xtrabackup --backup --datadir=/usr/local/mysql/data/ --tables='reports#P#p_2010'

...

[01] Copying ./ibdata1 to /data/backup/ibdata1[01] ...done[01] Copying ./test/reports#P#p_2010.ibd to /data/backup/./test/reports#P#p_2010.ibd[01] ...done[01] Skipping ./test/reports#P#p_2011.ibd

[01] Skipping ./test/reports#P#p_2012.ibd

[01] Skipping ./test/reports#P#p_cur.ibd

Page 116: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:individual partitions

To prepare:$ xtrabackup --prepare --export --innodb_file_per_table --target-dir=./

$ ls -1 test/

reports#P#p_2010.expreports#P#p_2010.ibd

Page 117: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:individual partitions

To restore:● no support for attaching partitions in MySQLmysql> CREATE TABLE reports_2010( d DATE, t TEXT);

mysql> ALTER TABLE reports_2010 DISCARD TABLESPACE;

Page 118: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Partial backups:individual partitionsTo restore (continued):

$ cp /data/backup/test/reports#P#p_2010.exp /usr/local/mysql/data/test/reports_2010.exp

$ cp /data/backup/test/reports#P#p_2010.ibd /usr/local/mysql/data/test/reports_2010.ibd

mysql> SET GLOBAL innodb_import_table_from_xtrabackup=1;

mysql> ALTER TABLE report_2010 IMPORT TABLESPACE;

● same procedure to backup/restore MyISAM partitions (except DISCARD/IMPORT TABLESPACE)

Page 119: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

End of part 2

Percona XtraBackup: install, usage, tricks

Questions ?

Page 120: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Part 3

Percona XtraBackup: install, usage, tricks

Page 121: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Streaming Unique for XtraBackup

Percona XtraBackup: install, usage, tricks

Page 122: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Local copy to

• Local partitions

• NFS partitions

Percona XtraBackup: install, usage, tricks

Page 123: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Unix pipes

• tar zcvf - /wwwdata | ssh [email protected] "cat > /backup/wwwdata.tar.gz“

• We want: • innobackupex | script1 | scrip2 | …

Percona XtraBackup: install, usage, tricks

Page 124: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 125: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Basic command

• innobackupex --stream=tar /tmpdir

Percona XtraBackup: install, usage, tricks

Page 126: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Stream output

Percona XtraBackup: install, usage, tricks

$ innobackupex --stream=tar /tmp > /data/backup.tar...[01] Streaming ./ibdata1[01] ...done[01] Streaming ./sakila/actor.ibd[01] ...done[01] Streaming ./sakila/address.ibd[01] ...done...xtrabackup: Streaming transaction log from a temporary file...xtrabackup: Done.xtrabackup: Transaction log of lsn (1597945) to (8440939) was copied.120925 13:59:21 innobackupex: All tables unlocked120925 13:59:21 innobackupex: Connection to database server closed

innobackupex: Backup created in directory '/tmp'innobackupex: MySQL binlog position: filename 'mysql-bin.000001', position 3253178innobackupex: You must use -i (--ignore-zeros) option for extraction of the tar stream.120925 13:59:21 innobackupex: completed OK!

Page 127: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Tar -i

• -i is IMPORTANT

Percona XtraBackup: install, usage, tricks

> tar --list -f backup.tar backup-my.cnf

> tar --list -i -f backup.tar backup-my.cnf ibdata1 ./sbtest/sbtest3.ibd ./sbtest/sbtest2.ibd ./sbtest/sbtest1.ibd ./sbtest/sbtest4.ibd xtrabackup_binlog_info mysql/time_zone_leap_second.MYI mysql/ndb_binlog_index.MYD mysql/plugin.frm mysql/time_zone_leap_second.MYD mysql/time_zone_name.frm mysql/tables_priv.frm mysql/event.MYD mysql/time_zone_name.MYI mysql/ndb_binlog_index.MYI mysql/proxies_priv.MYD mysql/proxies_priv.MYI mysql/func.MYI

Page 128: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Use cases:

Percona XtraBackup: install, usage, tricks

– compression:● innobackupex --stream=tar . | gzip - > /data/backup/backup.tar.gz

– encryption:● innobackupex --stream=tar . | openssl des3 -salt -k “password” > backup.tar.des3

– remote backup:● innobackupex --stream=tar . | ssh user@host “tar -xif - -C /data/backup”

– compressed + encrypted + remote backup:● innobackupex --stream=tar . | gzip - | openssl des3 -salt -k “password” | ssh @user@host “cat - > /data/backup.tar.gz.des3”

Page 129: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Incremental backups

Percona XtraBackup: install, usage, tricks

Page 130: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 131: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 132: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 133: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 134: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 135: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 136: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Log Sequential Number LSN

Percona XtraBackup: install, usage, tricks

Page 137: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

LSN

Percona XtraBackup: install, usage, tricks

SHOW ENGINE INNODB STATUS\G --- LOG --- Log sequence number 180661841734

Page 138: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 139: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 140: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 141: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 142: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 143: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Xtrabackup_checkpoints

Percona XtraBackup: install, usage, tricks

cat xtrabackup_checkpoints backup_type = full-backuped from_lsn = 0

to_lsn = 172573554953 last_lsn = 172728858570

Page 144: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 145: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 146: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Innobackupex incremental:

• Handles incremental changes to InnoDB

• DOES not handle MyISAM and other engines • Full copy instead

Percona XtraBackup: install, usage, tricks

Page 147: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 148: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Innobackupex commands

• innobackupex --incremental \ --incremental-basedir= /previous/full/or/incremental \ /data/backup/inc

Percona XtraBackup: install, usage, tricks

Page 149: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Innobackupex from LSN

• innobackupex --incremental \ --incremental-lsn=172573554953 \ /data/backup/inc

Percona XtraBackup: install, usage, tricks

Page 150: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Innobackup incremental output

Percona XtraBackup: install, usage, tricks

$ innobackupex --incremental --incremental-basedir=/mnt/backup/2012-09-30_17-21-56 /mnt/backup...incremental backup from 1706739 is enabled.[01] Copying ./ibdata1 to /mnt/backup/2012-10-01_17-21-56/ibdata1.delta[01] ...done[01] Copying ./incremental_sample/t2.ibd to /mnt/backup/2012-10-01_17-21-56/./incremental_sample/t2.ibd.delta[01] ...done[01] Copying ./incremental_sample/test.ibd to /mnt/backup/2012-10-01_17-21-56/./incremental_sample/test.ibd.delta[01] ...done...innobackupex: Backup created in directory '/mnt/backup/2012-10-01_17-21-56'innobackupex: MySQL binlog position: filename 'mysql-bin.000001', position 418155121001 17:22:16 innobackupex: completed OK!

Page 151: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Incremental apply complications

Percona XtraBackup: install, usage, tricks

Page 152: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 153: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 154: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 155: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

No need for ROLLBACK if we apply incremental later

Percona XtraBackup: install, usage, tricks

Page 156: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

innobackupex --redo-only

• Full backup: apply with --redo-only

• innobackupex --apply-log --redo-only/mnt/backup/full

Percona XtraBackup: install, usage, tricks

Page 157: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Innobackupex apply incremental

• innobackupex --apply-log /mnt/backup/full --incremental-dir=/mnt/backup/incremental

Percona XtraBackup: install, usage, tricks

Page 158: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Incremental streaming New in Percona-XtraBackup 2.0

Percona XtraBackup: install, usage, tricks

Page 159: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Why not in 1.6?

• Tar format has limitations • We need to know filesize before sending to tar

• New streaming format xbstream

• Xbstream supports compression and incremental backups • Custom format

Percona XtraBackup: install, usage, tricks

Page 160: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Usage:

• innobackupex --incremental --incremental-basedir=/mnt/backup/full --stream=xbstream ./ > incremental.bak

• Unpack:

• xbstream –x < incremental.bak

• Incremental backup directly to remote server

• innobackupex --incremental --incremental-basedir=/mnt/backup/full --stream=xbstream ./ | ssh user@desthost "cat - | xbstream –x -C /mnt/backup/incremental"

Percona XtraBackup: install, usage, tricks

Page 161: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 162: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Point-In-Time Recovery PITR

Percona XtraBackup: install, usage, tricks

Page 163: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 164: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 165: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 166: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 167: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

XtraBackup provides binlog coordinates:

Percona XtraBackup: install, usage, tricks

innobackupex: Backup created in directory '/mnt/backup/full'innobackupex: MySQL binlog position: filename 'mysql-bin.000001', position 418155120925 17:22:16 innobackupex: completed OK!

Page 168: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup_binlog_info

• Master binary log position

• Result of SHOW MASTER STATUS

Percona XtraBackup: install, usage, tricks

binlog.000001 68212201

Page 169: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup_binlog_pos_innodb

• Master binary log position STORED IN InnoDB (ibdata1)

• Available only after –apply-log

• Can be used with –no-lock

Percona XtraBackup: install, usage, tricks

./binlog.000001 68212201

Page 170: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

In innobackupex output

• Master binary log position STORED IN InnoDB (ibdata1)

• Available only after –apply-log

• Can be used with –no-lock

Percona XtraBackup: install, usage, tricks

[notice (again)] If you use binary log and don't use any hack of group commit, the binary log position seems to be: InnoDB: Last MySQL binlog file position 0 68212201, file name ./binlog.000001

Page 171: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

mysqlbinlog

• mysqlbinlog to apply binary logs from position

• mysqlbinlog –start-position=68212201 binlog.000001

• mysqlbinlog --stop-datetime= • To stop at exact time

Percona XtraBackup: install, usage, tricks

Page 172: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Slave topics

Percona XtraBackup: install, usage, tricks

Page 173: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 174: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup_slave_info

• innobackupex –slave-info • Master binary log position for slave

• Result of SHOW SLAVE STATUS

Percona XtraBackup: install, usage, tricks

CHANGE MASTER TO MASTER_LOG_FILE='binlog.000001', MASTER_LOG_POS=68212201

Page 175: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup_slave_info

• Used for PITR when we backup from slave

• Slave cloning

Percona XtraBackup: install, usage, tricks

Page 176: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 177: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Using streaming

• innobackupex --stream=tar /tmp --slave-info | ssh user@NEWSLAVE "tar xfi - -C /DESTDIR"

• innobackupex --apply-log /DESTDIR

• Execute CHANGE MASTER from xtrabackup_slave_info

Percona XtraBackup: install, usage, tricks

Page 178: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 179: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

rsync for file copying

Percona XtraBackup: install, usage, tricks

Page 180: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 181: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Rsync

• innobackupex –rsync

• Two stage • 1. rsync before FLUSH TABLES WITH READ LOCK

• 2. rsync changes (if any) after FLUSH TABLES WITH READ LOCK

• Can’t be used with –stream

Percona XtraBackup: install, usage, tricks

Page 182: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Rsync output

Percona XtraBackup: install, usage, tricks

$ innobackupex --rsync /mnt/backup...120926 15:23:44 Starting rsync as: rsync -t "/var/lib/mysql" --files-from="/tmp/xtrabackup_rsyncfiles_pass1" "/mnt/backup/2012-09-26_15-23-36"120926 15:23:45 rsync finished successfully.120926 15:23:45 innobackupex: Finished a prep copy of .frm, .MRG, .MYD, .MYI, .TRG, .TRN, .ARM, .ARZ, .CSV, .CSM and .opt files...120926 15:23:55 innobackupex: All tables locked and flushed to disk120926 15:23:55 Starting rsync as: rsync -t "/var/lib/mysql" --files-from="/tmp/xtrabackup_rsyncfiles_pass2" "/mnt/backup/2012-09-26_15-23-36"120926 15:23:55 rsync finished successfully....120926 15:23:58 innobackupex: All tables unlocked

Page 183: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Statistics

Percona XtraBackup: install, usage, tricks

Page 184: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 185: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 186: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot
Page 187: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

xtrabackup --stats

Percona XtraBackup: install, usage, tricks

table: tpcc/order_line, index: PRIMARY, space id: 25, root page: 3, zip size: 0 estimated statistics in dictionary: key vals: 32471816, leaf pages: 264267, size pages: 302592 real statistics: level 2 pages: pages=1, data=8406 bytes, data/pages=51% level 1 pages: pages=467, data=4756806 bytes, data/pages=62% leaf pages: recs=35116662, pages=264267, data=2410713870 bytes, data/pages=55%

Page 188: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Optimize?

• OPTIMIZE TABLE for Primary Key (data)

• DROP INDEX / CREATE INDEX (via Fast Index Creation) for Secondary Keys

Percona XtraBackup: install, usage, tricks

Page 189: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Backup schemas

Percona XtraBackup: install, usage, tricks

Page 190: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Daily full backups

• /data/backup/2012-04-01

• /data/backup/2012-04-02

• /data/backup/2012-04-03

• /data/backup/2012-04-04

• /data/backup/2012-04-05

• …

• As many copies as you need

Percona XtraBackup: install, usage, tricks

Page 191: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Daily full backups + binary logs

• /data/backup/2012-04-01

• /data/backup/2012-04-02

• /data/backup/2012-04-03

• /data/backup/2012-04-04

• /data/backup/2012-04-05

• …

• As many copies as you need

• + backup of binary logs

Percona XtraBackup: install, usage, tricks

Page 192: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Daily backup with local copy

• /data/backup/2012-04-01

• /data/backup/2012-04-02

• /data/backup/2012-04-03

• /data/backup/2012-04-04

• /data/backup/2012-04-05

• …

• As many copies as you need

• The latest copy is stored also locally

• Compressed

Percona XtraBackup: install, usage, tricks

Page 193: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Incremental into to the mix

• For low rate of changes. ~10%

Percona XtraBackup: install, usage, tricks

Page 194: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Schema

• Full /data/backup/2012-04-01

• Incremental /data/backup/2012-04-02

• Incremental /data/backup/2012-04-03

• Incremental /data/backup/2012-04-04

• Full /data/backup/2012-04-05

• …

Percona XtraBackup: install, usage, tricks

Page 195: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

We do not provide scripts DIY or use third-party tools

Percona XtraBackup: install, usage, tricks

Page 196: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Third-party tools

• Zmanda Recovery Manager

• XtraBackup manager

Percona XtraBackup: install, usage, tricks

Page 197: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

End of part 3

Percona XtraBackup: install, usage, tricks

Questions ?

Page 198: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Part 4: New Features● XtraBackup 2.0

– streaming incremental backups

– parallel compression

– xbstream

– LRU dump backups

● Future releases– true incremental backups

– log archiving

– compact backups

– encrypted backups

– merged innobackupex + xtrabackup

Page 199: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Streaming incremental backupsProblem: send an incremental backup to a remote host:

innobackupex --stream=tar --incremental ... | ssh ...

● didn't work in XtraBackup 1.6

– innobackupex used external utilities to generate TAR streams, didn't invoke xtrabackup binary

– xtrabackup binary must be used for incremental backups to scan data files and generate deltas

● in XtraBackup 2.0:

– xtrabackup binary can produce TAR or XBSTREAM streams on its own

Page 200: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Compression● disk space is often a problem● compression with external utilities has some

limitations● built-in parallel compression in XtraBackup 2.0

Page 201: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Compression:external utilities

● local backups:– create a local uncompressed backup first, then gzip files– must have sufficient disk space on the same machine– data is read and written twice

● streaming backups:– innobackupex --stream=tar ./ | gzip - > /data/backup.tar.gz

– innobackupex --stream=tar ./ | gzip - | ssh user@host “cat - > /data/backup.tar.gz”

– gzip is single-threaded

– pigz (parallel gzip) can do parallel compression, but decompression is still single-threaded

– have to uncompress the entire .tar.gz even to restore a single table

Page 202: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Compression:XtraBackup 2.0

● new --compress option in both innobackupex and xtrabackup

● QuickLZ compression algorithm: http://www.quicklz.com/– “the world's fastest compression library, reaching 308 Mbyte/s per core”– combines excellent speed with decent compression

(8x in tests)– more algorithms (gzip, bzip2) will be added later

● qpress archive format (the native QuickLZ file format)● each data file becomes a single-threaded .qp archive

– no need to uncompress entire backup to restore a single table as with .tar.gz

Page 203: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Compression:XtraBackup 2.0

● parallel! --compress-threads=N

● can be used together with parallel file copying:xtrabackup --backup --parallel=4 --compress --compress-threads=8

Data directoryData directory

ibdata1ibdata1

actor.ibdactor.ibd

customer.ibdcustomer.ibd

film.ibdfilm.ibd

I/O threads Compressionthreads

1

2

3

4

1

8765432

Page 204: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Compression:XtraBackup 2.0

● qpress file archiver

– with --compress every file in the backup is a single-file qpress archive (e.g. customer.ibd.qp)

– available from http://www.quicklz.com/ – multi-threaded decompression:

● qpress -d -TN customer.ibd.qp ./● support for gzip/bzip2 to be added later

Page 205: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

xbstream● new streaming format in XB 2.0 in addition to --stream=tar

● Problems with traditional archive formats (tar, cpio, ar):– simultaneous compression and streaming is impossible

.tar file format

header actor.ibd.gz header customer.ibd.gz header film.ibd.gz ...

File name

File mode

File size

Checksum

...

have to know the file size beforeadding a file to archive => have to writeto a temporary file first before streaming

Page 206: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

xbstream (XtraBackup 2.0)● Problems with traditional archive formats:

– no parallel streaming => can only backup single file at a time

● meant as file archiving utilities:

– not really suitable for dynamically generated data streams like backups

– XtraBackup 1.6 didn't support compression, parallel streaming, so using TAR streams was possible

Page 207: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

xbstream (XtraBackup 2.0)

...

actor.ibd.gz chunk #1

chunk header

chunk header

film.ibd.gz chunk #1

header

actor.ibd.gz chunk #2

actor.ibd.gz chunk #1actor.ibd.gz chunk #1

chunk header

chunk header

film.ibd.gz chunk #2

● the XBSTREAM format– files are written in chunks

=> can stream dynamically generated files

– support interleaving chunks from different files => can do parallel file streaming

– CRC32 checksums =>can be optimized to use CPU instructions

Page 208: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

xbstream (XtraBackup 2.0)The xbstream utility

● can be used to extract files from streams produced by xtrabackup

● tar-like interface:● xbstream -x [-C directory)

read stream from the standard input and extract files to the current (or specified with -C) directory

Page 209: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

xbstream (XtraBackup 2.0)xbstream usage example

$ innobackupex --parallel=8 --compress --compress-threads=4 --stream=xbstream ./ | ssh user@host ”xbstream -x -C /data/backup”

...

xtrabackup: Starting 8 threads for parallel data files transfer[01] Compressing and streaming ./ibdata1[02] Compressing and streaming ./sakila/actor.ibd[03] Compressing and streaming ./sakila/address.ibd[04] Compressing and streaming ./sakila/category.ibd[05] Compressing and streaming ./sakila/city.ibd[06] Compressing and streaming ./sakila/country.ibd[07] Compressing and streaming ./sakila/customer.ibd

Page 210: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

LRU dump backup(XtraBackup 2.0)

LRU dumps in Percona Server:

page1page1 page2page2 page3page3 ......

InnoDB buffer poolmost recently used least recently used

ib_lru_dump

id id id id

● Reduced warmup time by restoring buffer pool state from ib_lru_dump after restart

Page 211: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

LRU dump backup(XtraBackup 2.0)

● XtraBackup 2.0 discovers ib_lru_dump and backs it up automatically

– buffer pool is in the warm state after restoring from a backup!

– make sure to enable buffer pool restore in my.cnf after restoring on a different server

● innodb_auto_lru_dump=1 (PS 5.1)

● innodb_buffer_pool_restore_at_startup=1 (PS 5.5)

Page 212: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

True incremental backups(XtraBackup 2.x)

● slow incremental backups

100 15003001100200 800 ibdata1...

scan page LSNs

1100 1500 ... ibdata1.delta

incremental LSN = 1000

● 2 solutions in future releases:– changed page tracking– log archiving

Page 213: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

True incremental backups(XtraBackup 2.x)

● changed page tracking (new feature in Percona Server)datadirdatadir

ibdata1ibdata1

actor.ibdactor.ibd

customer.ibdcustomer.ibd

film.ibdfilm.ibd

ib_logfile0ib_logfile0

ib_logfile1ib_logfile1

follow page updates

changed page bitmapschanged page bitmaps

ib_modified_log.*

incremental backup

xtrabackup--backup--incremental-lsn

ibdata1.deltaibdata1.delta

actor.ibd.deltaactor.ibd.delta

customer.ibd.deltacustomer.ibd.delta

film.ibd.deltafilm.ibd.delta

Page 214: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

True incremental backups:log archiving

● log archiving (a new feature in Percona Server)

datadirdatadir

ibdata1ibdata1

actor.ibdactor.ibd

customer.ibdcustomer.ibd

film.ibdfilm.ibd

ib_logfile0ib_logfile0

ib_logfile1ib_logfile1

log_archive_dirlog_archive_dir

ib_log_archive_LSN1

ib_log_archive_LSN2

ib_log_archive_LSN3

ib_log_archive_LSN...

...

copy

full backupfull backupxtrabackup--apply-log

ibdata1

actor.ibd

customer.ibd

film.ibd

Page 215: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Compact backupsIdea: skip certain pages on backup, recreate on prepare

1 5432 6 ibdata1...

pages

backup

421 page map file

Legend:

primary key page

secondary key page

unused page

Page 216: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Compact backups

421 page map file

prepare

1 5432 6 ...

- recreate secondary keys

no unused pages

backup copy

Legend:

primary key page

secondary key page

unused page

On prepare:- fix PK page offsets

- use page map to translatepage offsets when applying log records

- update pointers to root pages in data dictionary

Page 217: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Compact backups● Pros:

– smaller backups (sometimes a lot!)

– reclaim unused space (a very old problem, MySQL bug #1341)

● Cons:– --apply-log takes much longer

Page 218: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Built-in encryption:XtraBackup 2.x

● parallel! ● decrypt/restore individual tables

Data directoryData directory

ibdata1ibdata1

actor.ibdactor.ibd

customer.ibdcustomer.ibd

film.ibdfilm.ibd

I/O threads Encryptionthreads

1

2

3

4

1

8765432

libgcrypt.so

Page 219: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Merging innobackupex and xtrabackup● problems with innobackupex:

– legacy code inherited from InnoDB Hot Backup tool

– duplicates features from xtrabackup binary

– non-portable

● the plan is to rewrite it in C and merge with the xtrabackup binary (XtraBackup 2.x/3.0)

Page 220: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

XtraBackup 2.x● changed page tracking● log archiving● compact backups● built-in encryption● obsoleted innobackupex

Page 221: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

Percona XtraBackup: install, usage, tricks

Resources, further reading & feedback● XtraBackup documentation:

http://www.percona.com/doc/percona-xtrabackup/● Downloads:

http://www.percona.com/software/percona-xtrabackup/downloads/

● Google Group: http://groups.google.com/group/percona-discussion

● #percona IRC channel on Freenode

● Launchpad project:https://launchpad.net/percona-xtrabackup

● Bug reports:https://bugs.launchpad.net/percona-xtrabackup

Page 222: Do I/you need a backup? - Percona · productivity Percona XtraBackup: install, ... MySQL 5.1 + InnoDB-plugin ... you created 100GB MyISAM table and forgot

[email protected]

Percona XtraBackup: install, usage, tricks

http://bit.ly/PLNY2012-XB

This talk online:

Questions ?