Physical backup with myrocks_hotbackup

Facebook's MySQL 5.6 includes MyRocks hot backup (online physical backup) tool "myrocks_hotbackup". With myrocks_hotbackup, you can easily take a physical copy from a running MyRocks instance to local or remote server, without stopping the source instance.

Usage

Here is an example usage.

[src_host]$ myrocks_hotbackup --user=root --password=pw --port=3306 --stream=tar --checkpoint_dir=$backup_dir | ssh -o NoneEnabled=yes dst_host 'tar -xi -C $dst_host_dir/backup_from_src'

[dst_host]$ myrocks_hotbackup --move_back --datadir=$datadir --rocksdb_datadir=$datadir/.rocksdb --rocksdb_waldir=$datadir --backup_dir=$dst_host_dir/backup_from_src

How it works

• myrocks_hotbackup is a tiny script (~700 lines), written in Python. It's located at scripts/myrocks_hotbackup.
• myrocks_hotbackup supports streaming backup, which is similar to xtrabackup. With streaming backup, you can easily send backups to remote servers via other transfer protocols such as SSH.
• myrocks_hotbackup supports tar and WDT (https://github.com/facebook/wdt).
• myrocks_hotbackup instructs MyRocks to create a checkpoint -- consistent snapshot via hard links. Hard links are stored at --checkpoint_dir, so it is important for --checkpoint_dir to be on the same filesystem as the --rocksdb_datadir directory. Then myrocks_hotbackup starts sending files under --checkpoint_dir. MyRocks must have write access under --checkpoint_dir.
• myrocks_hotbackup renews checkpoint every --interval seconds. We call it "Rolling Checkpoints". When renewing a checkpoint, myrocks_hotbackup discards old checkpoint then sending files from new checkpoint. myrocks_hotbackup remembers files sent and skips sending same file multiple files. Rolling Checkpoints has the following advantages.

1. Total checkpoint size is limited. If creating a checkpoint at starting backup then keeping it until backup ends, checkpoint size might become huge because old SST files remain after multiple compaction.
2. Starting MyRocks destination instance doesn't take much time. After taking backups, on destination instance, applying WAL files and catching up replication is needed. With Rolling Checkpoints, replication starts from the last round of the checkpoint, which is short enough period, independent from backup size. Without Rolling Checkpoint, you need to start replication from starting time of the backup, which may take very long time to catch up.

• myrocks_hotbackup sends MySQL meta files (*.frm) too. myrocks_hotbackup does not support InnoDB files.
• --move-back is needed to place SST (.sst), WAL (.log), *.frm files properly. With streaming backup, all files are sent to single destination directory (--backup_dir), so moving back (--move-back) is needed to move them. xtrabackup streaming has the same requirement.