Jehops / zap

Overview

zap automates the management of zfs snapshots. With a few crontab entries, it can be used to create a comprehensive zfs backup system. There are no configuration files. Parameters are supplied on the command line or in zfs properties and all snapshot information is stored in snapshot names.

zap will not interfere with manually created snapshots or snapshots from other tools. It will only operate on snapshots it creates.

Synopsis

# zap snap|snapshot [-DLSv] TTL [[-r] dataset]...

# zap rep|replicate [-DFLSv] [-h host] [[[user@]host:]parent_dataset [-r] dataset [[-r] dataset]...]

# zap destroy [-Dlsv] [host[,host]...]

# zap -v | -version | --version

Examples

Create snapshots that will expire after 3 weeks. The # prompt indicates commands that are run as root. A solution that delegates permissions for most of these commands to an unprivileged user is described here.

# zfs set zap:snap=on zroot/usr/home/nox zroot/var/
# zfs set zap:snap=off zroot/var/crash zroot/var/tmp zroot/var/mail
# zap snap 3w

Create snapshots specifying the datasets on the command line.

# zap snap 3w zroot/usr/home/nox -r zroot/var

Create snapshots that will expire after one day. Be verbose.

# zfs set zap:snap=on zroot/usr/home
# zap snap -v 1d

Replicate datasets to the remote host bravo, under the zback/phe dataset. If you use a non-default ssh port, specify it in ~/.ssh/config.

# zfs set zap:rep='zap@bravo:zback/phe' zroot/ROOT zroot/usr/home/jrm
# zap rep -v

Replicate datasets (recursively for zroot/ROOT) to the remote host bravo, under the rback/phe dataset, but this time specify the datasets on the command line. If you use a non-default ssh port, specify it in ~/.ssh/config.

# zap rep zap@bravo:rback/phe -r zroot/ROOT zroot/usr/home/jrm

Replicate datasets that originated from the host awarnach to the remote host bravo, under the zback/phe dataset. If you use a non-default ssh port, specify it in ~/.ssh/config. Filter the transfer through ~mbuffer~ by setting the ~ZAP_FILTER~ environment variable. Note that ~mbuffer~ must be installed on both the sending and receiving hosts.

# zfs set zap:rep='zap@bravo:zback/phe' zroot/ROOT zroot/usr/home/jrm
# ZAP_FILTER="mbuffer -s 128k -m 10M" zap rep -v -h awarnach

Destroy expired snapshots. Be verbose.

# zap destroy -v

Destroy expired snapshots that originated from either the host awarnach or the host gly. Be verbose.

# zap destroy -v awarnach,gly

Example crontab entries for rolling snapshots and remote replication.

• Refer to http://ftfl.ca/blog/2016-12-27-zfs-replication.html for a detailed description of a backup and replication strategy.
• Snapshots are created for datasets with the zap:snap property set to on.
• Datasets with the zap:rep property set are replicated.
• Taking snapshots is normally cheap, so do it often. Destroying snapshots can thrash disks, so only do it every 24 hours.

# crontab -e

#minute	hour	mday	month	wday	command

# take snapshots
*/5	*	*	*	*	zap snap 1d
14	*/4	*	*	*	zap snap 1w
14	00	*	*	1	zap snap 1m

# replicate datasets
54	*/1	*	*	*	zap rep -v

# destroy snapshots
44	04	*	*	*	zap destroy

Subcommands

snap | snapshot

Use the snap subcommand to create snapshots that will expire after TTL (time to live) has elapsed. An expired snapshot will be destroyed the next time zap destroy is run. TTL takes the form [0-9]{1,4}[dwmy]. That is, one to four digits followed by a character to represent the time unit (day, week, month, or year). If datasets are not not supplied on the command line, snapshots will be created for datasets with the property zap:snap set to on.

rep | replicate

Use the rep subcommand to replicate datasets. If a destination and datasets are not supplied on the command line, datasets with a destination set in the zap:rep user property are replicated. If the destination does not contain a host, or if the supplied host is one of localhost, 127.x.x.x, or ::1, then any user@ is ignored and ssh is not be used. If the canmount property of the local dataset is set to on, after replication an attempt is made to set canmount to noauto on the remote side. This is done to prevent mountpoint collisions. Set the ~ZAP_FILTER~ environment variable to a command to filter the transfer. For example, ~ZAP_FILTER“mbuffer -s 128k -m 10M”~= will filter the transfer through ~mbuffer~. Note that the filter command must be installed on both the sending and receiving hosts.

destroy

Use the destroy subcommand to destroy expired snapshots. By default, only snapshots originating from the local host are destroyed. If a comma separated list of hosts are specified, then only snapshots originating from those hosts are destroyed. Hosts must be specified without any domain information, that is, as returned by hostname -s.

Options

-v | -version | --version Show the version.

Subcommand options

-D Do not operate on snapshots when the pool is in a DEGRADED state.

-F Supply -F to zfs receive, which destroys remote changes that do not exist on the sending side.

-L Do not operate on snapshots if the pool has a resilver in progress. This is the default for the destroy subcommand.

-l Operate on snapshots, even if the pool has a resilver in progress. This is the default for the snap and rep subcommands.

-S Do not operate on snapshots if the pool is being scrubbed. This is the default for the destroy subcommand.

-s Operate on snapshots, even if the pool is being scrubbed. This is the default for the snap and rep subcommands.

-r Recursively create or replicate snapshots of all descendants.

-v Be verbose.

Author and Contributors

• Joseph Mingrone <[email protected]>
• Tobias Kortkamp <[email protected]>
• David Samms <[email protected]>
• Victor Naumov <[email protected]>
• Dries Michiels <[email protected]>
• HIROSE yuuji <[email protected]>
• Maxime Soulé
• sevmonster

License

zap is released under a BSD 2-Clause License. Refer to the header of each source file for details.

zap was influenced by zfSnap, which is under a BEER-WARE license. We owe the author a beer.